How to select the adjust layer num K?

[jqliu42]

Hi,I'd like to ask a question about the hyperparameter K. How to select a suitable K for models with different depth? Do you use the same k for resnet20 and resnet56 ?
Thanks for your help!

[danczs]

Thanks for your question.
We use different k for networks with different detphs. For a network with L searched layers, we use k = L/3 , by which the number of the increased, decreased and unchannged layers is all L/3.
By the way, if you wants to get better results, it is suggested to tune k in the range of L/6~L/2 and use 'update_num_decay' to reudce k during search.