i have read a lot about NAS but I still do not understand one concept: When setting up a neural network, hyperparameters need to set up, like for example the learning rate, dropout rate, batch size, filter size... etc.

In NAS only the best architecture is decided, e.g. how many layers and neurons.. . But what about the hyperparameters? Are they randomly decided?

Thank you for your help


Great question! It's not clearly stated (it's not stated at all on wikipedia), but after a bit of searching, I found an answer here about a third of the way down the page:

The best performing architecture observed during the training of the controller is taken, and a grid search is performed over some basic hyperparameters such as learning rate and weight decay in order to achieve near STOTA (state of the art) performance.

So as a direct answer: The norm; A grid search.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.