Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
i have read a lot about NAS but I still do not understand one concept: When setting up a neural network, hyperparameters need to set up, like for example the learning rate, dropout rate, batch size, filter size... etc.
In NAS only the best architecture is decided, e.g. how many layers and neurons.. . But what about the hyperparameters? Are they randomly decided?
Thank you for your help
Great question! It's not clearly stated (it's not stated at all on wikipedia), but after a bit of searching, I found an answer here about a third of the way down the page:
The best performing architecture observed during the training of the controller is taken, and a grid search is performed over some basic hyperparameters such as learning rate and weight decay in order to achieve near STOTA (state of the art) performance.
So as a direct answer: The norm; A grid search.
