5
$\begingroup$

I have come across something that IBM offers called neural Architecture search. You feed it a data set and it outputs an initial neural Architecture that you can train.

How is neural architecture search (NAS) performed? Do they use heuristics or is this meta machine learning?

If you have any papers on NAS, I would appreciate if you can provide a link to them.

$\endgroup$
6
$\begingroup$

You could say that NAS fits into the domain of Meta Learning or Meta Machine learning.

I've pulled the NAS papers from my notes, this is a collection of papers/lectures that I personally found very interesting. It's sorted in rough chronological descending order, and *** means influential / must read.

Quoc V. Le and Barret Zoph are to good authors on the topic.

$\endgroup$
1
$\begingroup$

Here are two review articles:

$\endgroup$
  • $\begingroup$ The question is not about AutoML, but only NAS, so I suggest you remove your second suggestion. $\endgroup$ – nbro Nov 11 at 14:17
  • $\begingroup$ @nbro 1. about half of the paper is about NAS. 2. if there were tons of reviews one could get finicky, but losing one out of two? $\endgroup$ – Mathieu Bouville Nov 11 at 15:48
  • $\begingroup$ I don't get what you mean by "but losing one out of two" in this context. $\endgroup$ – nbro Nov 11 at 15:51
0
$\begingroup$

Neural architecture search (NAS) is a method of automating the design (that is, the choice of the values of the hyper-parameters) of artificial neural networks. There are different approaches to search the space of neural network architectures. For example, you can use reinforcement learning or evolutionary (or genetic) algorithms.

Check out the paper Neural Architecture Search with Reinforcement Learning (2017), by Barret Zoph and Quoc V. Le, where the authors train, using reinforcement learning (specifically, REINFORCE), a recurrent neural network (the "controller") to generate (convolutional and recurrent) neural network architectures, so that to maximise the expected accuracy of the generated architectures on a validation dataset. They achieve some good results using this approach.

See also Efficient Neural Architecture Search via Parameter Sharing (2018), by Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le and Jeff Dean (which thus includes some of the authors of NAS), which is similar to NAS, but more efficient, hence the acronym ENAS (efficient NAS).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.