Why not go another layer deeper with Auto-AutoML?

Question

So I'm finding AutoML to be pretty interesting but I'm still learning how it all works. I've played with the incredibly broken AutoKeras and got some decent results.

The question is, if you are using a NN to optimize the architecture of another network, why not take it another layer deeper and use another network to find the optimum architecture for your Parent network with a grand-parent network?

The problem doesn't necessarily need to expand exponentially as the grand-parent network could do few-shot training sessions on the parent network which itself is doing few or one-shot training.

Answer 1

Logically it is possible, but you will just end up complicating the entire task.

The aim of AutoML is to provide a drop in solution to the customers. To do this, a trained network decides and generates the model architecture. This is done so that anyone with basic experience is able to integrate the solution into their systems.

Currently, the complicated architectures and networks require experienced data scientist to build, train and deploy. To overcome this bottleneck and make ML accessible to all, AutoML is being developed.

So adding another grand-parent network to optimize the autoML network will just complicate the task in terms to computation time and hyper parameter optimization.

In case we decide to add another network, now the researchers must look at this network and tune it in regards to the inner network and the model both. This means more work and no direct way to understand how the hyper parameters are affecting the final results.