How does one make it obvious that the structure of a neural network should be what it is?

Question

I am a beginner: I've only read a book about neural network and barely implemented one in C.

In short:

• A neural network is built out of nodes,
• Each node holds an output: activation.(sum.(x * w)),
• We then compute the total error out of the network output.

From a beginner perspective, hyper-parameters, such as the number of layers needed, seem to be defined arbitrarily in most tutorials, books. In fact, the whole structure seems to be quite arbitrarily defined. In practice, hyper-parameters are often defined based on some standards.

My question is, if you were to talk to a total beginner, how would you explain to him the structure of a neural network in such a way that the whole thing would appear as obvious ? Is that even possible ?

Here, the word structure refers to a neural network being a configuration of nodes inside layers.

Thanks to anyone pointing out ambiguities or spelling errors.

Edit: note that I actually understand the whole back-propagation algorithm. I have no problem visualizing a nn.

Answer 1

While, as you begin to hit on, there are general guidelines to follow when building a neural network, they are far from standardized. This is because even though AI is a reasonably old field(1950s), neural networks have only been the tool of choice for less than a decade. Before, NNs did horribly, due to lack of data, and computation along with some less than efficient architectures.

With that being said Hinton's general rule is to add nodes/layers till the model begins to overfit and then add dropout(pretty reasonable in practice).

As such, the whole field is essentially an art as much as a science currently with only basic guidelines to follow based on your problem and data. This is part of the beauty though imo, with there being so much left to discover.

Hope that helped answer your question!