I am quite new to neural networks, and would like to save myself some of the learning curve by having some rules of thumb about hidden layer sizes.
I would also like to have a rule of thumb for the ratio between the size of the dataset and the size or depth of the network.
Currently, for example, I need to create an autoencoder for a dataset which is about 3000 images of a small size, say 25x25.
I would assume the innermost layer would be of size ~10, but I don't really know why would it be either more or less, or what I should see in order to want to change it in either direction.
Same goes for any other layer.
Are there any thumb rules for network sizes? Are there any thumb rules for optimization of sizes? I find it hard to believe trial and error is all there is, but I found now course that teaches how to do that.