New answers tagged


Can residual connections be beneficial when we have a small training dataset? The usual rule of data science investigations applies here: Try it, measure the results, then you will know. It is very hard to tell, a priori, whether a specific architectural or hyperparameter choice will impact the performance of a neural network on a given problem. In this ...


I'm not aware of a direct way for finding the best NN architecture for a given task, but the recommended way, as far as I know, is to devise a network that can overfit the training data, and then apply regularization on top of it. That way, you can be almost sure you're not underfitting/underperforming due to network capacity.

Top 50 recent answers are included