I have an idea to find the optimal number of hidden neurons required in a neural network but I'm not sure how accurate it is.

Assuming that it has only one hidden layer, it is a classification problem with one output node, has N input nodes for N features in the data set and every node is connected to every node in the next layer.

I’m thinking that to ensure that the network is able to extract all of the useful relations between the data, then every piece of data must be linked to every other piece of data like in a complete graph. So if you have 6 inputs, there must therefore be 15 edges to make it complete. Any more and it will be recomputing previously computed information and any less will be not computing every possible relation.

So if a network has 6 input nodes, 1 hidden node, 1 output node. There will be 6 + 1 connections. With 6 input nodes, 2 hidden nodes and 1 output node, there will be 12 + 2 connections. With 3 hidden nodes there will be 21 connections. therefore the hidden layer should have 3 hidden nodes to ensure all possibilities are covered.

 def complete_graph_edges(features):
    if features > 2: return complete_graph_edges(features-1) + features - 1
    else: return 1

# maximum is the maximum edges required to ensure all paths are covered, features is the amount of input features
def hidden_neuron_count(maximum, features):
    count = 1
    connections = features
    while(connections < maximum):
        count += 1
        connections = (count * features) + (count)
    return count

features = 6
print(hidden_neuron_count(complete_graph_edges(features), features)) # 3

https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw

The second answer from here discusses another method. For the sake of argument I’ve tried to keep both examples using the same data. If this idea is computed with 6 input features, 1 output node, alpha of 2 and 60 samples in the training set, this would result in a maximum of 4 hidden neurons. As 60 samples is very small, increasing this too 600 would result in a maximum of 42 hidden neurons.

Based on my idea I think there should be at most 3 hidden nodes and I can’t imagine anymore being useful, but would there be any reason to go beyond 3 and up to 42 like in the second example?