This might sound like a slightly odd question, but is there a good technique to make a fully connected neural network act as if it's partially connected - by having certain edges not propagate a signal either forward or back?
I have several small neural networks, each of which is structurally identical - they all have the same inputs, all have a single output, and the same number and size of hidden layers.
I was hoping to take advantage of hardware acceleration by putting them in the same NN, so rather than having five networks with 5 input, 1 output and a single hidden layer of 20 neurones each, I'd have a single network with 5 inputs, 5 outputs, and a hidden layer with 100 neurones.
But I need them to not cross-contaminate one-another - to operate as if they were distinct networks and not learn based on one-another.
I know I can manually set the initial weights to zero between the different sub-networks, which works fine initially - but during training the error-correction ends up connecting the various networks together.
It occurs to me that I might be able to have another matrix very similar to weights, but this time being a simple '1' (open) or '0' (closed) multiplier I could attach to the learning rate - 'closed' links would be multiplied by 0 during training, resulting in no change.
But if anyone knows if there's a different technique that works, I'd love to know.
