How to create Partially Connected NNs with prespecified connections using Tensorflow?

Question

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer. So I want the architecture to be highly specified/customized from the get-go and I want the neural network to optimize the weights of the specified connections, while keeping everything else 0 during the forward pass AND the backpropagation (connection does not ever exist).

I am a complete beginner in neural networks. I have been recently working with tensorflow & keras to construct fully connected deep networks. Is there anything in tensorflow (or something else) that I should look into that might allow me to do this? I think with tf, I should be able to specify the computational graph such that only certain connections exist but I really have no idea yet where to start from to do this...

I came across papers/posts on network pruning, but it doesn't seem really relevant to me. I don't want to go back and prune my network to make it less over-parameterized or eliminate insignificant connections.

I want the connections to be specified and the network to be relatively sparse from the initialisation and stay that way during the back-propagation.

Answer 1

So I've done some more research after posting this and it looks like treating things as separate neural nets and connecting them later in subsequent hidden layers is a good idea. If anyone has other ideas that might be helpful, I'd love to hear, though!

Answer 2

In case that you want to connect a placeholder to a new layer you can do as below

x = tf.placeholder(shape=[None, 784], dtype=tf.float32) # mnist for example

x = tf.reshape(x, (-1, 784, 1)) # change to new shape (784 ,1)

x = tf.unstack(self._input,axis=1) # getting a list with 784 elements of 1

con = tf.concat(x[1,2,3,4,5]) # for instance you want only from 1 to 5 inputs/neurons

you can feed your new layer with above con which is only corresponds from the input 1 to 5. you can apply same technique to a layer instead of tf.placeholder