# Make an NN utilize other NNs as part of its decision process

Suppose I have a NN that learns to predict the time it takes a robot to move between two jobs. That's three inputs (for starters): robot, job A, job B. Not all robots travel at the same speed, and jobs are not symmetrically spaced.

Suppose I have another NN that learns how long a robot takes to do some given job. All robots can do all jobs, but the robots are not equally effective at all jobs.

Now suppose that I want to make some new NN that takes a list of robots and a list of jobs and gives me robot-to-job map and sequence that minimizes the time to accomplish all jobs. Conceptually, how can I make this third NN utilize the knowledge contained in the other two NNs? What do you call this type of architecture? What structural form for this third NN do you recommend?

In tensorflow models can be their own layers as part of a larger architecture.
(If you want to see an example you can check out a notebook I have here and here)

Specifically what you are trying to do is called transfer learning but in general, a NN model can take any numerical input set and use it to output another numerical output set. The input can even be the outputs of other models, and their hidden layers.

Suppose you have 2 models each with 4 layers, that take 2 datasets X1 & X2 and output 2 other sets:<br

X1 -> M1(L1,L2,L3,L4) -> Y1
X2 -> M2(L1,L2,L3,L4) -> Y2

You can directly concatenate the outputs (Y's) of each model into a new input to a 3rd model:

[Y1, Y2] = X3 -> M3(L1,...) -> Y3

or you can slice each model and at one of the hidden layers and feed those outputs to M3:

[M1L3, M2L3] = X3 -> M3(L1,...) -> Y3

You can even mix and match in any way you find that works:

[M1L3, M2L2, Y1, Y2] = X3 -> M3(...) -> Y3

If you use a TensorFlow model, 'layers' is a property of the model and you can slice them as needed to build custom layers.

However, if you just want to feed the outputs of each model [Y1,Y2] to M3, I would probably do this using tensorflow's functional api creating two branches:

Input1 = tf.keras.Layers.Input(...)
Input2 = tf.keras.Layers.Input(...)
x1 = M1(Input1)   #pretrained model 1
x2 = M2(Input2)   #pretrained model 2
Output = M3(x1)
Output = M3(x2)
model = tf.keras.Model(inputs=[Input1,Input2],
outputs=[Output]


Hope this helps! :)

• How do I tell TensorFlow to not update the borrowed models when training the conglomerate? Jun 17 at 2:25
• to prevent layers from being trained you can explicitly set a 'trainable' parameter for the models . Jun 17 at 10:33