2
$\begingroup$

For a network of the form:

Input(10)
Dense(200)
Dense(100+10)
Dense(20)
Output()

Those +10 outputs are what I want to add to the standard 20 outputs, for my loss function.

Is this possible - in theory or even with some pre-existing library?

$\endgroup$
0
2
$\begingroup$

Yes, you can do that, and it is a standard practice. One famous example is the "Inception" network architecture. To keep inner subnets from "dying out", several outputs from inner layers are extracted and passed into FC->Softmax. Then all the outputs are averaged in the loss function.

From practical point of view, you won't be able to implement such things with basic Sequential model construction. Most libraries allow you to move beyond it, though, and this distinction is usually well-documented. For example in the tensorflow guide on functional API it states right in the introduction:

The functional API is a way to create models that are more flexible than the tf.keras.Sequential API. The functional API can handle models with non-linear topology, shared layers, and even multiple inputs or outputs.

Hope that'll help you to get started.

$\endgroup$
1
  • $\begingroup$ Thanks I'll have a look at this $\endgroup$ – Tobi Akinyemi Apr 11 at 22:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.