# Is it possible to use an internal layer's outputs in a loss function?

For a network of the form:

Input(10)
Dense(200)
Dense(100+10)
Dense(20)
Output()


Those +10 outputs are what I want to add to the standard 20 outputs, for my loss function.

Is this possible - in theory or even with some pre-existing library?

## 1 Answer

Yes, you can do that, and it is a standard practice. One famous example is the "Inception" network architecture. To keep inner subnets from "dying out", several outputs from inner layers are extracted and passed into FC->Softmax. Then all the outputs are averaged in the loss function.

From practical point of view, you won't be able to implement such things with basic Sequential model construction. Most libraries allow you to move beyond it, though, and this distinction is usually well-documented. For example in the tensorflow guide on functional API it states right in the introduction:

The functional API is a way to create models that are more flexible than the tf.keras.Sequential API. The functional API can handle models with non-linear topology, shared layers, and even multiple inputs or outputs.

Hope that'll help you to get started.

• Thanks I'll have a look at this Apr 11 at 22:21