3
$\begingroup$

Let's say that I have a neural network with 2 heads. The first consists of X neurons. The second consists of Y neurons. I have these 2 heads because I want to predict 2 different variables. And I can see the loss for each head during training. Now, let's say that I have only one head that consists of X+Y neurons. I can interpret the output because I know that the first X neurons describe some variable and the latter Y neurons describe the second variable. I want to know if there is any difference between these 2 methods (maybe in performance or something). What are the pros and cons? Are there any advantages of one method over another for some particular tasks?

$\endgroup$
1
$\begingroup$

It depends on what your outputs are. For example, if both outputs are similar then you can use one output branch. However, what if the two outputs are different? With two output branches you can used two different loss functions. Now your model will optimize the two branches separately.

Imagine if you have a model that has to output a class label for the input and a real value describing something in the input. One is a classification task and the other is a regression task. And the two will require different loss functions, so you would use two branches.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.