I am trying to understand the best loss function to be used with a convolutional neural network. I came to know that we can mix two loss functions. Can any body share in what case was it done and how?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ In cases like regularization loss functions are mixed (although its not very effective in case of NN's). You can look into generative NNs like VAE's where 2 loss functions are mixed and used each with a different purpose. $\endgroup$– user9947Commented Oct 15, 2019 at 12:22
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
Mixing loss functions is very possible. For example, in the case of neural style transfer, there is a style loss and a content loss. Both of them are backpropagated through the network. The final loss used for the backpropagation is a weighted sum of the losses. In the case of style transfer, it ensures that the image generated is not only imitating the style of the style image, but also keeping the original content. Sometimes there is also a third loss called variation loss. This is also applied to a weight and summed. Note the weight is a hyperparameter and is not changed during training. Also, the loss should not measure the same purpose, but rather a different purpose that you want to optimize together. Example code using PyTorch:
loss1 = torch.nn.MSELoss(a,b)
loss2 = torch.nn.MSELoss(b,c)
loss = loss1 * alpha + loss2 * beta
