Questions tagged [loss]
For questions related to the concept of loss (or cost) in machine learning or other AI sub-fields.
20
questions with no upvoted or accepted answers
3
votes
1answer
120 views
How to perform back-propagation in Decoupled Neural Interfaces?
I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces (DNIs) as explained in the paper Decoupled Neural Interfaces using Synthetic Gradients (...
2
votes
1answer
86 views
Do smaller loss values during DQN training produce better policies?
During the training of DQN, I noticed that the model with prioritized experience replay (PER) had a smaller loss in general compared to a DQN without PER. The mean squared loss was an order of ...
2
votes
1answer
34 views
Data scan not making sense for coco dataset
I am doing a simple scan to see how dataset size affects training. Basically, I took 10% of the coco dataset and trained a yolov3 net (from scratch) to just look for people. Then I took 20% of the ...
2
votes
0answers
19 views
Why is the loss of one of the outputs of a model with multiple outputs increasing while the others are decreasing?
I'm a newbie in neural networks. I'm trying to fit my neural network that has 3 different outputs:
semantic segmentation,
box mask and
box coordinates.
When my model is training, the loss of ...
1
vote
0answers
33 views
Could the inputs of the mean squared-error loss function be transformed to allow larger learning rates?
In the context of a neural network $\hat{y} = f_\theta(\mathbf{x})$ with parameters $\theta$ that is trained to perform regression such that the prediction $\hat{\mathbf{y}} = [\hat{y}_1,\hat{y}_2,...,...
1
vote
0answers
11 views
How does the loss landscape look like or change when a model is overfitting?
My understanding is that when a model starts overfitting, it no longer learns useful features and starts remembering the training data set. Given enough epochs and sufficient parameters, a model can ...
1
vote
1answer
63 views
Can someone explain me what does this loss curve says?
I was training a CNN model on TensorFlow. After a while I came back and saw this loss curve:
The green curve is training loss and the gray one is validation loss. I know that before epoch 394 the ...
1
vote
0answers
30 views
WGAN-GP Loss formalization
I have to write the formalization of the loss function of my network, built following the WGAN-GP model. The discriminator takes 3 consecutive images as input (such as 3 consecutive frames of a video) ...
1
vote
0answers
36 views
Why does loss and accuracy for a multi label classification ann does not change overtime?
I have run into a strange behavior of my multi label classification ANN
...
1
vote
0answers
65 views
LSTM - MAPE Loss Function gives Better Results when Data is De-Scaled before Loss Calculation
I am building an LSTM for predicting a price chart. MAPE resulted in the best loss function compared to ...
1
vote
0answers
231 views
what will be the best loss function for unet to predict the each pixel values?
I'm predicting the used 9 pictures to predict the last picture
so (40,40,9) -> unet -> (40,40,1)
but as you see the predict picture
It's not just a mask(0or 1) its float
so which loss function ...
1
vote
0answers
48 views
How MSE should be appliead with multi target deep network?
I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical)
Let us say I have a ...
1
vote
0answers
28 views
Crossing of training and validation loss
During training of my models I often encounter the following situation about training (green) and validation (gray) loss:
Initially, the validation loss is significantly lower than the training loss. ...
1
vote
0answers
32 views
Deduce properties of the loss functions from the training loss curves
I have two convex, smooth loss functions to minimise. During the training (a very simple model) using batch SGD (with tuned optimal learning rate for each loss function), I observe that the (log) loss ...
0
votes
0answers
14 views
KL divergence coefficient update doesn't make sense in RLlib's PPO implementation
I am using RLlib (Ray 1.4.0)'s implementation of PPO for a multi-agent scenario with continuous actions, and I find that the loss includes the KL divergence penalty term, apart from the surrogate loss,...
0
votes
0answers
23 views
How to interpret the training loss curves in Soft-Actor-Critic (SAC)?
I am using stable-baseline3 implementation of the Soft-Actor-Critic (SAC) algorithm. The plotted training curves look promising. However, I am not fully sure how to interpret the actor and critic ...
0
votes
0answers
23 views
Incorporating regularization for the kernel perceptron
To my understanding, the following is how the kernel perceptron works.
Kernel perceptron algorithm
The parameters to be calculated are $\alpha = \begin{pmatrix} \alpha_1 &\ldots &\...
0
votes
0answers
17 views
Loss function for better class separability in multi class classification
So I am trying to enforce better separability in my deep learning model and was wondering what I can use besides cross entropy loss to do that? Could maybe using logarithm with different basis in ...
0
votes
0answers
46 views
Variance of the Gaussian policy is not decreasing while training the agent using Soft Actor-Critic method
I've written my own version of SAC(v2) for a problem with continuous action space. While training, the losses for the value network and both q functions steadily decrease down to 0.02-0.03. The loss ...
0
votes
0answers
44 views
Non-trainable regularizer in loss function
I train a fully convoluted network for semantic segmentation. To each convolution blocks, I associate a module pruning feature maps to reduce the quantity of information generated by the network. From ...
