Questions tagged [loss]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
23 views

WGAN-GP Loss formalization

I have to write the formalization of the loss function of my network, built following the WGAN-GP model. The discriminator takes 3 consecutive images as input (such as 3 consecutive frames of a video) ...
1
vote
1answer
46 views

Why does the accuracy drop while the loss decrease, as the number of epochs increases?

I've been trying to find the optimal number of epochs that I should train my neural network (that I just implemented) for. The visualizations below show the neural network being run with a variable ...
0
votes
0answers
11 views

Loss function for better class separability in multi class classification

So I am trying to enforce better separability in my deep learning model and was wondering what I can use besides cross entropy loss to do that? Could maybe using logarithm with different basis in ...
0
votes
0answers
38 views

Variance of the Gaussian policy is not decreasing while training the agent using Soft Actor-Critic method

I've written my own version of SAC(v2) for a problem with continuous action space. While training, the losses for the value network and both q functions steadily decrease down to 0.02-0.03. The loss ...
1
vote
0answers
32 views

Why does loss and accuracy for a multi label classification ann does not change overtime?

I have run into a strange behavior of my multi label classification ANN ...
2
votes
1answer
92 views

Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
0
votes
0answers
44 views

Non-trainable regularizer in loss function

I train a fully convoluted network for semantic segmentation. To each convolution blocks, I associate a module pruning feature maps to reduce the quantity of information generated by the network. From ...
0
votes
0answers
39 views

VAE KL divergence loss decreases really fast

I am new to training VAEs and I am using it on some 16x16 images, that contains some images from a physics experiment with one or 2 events i.e. the images are mainly black, except for one or 2 regions ...
1
vote
0answers
37 views

LSTM - MAPE Loss Function gives Better Results when Data is De-Scaled before Loss Calculation

I am building an LSTM for predicting a price chart. MAPE resulted in the best loss function compared to ...
1
vote
0answers
93 views

what will be the best loss function for unet to predict the each pixel values?

I'm predicting the used 9 pictures to predict the last picture so (40,40,9) -> unet -> (40,40,1) but as you see the predict picture It's not just a mask(0or 1) its float so which loss function ...
1
vote
1answer
67 views

Do smaller loss values during DQN training produce better policies?

During the training of DQN, I noticed that the model with prioritized experience replay (PER) had a smaller loss in general compared to a DQN without PER. The mean squared loss was an order of ...
1
vote
0answers
43 views

How MSE should be appliead with multi target deep network?

I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical) Let us say I have a ...
2
votes
1answer
32 views

Data scan not making sense for coco dataset

I am doing a simple scan to see how dataset size affects training. Basically, I took 10% of the coco dataset and trained a yolov3 net (from scratch) to just look for people. Then I took 20% of the ...
2
votes
0answers
19 views

Why is the loss of one of the outputs of a model with multiple outputs increasing while the others are decreasing?

I'm a newbie in neural networks. I'm trying to fit my neural network that has 3 different outputs: semantic segmentation, box mask and box coordinates. When my model is training, the loss of ...
1
vote
0answers
28 views

Crossing of training and validation loss

During training of my models I often encounter the following situation about training (green) and validation (gray) loss: Initially, the validation loss is significantly lower than the training loss. ...
1
vote
0answers
28 views

Deduce properties of the loss functions from the training loss curves

I have two convex, smooth loss functions to minimise. During the training (a very simple model) using batch SGD (with tuned optimal learning rate for each loss function), I observe that the (log) loss ...
3
votes
1answer
76 views

What is the best way to smoothen out a loss curve plot

I am currently using a loss averaged over the last 100 iterations, but this leads to artifacts like the loss going down even when the current iteration has an average loss, because the loss 100 ...
1
vote
1answer
45 views

How to explain peak in training history of a convolutional neural network?

I am training a simple convolutional neural network to recognize two types of 1024-point frequency spectra (FFT). This is the model I'm using: ...
1
vote
1answer
46 views

How to plot Loss Landscape with more than 2 weights in the network

For a single neuron with 2 weights, I can plot the loss landscape and it looks like this (OR data, sigmoid activation, MAE loss): But, when the neuron accepts more inputs, which means more than 2 ...
2
votes
1answer
102 views

How to perform back-propagation in Decoupled Neural Interfaces?

I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces (DNIs) as explained in the paper Decoupled Neural Interfaces using Synthetic Gradients (...