Questions tagged [loss-functions]

For questions related to the concept of loss (or cost) function in the context of machine learning.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

What is the intuition behind equations 10, 11 and 12 of the paper “Noise2Noise: Learning Image Restoration without Clean Data”?

Can anyone help me understand these functions described in the paper Noise2Noise: Learning Image Restoration without Clean Data I have read the portion A.4 in the appendix but need a more detailed and ...
0
votes
1answer
26 views

Explanation of this L2 minimization equation

I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
0
votes
0answers
14 views

Good resources on building your own objective function? [closed]

So my understanding is ML is great for prediction, but seems like in a lot of cases, the focus is on minimizing MSE or something like entropy for a single predicted variable. For situations where ...
3
votes
1answer
103 views

How should we interpret this figure that relates the perceptron criterion and the hinge loss?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is ...
1
vote
1answer
39 views

Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is ...
1
vote
0answers
16 views

Loss function decays linearly in segmentation MRI fascia

I am working on a segmentation of MRI images of the thigh. I am trying to segment the fascia, there is a slight imbalance between the background and the mask. I have about 1400 images from 30 patients ...
1
vote
1answer
31 views

Explain the difference in graphical patterns between discriminator fake loss and generator loss in GAN

In GAN (generative adversarial networks), let us take "binary cross-entropy" as the loss function for discriminator $$(overall \; loss = -\sum log(D(x_i)) -\sum log(1-D(G(z_i))) $$ $$ where \...
0
votes
0answers
11 views

Understanding the MuZero loss function for a two-player game

This question is connected to a question that I asked some time ago. This is how I understood the training procedure takes place (please correct any conceptual mistakes here): Many complete games are ...
1
vote
0answers
22 views

What is a “center loss”?

I have seen that a center loss is beneficial in computer vision, especially in face recognition. I have tried to understand this concept from the following material A Discriminative Feature Learning ...
2
votes
1answer
46 views

What is the impact of scaling the KL divergence and reconstruction loss in the VAE objective function?

Variational autoencoders have two components in their loss function. The first component is the reconstruction loss, which for image data, is the pixel-wise difference between the input image and ...
-1
votes
1answer
22 views

Loss function definition

I have read what the loss function is but I am not sure if I have understood it. For each neuron in the output layer the loss function is equal most usually to the square of the difference value of ...
0
votes
0answers
17 views

How to afine the extremity values in regression prediction with Keras?

I made a stack of bidirectional LSTM layers following by Dense layers (with swish activation functions) in order to predict a continuous value between 0 and 2. I compiled the model with ...
1
vote
0answers
20 views

Could the data augmentation lead to the model learning features which corresponds to data augmented data and not to the real data?

I am trying to train a Unet network with Synthetic data to do binary segmentation due to the fact that is is not easy to collect real data. And there is something in the training process that I do not ...
0
votes
0answers
19 views

Wasserstein GAN with gradient penality - Loss values

I have trained a WAN with gradient penalty and the loss values ​​seem to me much higher than the examples I have seen on the net. The generator receives 2 images as input and must generate a ...
2
votes
1answer
34 views

Evaluate model multiple times in loss function? Is this reinforcement learning?

I am interested in models that exhibit behavior. My goal is a model that survives indefinitely on a two dimensional resource landscape. One dimension represents the location (0 to 1) and the second ...
0
votes
0answers
11 views

Loss function for better class separability in multi class classification

So I am trying to enforce better separability in my deep learning model and was wondering what I can use besides cross entropy loss to do that? Could maybe using logarithm with different basis in ...
0
votes
0answers
23 views

Choice of loss function for semantic segmentation

I am training a U-Net for semantic segmentation of large medical images (4096x4096px). The two classes are "too" unbalanced. The white pixels are just about 0.1% (or less) of the whole image....
1
vote
0answers
27 views

How to find distance between 2 points when dimensions are all of different nature?

I have a dataset with four features: the x coordinate the y coordinate the velocity magnitude angle Now, I want to measure the distance between two points in the dataset, taking into account the ...
1
vote
0answers
48 views

Which loss function to choose for imbalanced datasets?

For imbalanced datasets (either in the context of computer vision or NLP), from what I learned, it is good to use a weighted log loss. However, in competitions, the people who are in top positions are ...
2
votes
0answers
30 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
1
vote
1answer
24 views

Back propagation approach to logistic regression: why is cost diverging but accuracy increasing?

Background I have tried to fit a logistic regression model - written using a forward / back propagation approach (as part of Andrew Ng's deep learning course) - to a very non-linear data set (see ...
1
vote
0answers
27 views

What is the purpose of the DAMSM loss for the generators in AttnGAN?

I am confused about the training part in AttnGan. If you observe page 3. There are two types of losses for generator network: one involving the Deep Attentional Multimodal Similarity Model (DAMSM) ...
2
votes
1answer
27 views

Generation of 'new log probabilities' in continuous action space PPO

I have a conceptual question for you all that hopefully I can convey clearly. I am building an RL agent in Keras using continuous PPO to control a laser attached to a pan/tilt turret for target ...
0
votes
1answer
42 views

Is the error function known or unknown?

What is the error function? Is it the same as the cost function? Is the error function known or unknown? When I get the outcome of a neural net I compare it with the target value. The difference ...
2
votes
1answer
32 views

Why is the mean used to compute the expectation in the GAN loss?

From Goodfellow et al. (2014), we have the adversarial loss: $$ \min_G \, \max_D V (D, G) = \mathbb{E}_{x∼p_{data}(x)} \, [\log \, D(x)] \\ \quad\quad\quad\quad\quad\quad\quad + \, \mathbb{E}_{z∼p_z(...
0
votes
0answers
17 views

How to calculate the attention loss in the paper “Tell Me Where to Look: Guided Attention Inference Network”?

I have been reading the research paper Tell Me Where to Look: Guided Attention Inference Network. In this paper, they calculate the attention loss, but I didn't understand how to calculate it. Do we ...
1
vote
0answers
35 views

Why would the loss increase on a single fixed input?

I'm training a neural network on some input data. I know that loss increasing may be related to: overfitting, if the loss increases on test data (while still decreases on training data) oscillations ...
1
vote
1answer
46 views

How could logistic loss be used as loss function for an ANN?

Normally, in practice, people use those loss functions with minima, e.g. $L_1$ mean absolute loss, $L_2$ mean squared error, etc. All those come with a minimum to optimize to. However, there's ...
2
votes
1answer
92 views

Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
0
votes
0answers
44 views

Non-trainable regularizer in loss function

I train a fully convoluted network for semantic segmentation. To each convolution blocks, I associate a module pruning feature maps to reduce the quantity of information generated by the network. From ...
1
vote
0answers
53 views

Classification or regression for deep Q learning

DQN implemented at https://github.com/PacktPublishing/PyTorch-1.x-Reinforcement-Learning-Cookbook/blob/master/Chapter07/chapter7/dqn.py uses the mean square error loss function for the neural network ...
1
vote
0answers
54 views

Loss Function In Units Of Bits?

Where can I find a machine learning library that implements loss functions measuring the Algorithmic Information Theoretic-friendly quantity "bits of ...
2
votes
1answer
49 views

Should illegal moves be excluded from loss calculation in DQN algorithm?

I'm implementing DQN algorithm to train my agent to play a turn-based game. The action space for the game is small, but not all moves are available at all the states. Therefore, when deciding on which ...
0
votes
0answers
34 views

Which loss function and evaluation metric should I use for a multiple output prediction problem?

I was running into a situation with a data set like this I have 4 events and and they might happen together in pairs. I want to use 3 features to predict the coupling between event. I am building a ...
0
votes
0answers
16 views

How to figure out loss weight for label-imbalanced regression problems?

In classification, suppose you have 1 image labeled as cancer and 99 labeled as not cancer, you can just divide the loss weight of "not cancer" by 99. Then you can train the model as this ...
0
votes
1answer
86 views

Why is it useful to track loss while model is being trained?

Why is it useful to track loss while the model is being trained? Options are: Loss is only useful as a final metric. It should not be evaluated while the model is being trained. Loss dictates how ...
1
vote
0answers
37 views

LSTM - MAPE Loss Function gives Better Results when Data is De-Scaled before Loss Calculation

I am building an LSTM for predicting a price chart. MAPE resulted in the best loss function compared to ...
1
vote
0answers
33 views

What are some good loss functions used to minimize extreme errors in regression and time series forecasting?

I'm working on a time series forecasting task, and, in some specific cases, I don't need perfect accuracy, but the network cannot by any means miss by a lot. So, in detriment of a smaller mean error, ...
1
vote
0answers
71 views

How can the target rely on untrained parameters?

I'm trying to understand DQN. I understand where the loss function comes from. I'm just unsure about why the target function works in practice. Given the loss function $$ L_i(\theta_i) = [(y_i - Q(s,a;...
2
votes
1answer
54 views

Is there a way of deriving a loss function given the neural network and training data?

There is some sort of art to using the right loss function. However, I was wondering if there is a way to derive the loss function if I gave you a neural network model (the weights) as well as the ...
1
vote
0answers
33 views

Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
1
vote
0answers
43 views

Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
1
vote
0answers
100 views

what will be the best loss function for unet to predict the each pixel values?

I'm predicting the used 9 pictures to predict the last picture so (40,40,9) -> unet -> (40,40,1) but as you see the predict picture It's not just a mask(0or 1) its float so which loss function ...
0
votes
0answers
18 views

Why do code implementations average the loss over a batch instead of finding the expected sample of that batch (using sampling probabilities)

Usually, our training objective over a batch is written in terms of the expected value of a sample in that batch such as $objective = E_{x \sim data} * log(P(x))$ But in the code implementations, ...
1
vote
0answers
43 views

How MSE should be appliead with multi target deep network?

I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical) Let us say I have a ...
1
vote
1answer
75 views

How does the DQN loss from td_targets against q_values make sense?

Why td_loss is calculated from (td_targets against q_values)? Why I am lost is because: <...
0
votes
0answers
58 views

Can neural networks handle redundant inputs?

I have a fully connected neural network with the following number of neurons in each layer [4, 20, 20, 20, ..., 1]. I am using TensorFlow and the 4 real-valued ...
4
votes
1answer
92 views

Loss function for choosing a subset of objects

I'm trying to train a neural net to choose a subset from some list of objects. The input is a list of objects $(a,b,c,d,e,f)$ and for each list of objects the label is a list composed of 0/1 - 1 for ...
3
votes
1answer
46 views

How to add weights to one specific input feature to ensure fair training in the network?

I am trying to create a multiclass product-rating network based on product reviews and other input features. Two of the other input features are "product category" and "gender". However, I want to ...
2
votes
0answers
30 views

Single label classification into hierarchical categories using a neural network

I am working on a classification problem into progressive classes. In other words, there is some hierarchy of categories in such a way, that A < B < C, e.g. low, medium, high, very high. What ...