# Questions tagged [loss-functions]

For questions related to the concept of loss (or cost) function in the context of machine learning.

157 questions
Filter by
Sorted by
Tagged with
9 views

### What is the intuition behind equations 10, 11 and 12 of the paper “Noise2Noise: Learning Image Restoration without Clean Data”?

Can anyone help me understand these functions described in the paper Noise2Noise: Learning Image Restoration without Clean Data I have read the portion A.4 in the appendix but need a more detailed and ...
26 views

### Explanation of this L2 minimization equation

I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
14 views

### Good resources on building your own objective function? [closed]

So my understanding is ML is great for prediction, but seems like in a lot of cases, the focus is on minimizing MSE or something like entropy for a single predicted variable. For situations where ...
103 views

### How should we interpret this figure that relates the perceptron criterion and the hinge loss?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is ...
39 views

### Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is ...
16 views

### Loss function decays linearly in segmentation MRI fascia

I am working on a segmentation of MRI images of the thigh. I am trying to segment the fascia, there is a slight imbalance between the background and the mask. I have about 1400 images from 30 patients ...
31 views

17 views

### How to calculate the attention loss in the paper “Tell Me Where to Look: Guided Attention Inference Network”?

I have been reading the research paper Tell Me Where to Look: Guided Attention Inference Network. In this paper, they calculate the attention loss, but I didn't understand how to calculate it. Do we ...
35 views

### Why would the loss increase on a single fixed input?

I'm training a neural network on some input data. I know that loss increasing may be related to: overfitting, if the loss increases on test data (while still decreases on training data) oscillations ...
46 views

### How could logistic loss be used as loss function for an ANN?

Normally, in practice, people use those loss functions with minima, e.g. $L_1$ mean absolute loss, $L_2$ mean squared error, etc. All those come with a minimum to optimize to. However, there's ...
92 views

### Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
44 views

### Non-trainable regularizer in loss function

I train a fully convoluted network for semantic segmentation. To each convolution blocks, I associate a module pruning feature maps to reduce the quantity of information generated by the network. From ...
53 views

### Classification or regression for deep Q learning

DQN implemented at https://github.com/PacktPublishing/PyTorch-1.x-Reinforcement-Learning-Cookbook/blob/master/Chapter07/chapter7/dqn.py uses the mean square error loss function for the neural network ...
54 views

### Loss Function In Units Of Bits?

Where can I find a machine learning library that implements loss functions measuring the Algorithmic Information Theoretic-friendly quantity "bits of ...
49 views

### Should illegal moves be excluded from loss calculation in DQN algorithm?

I'm implementing DQN algorithm to train my agent to play a turn-based game. The action space for the game is small, but not all moves are available at all the states. Therefore, when deciding on which ...
34 views

### Which loss function and evaluation metric should I use for a multiple output prediction problem?

I was running into a situation with a data set like this I have 4 events and and they might happen together in pairs. I want to use 3 features to predict the coupling between event. I am building a ...
16 views

### How to figure out loss weight for label-imbalanced regression problems?

In classification, suppose you have 1 image labeled as cancer and 99 labeled as not cancer, you can just divide the loss weight of "not cancer" by 99. Then you can train the model as this ...
86 views

### Why is it useful to track loss while model is being trained?

Why is it useful to track loss while the model is being trained? Options are: Loss is only useful as a final metric. It should not be evaluated while the model is being trained. Loss dictates how ...
37 views

### LSTM - MAPE Loss Function gives Better Results when Data is De-Scaled before Loss Calculation

I am building an LSTM for predicting a price chart. MAPE resulted in the best loss function compared to ...
33 views

### What are some good loss functions used to minimize extreme errors in regression and time series forecasting?

I'm working on a time series forecasting task, and, in some specific cases, I don't need perfect accuracy, but the network cannot by any means miss by a lot. So, in detriment of a smaller mean error, ...
71 views

### How can the target rely on untrained parameters?

I'm trying to understand DQN. I understand where the loss function comes from. I'm just unsure about why the target function works in practice. Given the loss function  L_i(\theta_i) = [(y_i - Q(s,a;...
54 views

### Is there a way of deriving a loss function given the neural network and training data?

There is some sort of art to using the right loss function. However, I was wondering if there is a way to derive the loss function if I gave you a neural network model (the weights) as well as the ...
33 views

### Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
43 views

### Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
100 views

### what will be the best loss function for unet to predict the each pixel values?

I'm predicting the used 9 pictures to predict the last picture so (40,40,9) -> unet -> (40,40,1) but as you see the predict picture It's not just a mask(0or 1) its float so which loss function ...
18 views

### Why do code implementations average the loss over a batch instead of finding the expected sample of that batch (using sampling probabilities)

Usually, our training objective over a batch is written in terms of the expected value of a sample in that batch such as $objective = E_{x \sim data} * log(P(x))$ But in the code implementations, ...
43 views

### How MSE should be appliead with multi target deep network?

I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical) Let us say I have a ...
75 views

### How does the DQN loss from td_targets against q_values make sense?

Why td_loss is calculated from (td_targets against q_values)? Why I am lost is because: <...
58 views

### Can neural networks handle redundant inputs?

I have a fully connected neural network with the following number of neurons in each layer [4, 20, 20, 20, ..., 1]. I am using TensorFlow and the 4 real-valued ...
92 views

### Loss function for choosing a subset of objects

I'm trying to train a neural net to choose a subset from some list of objects. The input is a list of objects $(a,b,c,d,e,f)$ and for each list of objects the label is a list composed of 0/1 - 1 for ...