# Questions tagged [loss-functions]

For questions related to the concept of loss (or cost) function in the context of machine learning.

117 questions
Filter by
Sorted by
Tagged with
36 views

### Is there a way of deriving a loss function given the neural network and training data?

There is some sort of art to using the right loss function. However, I was wondering if there is a way to derive the loss function if I gave you a neural network model (the weights) as well as the ...
17 views

### Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
8 views

### Keras: how to comiple a model with a custom loss and evaluate the model with another loss? [migrated]

I'm using Keras to do some experiments. As we said in the subject, is it possible to model.compile() with a custom loss and, then, use another loss with model.evaluate()?
29 views

### Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
10 views

### what will be the best loss function for unet to predict the each pixel values?

I'm predicting the used 9 pictures to predict the last picture so (40,40,9) -> unet -> (40,40,1) but as you see the predict picture It's not just a mask(0or 1) its float so which loss function ...
10 views

### Why do code implementations average the loss over a batch instead of finding the expected sample of that batch (using sampling probabilities)

Usually, our training objective over a batch is written in terms of the expected value of a sample in that batch such as $objective = E_{x \sim data} * log(P(x))$ But in the code implementations, ...
34 views

### How MSE should be appliead with multi target deep network?

I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical) Let us say I have a ...
45 views

### How does the DQN loss from td_targets against q_values make sense?

Why td_loss is calculated from (td_targets against q_values)? Why I am lost is because: <...
22 views

### Can neural networks handle redundant inputs?

I have a fully connected neural network with the following number of neurons in each layer [4, 20, 20, 20, ..., 1]. I am using TensorFlow and the 4 real-valued ...
73 views

### Loss function for choosing a subset of objects

I'm trying to train a neural net to choose a subset from some list of objects. The input is a list of objects $(a,b,c,d,e,f)$ and for each list of objects the label is a list composed of 0/1 - 1 for ...
34 views

### How to add weights to one specific input feature to ensure fair training in the network?

I am trying to create a multiclass product-rating network based on product reviews and other input features. Two of the other input features are "product category" and "gender". However, I want to ...
22 views

### Single label classification into hierarchical categories using a neural network

I am working on a classification problem into progressive classes. In other words, there is some hierarchy of categories in such a way, that A < B < C, e.g. low, medium, high, very high. What ...
72 views

### Why does TensorFlow docs discourage using softmax as activation for the last layer?

The beginner colab example for tensorflow states: Note: It is possible to bake this tf.nn.softmax in as the activation function for the last layer of the network....
35 views

### Is Mean Squared Error Loss function a good loss function for continuous variables $0 < x < 1$

Suppose I am utilising a neural network to predict the next state, $s'$ based on the current $(s, a)$ pairs. all my neural network inputs are between 0 and 1 and the loss function for this network ...
42 views

### What do these numbers represent in this picture of a surface?

The following image is a screenshot from a video tutorial that illustrates the concept of gradient descent algorithm with a 3D animation. Do the numbers on the top of the balls pointed out by the red ...
25 views

### How should I define the loss function when using DQN to estimate the probability density?

I'm doing a Deep Q-learning project. All of my rewards are positive and there are two terminal states. One of them has a zero reward and the other has a high positive reward. The rewards are ...
25 views

### How are the weights retained for filters for a particular class in a CNN?

I am new to CNN. What I have learned so far about the filters is that when we are giving a training example to our model, our model updates the weights by gradient descent to minimize the loss ...
20 views

242 views

### What's the function that SGD takes to calculate the gradient?

I'm struggling to fully understand the stochastic gradient descent algorithm. I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that ...
21 views

### Deduce properties of the loss functions from the training loss curves

I have two convex, smooth loss functions to minimise. During the training (a very simple model) using batch SGD (with tuned optimal learning rate for each loss function), I observe that the (log) loss ...
34 views

### How to understand my CNN's training results?

I created a multi-label classification CNN to classify chest X-ray images into zero or more possible lung diseases. I've been doing some configuration tests on it and analyzing its results and I'm ...
29 views

### Outliers detection problem in neural networks

Assuming we have big m x n input dataset with m x 1 output vector. It's a classification problem with only two possible values: either 1 or 0. Now the problem is that almost all elements of the output ...
6 views

### Should you use the log of the independent variable to train if you're using RMSLE?

So I'm working on an old Kaggle competition which requires you to predict the price of something, and the evaluation metric used is RMSLE. I found a tutorial for that data set, and the person in the ...
20 views

### Using U-NET for image semantic segmentation

If it is not the right place to ask this question, please tell me and I move it to the right place. I'm getting literally crazy trying to understand how U-NET works. Maybe it is very easy but I'm ...
111 views

### How do you interpret this learning curve?

Loss is MSE; orange is validation loss, blue training loss. The task is NN regression (18 inputs, 2 outputs), one layer 300 hidden units. Tuning the lr, mom, l2 regularization parameters this is the ...
42 views

### A generalized quadratic loss and Newton iteration for Support Vector Regression, why doesn't it generalize well?

I'm comparing the results of an Newton optimizer for a modified version of SVM ( a generalized quadratic loss, similar to the one stated in: A generalized quadratic loss for SVM ) with classic SVM^...
21 views

### How would the “best function” been constructed if there are no computationally limitations?

I am reading the Wikipedia article on gradient boosting. There is written: Unfortunately, choosing the best function $h$ at each step for an arbitrary loss function $L$ is a computationally ...
13 views

### Loss function for increasing the quality of the image when labels are not perfectly alligned

I am trying to increse the quality of the images that I gather from the microscope. That is a acoustic microscope and there are lots of technical details but in a nutshell the low quality images and ...
389 views

### What's the difference between RMSE and Euclidean distance, and when to use a custom loss? [closed]

I'm searching for a loss function that fits my project. Actually, I have two questions, but they are in the same direction. I take a look at the definition of the root mean squared error (RMSE) and ...
68 views

### Why is Jensen-Shannon divergence preferred over Kullback-Leibler divergence in measuring the performance of a generative network?

I have read articles on how Jensen-Shannon divergence is preferred over Kullback-Leibler in measuring how good a distribution mapping is learned in a generative network because of the fact that JS-...