Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

# Questions tagged [loss-functions]

For questions related to the concept of loss (or cost) function in the context of machine learning.

75 questions
Filter by
Sorted by
Tagged with
22k views

### Understanding GAN loss function

I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita). In the standard cross-entropy loss, we have an ...
3k views

### Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch

I'm training an auto-encoder network with Adam optimizer (with amsgrad=True) and ...
284 views

57 views

### How to stop DQN Q function from increasing during learning?

Following the DQN algorithm with experience replay: We calculate the $loss=(Q(s,a)-(r+Q(s+1,a)))^2$. Assume I have positive but changing rewards. Meaning, $r>0$. Thus, since the rewards are ...
66 views

### How to obtain a formula for loss, when given an iterative update rule in gradient descent?

From the reinforcement learning book section 13.3: Using pytorch, I need to calculate a loss, and then the gradient is calculated internally. How to obtain the loss from equations which are stated ...
27 views

### How is the percentage or the probablity calculated using Loss function in Facenet Model?

This question is related to What is the formula used to calculate the accuracy in the FaceNet model? . I know how loss is calculated in the FaceNet model , but how the loss function is used to ...
33 views

### Why is image classification tasks are dominated by minimizing cost function instead of maximizing ones?

I was watching a video of policy gradient by Andrej Karpathy at 10:00 there shows an equation for supervised learning for image classification. $max\sum _{i}log \:p(y_i|x_i)$ I have worked with ...
30 views

### What loss function is appropriate for finding “points of interest” in a array of x,y inputs

I am looking into whether a neural network is appropriate to detect "points of interest" (POI) in a set of tuples (say length, and some sensor value). A POI is essentially a quick change in the value ...
48 views

### What is the best loss function for convolution neural network and autoencoder?

What is the best choice for loss function in Convolution Neural Network and in Autoencoder in particular - and why? I understand that the MSE is probably not the best choice, because little ...
42 views

### Train and Test Accuracy of GRU network not increasing after 2nd epoch

So I´m currently implementing my first neural network using GRUs as a model and Keras as an implementation since it´s pretty highlevel. My problem is about the classification of 8 hour long timeseries ...
30 views

### Understanding how the loss was calculated for the SQuAD task in BERT paper

In the BERT paper, section 4.2 covers the SQuAD training. From my understanding, there are two extra parameters trained, they are two vectors with the same dimension as the hidden size, so the same ...
65 views

### Heavy loss and inaccurate answer in pytorch

As my first AI model I have decided to make an AI model to predict multiplication of two numbers EX - [2,4] = . I wrote the following code, but the loss is very high, around thousands, and it's ...
39 views

### Which local minima to choose according to the shape of the error surface?

The following plot shows error function output based on system weights. Two equal local minima are shown in green pointers. Note that the red dots are not related to the question. Considering the ...
49 views

### How do I calculate $max_{a′}Q(s′,a′,w−)$ when it is represented as a neural network?

Consider the following loss function $$L(\mathbf{w}) = [(r + \gamma max_{a'} Q(s', a', \mathbf{w^-})) - Q(s, a, \mathbf{w})]^2$$ where $Q(s, a, \mathbf{w^-})$ and $Q(s, a, \mathbf{w})$ are ...
245 views

### What is the derivative function used in backpropagration?

I'm learning AI, but this confuses me. The derivative function used in backpropagation is the derivative of activation function or the derivative of loss function? These terms are confusing: ...
24 views

### Training by one batch of examples, what does it mean

Say I have a batch of examples, each examples represent a state: ...
164 views

### Chess policy network

I am interested in making a simple chess engine using neural networks. I already have a fairly good value network but I can't figure out how to train a policy network. I know that Leela chess zero ...
I am reading the wiki: https://en.wikipedia.org/wiki/Gradient_boosting There there is written: Unfortunately, choosing the best function $h$ at each step for an arbitrary loss function $L$ is a ...