# Questions tagged [loss-functions]

For questions related to the concept of loss (or cost) function in the context of machine learning.

137 questions
Filter by
Sorted by
Tagged with
195 views

### Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The MSE can be defined as $(\hat{y} - y)^2$, which should be equal to $(y - \hat{y})^2$, but I think their derivative is different, so I am confused of what derivative will I use for computing my ...
74 views

162 views

### Train and Test Accuracy of GRU network not increasing after 2nd epoch

So I´m currently implementing my first neural network using GRUs as a model and Keras as an implementation since it´s pretty highlevel. My problem is about the classification of 8 hour long timeseries ...
187 views

### Artifacts After pruning Unet CNN

Im trying to make a dark image brighter using CNN-UNet arcitecture. When I train the network I get the following results: When I cut the features in half for pruning, and do full train again, I get ...
40 views

304 views

### What is the derivative function used in backpropagration?

I'm learning AI, but this confuses me. The derivative function used in backpropagation is the derivative of activation function or the derivative of loss function? These terms are confusing: ...
113 views

### SSE involving power of 2 eliminates negative gradient?

I think I missunderstood something when it comes to the SSE (= sum of squared errors) as a loss function. So: the formula for the SSE is: But, if ti - oi is negative, doessn't the power of 2 ...
26 views

### Training by one batch of examples, what does it mean

Say I have a batch of examples, each examples represent a state: ...
3k views

### Can the mean squared error be negative?

I'm new to machine learning. I was watching a Prof. Andrew Ng's video about gradient descent from the machine learning online course. It said that we want our cost function (in this case, the mean ...
342 views

### Chess policy network

I am interested in making a simple chess engine using neural networks. I already have a fairly good value network but I can't figure out how to train a policy network. I know that Leela chess zero ...
48 views

### How to understand marginal loglikelihood objective function as loss function (explanation of an article)?

I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned ...
70 views

### How to define a loss function for a classifier where the confusion between some classes is more important than the confusion between others?

I have a dataset of images belonging to $N$ classes, $A_1, A_2...A_n,B_1,B_2...B_m$ and I want to train a CNN to classify them. The classes can be considered as subclasses of two broader classes $A$ ...
1k views

### How do I calculate the gradient of the hinge loss function?

With reference to the research paper entitled Sentiment Embeddings with Applications to Sentiment Analysis, I am trying to implement its sentiment ranking model in Python, for which I am required to ...
3k views

### Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch

I'm training an auto-encoder network with Adam optimizer (with amsgrad=True) and ...
130 views

### Should the input to the negative log likelihood loss function be probabilities?

I am trying to train a supervised model where the output from the model is output of a linear function $WX + b$. Kindly note that I'm not using any softmax or $\log$ softmax on the result of the ...
244 views

### Extend the loss function from the single action to the n-action case per time step

My question concerns a side question (which was not answered) asked here: Policy gradients for multiple continuous actions I am trying to implement a simple policy gradient algorithm for a discrete ...