# Questions tagged [loss-functions]

For questions related to the concept of loss (or cost) function in the context of machine learning.

189 questions
Filter by
Sorted by
Tagged with
35 views

### Add a layer derivative in the loss function

I am writing a NN in pytorch and I want to add the derivative of the output with respect to one of the inner layers in the loss. Here is a simple example of what I mean: ...
202 views

### Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The MSE can be defined as $(\hat{y} - y)^2$, which should be equal to $(y - \hat{y})^2$, but I think their derivative is different, so I am confused of what derivative will I use for computing my ...
86 views

234 views

### Train and Test Accuracy of GRU network not increasing after 2nd epoch

So I´m currently implementing my first neural network using GRUs as a model and Keras as an implementation since it´s pretty highlevel. My problem is about the classification of 8 hour long timeseries ...
47 views

27 views

### Training by one batch of examples, what does it mean

Say I have a batch of examples, each examples represent a state: ...
4k views

### Can the mean squared error be negative?

I'm new to machine learning. I was watching a Prof. Andrew Ng's video about gradient descent from the machine learning online course. It said that we want our cost function (in this case, the mean ...
8k views

### What is an objective function?

Local search algorithms are useful for solving pure optimization problems, in which the aim is to find the best state according to an objective function. My question is what is the objective function?
503 views

### Chess policy network

I am interested in making a simple chess engine using neural networks. I already have a fairly good value network but I can't figure out how to train a policy network. I know that Leela chess zero ...
1k views

### How do we design a neural network such that the $L_1$ norm of the outputs is less than or equal to 1?

What are some ways to design a neural network with the restriction that the $L_1$ norm of the output values must be less than or equal to 1? In particular, how would I go about performing back-...
74 views

### How to understand marginal loglikelihood objective function as loss function (explanation of an article)?

I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned ...
81 views

### How to define a loss function for a classifier where the confusion between some classes is more important than the confusion between others?

I have a dataset of images belonging to $N$ classes, $A_1, A_2...A_n,B_1,B_2...B_m$ and I want to train a CNN to classify them. The classes can be considered as subclasses of two broader classes $A$ ...
2k views

### How do I calculate the gradient of the hinge loss function?

With reference to the research paper entitled Sentiment Embeddings with Applications to Sentiment Analysis, I am trying to implement its sentiment ranking model in Python, for which I am required to ...
4k views

### Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch

I'm training an auto-encoder network with Adam optimizer (with amsgrad=True) and ...
225 views

### Should the input to the negative log likelihood loss function be probabilities?

I am trying to train a supervised model where the output from the model is output of a linear function $WX + b$. Kindly note that I'm not using any softmax or $\log$ softmax on the result of the ...
259 views

### Extend the loss function from the single action to the n-action case per time step

My question concerns a side question (which was not answered) asked here: Policy gradients for multiple continuous actions I am trying to implement a simple policy gradient algorithm for a discrete ...