# Questions tagged [objective-functions]

For questions related to the concept of loss (or cost) function in the context of machine learning.

214 questions
Filter by
Sorted by
Tagged with
153 views

### How can the sum of squared errors have negative gradient if it's defined as the squared of the error?

The formula for the sum of squared errors (SSE) is: $$\frac{1}{2} \sum_{i=1}^n (t^i - o^i)^2$$ I have a few related questions. If $t^i - o^i$ is negative, doesn't the power of 2 eliminate any ...
30 views

### Training by one batch of examples, what does it mean

Say I have a batch of examples, each examples represent a state: ...
5k views

### Can the mean squared error be negative?

I'm new to machine learning. I was watching a Prof. Andrew Ng's video about gradient descent from the machine learning online course. It said that we want our cost function (in this case, the mean ...
9k views

### What is an objective function?

Local search algorithms are useful for solving pure optimization problems, in which the aim is to find the best state according to an objective function. My question is what is the objective function?
624 views

### Chess policy network

I am interested in making a simple chess engine using neural networks. I already have a fairly good value network but I can't figure out how to train a policy network. I know that Leela chess zero ...
1k views

### How do we design a neural network such that the $L_1$ norm of the outputs is less than or equal to 1?

What are some ways to design a neural network with the restriction that the $L_1$ norm of the output values must be less than or equal to 1? In particular, how would I go about performing back-...
96 views

### How to understand marginal loglikelihood objective function as loss function (explanation of an article)?

I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned ...
103 views

### How to define a loss function for a classifier where the confusion between some classes is more important than the confusion between others?

I have a dataset of images belonging to $N$ classes, $A_1, A_2...A_n,B_1,B_2...B_m$ and I want to train a CNN to classify them. The classes can be considered as subclasses of two broader classes $A$ ...
3k views

### How do I calculate the gradient of the hinge loss function?

With reference to the research paper entitled Sentiment Embeddings with Applications to Sentiment Analysis, I am trying to implement its sentiment ranking model in Python, for which I am required to ...
4k views

### Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch

I'm training an auto-encoder network with Adam optimizer (with amsgrad=True) and ...
252 views

### Should the input to the negative log likelihood loss function be probabilities?

I am trying to train a supervised model where the output from the model is output of a linear function $WX + b$. Kindly note that I'm not using any softmax or $\log$ softmax on the result of the ...
331 views

### Extend the loss function from the single action to the n-action case per time step

My question concerns a side question (which was not answered) asked here: How can policy gradients be applied in the case of multiple continuous actions? I am trying to implement a simple policy ...