# What is the definition of the hinge loss function?

I came across the hinge loss function for training a neural network model, but I did not know the analytical form for the same.

I can write the mean squared error loss function (which is more often used for regression) as

$$\sum\limits_{i=1}^{N}(y_i - \hat{y_i})^2$$

where $$y_i$$ is the desired output in the dataset, $$\hat{y_i}$$ is the actual output by the model, and $$N$$ is the total number of instances in our dataset.

Similarly, what is the (basic) expression for hinge loss function?

• The idea behind hinge loss (not obvious from its expression) is that the NN must predict with confidence i.e.its prediction score must exceed a certain threshold (a hyperparameter) for the loss to be 0. Hence while training the NN tries to predict with maximum confidence or exceed the threshold so that loss is 0.
– user9947
Feb 11 at 15:57

$$\ell(y) = \max(0, 1-t \cdot y) \tag{1}\label{1},$$ where
• $$t = \{-1, 1\}$$ is the label (so, if your labels are in the set $$\{0, 1 \}$$, you will have to first map them to $$\{-1, 1\}$$)
• $$y$$ is the output of the classifier (e.g. in the context of the linear SVM, $$y=\mathbf{w} \cdot \mathbf{x}+b$$, where $$\mathbf{w}$$ and $$b$$ are the parameter of the hyper-plane)