In a course that I am attending, the cost function of a support vector machine is given by

$$J(\theta)=\sum_{i=1}^{m} y^{(i)} \operatorname{cost}_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\right) \operatorname{cost}_{0}\left(\theta^{T} x^{(i)}\right)+\frac{\lambda}{2} \sum_{j=1}^{n} \Theta_{j}^{2}$$

where $\operatorname{cost}_{1}$ and $\operatorname{cost}_{0}$ look like this (in Magenta):

enter image description here

enter image description here

What are the values of the functions $\operatorname{cost}_{1}$ and $\operatorname{cost}_{0}$?

For example, if using logistic regression the values of $\operatorname{cost}_{1}$ and $\operatorname{cost}_{0}$ would be $-\log* \operatorname{sigmoid}(-z)$ and $-\log*(1-\operatorname{sigmoid}(-z))$.


1 Answer 1


That is the hinge loss, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:

$$ \ell(y)=\max (0,1-t \cdot y), $$

which, in your use case, is something like this:

$$ \operatorname{cost}\left(h_{\theta}(x), y\right)=\left\{\begin{array}{ll} \max \left(0,1-\theta^{T} x\right) & \text { if } y=1 \\ \max \left(0,1+\theta^{T} x\right) & \text { if } y=0 \end{array}\right. $$

Check this article on it.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .