In a course that I am attending, the cost function of a support vector machine is given by

$$J(\theta)=\sum_{i=1}^{m} y^{(i)} \operatorname{cost}_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\right) \operatorname{cost}_{0}\left(\theta^{T} x^{(i)}\right)+\frac{\lambda}{2} \sum_{j=1}^{n} \Theta_{j}^{2}$$

where $\operatorname{cost}_{1}$ and $\operatorname{cost}_{0}$ look like this (in Magenta):

enter image description here

enter image description here

What are the values of the functions $\operatorname{cost}_{1}$ and $\operatorname{cost}_{0}$?

For example, if using logistic regression the values of $\operatorname{cost}_{1}$ and $\operatorname{cost}_{0}$ would be $-\log* \operatorname{sigmoid}(-z)$ and $-\log*(1-\operatorname{sigmoid}(-z))$.


That is the hinge loss, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:

$$ \ell(y)=\max (0,1-t \cdot y), $$

which, in your use case, is something like this:

$$ \operatorname{cost}\left(h_{\theta}(x), y\right)=\left\{\begin{array}{ll} \max \left(0,1-\theta^{T} x\right) & \text { if } y=1 \\ \max \left(0,1+\theta^{T} x\right) & \text { if } y=0 \end{array}\right. $$

Check this article on it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.