I am reading a paper "Tracking-by-Segmentation With Online Gradient Boosting Decision Tree". In Section 2.1, the paper says

enter image description here

I cannot understand the exponential loss function. In my opinion, the loss function should get the smallest value when $y_i=f(x_i)$. But the loss function in the image obtains a smaller value if $(-y_i f(x_i))$ becomes smaller.

  • 1
    $\begingroup$ Hello, welcome to Artificial Intelligence Stack Exchange. If possible, please try to quote the data provided in text rather than image. $\endgroup$
    – hanugm
    Jul 25 at 11:32
  • $\begingroup$ @hanugm Thanks for your suggestions! $\endgroup$
    – Zhang Liao
    Jul 25 at 12:27

The loss is

$$\mathcal{L}=\sum_{i=1}^{N} \ell\left(y_{i}, f\left(\mathbf{x}_{i}\right)\right) \equiv \sum_{i=1}^{N} \exp \left(-y_{i} f\left(\mathbf{x}_{i}\right)\right),$$

which can also be written as follows

$$\mathcal{L} = \sum_{i=1}^{N} e^{-y_{i} f\left(\mathbf{x}_{i}\right)} \tag{1}\label{1}$$

The important thing to note here is the $-$ in the exponent, which allows us to write \ref{1} as follows (see this)

$$\mathcal{L} = \sum_{i=1}^{N} \frac{1}{e^{y_{i} f\left(\mathbf{x}_{i}\right)}} \tag{2}\label{2}$$

So, the loss becomes smaller the higher $y_{i}$ and $f\left(\mathbf{x}_{i}\right)$ are. If, for example, $y_{i}$ is negative and $f\left(\mathbf{x}_{i}\right)$ positive (or vice-versa), then \ref{2} would be higher. If both are positive or negative, then the loss will be smaller, as we will be summing fractions of the form $\frac{1}{e^k} < 1$ for some positive constant $k$: the higher the $k$, the smaller the loss.

So, if you use this loss, it seems that you want that

  1. $y_{i}$ and $f\left(\mathbf{x}_{i}\right)$ have the same sign (either both negative or both positive)
  2. $f\left(\mathbf{x}_{i}\right)$ is as high as possible (note that you cannot change $y_{i}$), which could even be much higher (in terms of magnitude) than $y_{i}$ (not sure why they would want this)

I don't know why they chose this loss function because I didn't read the paper (yet), but it seems to me that this is how you should interpret this loss function.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.