I am reading the paper Tracking-by-Segmentation With Online Gradient Boosting Decision Tree. In Section 2.1, the paper says

Given training examples, $\left\{\left(\mathbf{x}_{i}, y_{i}\right) \mid \mathbf{x}_{i} \in \mathbb{R}^{n}\right.$ and $y_{i} \in$ $\mathbb{R}\}_{i=1: N}, f(\cdot)$ is constructed in a greedy manner by selecting parameter $\theta_{j}$ and weight $\alpha_{j}$ of a weak learner iteratively to minimize an augmented loss function given by $$ \mathcal{L}=\sum_{i=1}^{N} \ell\left(y_{i}, f\left(\mathbf{x}_{i}\right)\right) \equiv \sum_{i=1}^{N} \exp \left(-y_{i} f\left(\mathbf{x}_{i}\right)\right) $$ where an exponential loss function is adopted ${ }^{1}$. The greedy optimization procedure is summarized in Algorithm 1.

I cannot understand the exponential loss function. In my opinion, the loss function should get the smallest value when $y_i=f(x_i)$. But the loss function in the image obtains a smaller value if $(-y_i f(x_i))$ becomes smaller.


1 Answer 1


The loss is

$$\mathcal{L}=\sum_{i=1}^{N} \ell\left(y_{i}, f\left(\mathbf{x}_{i}\right)\right) \equiv \sum_{i=1}^{N} \exp \left(-y_{i} f\left(\mathbf{x}_{i}\right)\right),$$

which can also be written as follows

$$\mathcal{L} = \sum_{i=1}^{N} e^{-y_{i} f\left(\mathbf{x}_{i}\right)} \tag{1}\label{1}$$

The important thing to note here is the $-$ in the exponent, which allows us to write \ref{1} as follows (see this)

$$\mathcal{L} = \sum_{i=1}^{N} \frac{1}{e^{y_{i} f\left(\mathbf{x}_{i}\right)}} \tag{2}\label{2}$$

So, the loss becomes smaller the higher $y_{i}$ and $f\left(\mathbf{x}_{i}\right)$ are. If, for example, $y_{i}$ is negative and $f\left(\mathbf{x}_{i}\right)$ positive (or vice-versa), then \ref{2} would be higher. If both are positive or negative, then the loss will be smaller, as we will be summing fractions of the form $\frac{1}{e^k} < 1$ for some positive constant $k$: the higher the $k$, the smaller the loss.

So, if you use this loss, it seems that you want that

  1. $y_{i}$ and $f\left(\mathbf{x}_{i}\right)$ have the same sign (either both negative or both positive)
  2. $f\left(\mathbf{x}_{i}\right)$ is as high as possible (note that you cannot change $y_{i}$), which could even be much higher (in terms of magnitude) than $y_{i}$ (not sure why they would want this)

I don't know why they chose this loss function because I didn't read the paper (yet), but it seems to me that this is how you should interpret this loss function.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .