Why is the exponential loss used in this case?

Question

I am reading the paper Tracking-by-Segmentation With Online Gradient Boosting Decision Tree. In Section 2.1, the paper says

Given training examples, $\left\{\left(\mathbf{x}_{i}, y_{i}\right) \mid \mathbf{x}_{i} \in \mathbb{R}^{n}\right.$ and $y_{i} \in$ $\mathbb{R}\}_{i=1: N}, f(\cdot)$ is constructed in a greedy manner by selecting parameter $\theta_{j}$ and weight $\alpha_{j}$ of a weak learner iteratively to minimize an augmented loss function given by $$ \mathcal{L}=\sum_{i=1}^{N} \ell\left(y_{i}, f\left(\mathbf{x}_{i}\right)\right) \equiv \sum_{i=1}^{N} \exp \left(-y_{i} f\left(\mathbf{x}_{i}\right)\right) $$ where an exponential loss function is adopted ${ }^{1}$. The greedy optimization procedure is summarized in Algorithm 1.

I cannot understand the exponential loss function. In my opinion, the loss function should get the smallest value when $y_i=f(x_i)$. But the loss function in the image obtains a smaller value if $(-y_i f(x_i))$ becomes smaller.

Answer 1

The loss is

$$\mathcal{L}=\sum_{i=1}^{N} \ell\left(y_{i}, f\left(\mathbf{x}_{i}\right)\right) \equiv \sum_{i=1}^{N} \exp \left(-y_{i} f\left(\mathbf{x}_{i}\right)\right),$$

which can also be written as follows

$$\mathcal{L} = \sum_{i=1}^{N} e^{-y_{i} f\left(\mathbf{x}_{i}\right)} \tag{1}\label{1}$$

The important thing to note here is the $-$ in the exponent, which allows us to write \ref{1} as follows (see this)

$$\mathcal{L} = \sum_{i=1}^{N} \frac{1}{e^{y_{i} f\left(\mathbf{x}_{i}\right)}} \tag{2}\label{2}$$

So, the loss becomes smaller the higher $y_{i}$ and $f\left(\mathbf{x}_{i}\right)$ are. If, for example, $y_{i}$ is negative and $f\left(\mathbf{x}_{i}\right)$ positive (or vice-versa), then \ref{2} would be higher. If both are positive or negative, then the loss will be smaller, as we will be summing fractions of the form $\frac{1}{e^k} < 1$ for some positive constant $k$: the higher the $k$, the smaller the loss.

So, if you use this loss, it seems that you want that

1. $y_{i}$ and $f\left(\mathbf{x}_{i}\right)$ have the same sign (either both negative or both positive)
2. $f\left(\mathbf{x}_{i}\right)$ is as high as possible (note that you cannot change $y_{i}$), which could even be much higher (in terms of magnitude) than $y_{i}$ (not sure why they would want this)

I don't know why they chose this loss function because I didn't read the paper (yet), but it seems to me that this is how you should interpret this loss function.