# How could logistic loss be used as loss function for an ANN?

Normally, in practice, people use those loss functions with minima, e.g. $$L_1$$ mean absolute loss, $$L_2$$ mean squared error, etc. All those come with a minimum to optimize to.

However, there's another thing, logistic loss, I'm reading about, but don't get it why the logistic function could be used as a loss function, given that it has the so-called minimum at infinity, but that isn't a normal minimum. Logistic loss function (black curve):

How can an optimizer minimize the logistic loss?

## 1 Answer

I see why you might be confused. First, the logistic-loss or log-loss is technically called cross-entropy loss. This function is very simple:

$$CE = -[y \log(p) + (1 - y) \log(1 - p)]$$

This tells basically if the predicted class $$y$$ was right $$y=1$$ then the loss is $$CE=-\log(p)$$, if the predicted class was not the right one then the loss is $$CE=-\log(1-p)$$.

If we look at the function as a pure math concept we see that:

$$CE = f(x) = - \log(x)$$

And as you point out, that function is minima-unbounded as its domain is $$D(f(x)) = [0, +\infty]$$. You can check that in here:

However the trick is that the inputs must be bounded, meaning, the inputs to the loss function must be in range $$[0, 1]$$. This bounding is achieved by applying a sigmoid activation function as the final "layer" of the network. Then if the inputs to the loss function are bounded the function has a clear minima.

Check how the function looks in reality from one of the most important papers in loss function in the AI world: Focal Loss (I really encourage you to read it as the first section explains in detail the cross-entropy loss). The blue curve is the one you are looking for.

Finally, you might want to review your log-loss/CE function since it should have an asymptote for $$f(x=0) = \infty$$