The question looks foolish, but I think cross-entropy is somewhat weird as a cost function.

What I mean is, for example mean square error as a cost function for linear regression, it seems quite rational. Because it literally measures error between real value and predicted value, directly.

For mathematic expression : $ \sum_{i=1}^{n} (y_i - (ax_i+b)) ^2$

But about cross-entropy, I do not understand what it is.

For multi-class classification, for example, with 3 classes:

true target =[ 0 0 1 ]

output of model = [ 0.2 0.3 0.5 ] (maybe with softmax activation at last layer)

So the error of it is : $C(x) = -(0*log(0.2) + 0*log(0.3) + 1*log(0.5))$

It looks... I don't know, why it is a "Error?" and how can it make update with backpropagation?

Also, what is the objective of it? Maybe optimization, so maybe minimizing error? then what happens?