# Where can I find authentic references on "categorical cross entropy" and "categorical accuracy metric"?

My Python source code uses TensorFlow and Keras to implement a neural network.

The Keras source code uses something called "categorical cross-entropy" and "categorical accuracy metric". I have searched a lot of books on NN theory, and no one talks about these two specific terms. Yes, they talk about "cross-entropy" and "accuracy metric" but there are no mentions of "categorical ...".

N.B. These terms can be found only in the so-called "Hands-on" books.

Can anyone please supply me with authentic references on these two specific terminologies?

• – D.W.
Feb 14 at 3:41

Categorical just means that we will conduct multiclass classification. The output of the classifier is a binary vector. Each entry $$x_i$$ in the binary vector is a prediction whether or not the input is part of class $$C_i$$.
In that sense, categorical accuracy introduces nothing new: it is just the accuracy of a multiclass classifier. On the other hand, categorical cross-entropy refers to the joint entropy: $$H(X_1, X_2) = - \sum{p(x_1,x_2)\log_2p(x_1, x_2)}$$, where each random variable $$X_i$$ expresses whether or not the input is to be classified into class $$C_i$$. Kevin Murphy's book "Probabilistic Machine Learning: An introduction" is a great source of reference for many topics of machine learning, including cross-entropy and joint entropy.