Categorical just means that we will conduct multiclass classification. The output of the classifier is a binary vector. Each entry $x_i$ in the binary vector is a prediction whether or not the input is part of class $C_i$.
In that sense, categorical accuracy introduces nothing new: it is just the accuracy of a multiclass classifier. On the other hand, categorical cross-entropy refers to the joint entropy: $H(X_1, X_2) = - \sum{p(x_1,x_2)\log_2p(x_1, x_2)}$, where each random variable $X_i$ expresses whether or not the input is to be classified into class $C_i$. Kevin Murphy's book "Probabilistic Machine Learning: An introduction" is a great source of reference for many topics of machine learning, including cross-entropy and joint entropy.
The random variables could be mutually exclusive or not, depending on the problem.
- If they are mutually exclusive, we would allow only one of the classes to have value 1 during training and use the softmax activation function.
- If they are not mutually exclusive, we would allow multiple classes to have value 1 during training and use the sigmoid activation function.