When does a neural network have a single and when does it have multiple outputs?

What I understand is, each input in a neural network is a feature.

However, what I don't understand is, when we need multiple outputs in a neural network.

For example, say, if we are classifying cats and dogs, only one output is enough. 0 = cat, 1 = dog.

When does a neural network have a single and when does it have multiple outputs?

The output depends on what answer you want from the network. Think of the network as a function $$f$$ with weights $$\theta$$, that takes an input $$x$$ and gives some output $$y$$:
$$f_\theta(X) \rightarrow y$$
The example you give (dog=1 or cat=0) is binary classification -- the answer from the network is "this looks more like a dog than a cat" if the output $$y > 0.5$$ or vice versa. So yes, $$y$$ is a scalar between 0 and 1.
If you had three or more classes e.g [dog, cat, banana] (multimomial classification), you want the network to answer how "confident" it is that the $$x$$ is a certain class. It will give a prediction for each class. In this example $$y$$ will be a vector of size 3, each element being a prediction for a class e.g y = $$[dog=0.3, cat=0.01, banana=0.7]$$
For a non-classification example: say a robot's action comprises the velocity $$v$$ and a joint angle $$\phi$$ (action = $$[v, \phi]$$). The robot's questions a neural network with "what action should I take?". The network's answer $$y$$ might looks like $$y = [v=3.0, \phi=23]$$. So the size of the output will be dependant on the dimensions of the action. In this case we have a multiple-dimension output, one dimension for velocity, and one for the joint angle.