By Unsupervised they refer to the learning process, which is Unsupervised. Although, it does not appear so, but Unsupervised loosely means you work with data points and data points only without the use of any information about those data points.
I do not have idea about how AE's are used for anomaly detection, but from the point of view of AE's, I will explain what might be happening.
In AE's the hidden representations (if hidden units are less than visible units) the weights for the encoder network are not independent of each other, thus the values coming out of a hidden node after encoding operation are not independent of each other, which if speaking mathematically will mean each hidden node is modelling complex probabilistic interactions between the input features. Thus, the AE encoder learns that if the given feature $x_1$ is of this magnitude, it also expects $x_2,...x_n$ to be of certain other magnitudes. Similarly, the decoder also learns how to decode this complex interactions to original data.
The point to note is that in AE's with hidden nodes less than input nodes, if encoder function is given by $e$ and decoder by $d$ then the operation you are doing is $$d(e(x))=x$$ and not $$e^{-1}(e(x))=x$$
Decoder is reversing the inputs as "learned" and "generalized" from inputs, and not by "learning" to perform the inverse operation (this might be the case where $nodes_{hidden}$ $\geq$ $nodes_{input}$).
Now, say there is an anomaly. The encoder, outputs something which the decoder will see does not match the way it should match with other hidden nodes output (normally if hidden node1 is outputing $h_1$ the decoder expects $h_2$ from hidden node 2, but now the decoder is getting $h_1$ but not getting $h_2$ instead getting $h_2 + \epsilon$ where $\epsilon$ is noise and as a result fails to understand and decode the hidden representation properly.
