2
$\begingroup$

Autoencoders are used for unsupervised anomaly detection by at first learning the features of the data set with mainly "normal" data points. Then new data can be considered anomalous, if the new data has a large reconstruction error, i.e. it was hard to fit the features as in the normal data.

But how can Autoencoders provide a reconstruction error when unsupervised? Even if the training is supervised by learning to reconstruct the same data, how is the reconstruction error retrieved from new data?

Improve this question
$\endgroup$
  • 1
    $\begingroup$ I assume you know the basic theory about how AE;s are trained? $\endgroup$ – DuttaA May 29 '19 at 8:56
  • $\begingroup$ Hi! Was my answer unclear or were you looking for something else and I answered something else? $\endgroup$ – DuttaA Jun 5 '19 at 6:09
  • 1
    $\begingroup$ Hi, thanks for the detailed answer, I appreciated it. It guided me towards the right direction. However, it was not directly answering my question. When you say, the AE expects specific features to be of a specific magnitude compared to other features, I feel like that is misunderstanding a bit. The AE itself does not reason about the magnitude. I think the point I was missing is that the recon. error is actually just a value for how different the output is to the (ideally identical) input. If the magnitude of features is out of order as you describe, the recon. error most probably increases. $\endgroup$ – Brian Jun 6 '19 at 7:11
0
$\begingroup$

By Unsupervised they refer to the learning process, which is Unsupervised. Although, it does not appear so, but Unsupervised loosely means you work with data points and data points only without the use of any information about those data points.

I do not have idea about how AE's are used for anomaly detection, but from the point of view of AE's, I will explain what might be happening.

In AE's the hidden representations (if hidden units are less than visible units) the weights for the encoder network are not independent of each other, thus the values coming out of a hidden node after encoding operation are not independent of each other, which if speaking mathematically will mean each hidden node is modelling complex probabilistic interactions between the input features. Thus, the AE encoder learns that if the given feature $x_1$ is of this magnitude, it also expects $x_2,...x_n$ to be of certain other magnitudes. Similarly, the decoder also learns how to decode this complex interactions to original data.

The point to note is that in AE's with hidden nodes less than input nodes, if encoder function is given by $e$ and decoder by $d$ then the operation you are doing is $$d(e(x))=x$$ and not $$e^{-1}(e(x))=x$$

Decoder is reversing the inputs as "learned" and "generalized" from inputs, and not by "learning" to perform the inverse operation (this might be the case where $nodes_{hidden}$ $\geq$ $nodes_{input}$).

Now, say there is an anomaly. The encoder, outputs something which the decoder will see does not match the way it should match with other hidden nodes output (normally if hidden node1 is outputing $h_1$ the decoder expects $h_2$ from hidden node 2, but now the decoder is getting $h_1$ but not getting $h_2$ instead getting $h_2 + \epsilon$ where $\epsilon$ is noise and as a result fails to understand and decode the hidden representation properly.

Improve this answer
$\endgroup$
  • $\begingroup$ This answer is a little confusing, especially the last paragraph, where I suppose you attempt to answer the actual question "how can Autoencoders provide a reconstruction error when unsupervised?"; otherwise, it does not seem you answer this question. $\endgroup$ – nbro♦ May 29 '19 at 18:20
  • $\begingroup$ @nbro which part is confusing or what does not add up with the question? $\endgroup$ – DuttaA May 29 '19 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.