5
votes
Accepted
What is the difference between out of distribution detection and anomaly detection?
You observation is correct although the terminology needs a little explaining.
The term 'out-of-distribution' (OOD) data refers to data that was collected at a different time, and possibly under ...
- 1,987
4
votes
How to detect outlier images?
Hi there @pookie you can approach this problem using unsupervised anomaly detection techniques. One such technique is to use an autoencoder neural network.
The idea is to train an autoencoder on a ...
- 41
4
votes
Accepted
Which unsupervised learning technique can be used for anomaly detection in a time series?
So if I understood correctly:
You have data from 2 sensors in time: Ar flow and BackGas Flow (SCCM, what is that?)
You have that data for multiple products.
1 - Since it is relatively low dimensional, ...
3
votes
Accepted
Which unsupervised learning algorithm can be used for peaks detection?
If your anomalies are simply peaks, why should you be using machine learning methods? You could use peak detection algorithms for the purpose.
If you still insist on ML, isolation forest is a good ...
- 441
2
votes
Find anomalies from records of categorical data
First of all, you mention that you have categorical data. I don't see how you can define similarity so that you can also define the distance between the predicted value and the ground truth (error). ...
- 241
2
votes
Accepted
How to calculate a meaningful distance between multidimensional tensors
You could try an earth mover distance in 2d or 3d over the image? For example you could follow this example, but call it sequentially. The idea would be something like the following (untested and ...
- 206
2
votes
How to detect outlier images?
I agree with Triple S' answer, but as a preprocessing step I suggest to use some pre-trained image classification network without the last fully-connected layer. This will give you robust features ...
- 842
1
vote
How is it possible to detect anomalies in batches of 2 minutes of web access logs?
Do you have a labeled dataset that specifies which connections are anomalous and which are not? You will need that to train the model.
Yes, you will need all of the individual labeled data rows for ...
- 21
1
vote
How to train a model for 1 image class to detect anomaly?
It sounds like you only have "normal" examples with which to train your model, so this makes the problem feel like an application for outlier detection algorithms. There are a variety of ...
- 411
1
vote
Accepted
Understanding the reconstruction loss in the paper "Anomaly Detection using Deep Learning based Image Completion"
$F$ in this context is the output of the Convolutional Neural Network that's being trained, which is of the same size as $X$.
- 2,389
1
vote
How can auto-encoders compute the reconstruction error for the new data?
It is computed just like in training. You take an MSE or something along these lines between the input and the output. You set a threshold for it. If new data's reconstruction error is higher than ...
- 868
1
vote
Which unsupervised anomaly detection algorithms are there?
Hierarchical Temporal Memory is a model well suited for anomaly detection. It is also pretty interesting and different from currently typical Deep Learning models.
- 4,295
1
vote
Accepted
Which unsupervised anomaly detection algorithms are there?
If you are OK to use python, thy novelty-detection with sklearn:
https://scikit-learn.org/stable/modules/outlier_detection.html
- 126
Only top scored, non community-wiki answers of a minimum length are eligible