Which unsupervised learning technique can be used for anomaly detection in a time series?

Question

I've started working on anomaly detection in Python. My dataset is a time series one. The data is being collected by some sensors which record and collect data on semiconductor-making machines.

My dataset looks like this:

ContextID   Time_ms Ar_Flow_sccm    BacksGas_Flow_sccm
7289973 09:12:48.502    49.56054688 1.953125
7289973 09:12:48.603    49.56054688 2.05078125
7289973 09:12:48.934    99.85351563 2.05078125
7289973 09:12:49.924    351.3183594 2.05078125
7289973 09:12:50.924    382.8125    1.953125
7289973 09:12:51.924    382.8125    1.7578125
7289973 09:12:52.934    382.8125    1.7578125
7289999 09:15:36.434    50.04882813 1.7578125
7289999 09:15:36.654    50.04882813 1.7578125
7289999 09:15:36.820    50.04882813 1.66015625
7289999 09:15:37.904    333.2519531 1.85546875
7289999 09:15:38.924    377.1972656 1.953125
7289999 09:15:39.994    377.1972656 1.7578125
7289999 09:15:41.94     388.671875  1.85546875
7289999 09:15:42.136    388.671875  1.85546875
7290025 09:18:00.429    381.5917969 1.85546875
7290025 09:18:01.448    381.5917969 1.85546875
7290025 09:18:02.488    381.5917969 1.953125
7290025 09:18:03.549    381.5917969 14.453125
7290025 09:18:04.589    381.5917969 46.77734375

What I have to do is to apply some unsupervised learning technique on each and every parameter column individually and find any anomalies that might exist in there. The ContextID is more like a product number.

I would like to know which unsupervised learning techniques can be used for this kind of task at hand since the problem is a bit unique:

1. It has temporal values.
2. Since it has temporal values, each product will have many (similar or different) values as can be seen in the dataset above.

Answer 1

So if I understood correctly: You have data from 2 sensors in time: Ar flow and BackGas Flow (SCCM, what is that?) You have that data for multiple products.

1 - Since it is relatively low dimensional, you may try using raw data with K-Means or Self Organizing Maps.

2 - If you searching for anomalies in time, you might try using feature engineering with things like flow change:

• i) Take 2 points in time and "derivate" (Point A - Point B)/(Time A - Time B)
• ii) Take the result for A and repeat the process as a second derivative, try that for many levels of "derivative"
• iii) Machine learning application for sensor failure detection in polymerization process. In: Simpósio Brasileiro de Telecomunicações e Processamento de Sinais.

3 - You might want to check other Time-Series related research, even on regression and classification, since they may give you ideas on relevant features or approaches:

• i) An enseble approach to time dependent classification (10.1109/ICMLA.2018.00164)
• ii) Multivariate Time Series for Data-driven Endpoint Prediction in the Blast Oxygen Furnace (DOI: 10.1109/ICMLA.2018.00231)

4 - Note that anomalies are outliers, so analysing data distant from the mean of any features you might engineer from the time. An model based on RBF kernels might pick up some information.

5 - Since you don't know the amount of clusters you can try hierarchical clustering.

6 - Don't forget to talk to people in your field about what to expect from this.

If you can elaborate on the type of anomalies you're looking for I may come with better ideas. This is pretty general tips.

I may have misunderstood what you wanted to cluster:

7 - If you want to detect anomalies in:

• i) products behavior tips above were mostly for it and you might use all 23 features in short time intervals and cluster like this:

    For each product in dataset {

      samples = {} # empty set

      For each time interval in product {

        features in interval add to samples

      }

      perform clustering method using samples

    }

This algorithm would help cluster periods of time where the product behavied weirdly

• ii) sensors behavior then for something like that you might try tip 8

• iii) products itself create a feature vector with all avaiable information for each product, then cluster that. Since it is composed of variable time interval and 23 features per product (plus time) you might want to use a bit of dimensionality reduction (PCA should work fine, since it gives you the direction of greater variance and you're look for that). Using a regressor to model products behavior and cluster that could all be usefull (create a regressor in time for each product to model its behavior, then cluster the weights of the regressors as the product representation)

8 - Let's say you want to identify sensors behavior, it might be helpful to model the sensors output as a function f(time, extravariables) and do regression over it (try linear, then try non-linear). Points in time that the regressor had bad performance predicting the answer might indicate that that behavior is anomalous