I'm fairly new to the field of deep learning and would like to ask which deep learning techniques can be used for anomaly detection in unlabeled data. For example, let's say I want to detect anomalous transactions within financial transaction data that shows the number of transactions and the amount in transactions. The data looks like this:
Transaction_ID Count Amount
1241 9 425
1563 7 354
4456 142 45683
7895 452 75358
As you can see, the data itself is not labeled and I'd like to use some deep learning techniques for anomaly detection within this data. I'd say a transaction is anomalous if it has a too large number in the Count or Amount columns like the third and last row.
My questions are:
Do I need to label the data first to separate the normal transactions from the anomalous transactions before I apply any deep learning technique? For example, if I want to use AutoEncoder, I need to train the model with "normal" transaction data first. Then, how do I decide which data points are normal or not? Can I use another machine learning method like K-Means clustering to determine which transactions are normal? What are some other ways to differentiate normal transactions from anomalous transactions to feed the data into a deep learning model?
I've tried the following approach: I labeled the data using K-means clustering first and built a classification model using Neural networks such as CNN, RNN, and LSTM. The labeled data has a new label column(1:anomalous, 0: normal) and I used this label as my "Y" in the deep learning model. I was able to get a high accuracy rate but I wonder if this approach has any issues!. Is it okay to use one machine learning method to label the data and feed that data into another deep learning model for classification? Or will bias from the first ML model be carried over into the second DL model?
Besides what I've mentioned above, could you suggest any Deep Learning approach that is more suitable for this type of analysis?
