I'm working on Unsupervised Video Anomaly Detection, and I've tried implementing the Generative Cooperative Learning method, with the help of this paper.
The method uses a fixed backbone (ResNext-101) for video feature extraction. The videos are divided into segments of 16 frames, and a feature vector is computed for each segment. A generator (A simple Autoencoder) provides pseudo labels (based on the reconstruction error) for the discriminator which is a simple fully connected classifier. Pseudo labels from the discriminator are used to improve the generator using a process called negative learning, and in this fashion, the Generator and Discriminator are put in a collaborative learning loop, and the loss eventually converges.
I've recently come across the Deep Clustering paper , and was wondering if we can use a clustering method instead of the autoencoder as part of the generator. I think we can use the cosine distance as a good distance metric. The troublesome part however, is thinking of a good criteria for generating the pseudo labels.
With the autoencoder, the reconstruction error is a pretty intuitive criteria for pseudo labelling. Since anomalies are sparse, the autoencoder will not be able to reconstruct them properly and so they will have large(r) reconstruction errors.
What can be a similar criteria that we can use for pseudo labelling if we use a clustering method instead of an Autoencoder?
