0
$\begingroup$

I have a scanned image, and they need to be classified in one of the pre-defined image classes, so that it can be sorted. However, the problem is the open nature of the classes. At testing time, new classes of scanned images can be added and the model should not only classify them as unseen (open set image recognition), but it should be able to tell in which new class it should belong (not able to figure out the implementation for this.)

So, I am thinking that the below option can work for the classification of unseen classes

  1. Zero-shot learning: Once the image is classified as unseen, we can then apply zero-shot learning to find its respective class for sorting.

  2. Template matching: Match the test image of unseen classes with all available class images, and, once we have a match, we can do sorting of images.

  3. Meta learning-based approach: I am not sure how to implement this, suggestions are much appreciated.

Note: I already tried the classical computer vision approach, but it's not working out. So, more open for neural net-based approach.

Is my approach to solving the problem correct? If possible, suggest some alternative to find the corresponding match/classification of the unseen class image. As I could think of these 2 alternative solutions only.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

I have one possible solution for you inspired by real-time face recognition system. It is a similar case to you.

  1. Create embeddings for each class i.e. person or in your case, a class. (Using Siamese network with ArcFace loss)
  2. When a new image comes, take L1 or cosine distance with the embeddings. You will need a threshold above which you create a embedding for the new guy and give it a new class.

One issue you will think at this point of time is how do you do that when have billions of data points. The search for the matching embeddings can then be done using Hierarchical Navigable Small World graphs (see https://github.com/nmslib/hnswlib).

This solution is loosely connected to the plausible techniques you have mentioned here. And if you still want something in that regard, I shall add the keyword Active Learning.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for suggesting facial recognition based approach. I have thought the same but input images are very noisy, say lot of extra things are written on the image(noise) before scan, so I was thinking that Face recog based approach will have hard time finding similarity. ON top of that I have many classes in which image can go, so timing is also a problem with face recog based approach. I can compromise with time issue but not sure if 1st issue(noise) can be taken care by Siamese based network. $\endgroup$
    – Rambo_john
    Feb 12, 2021 at 8:37
  • $\begingroup$ Try cleaning your images with a GAN or VAE based approach and then move on with this one. Apart from that, it will be a good idea to give this a try since facial recognition system really get a lot of different kind of noises still go on. $\endgroup$
    – Abhishek Verma
    Feb 13, 2021 at 2:57
  • $\begingroup$ Many thanks for VAN /GAN based cleaning approach suggestion. Will try first FR based and then will go for GAN/VAN based approach. $\endgroup$
    – Rambo_john
    Feb 13, 2021 at 16:11
0
$\begingroup$

Matching network and prototypical network few-shot learning can be a better option than Siamese network, There are many implementation available online. Also, these methods are fast in computation or classification.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .