4
$\begingroup$

I am trying to build a ML Agent to find the closest matching image from a given set. The user will draw something and the agent should list the closest matching images.

Very similar to these examples

  1. https://sketchx.eecs.qmul.ac.uk/
  2. Emoji search in Android keyboard

One unique problem I've is, each image will represent a category. Imagine we have product images and user will draw something and we have to find the products close to the drawing. So category in my case will be product Id.

enter image description here

I would like to evaluate the approach before trying out.

There are lot of examples to classify images, however if I use the item identifier instead of category it should work. But am trying to find the best approach for this problem.

|improve this question
$\endgroup$
2
$\begingroup$

There is a good project you can research to find information on this specific task, it is Google quickdraw. It has 50 million drawings across 345 categories and it was used in Google autoDraw. AutoDraw guesses what you’re trying to draw.

You can use quickdraw dataset to train a neural network (Sequence-to-Sequence Variational Autoencoder) in order to get draws from your Sketches. for more info on this read this papper

Classified Sketches into categories is a difficult task because Sketches are a high-level representation that does not always convey enough information to distinguish between different categories.

The the best approach will depend on your specific needs

|improve this answer
$\endgroup$
  • $\begingroup$ Hi Jony, Thanks for your response. I've a unique problem in image classification. I've updated my question with more information. In my case, I am trying to identify the product based on the user's hand art. So there is only one image for a product from which the ML agent can learn $\endgroup$ – codetiger Jul 13 '17 at 6:49
  • $\begingroup$ in that case, you are not going to get a good performance with natural photos classification algorithm, take a look at Sketch-a-Net arXiv:1501.07873v3 This architecture Beats Humans and can help with your problem. $\endgroup$ – Jony Raverta Jul 13 '17 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.