Set is a card game and is Nicely described here.

Each set-card has 4 properties:

  1. The number(1,2 or 3)
  2. the color (Red, Green or Purple)
  3. Fill (Full, Stripes, None)
  4. Form (Wave, Oval or Diamond)


converts to 2 Purple Waves No fill (code: 2PWN)

enter image description here and enter image description here

convert to codes 1RON and 3GDN

For every combination there is one card so in total there are 3^4 = 81 cards. The goal is to identify 3 cards (set) out of collection of 12 displayed randomly chosen set cards where all properties occur 0,1 or 3 times.

As a hobby project I want to create an android app which can -with the camera- capture the 12 (less or more) set cards and indicate the sets present in the collection of 12. I'm looking for ways to leverage image recognition as efficient as possible.

I've been thinking of taking multiple pictures of all the individual cards, label them and feeding them to a trainer (firebase ML KIT AutoML Vision Edge) But I have the feeling that this a bit of brute force and takes a lot of time and effort photographing and labeling. I could also take pictures of multiple set cards and provide the different codes as labels.

What would be the best (most efficient) approach to have a model for labelling all cards?


Since you only have fixed types.

For colour, I think it is fairly straight forward.
For number, simplest way is to plot a projection histogram and count the points of discontinuity.

An example of the projection histogram

enter image description here

For fill, You can find the number of islands. Islands of background colour.
For shape, Like Clement Hui suggested you can use shape detection

| improve this answer | |
  • $\begingroup$ However the shape detection uses ML as far as I am concerned. One concern over the shape detection of openCV is that it may not feature the shapes of the cards.. the wave shape maybe too peculiar to be in the shapes of opencv library. Also I am not sure if the shape detection will work on non-filled or striped shapes. Have you tried it out? Will it work? $\endgroup$ – Clement Hui Jan 11 at 4:32
  • $\begingroup$ Edited my answer. And you can erode the image before performing a shape detection. Then it should work, but haven't tried it personally. $\endgroup$ – codeblooded Jan 11 at 13:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.