Set is a card game and is Nicely described here.

Each set-card has 4 properties:

  1. The number(1,2 or 3)
  2. the color (Red, Green or Purple)
  3. Fill (Full, Stripes, None)
  4. Form (Wave, Oval or Diamond)


converts to 2 Purple Waves No fill (code: 2PWN)

enter image description here and enter image description here

convert to codes 1RON and 3GDN

For every combination there is one card so in total there are 3^4 = 81 cards. The goal is to identify 3 cards (set) out of collection of 12 displayed randomly chosen set cards where all properties occur 0,1 or 3 times.

As a hobby project I want to create an android app which can -with the camera- capture the 12 (less or more) set cards and indicate the sets present in the collection of 12. I'm looking for ways to leverage image recognition as efficient as possible.

I've been thinking of taking multiple pictures of all the individual cards, label them and feeding them to a trainer (firebase ML KIT AutoML Vision Edge) But I have the feeling that this a bit of brute force and takes a lot of time and effort photographing and labeling. I could also take pictures of multiple set cards and provide the different codes as labels.

What would be the best (most efficient) approach to have a model for labelling all cards?

  • 1
    $\begingroup$ You might want to try Open CV's shape detector though it might not work for you because your shapes are custom. pyimagesearch.com/2016/02/08/opencv-shape-detection $\endgroup$
    – Clement
    Jan 9, 2020 at 16:04
  • $\begingroup$ For data labelling approach amount of data and accuracy maybe a problem here. $\endgroup$
    – Clement
    Jan 9, 2020 at 16:07

1 Answer 1


Since you only have fixed types.

For colour, I think it is fairly straight forward.
For number, simplest way is to plot a projection histogram and count the points of discontinuity.

An example of the projection histogram

enter image description here

For fill, You can find the number of islands. Islands of background colour.
For shape, Like Clement Hui suggested you can use shape detection

  • $\begingroup$ However the shape detection uses ML as far as I am concerned. One concern over the shape detection of openCV is that it may not feature the shapes of the cards.. the wave shape maybe too peculiar to be in the shapes of opencv library. Also I am not sure if the shape detection will work on non-filled or striped shapes. Have you tried it out? Will it work? $\endgroup$
    – Clement
    Jan 11, 2020 at 4:32
  • $\begingroup$ Edited my answer. And you can erode the image before performing a shape detection. Then it should work, but haven't tried it personally. $\endgroup$ Jan 11, 2020 at 13:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .