Let's assume I have dataset of image-like 2D samples where values can be divided into few discrete levels (for example 1, 2, 3 and 4) like in the image below, where each color maps different value, from 1 to 4. Number of how many times given color occurs on the picture varies from sample to sample though.

enter image description here

I would like to classify these images into different classes but based on the spatial relations of these values between each other (not the values themselves). By spatial relations I mean basically (left, right, up, down), for example:

  • If blue is above and to the right of the red
  • Another blue is above and to the left of the same red
  • Yellow is to the right of one blue (same height)
  • One green is below red
  • ...

My question is, what algorithm (probably some deep neural network) I should use for that task? I would appreciate even just some keywords or clues of what might help.


1 Answer 1


The spatial relationships that you describe would correspond to features, and it's not clear that you need to use a neural network for detecting or discovering these features since you have just described them. Could you instead define a feature extractor that detects the correct patterns and returns you a vector of counts of feature occurrences across the image?

In fact gray level co-occurrance matrices might be adaptable to this problem. Since there are only four colours, each matrix would be very small, and you would define one matrix for each local spatial configuration. For example, one matrix might encode the number of times that colour x is below colour y. From these you could then extract counts of interest to use for further downstream processing.

  • $\begingroup$ I think "discovering" them would be proper term here, since I do not know them a priori. Discovering them and then, based on that, doing the classification or clusterisation. $\endgroup$
    – GKozinski
    Apr 23, 2021 at 6:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .