Suppose I have a problem where I want to classify the color of LEDs seen in the image. I can use OpenCV to pinpoint the exact location of these LEDs but I do not know their color for sure because the image can appear too bright or too dim. And the LED itself can have so low resolution that the color appears distorted.

If I design a simple CNN model that takes in a cropped LED image and classify it as GREEN or BLUE independently it will be likely wrong. The model need to know the overall brightness of the image by looking at how the other LEDs are like in the image.

Now you can say that maybe I just need to compute the overall brightness of the image and feed it to the model, but think of this as a general problem. Suppose I want to do something more complicated that I cannot extract features for the model, how do I write this kind of model in tensorflow/keras ?

enter image description here

The model architecture is inspired by PointNet architecture where max pooling is used to extract global features from all the points. In the image, the top and bottom models are simply the same. They share all the weights, and the number of cropped images doesn't need to be constant. The idea is to extract 2 dense layers, where one of them will be used for global max pooling. This global feature represents values like the "overall brightness" of the image. So theoretically, the model should use this info to classify the color better.

Again, how do I implement this kind of thing in a keras/tensorflow? Any paper/model/link suggestions?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.