0
$\begingroup$

I want to make a model that outputs the centre pixel of objects appearing in an image.

My current method involves using a CNN with L2 loss to output an image of equivalent size to the input where each pixel has a value of 1 if it is the center of an object and 0 otherwise. Each input image has roughly ~80 objects.

The problem with this is the CNN learns the easiest way to reduce the error, which is having the entire output be 0, because for 97% of cases that's correct. As such, error decreases but it learns nothing.

What is another potential method for training a network to do something similar? I also tried adding dropout, which made the output a lot more noisy and it seemed to learn ok, but eventually ended up in the same state as before with the entire output being 0, never really seeming to learn how to output the locations of objects.

Improve this question
$\endgroup$
4
  • $\begingroup$ I didn't saw your dataset but problem can also occur when one time pixel RGB(10, 10, 10) is 1, another time is 0 $\endgroup$ – Adrian Grygutis Feb 7 '20 at 10:57
  • $\begingroup$ Do you have to train a new model or can you use a pre-trained model - it might be enough to remove the last one or two layers and adjust those to your needs? For object detection there are basically two approaches: Instance Segmentation that try to find a pixel-by-pixel classification and Bounding Box models that find the minimum box to contain the entire object. A centre pixel of an object can have different meaning for these approaches: is the centre pixel of the crescent the void in the middle of the bounding box? $\endgroup$ – Gregor Feb 12 '20 at 15:49
  • $\begingroup$ @Gregor Yes the centre pixel would be the middle of the void in the bounding box. I could use a pre-trained model, but most of those models feel much to large for the task at hand $\endgroup$ – Recessive Feb 12 '20 at 23:09
  • $\begingroup$ A pre-trained model is indeed probably fairly large for your purpose (usually trained to predict 1000 classes) but they come almost production-ready: set up a machine with TensorFlow, OpenCV, ... and load the model. That's it. The outputs are the probabilities of the different classes (this is where you might want to adjust your final layer to predict only those 80 classes) and the bounding box or pixel mask. The centre of a bounding box is simply the middle of the height and width. You can also find a trade-off between accuracy and speed by adjusting the precision from FP32 to maybe FP16. $\endgroup$ – Gregor Feb 13 '20 at 12:18
0
$\begingroup$

from what I understand you are building you own model for this specific use case. From my perspective I would try not to reinvent the wheel, as it is said, and use an already proven and working model such as the YOLOs (v1, v2 and v3).

YOLO does not tell you the center pixel of the image directly but it tells you the center cell, with respect to the predicted object, of a grid (which is built on top of the image) responsible for predicting each object. See on the left image how the grid is built on top of the input image, then YOLO computes a probability map, and then the bounding boxes are predicted from the cell in the object's center. I have highlighted which cells would be responsible for predicting each object in the rightmost image. (Image from YOLOv1 paper)

YOLO cell approach

The grid can have different resolutions but if you make it equal to the image size, then you would have the center pixel of each object predicted. This is because YOLO predicts objects in each cell of the grid, so if the grid is equal to the image size, YOLO will predict objects in each pixel.

As an example, imagine you have an input image of $[H \times W] = [416 \times 416]$ then YOLO would compute a grid of $[S_1 \times S_2]=[52 \times 52]$ on top of it. And predict objects in the center cell of the $[S_1 \times S_2]$. So, if you tune YOLO for computing a grid such as $[S_1 \times S_2] = [H \times W]$, then YOLO would output objects prediction with respect to the image pixels, in other words, YOLO would predict bounding boxes centered in the image pixel on the object's center.

This is how I would proceed for this use case, I hope it helps you or at least give you some clues about how to proceed further! Cheers! :)

NOTE: I chose the image size and grid size with numbers I usually see at work. Specifically, using YOLOv3. In YOLOv3, for aninput image of $[H \times W] = [416 \times 416]$, 3 grids are built, with different resolutions (for predicting big and small objects), with the following grids sizes are: $[13 \times 13], [26 \times 26], [52 \times 52]$

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your reply. Is there any faster way to do this than with YOLO that you might know of? My dataset is very simplistic and could almost be solved by using a heuristic approach, were it not for minor differences in dimensions and proportions. $\endgroup$ – Recessive Feb 11 '20 at 2:20
  • $\begingroup$ Yes, by building a custom model as you suggest, but you did not provide enough information about 2 key things: how you do the bounding box regression and what your loss function is (very important for custom models). By predicting the bbox coordinates (x,y,w,h) you can get the (x,y) pixel in the image. But before doing that you need to write an adequate loss function that facilitates that prediction. $\endgroup$ – JVGD Feb 11 '20 at 10:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.