What loss function should one use, knowing that input image contains exactly one target object?

I am currently using MSE to predict center of ROI coordinates and it's width and height. All values are relative to image size. I think that such approach does not put enough pressure on fact, that those coordinates are related.

I am aware of existence of algorithms like YOLO or UnitBox, and am just wondering if there might be some shortcut for such particular case.


The terms error and loss are always relative to some ideal of fitness, and it is always against some model, the extension of the concept of curve fitting. The ideal is the optimized state.

When we are analyzing system behavior and optimizing PROCESS parameters as we do in machine learning, we don't always use the same terms. Just as people in medicine and alternative medicine speak of wellness, systems that process data instead of food also can be considered either well or dysfunctional. With machine vision, we optimize the processing of an arbitrary signal, an image with an object depicted in your case, not the parameters of a curve to fit a set of data points.

It is common to wish to judge the wellness of the learning state across a range of sizes. You cannot normalize the image by re-sizing before you recognize the image boundaries, so you have to design the process (circuit) that must be trained accordingly.

It is not in the error function that this can be done. There are only two places where it can be done.

  • The model of the object you are trying to recognize
  • The process topology you are attempting to train

The standard approach for size-independent recognition involves a sequence in the process being trained, reflected in activation and interconnection choices.

  • Edge detection
  • Corner detection
  • Shape detection
  • Illumination detection (for higher reliability)

Note that edge detection and corner detection can overlap in learning layers but not necessarily.

The key is obviously between corner and shape detection. The feed of corners into the layer that detects shape has to be angular rather than Cartesian yielding this sequence.

  • Edge detection
  • Corner detection
  • Angles at corners
  • Shape/form detection
  • Illumination detection (for higher reliability)

A designer can hope a deep network will learn these through back-propagation, but convergence upon the ideal behavior (wellness) will be faster and more reliable if the layers are designed to perform those functions. That means your model for the shape or form you hope to detect must be in terms of angles not coordinates.

Florian Pierre and Joseph Piewak seem to be doing it correctly: https://arxiv.org/pdf/1709.03138.pdf. Search the document with ctrl-f for "angl". These researchers show excellence in their treatment: https://arxiv.org/pdf/1806.07996.pdf [2]. Search for "size indep" in that one.


[1] Fully Convolutional Neural Networks for Dynamic Object Detection in Grid Maps; Master’s Thesis of Florian Pierre, Joseph Piewak; Department of Computer Science, Institute for Anthropomatics and FZI Research Center for Information Technology respectively; 2016

[2] Novel Convolution Kernels for Computer Vision and Shape Analysis based on Electromagnetism; Beaini, Achiche, Cio, Raison; 2018; Polytechnique Montreal, 2900 Edouard Montpetit Blvd, Montreal, H3T 1J4, Canada

  • $\begingroup$ Please correct me if I understand this wrong: You suggest, that I should focus on splitting process of object detection into smaller tasks, rather than trying to achieve it by singular neural network? Am I right to say that what I was trying to do is end to end learning? $\endgroup$ – don_pablito Jul 23 '18 at 12:21
  • $\begingroup$ @paffciu, You are thinking in terms of lambda calculus (algorithm call hierarchy) when you say, "spliting process ... smaller tasks." That's not how people have designed effective CNNs to win contests or deploy in apps and web services. Think of it more like a sequence of processes in a pipeline. Just as we wouldn't bottle orange juice before we filter it to create pulp free juice, we wouldn't try to detect shapes before detecting edges. In well designed computer vision, the pipeline is usually heterogeneous rather than homogeneous. Was that clear? $\endgroup$ – FauChristian Jul 24 '18 at 12:06
  • 1
    $\begingroup$ (1) One cannot detect objects of a known shape or contour independent of their size or camera lens proximity to the object without using angular modelling. Triangles are triangles because they have three vertices the interior angles of which add to Pi radians. Rectangles, even when viewed with some distortion due to lens perspective, have two pair of adjacent interior angles each of which add to Pi radians. Size independence depends on angular modeling, which means the Cartesian coordinates of the object vertices must implicitly or explicitly be translated to angles and proximities. $\endgroup$ – FauChristian Jul 24 '18 at 23:44
  • 1
    $\begingroup$ (2) To understand how machine learning solves problems, forget everything you know from Programming 101 and structured programming (based on Alonso Church's lambda calculus) and think more like a social leader or a manufacturing process engineer or a therapist. You want to design a process, then look for algorithms, services, frameworks, and software tricks to accomplish the flow you've already designed. If you do that, when you do an article search or web search for "size independent object recognition" you will comprehend what you are reading. $\endgroup$ – FauChristian Jul 24 '18 at 23:49
  • 1
    $\begingroup$ Yes I think i get it now $\endgroup$ – don_pablito Jul 25 '18 at 7:43

I am currently using MSE to predict center of ROI coordinates and it's width and height. All values are relative to image size. I think that such approach does not put enough pressure on fact, that those coordinates are related.

At first glance this looks quite reasonable. Computer vision is not really my main area of expertise, so I did some googling around, and one of the first repositories I ran into does something very similar. It may be interesting for you to look into the code and the references in that repository in more detail.

It looks to me like they're also using the MSE loss function. I'm not 100% sure how they define the bounding boxes, maybe you can figure it out by digging through the code. You currently define bounding boxes by:

  1. X coordinate of center of bounding box
  2. Y coordinate of center of bounding box
  3. Width of bounding box
  4. Height of bounding box

You are right in that these coordinates are quite closely related. If the center is incorrect (for example, a bit too far to the right), that mistake could partially be "fixed" by taking a greater width (the bounding box would go a bit too far to the right, but still encapsulate the object). I don't know if this is necessarily a problem, or a fact that should be exploited in some way or something that should be "put pressure on". If this is something you are concerned about, I suppose you could alternatively define the bounding box as follows (I'm not sure whether or not this is what's done in the repository linked above):

  1. X coordinate of top-left corner of bounding box
  2. Y coordinate of top-left corner of bounding box
  3. X coordinate of bottom-right corner of bounding box
  4. Y coordinate of bottom-right corner of bounding box

Intuitively, I suspect the relation between those two corner points will be less strong than the relation you identified exists between center + width + height. A "mistake" in coordinates of the top-left corner cannot be partially "fixed" by placing the bottom-right corner somewhere else.

  • $\begingroup$ I will try out this kind of loss and compare it with mine on same architecture, maybe there will be difference in convergence time. $\endgroup$ – don_pablito Jul 24 '18 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.