# Understanding the loss function of You Only Look Once(YOLO) network

I'm trying to implement a custom version of YOLO neural network. Originally it was described in this paper. I have some problems understanding the loss function they used.

Basic information:

• An input image is divided into S by S grid (that gives the total of S^2 cells) and each cell predicts B bounding boxes and c conditional class probabilities. Each bbox predicts 5 values: x,y,w,h,C (center of bbox, width and height and confidence score). This makes the output of yolo a SxSx(5B*c) tensor.
• The (x,y) coordinates are calculated relative to the bounds of the cell and (w,h) is relative to the whole image.
• I understand that the first term penalizes the wrong prediction of the center of a bbox; 2-nd term penalizes wrong width and height prediction, 3-rd term - wrong confidence prediction, 4-th is responsible for pushing confidence to zero when there is no object in a cell; the last term penalizes wrong class prediction.

My problem:

I don't understand when 1^obj_ij should be 1 or 0. In the paper, they write: "1^obj_ij denotes that the j-th bbox predictor in i-th cell is responsible for that prediction" and also "Loss function only penalizes bounding box coordinate error if that predictor is responsible for ground truth box".

So is it right that for every object in the image there should be exactly one pair of ij such taht 1^obj_ij=1? And if this is correct, this means that the center of the ground truth bbox should fall into i-th cell, right?

If this is not the case, what are other possibilities when 1^obj_ij=1 and what ground truth labels x_i and y_i should be in these cases?

Also, I assume that ground truth p_i(c) should be 1 if there is an object of class c in the cell i, but what ground truth p_i(c) should be equal to in case there are several objects of different classes in the cell? The correct interpretation (based on the comment by author of the question below):

Yep, you are right. It is actually only a single cell per object that contributes to the loss with 1^{obj}_{ij} factor. That cell is identified as the one that contains the centre(oid) of the ground truth box of the corresponding object.

My original (incorrect) interpretation of the paper:

From what I read in the paper it is not a single bounding box for all cells in the grid for a particular object (your original suggestion).

Rather for every object and every cell that is assigned to this object only one bounding box contributes to the loss. The network generates B bounding boxes for every cell in the grid, but we pick only one and only when that cell actually belongs to an object.

Every cell i, which is not a background cell, will have exactly one box j, such that 1^{obj}_{ij} = 1

This is based on the following paragraph from the paper:

Note that the loss function only penalizes classification error if an object is present in that grid cell (hence the conditional class probability discussed earlier). It also only penalizes bounding box coordinate error if that predictor is “responsible” for the ground truth box (i.e. has the highest IOU of any predictor in that grid cell).

• Thank you for your reply. I've done some digging after I posted this question and found one video where Joseph Redmon(author of the paper) gives a presentation about YOLO: youtu.be/NM6lrxy0bxs . Somewhere about 8 min he starts to explain how they train the network and it seems like my initial assumption is rights: 1^{obj}_{ij}=1 only when the center of the ground truth box falls into i cell. Also, it seems like 1^{noobj}_{ij}=1 for every cell where there is no center of ground truth box. – Andrew Mar 8 '18 at 15:47
• yep, looking at the video you are totally right. Still it was nice to clarify how YOLO objective works:) – hellmean Mar 8 '18 at 15:58