How is the data labelled in order to train a region proposal network?

I don't get how the training of the RPN works. From the forward propagation, I have $$W \times H \times k$$ outputs from the RPN.

How is the training data labeled such that I can use the loss function and update the weights through bach propagation? Is the training data labeled in the same shape of the output, as there are $$W \times H \times k$$ anchor boxes and we use the loss function directly or what?