I would like to know if my understanding of RPN training is correct, and if never training the RPN on some specific anchor box is bad (i.e if the anchor never sees good nor bad examples).

To make my point clear, assume we have two functions. $f_{\theta_1}$ which represents the backbone that outputs a feature map of size $n$ (assume flattened) for an image of size $m$ (WLOG assume the image is flattened) $$ f_{\theta_1}: \mathbb{R}^m \to \mathbb{R}^n $$ and $f_{\theta_2}$ that represents the 'objectness' of each anchor box. We can suppose that $f_{\theta_2}$ and $f_{\theta_1}$ are convolutional neural networks, where $\theta_1, \theta_2$ are the networks' parameters. For simplicity, assume the RPN does not output bounding boxes correction, and only outputs the probabilities that an anchor box is an object or not. $$ f_{\theta_1}: \mathbb{R}^n \to \mathbb{R}^{k \cdot n}$$ We can assume $k=1$, which is the number of boxes per anchor.

If my understanding is correct, we select $p$ good proposals $G_p$, and $p$ bad proposals $B_p$ for training the RPN, which are indices of good and bad predictions. In other terms, if $x$ is an image (assume flattened), then $f_{\theta_2}(f_{\theta_1}(x)) = y$, next we only back-propagate the loss for coordinates $B_p$ in $y$ and $G_p$ in y. For instance, if $p=1$, and $G_p = \{i\}$ and $B_p = \{j\}$ and $ 1\leq i \neq j \leq n$ then we only compute the loss of the RPN for coordinates $i$ and $j$ in $y$. My questions are:

1- Is my understanding correct? and if not, how do we perform training?

2- Assuming my understanding is right or partially right about the last step, What happens if we never train component $y_0$ from the RPN's output for example? (i.e we never back propagate the loss through some components for $y$) woudn't this be a problem (i.e hurt performance or network training does not go well at all in some cases?)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.