0
$\begingroup$

The ROC-AUC curve is invariant under a flip of the labels. I don't know if it's a famous result, so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not been able to prove or disprove it yet.

The reason this is important is that when data sets are highly imbalanced ROC-AUC curves perform well mechanically and it is better to look at PR-AUC curves. This obviously raises the question of a better curve for PR-AUC for one class being better for the other class as well or not.

Proof that ROC-AUC is invariant under label flip

$$ \begin{array}{|c|c|c|} \hline & \text{Predicted positive} & \text{Predicted negative} \\ \hline \hline \text{Positives} & TP(\xi) & FN(\xi) \\ \hline \text{Negatives} & FP(\xi) & TN(\xi) \\ \hline \end{array} $$ where $\xi \in [0,1]$ is the output of a classifier.

For a fixed dataset we have

$$ TP(\xi) + FN(\xi) = P \\ FP(\xi)+TN(\xi) = N $$ where $P$ and $N$ are actual number of negatives and positives. The ROC-AUC defined wrt to this definition of positives and negatives is

$$ A = \frac{1}{PN} \int_0^1 TP(\xi)~d FP(\xi) $$

Now lets look at what happens under a label flip. We get

$$ \begin{eqnarray} \tilde A &=& \frac{1}{PN} \int_1^0 TN(\xi)~d FN(\xi) \\ &=& - \frac{1}{PN} \int_1^0 (N-FP(\xi)~d TP(\xi) \\ &=& \frac{1}{PN} \int_0^1 (N-FP(\xi)~d TP(\xi) \\ &=& \frac{1}{P} TP(\xi) |_0^1 - \frac{1}{PN} (TP(\xi))(FP(\xi))|_0^1 + A \\ &=& A \end{eqnarray} $$ where the last equality follows from $TP(0)=0$, $TP(1)=P$ and $TN(1)=N$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.