4
$\begingroup$

Usually, it is said in ML that there is a trade-off between Precision and Recall. I wonder if it is possible that Precision and Recall can increase together?

$\endgroup$

4 Answers 4

5
$\begingroup$

They can increase together if your new classifier is indeed way better than your older one in terms of almost every metric you can imagine including the two scores, together with the F1-score, or even the overall accuracy. In the simplest case where you started from a negative-only extremely poor classifier with bad performance on nearly all the mentioned measures, then any reasonable classifier, say, a logistic regressor would produce much better precision and recall out of the matrix of any confusion.

In a practical scenario, say you trained an original nearest neighbor binary classifier $g_N$ with some balanced representative training data, and later you trained an optimal Bayes classifier $f$ with the same dataset. And from one of your previous questions you've probably already known the non-optimal $g_N$ is 2-optimal which means its out-of-sample misclassification error is at most twice the minimum possible out-of-sample error which is obtained only by the optimal classifier $f$. Since accuracy is the complement of misclassification error, so $f$'s true positives and true negatives most likely will both increase and the sum of false positives and false negatives possibly cut in half which implies precision and recall can both increase for $f$.

$\endgroup$
1
  • 2
    $\begingroup$ Perfect answer! $\endgroup$ Commented Nov 26, 2023 at 2:34
2
$\begingroup$

Often references to the precision-recall trade-off are discussing setting the classification threshold for a probabilistic classifier. A probabilistic classifier is one that returns a probability of membership of the positive class. With a probabilistic classifier we then need to chose a threshold value (by default a threshold of 0.5 is often used, but any threshold can be used). If the classifier returns a value greater than or equal to the threshold, the sample is classified as positive. If the value is less than the threshold, the sample is classified as negative.

We can calculate the precision and recall for each possible threshold value. If the threshold is high (close to 1), then very few samples will be classified as positive, so recall will be low. If the threshold is low (close to 0), then almost all samples will be classified as positive, so recall will be high. Note that as the threshold is decreased, recall must either increase or stay the same. Recall will be 0 when the threshold is > 1 and 1 when the threshold is 0.

If the classifier is reasonably good, precision will generally follow the opposite trend - when the threshold is set to a high value, precision will generally be high and when the threshold is set to a low value, precision will be low. But precision can increase, as shown in this example:

true label 1 1 0 1 0 1 1 0 0 0
classifier prediction 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
Threshold 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
TP 1 2 2 3 3 4 5 5 5 5
FP 0 0 1 1 2 2 2 3 4 5
Precision 1.0 1.0 0.67 0.75 0.6 0.67 0.71 0.63 0.56 0.5
FN 4 3 3 2 2 1 0 0 0 0
Recall 0.2 0.4 0.4 0.6 0.6 0.8 1.0 1.0 1.0 1.0

So, between the 0.5 and 0.3 thresholds (also between the 0.7 and 0.6 thresholds), both precision and recall increase as the threshold reduces.

$\endgroup$
1
$\begingroup$

PR curves need not be monotonic.

Suppose some model, such as a logistic regression, makes predictions p below and that the true outcomes are y below,

set.seed(2023)
N <- 1000
p <- rbeta(N, 1/4, 1)
y <- rbinom(N, 1, p)
pr <- PRROC::pr.curve(
    p,
    weights.class0 = y,
    curve = T
)
plot(pr)

The PR curve has increasing sections, though it will smooth out if you increase the sample size (quite smooth for even N <- 10000).

pr curve

The relationship between precision and recall is complicated.

$\endgroup$
0
$\begingroup$

Yes, just look at the fomulas:

$$ \begin{align} Prec &= \frac{TP}{TP+FP}\\ Recall &= \frac{TP}{TP+FN}\\ \end{align} $$

And the fixed "constraints":

$$ \begin{align} \text{Actual positive} &= TP + FN\\ \text{Actual negative} &= TN + FP\\ \end{align} $$

Also note:

$$ \begin{align} \text{Predicted positive} &= TP + FP\\ \text{Predicted negative} &= TN + FN\\ \text{Sample size} &= TP + TN + FN + FP \\ \end{align} $$

So for some "shared" value of $TP$, you can always vary $FP$ and $FN$ freely.

Here's a visual example to aid your intuition.

precision vs. recall

You can play around with the settings here.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .