Questions tagged [roc-auc]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Defining optimal false-positives and false-negatives balance with a cost function

My attempt to solve the problem below: $$\text{cost function} = C = (TP \cdot CTP) + (FN \cdot CFN) + (FP \cdot CFP) + (TN \cdot CTN) = ((1 - (1 - FP)^2) \cdot 1000) + (FN \cdot CFN) + (FP \cdot ...
4
votes
0answers
94 views

When computing the ROC-AUC score for multi-class classification problems, when should we use One-vs-Rest and One-vs-One?

The sklearn's documentation of the method roc_auc_score states that the parameter multi_class can take the value ...
0
votes
0answers
17 views

Is the PR AUC invariant under label flip?

The ROC-AUC curve is invariant under a flip of the labels. I don't know if it's a famous result, so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not ...
0
votes
0answers
15 views

Why does 0.8:0.2 divided dataset have a much greater AUROC than its 5-fold cross validated counterpart's mean AUROC?

I trained a dataset with a 5-fold cross validation to search for hyper-parameters by an AUROC metric. For splitting I used ...
0
votes
0answers
18 views

Is it a fair evaluation if each model of k-fold cross validation is trained with different epochs and the mean AUC score gathered out of k folds?

Let's say I have a dataset for binary classification. And I am going to conduct 5-fold cross-validation and get AUC scores for each fold (mean AUC score too). However, if I set the training epoch to $...