ROC and precision-recall curves quiz

Question 1

What does ROC stand for in machine learning, and what kind of curve does it produce?

Accepted Answer

Receiver Operating Characteristic

Answer

Rate of Correct Classification

Answer

Relative Output Curve

Answer

Ranked Output Comparison

Question 2

What quantities are plotted on the x-axis and y-axis of an ROC curve?

Accepted Answer

TPR on y-axis, FPR on x-axis

Answer

Precision on y-axis, Recall on x-axis

Answer

Accuracy on y-axis, F1 on x-axis

Answer

Specificity on y-axis, Sensitivity on x-axis

Question 3

What AUC value does a purely random classifier achieve, and what does its ROC curve look like?

Accepted Answer

0.5 — a diagonal line from (0,0) to (1,1)

Answer

0.0 — a flat line at TPR = 0

Answer

1.0 — a perfect L-shaped curve

Answer

0.75 — a gentle curve above the diagonal

Question 4

How is the True Positive Rate (TPR) calculated, and what does it measure?

Accepted Answer

TP / (TP + FN)

Answer

TP / (TP + FP)

Answer

TN / (TN + FP)

Answer

FP / (FP + TN)

Question 5

How is the False Positive Rate (FPR) calculated, and what does it represent?

Accepted Answer

FP / (FP + TN)

Answer

FP / (FP + TP)

Answer

FP / (FP + FN)

Answer

TN / (TN + FN)

Question 6

What does AUC stand for, and what does it measure in the context of an ROC curve?

Accepted Answer

Area Under the Curve — probability that a positive is ranked above a negative

Answer

Average Uncertainty Coefficient — measures model uncertainty

Answer

Accuracy of Unified Classification across datasets

Answer

Asymmetric Uncertainty Criterion for skewed data

Question 7

What is precision in a binary classifier, and how is it calculated?

Accepted Answer

TP / (TP + FP)

Answer

TP / (TP + FN)

Answer

TN / (TN + FP)

Answer

TP / (TP + FP + FN)

Question 8

What is recall in binary classification, and how does it differ from precision?

Accepted Answer

TP / (TP + FN)

Answer

TP / (TP + FP)

Answer

FP / (FP + TN)

Answer

TN / (TN + FP)

Question 9

What does the point (0, 0) represent on an ROC curve?

Accepted Answer

The model predicts all samples as negative

Answer

A perfectly accurate classifier

Answer

The model predicts all samples as positive

Answer

A random classifier with no skill

Question 10

What does the point (1, 1) represent on an ROC curve?

Accepted Answer

The model predicts all samples as positive

Answer

The model predicts all samples as negative

Answer

A perfect classifier with AUC = 1

Answer

A random classifier with AUC = 0.5

Question 11

Does changing the classification threshold redraw the entire ROC curve, or does it just move a single point along the curve?

Accepted Answer

The threshold moves the operating point along a fixed curve

Answer

The threshold change redraws the entire curve each time

Answer

Threshold changes only affect precision, not the ROC curve

Answer

The ROC curve is only defined at threshold = 0.5

Question 12

Why is the precision-recall (PR) curve preferred over the ROC curve when the dataset is highly imbalanced?

Accepted Answer

FPR is not affected by class imbalance, masking poor precision; PR exposes it

Answer

PR curves are always easier to compute than ROC curves

Answer

ROC curves cannot handle binary classification tasks

Answer

PR curves only work for balanced datasets

Question 13

What three values does scikit-learn's `roc_curve` function return, and how are they used?

Accepted Answer

fpr, tpr, and thresholds arrays

Answer

Only the scalar AUC value

Answer

precision, recall, and thresholds arrays

Answer

accuracy, f1_score, and thresholds arrays

Question 14

What is the F1 score and how does it relate to the precision-recall curve?

Accepted Answer

F1 is the harmonic mean of precision and recall, a single point on the PR curve

Answer

F1 is the total area under the PR curve

Answer

F1 is the arithmetic mean of precision and recall

Answer

F1 score is plotted on the x-axis of a PR curve

Question 15

What is the probabilistic interpretation of AUC-ROC?

Accepted Answer

It is the probability that a random positive is ranked higher than a random negative

Answer

It is the average accuracy across all possible thresholds

Answer

It equals the maximum F1 score achievable

Answer

It measures how well predicted probabilities are calibrated

Question 16

What shape does a perfect classifier's ROC curve take, and what AUC does it have?

Accepted Answer

An L-shape hugging the top-left corner, AUC = 1.0

Answer

A diagonal from (0,0) to (1,1), AUC = 0.5

Answer

A curve bowing toward the bottom-right corner

Answer

A horizontal line at TPR = 0.5

Question 17

What is the Area Under the Precision-Recall Curve (AUC-PR), and in what situations is it most informative?

Accepted Answer

The weighted mean of precision at each recall level

Answer

The area under the ROC curve restricted to the positive class only

Answer

The F1 score averaged uniformly across all thresholds

Answer

The fraction of correctly ordered positive-negative pairs

Question 18

As you lower the classification threshold from 1.0 to 0.0, how do precision and recall typically change?

Accepted Answer

Recall increases and precision typically decreases as threshold decreases

Answer

Both precision and recall increase as threshold decreases

Answer

Precision increases and recall decreases as threshold decreases

Answer

Both precision and recall decrease as threshold decreases

Question 19

What exactly does sklearn's `average_precision_score` compute, and how does it differ from simply computing the area under the PR curve with the trapezoid rule?

Accepted Answer

A step-function integral: sum of precision × change-in-recall at each threshold

Answer

The precision at the point where recall equals 1.0

Answer

The arithmetic (unweighted) mean of all sampled precision values

Answer

The area under the ROC curve for the positive class

Question 20

How is ROC-AUC extended to multi-class classification problems?

Accepted Answer

One-vs-Rest: one curve per class, then average the per-class AUCs

Answer

ROC curves cannot be extended beyond binary classification

Answer

All class scores are merged into a single combined curve automatically

Answer

Multi-class problems use only the PR curve, never the ROC curve

Question 21

Toward which corner of the ROC plot should a better classifier's curve bow, and why?

Accepted Answer

Top-left corner, for high TPR and low FPR

Answer

Bottom-right corner, for maximum specificity

Answer

Center of the plot, for balanced performance

Answer

Bottom-left corner, to minimise all error types

Question 22

Write Python code that plots an ROC curve with the random-classifier baseline clearly shown.

Accepted Answer

The random baseline is a diagonal from (0,0) to (1,1)

Answer

The random baseline is a vertical line at FPR = 0.5

Answer

The random baseline is a horizontal line at TPR = 0.5

Answer

The random baseline passes through (0.5, 0) and (0, 0.5)

Question 23

Why does AUC-ROC remain stable under class imbalance while AUC-PR degrades significantly?

Accepted Answer

TPR and FPR are within-class rates unaffected by ratio; precision mixes classes so PR is imbalance-sensitive

Answer

ROC uses absolute counts while PR uses ratios

Answer

The PR curve only functions correctly with balanced classes

Answer

AUC-ROC always equals 0.5 under severe class imbalance

Question 24

Using Bayes' theorem, derive the formula for precision in terms of TPR, FPR, and class prevalence π.

Accepted Answer

Precision = (TPR × π) / (TPR × π + FPR × (1 − π))

Answer

Precision = TPR × FPR / prevalence

Answer

Precision = (TPR + FPR) / 2

Answer

Precision = 1 − FPR when prevalence equals 50%

Question 25

What is the mathematical connection between AUC-ROC and the Wilcoxon-Mann-Whitney (WMW) statistic?

Accepted Answer

AUC equals the normalised WMW statistic, counting concordant positive-negative pairs

Answer

AUC is mathematically related to the chi-squared statistic

Answer

AUC and WMW are conceptually similar but numerically different

Answer

WMW statistic measures calibration, not discrimination

Question 26

What is the concordance index (C-statistic) and how does it relate to AUC-ROC?

Accepted Answer

C-statistic equals AUC-ROC and measures probability of correct pair ranking

Answer

C-statistic equals precision at recall = 0.5

Answer

C-statistic is a different metric that complements, but differs from, AUC

Answer

C-statistic measures calibration of predicted probabilities

Question 27

If two models have ROC curves that cross each other, why is a single AUC comparison misleading, and what should you do instead?

Accepted Answer

Compare partial AUC (pAUC) in the operationally relevant FPR range

Answer

Always pick the model with higher overall AUC regardless of curve shape

Answer

Switch to accuracy as the comparison metric when curves cross

Answer

The model whose curve crosses the other is always the inferior one

Question 28

What is the difference between model calibration and model discrimination (AUC), and can a model have one without the other?

Accepted Answer

AUC measures rank ordering; calibration measures whether probabilities match actual frequencies

Answer

Calibration and AUC both measure ranking ability and always agree

Answer

Calibration is another name for precision in imbalanced datasets

Answer

High AUC guarantees that predicted probabilities are well-calibrated

Question 29

How do you find the optimal classification threshold using the Youden index, and how is it coded in Python?

Accepted Answer

Youden index: find the threshold maximising TPR − FPR (farthest from diagonal)

Answer

Always use threshold = 0.5 as the default optimal point

Answer

Choose the threshold where precision exactly equals recall

Answer

Choose the threshold that minimises FPR regardless of TPR

Question 30

A fraud-detection model achieves AUC-ROC = 0.92 on a dataset where only 0.05% of transactions are fraudulent. Why might this model still be operationally useless?

Accepted Answer

AUC averages over all thresholds; at the deployment threshold with tiny prevalence, precision may be extremely low

Answer

AUC of 0.92 always guarantees operationally useful fraud detection

Answer

AUC only measures training accuracy, not deployment performance

Answer

Fraud detection should use F1, making AUC irrelevant

Test yourself