Precision, recall and F1

Precision = TP / (TP + FP) — of all the samples the model predicted as positive, precision tells you what fraction were actually positive. A high precision means the model is accurate when it raises a positive alarm. It does not measure how many real positives were found — that is recall.

Recall = TP / (TP + FN) — of all the real positive cases in the dataset, recall tells you what fraction the model actually found. A high recall means the model misses very few true positives. In medical contexts the same quantity is called sensitivity.

F1 = 2 × (Precision × Recall) / (Precision + Recall), which is the harmonic mean of the two metrics. The harmonic mean is chosen because it gives a low score whenever either precision or recall is low, making it a balanced but demanding combined metric.

A False Positive happens when the model predicts positive but the true label is negative — an unwarranted alarm. For example, a healthy patient flagged as diseased is an FP. FPs hurt precision (more bad predictions in the positive bucket) but do not affect recall.

The arithmetic mean of 0.99 and 0.01 is 0.50, suggesting a decent model — but one metric being near zero means the model is practically useless. The harmonic mean of those same values is only about 0.02, truthfully revealing the failure. The harmonic mean penalises extreme imbalances far more severely.

Cancer screening should prioritise recall because a False Negative (missing a real cancer patient) can be fatal. It is far better to trigger a follow-up test on a healthy person (FP) than to miss someone who is truly sick. Maximising recall, even at the cost of some false alarms, saves lives.

A False Positive in spam filtering means a legitimate email (perhaps a job offer) is sent to the spam folder and the user may never see it — a serious problem. A False Negative (spam reaching the inbox) is inconvenient but harmless: the user just deletes it. This asymmetry pushes spam filters toward high precision.

F1 = 2 × (0.8 × 0.8) / (0.8 + 0.8) = 1.28 / 1.60 = 0.80. When precision and recall are equal, the harmonic mean equals the arithmetic mean, so F1 is simply the same value as both metrics. This is one useful sanity check for verifying your F1 calculation.

F1 = 2 × (1.0 × 0.0) / (1.0 + 0.0) = 0.0. This model never predicts positive (TP = 0, FP = 0), so precision is technically 0/0 but treated as 1.0 by convention; recall is 0.0 because it catches absolutely nothing. The harmonic mean correctly awards such a useless model a score of zero.

When β > 1 the β² factor inflates the weight on recall relative to precision — the model is penalised more for missing true positives than for making false alarms. For example, F2 (β=2) weights recall four times as much as precision (β²=4). Conversely, F0.5 emphasises precision over recall.

Macro F1 first computes an independent F1 score for each class, then takes a simple unweighted average over all classes. Every class contributes equally regardless of how many samples it contains, so a rare class that the model handles poorly will pull the macro score down noticeably.

Micro averaging pools the raw TP, FP, and FN counts from every class together and then computes one single precision, recall, and F1 from those combined totals. Because larger classes contribute more counts, micro F1 is dominated by the majority class — it is equivalent to overall accuracy when each sample belongs to exactly one class.

Macro F1 = (0.90 + 0.20 + 0.10) / 3 = 1.20 / 3 = 0.40. Even though the model does well on one class, macro averaging exposes the very poor performance on the other two. This is the key strength of macro F1: it cannot be hidden by good performance on a single dominant class.

The model gets 95% accuracy because 950 of 1000 samples are negative and it always says negative. But its recall for the positive class is 0 (no positive ever detected) and F1 on the positive class is 0. Accuracy of 95% is dangerously misleading; F1 immediately reveals the model is completely useless for finding positives.

Recall = TP / (TP + FN) = 70 / (70 + 30) = 0.70. The 30 patients missed by the model are the False Negatives. Importantly, recall depends only on the actual positive cases — knowing the number of False Positives is not required to compute recall (though FPs do affect precision).

Precision = TP / (TP + FP) = 40 / 50 = 0.80 — the model is accurate when it predicts positive. Recall = TP / (TP + FN) = 40 / 100 = 0.40 — the model only finds 40% of actual positives. The F1 = 2 × 0.8 × 0.4 / (0.8 + 0.4) ≈ 0.533, reflecting the low recall pulling the score down.

`classification_report` returns a formatted text table with per-class precision, recall, F1-score, and support (number of true samples), plus macro and weighted averages at the bottom. It is the quickest way to get a complete diagnostic picture of a classifier without calling multiple separate functions.

from sklearn.metrics import classification_report
print(classification_report(y_true, y_pred))

Lowering the threshold causes the model to flag many more samples as positive. This catches more true positives (recall increases) but also generates more false alarms (precision decreases). This is the fundamental precision-recall tradeoff: there is no free lunch — improving one metric generally hurts the other.

A perfect model has no False Positives (so Precision = TP/(TP+0) = 1.0) and no False Negatives (so Recall = TP/(TP+0) = 1.0). Consequently F1 is also 1.0. Any mistake — even a single FP or FN — will push at least one of these values below 1.0.

Since every patient is predicted positive, all 10% who truly have diabetes are correctly caught — Recall = 1.0. But 90% of the positive predictions are wrong (healthy people), so Precision = 0.10 (equal to the prevalence rate). F1 = 2×1.0×0.1/(1.0+0.1) ≈ 0.18, correctly flagging this as a poor model.

Both sensitivity and recall equal TP / (TP + FN) — the proportion of all actual positives that are correctly detected. The term sensitivity comes from clinical diagnostics while recall is the machine-learning name; the math is identical. The complement (1 − recall) is called the False Negative Rate.

Substituting Precision = TP/(TP+FP) and Recall = TP/(TP+FN) into F1 = 2PR/(P+R) and simplifying gives F1 = 2·TP / (2·TP + FP + FN). This compact form is very convenient when you already have the raw confusion matrix counts and want to avoid the intermediate steps.

Macro F1 averages the two per-class F1 scores with equal weight (50% each), so a poor score on rare Class A pulls the result down noticeably. Weighted F1 weights each class by its number of true samples (Class B gets 99% of the weight here), so a model that ignores Class A can still score high on weighted F1.

The harmonic mean of two equal numbers a and a simplifies to 2a²/2a = a. So whenever precision = recall = p, F1 = p as well. This is a useful sanity check: if your reported F1, precision, and recall are all the same number, you can quickly verify the formula is being applied consistently.

Precision = 50/60 ≈ 0.833; Recall = 50/70 ≈ 0.714. F1 = 2 × 0.833 × 0.714 / (0.833 + 0.714) ≈ 1.190/1.547 ≈ 0.769. Using the direct formula: F1 = 2×50 / (2×50 + 10 + 20) = 100/130 ≈ 0.769 — a quicker route to the same answer.

If either precision or recall is zero, the harmonic mean is also zero: 2×0×anything/(0+anything) = 0. This is exactly the right behaviour — a model that never predicts positive (zero recall) or that is always wrong in its positive predictions (zero precision) is useless and deserves a score of zero. The arithmetic mean would give 0.5 in these cases, which is misleading.

A False Negative in spam detection is a spam email that the model labelled as 'not spam,' allowing it to reach the inbox. While inconvenient, the user can simply delete it. This is considered less harmful than a False Positive (a legitimate email wrongly quarantined in the spam folder), which is why spam filters are tuned for high precision.

Substitute into F1 = 2PR/(P+R): 0.75 = 2×0.9×R/(0.9+R). Multiply both sides by (0.9+R): 0.675 + 0.75R = 1.8R. Rearranging: 0.675 = 1.05R, so R = 0.675/1.05 ≈ 0.643. This algebra shows you can always recover any one metric if you know the other two.

At a very high threshold, the model only predicts 'positive' when it is almost certain, so virtually all its positive predictions are correct — precision is very high. However, many true positives fall below the threshold and are missed — recall is very low. This conservative strategy avoids false alarms but at the cost of coverage.

average='macro' computes an independent F1 score for each class and then takes a simple unweighted average, so a rare class with 5 samples has the same influence as a large class with 5000 samples. average='weighted' weights each class by its sample count, making larger classes dominate the result.

from sklearn.metrics import f1_score
f1_macro    = f1_score(y_true, y_pred, average='macro')
f1_weighted = f1_score(y_true, y_pred, average='weighted')
f1_micro    = f1_score(y_true, y_pred, average='micro')

Precision = TP / (TP + FP). If no positive is ever predicted, then TP = 0 and FP = 0, giving 0/0 — mathematically undefined. Libraries like sklearn return 0.0 and emit a UndefinedMetricWarning. Recall is 0.0 (all true positives missed) and F1 is 0.0, correctly identifying the model as useless.

Weighted F1 ≈ 0.99 because 995 healthy patients dominate (99.5% weight), and the model handles that majority class very well. Macro F1 ≈ 0.50 means the per-class F1 scores averaged evenly are only 0.50 — the rare 'sick' class has a very low F1. For medical screening, this distinction is critical: the model is failing on the patients who most need detection.

A naive model that always predicts 'not fraud' achieves 99% accuracy because 99% of transactions truly are legitimate. But it catches zero fraudulent transactions — recall for fraud = 0 and F1 on the fraud class = 0. F1 immediately exposes this failure, whereas accuracy falsely suggests excellent performance.