Cheat sheet · Machine Learning

Supervised vs Unsupervised Learning cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

Supervised learning maps inputs to known labels (learning a function from X to Y), while unsupervised learning discovers structure in unlabeled data (density, clusters, latent factors). Modern systems increasingly blend the two via self-supervised pretraining followed by supervised or RLHF fine-tuning.

Core Definitions & The Key Distinction

Supervised: you have paired data (x_i, y_i); the goal is to learn f: X -> Y that generalizes to unseen x. Split into classification (discrete y) and regression (continuous y).
Unsupervised: you have only x_i (no labels); the goal is to model structure - clusters, density p(x), low-dimensional manifolds, or anomalies.
Probabilistic framing: supervised models the conditional p(y|x); unsupervised (generative) models the joint p(x,y) or marginal p(x).
Semi-supervised: few labels + many unlabeled points (e.g., pseudo-labeling, consistency regularization like FixMatch).
Self-supervised: labels are auto-generated from the data itself (next-token prediction, masked modeling, contrastive pairs) - it uses unlabeled inputs but trains with a supervised-style loss; the engine behind modern LLMs and vision foundation models.
Reinforcement learning is a separate paradigm: learning a policy from reward signals via interaction, not fixed labels.
Rule of thumb: 'Do I have a target column I'm trying to predict?' Yes -> supervised, no -> unsupervised.

Canonical Algorithms (know a one-liner for each)

Supervised classification: logistic regression, SVM, k-NN, decision trees, random forests, gradient boosting (XGBoost/LightGBM/CatBoost - still the go-to for tabular), neural nets.
Supervised regression: linear/ridge/lasso, gradient boosting regressors, SVR, neural nets.
Clustering: k-means (spherical, must pick k), GMM (soft, elliptical via EM), DBSCAN/HDBSCAN (density-based, finds k and outliers), hierarchical/agglomerative.
Dimensionality reduction: PCA (linear, variance-maximizing), t-SNE & UMAP (nonlinear, mainly visualization - global distances not reliable), autoencoders.
Density / anomaly: Isolation Forest, One-Class SVM, kernel density estimation, Gaussian mixtures.
Association rules: Apriori / FP-Growth (market-basket analysis).
Self-supervised exemplars: BERT (masked LM) / GPT (autoregressive LM), SimCLR/MoCo (contrastive), MAE (masked autoencoders), CLIP (image-text alignment).

Objectives, Losses & Math

Supervised minimizes regularized empirical risk: average loss over labeled data plus a complexity penalty.
Classification loss (cross-entropy, per example over C classes): L = -sum_c y_c * log(p_c); averaged over the batch.
Regression loss: MSE = (1/n) sum (y_i - yhat_i)^2 (sensitive to outliers; MAE/Huber are robust alternatives). RMSE = sqrt(MSE).
k-means objective: minimize within-cluster sum of squares (inertia) = sum_i ||x_i - mu_{c(i)}||^2, optimized by Lloyd's algorithm (local optimum, so use multiple inits).
PCA: find orthogonal directions maximizing variance = top eigenvectors of the covariance of centered data (equivalently top right singular vectors via SVD).
GMM/EM maximizes the data log-likelihood with latent cluster assignments via Expectation-Maximization.
Contrastive (InfoNCE) loss pulls positive pairs together and pushes negatives apart in embedding space - the self-supervised workhorse.

Evaluation - Where They Diverge Most

Supervised has ground truth: use held-out test sets / cross-validation. Classification: accuracy, precision, recall, F1, ROC-AUC, PR-AUC (prefer PR-AUC under class imbalance). Regression: RMSE, MAE, R^2.
Unsupervised has no ground truth, so evaluation is harder and often indirect.
Internal clustering metrics: silhouette score, Davies-Bouldin (lower is better), Calinski-Harabasz (higher is better), inertia/elbow.
External metrics (only if you have labels for validation): Adjusted Rand Index, Normalized Mutual Information - both invariant to label permutation.
Dimensionality reduction: explained variance ratio (PCA); for embeddings, downstream task performance or reconstruction error.
Choosing k: elbow method, silhouette, gap statistic, or BIC/AIC for GMMs.
Validate unsupervised output downstream (does the cluster/feature improve a real task?) rather than trusting a single internal score.

When to Use Which (Practical Decision Guide)

Use supervised when labels exist (or are cheap) and you have a concrete prediction target - fraud detection, churn, pricing, ranking.
Use unsupervised when labels are absent/expensive and you want exploration: customer segmentation, topic discovery, anomaly detection, compression/visualization.
Use unsupervised as a preprocessing step for supervised: PCA/autoencoders for features, clustering to engineer features or stratify.
Use semi/self-supervised when unlabeled data is abundant but labels are scarce - pretrain on unlabeled, fine-tune on the small labeled set (the dominant 2026 paradigm for NLP/vision/foundation models).
Labeling cost vs value: supervised pays a recurring annotation cost; unsupervised trades that for harder, more subjective evaluation.
For tabular business problems with labels, gradient boosting usually beats deep nets; reach for self-supervised pretraining mainly for text, images, audio, and large-scale representation learning.

Modern Context (2026)

Foundation models = self-supervised pretraining at scale + supervised/RLHF (and increasingly RLAIF/DPO) fine-tuning; the supervised/unsupervised line is now a continuum, not a binary.
Representation learning: self-supervised embeddings (text/image encoders) feed downstream supervised heads and RAG pipelines.
Transfer learning is the default: rarely train supervised from scratch - start from pretrained representations.
Active learning combines the two: the model selects the most informative unlabeled points to label, minimizing annotation cost.
Weak supervision (programmatic labeling, Snorkel-style) bridges the gap by generating noisy labels at scale.
Embeddings + vector search power semantic retrieval - an unsupervised-representation use case central to modern LLM apps.

Common pitfalls

Calling self-supervised learning 'unsupervised' without nuance - interviewers want to hear it uses auto-generated labels with a supervised-style loss.
Evaluating clustering with accuracy as if labels were ground truth; cluster IDs are arbitrary and need ARI/NMI or internal metrics.
Treating t-SNE/UMAP cluster sizes, densities, and inter-cluster distances as meaningful - they are not; these are visualization tools tuned by hyperparameters (perplexity/n_neighbors).
Running k-means without scaling/standardizing features (it is distance-based) or assuming it handles non-spherical/varying-density clusters.
Picking k arbitrarily instead of using elbow/silhouette/gap statistic; or using k-means when DBSCAN/HDBSCAN fits the data better.
Data leakage in supervised pipelines: fitting scalers/encoders/PCA on the full dataset before the train-test split (fit on train, transform on test).
Using accuracy on imbalanced classification (99% by predicting the majority class) instead of PR-AUC/F1/recall.
Forgetting PCA assumes linear correlations, needs centering, and is scale-sensitive; applying it blindly to categorical or strongly non-linear data.
Assuming more unlabeled data always helps supervised learning - without semi/self-supervised methods it usually does not.

Interview tips

Be ready to classify a posed problem instantly as supervised vs unsupervised vs semi/self-supervised and justify it.
Explain how you'd evaluate an unsupervised model with no labels - a favorite probe (silhouette, downstream task, stability across seeds).
Contrast k-means vs DBSCAN vs GMM: assumptions, when each fails, and how each picks (or doesn't pick) the number of clusters.
Explain PCA from first principles (center data, covariance, eigenvectors/SVD, explained variance) and its assumptions/limits.
Articulate why self-supervised pretraining works and how it relates to the supervised/unsupervised dichotomy (LLMs, CLIP, MAE).
Discuss the label-cost vs evaluation-difficulty trade-off and when you'd invest in labeling vs unsupervised exploration.
Name the right metric for an imbalanced classification scenario and explain ROC-AUC vs PR-AUC.
Show you can combine paradigms: unsupervised features feeding a supervised model, or active/weak supervision to cut labeling cost.

Read the full lesson