Cheat sheet · Computer Vision

Computer Vision Essentials cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

A dense, interview-grade reference covering image fundamentals, CNN building blocks, modern architectures (CNNs, ViTs, detection/segmentation), training and augmentation, evaluation metrics, deployment, and the 2025-2026 landscape (foundation models, diffusion, vision-language models). Tuned for ML engineers and data scientists preparing for work and interviews.

Image Fundamentals & Preprocessing

An image is a tensor of shape (H, W, C); channels are typically RGB (or grayscale C=1). Frameworks differ on layout: PyTorch uses NCHW, TensorFlow/Keras defaults to NHWC.
Pixel values are 0-255 (uint8). Normalize to [0,1] or standardize per-channel with dataset mean/std (ImageNet mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]) so inputs match the pretrained backbone's training distribution.
Classic CV operations: convolution/correlation, Gaussian blur (denoise), Sobel/Canny (edges), morphological ops (erode/dilate), histogram equalization (contrast). Still relevant for preprocessing and as interview baselines.
Color spaces matter: RGB for display, HSV/Lab for color-invariant tasks, YCbCr for compression. Convert deliberately rather than assuming RGB.
Resize/crop policy must be consistent train vs. inference. Aspect-ratio-preserving resize + center crop is common; letterbox (aspect-preserving pad) is standard for detectors (e.g., YOLO).
Spatial frequency intuition: edges/texture = high frequency, smooth regions = low frequency. Downsampling without anti-aliasing (low-pass filtering) causes aliasing artifacts.

Convolutional Building Blocks

Convolution slides a learnable kernel over the input, sharing weights spatially -> translation equivariance and far fewer params than a dense layer.
Output size: O = floor((I + 2P - D*(K-1) - 1)/S) + 1, where K=kernel, S=stride, P=padding, D=dilation. 'same' padding keeps O=I when S=1.
Receptive field grows with depth, stride, and dilation; it must cover the object/context the task needs. Stacking small (3x3) convs is cheaper and more expressive than one large kernel for the same RF.
Param count of a conv layer = K*K*C_in*C_out + C_out (with bias). 1x1 convs do channel mixing / dimensionality reduction cheaply (bottlenecks).
Pooling (max/avg) downsamples and adds local invariance; strided convs are a learnable alternative. Global Average Pooling replaces large FC heads and reduces overfitting.
Depthwise-separable convs (depthwise + 1x1 pointwise) cut compute roughly K*K times (~8-9x for 3x3) - the basis of MobileNet/EfficientNet for edge deployment.
BatchNorm normalizes activations per-channel over the batch (and spatial dims), stabilizing/accelerating training; use GroupNorm/LayerNorm when batch sizes are tiny (detection, segmentation).

Modern Architectures: CNNs vs. Vision Transformers

ResNet introduced residual connections (y = F(x) + x) to train very deep nets by easing gradient flow; still a strong, ubiquitous baseline backbone.
Efficiency families: MobileNetV3/V4, EfficientNet/EfficientNetV2 (compound depth/width/resolution scaling), and ConvNeXt/ConvNeXt V2 (CNNs modernized with transformer-era design, competitive with ViTs).
Vision Transformer (ViT): split image into patches (e.g., 16x16), linearly embed, add positional encodings, prepend a [CLS] token, feed through self-attention. Global receptive field from layer 1 but data-hungry - needs large pretraining or strong augmentation (DeiT).
Self-attention cost is O(N^2) in tokens; Swin Transformer uses shifted local windows for near-linear scaling and hierarchical features, making transformers practical for detection/segmentation.
Inductive bias trade-off: CNNs bake in locality + translation equivariance (good with less data); ViTs learn these from data (win at scale). Hybrids (CoAtNet, MaxViT) blend both.
Self-supervised pretraining is standard: contrastive (SimCLR, MoCo), self-distillation (DINO/DINOv2/DINOv3), and masked image modeling (MAE) produce strong transferable features without labels.

Detection, Segmentation & Core Tasks

Task ladder: classification (image->label), detection (boxes+labels), semantic segmentation (per-pixel class), instance segmentation (per-object mask), panoptic (semantic+instance), keypoint/pose.
Detectors: two-stage (Faster R-CNN: RPN proposes, head refines - accurate) vs. one-stage (YOLO family, SSD, RetinaNet - fast, real-time). Anchor-free (FCOS, CenterNet) and transformer-based (DETR, DINO-DETR, RT-DETR) remove hand-tuned anchors.
NMS (Non-Max Suppression) removes duplicate boxes: sort by score, keep the top, drop overlapping boxes with IoU > threshold. DETR-style models avoid NMS via set prediction + bipartite (Hungarian) matching.
Segmentation backbones: U-Net (encoder-decoder + skip connections, dominant in medical), DeepLabv3+ (atrous/dilated convs + ASPP for multi-scale context), Mask R-CNN (detection + per-RoI mask), and Mask2Former (unified mask attention for semantic/instance/panoptic).
Foundation models: SAM / SAM 2 do promptable, zero-shot segmentation (SAM 2 adds video tracking); open-vocabulary detection (GroundingDINO, OWLv2, YOLO-World) detects classes via text prompts.
IoU = area(intersection) / area(union) is the geometric overlap metric driving both NMS and detection/segmentation evaluation.

Training, Loss Functions & Augmentation

Losses by task: cross-entropy (classification), focal loss (class imbalance / dense detection), L1/Smooth-L1/IoU-family (GIoU, DIoU, CIoU) for box regression, Dice/Tversky (segmentation overlap), triplet/contrastive/InfoNCE (metric & representation learning).
Transfer learning is the default: start from a pretrained backbone, replace the head, fine-tune. Freeze early layers + low LR when data is scarce; full fine-tune when you have enough labels.
Augmentation is the cheapest accuracy lever: flips, crops, color jitter, RandAugment/AutoAugment, plus mixing - Mixup (blend images+labels), CutMix (paste patch + mix labels by area), Cutout/random-erasing. Avoid augmentations that break label semantics (e.g., vertical flip on digits).
Optimizers/schedules: SGD+momentum (classic CNNs), AdamW (transformers/most modern work). Use LR warmup + cosine decay; label smoothing and EMA of weights stabilize and improve generalization.
Use mixed precision (AMP/bf16) and gradient accumulation to fit larger effective batches; BatchNorm prefers larger batches, so switch to GroupNorm/SyncBN in distributed or small-batch settings.
Handle class imbalance with weighted/focal loss, oversampling, or balanced sampling - not just threshold tuning. Monitor a held-out val set and use early stopping.

Evaluation Metrics & Deployment

Classification: accuracy, precision/recall, F1, ROC-AUC, top-5. Report per-class metrics and a confusion matrix when classes are imbalanced - overall accuracy hides minority-class failure.
Detection: mAP - average precision (area under the precision-recall curve) per class, averaged over classes. COCO mAP averages over IoU thresholds 0.50:0.95 (step 0.05); mAP@0.5 (PASCAL VOC) is more lenient.
Segmentation: mIoU (mean IoU over classes), Dice/F1 coefficient (Dice = 2|A intersect B| / (|A| + |B|)), pixel accuracy (misleading with imbalance), and PQ (panoptic quality).
Deployment: export to ONNX / TorchScript / TensorRT; quantize (INT8/FP16) and prune for latency. Quantization-aware training (QAT) recovers accuracy lost by post-training quantization.
Edge/real-time: track latency, throughput (FPS), memory, and model size, not just accuracy. NMS, preprocessing, and CPU<->GPU transfers are common hidden bottlenecks.
Robustness/monitoring: watch for distribution shift, corrupted inputs (ImageNet-C style), and adversarial examples; log input stats and confidence in production to catch drift.

2025-2026 Landscape: Foundation Models, Diffusion & VLMs

Vision foundation backbones: DINOv2/DINOv3 and EVA produce general-purpose frozen features; CLIP/SigLIP align image-text embeddings for zero-shot classification and retrieval.
Promptable & open-vocabulary models: SAM 2 (image+video segmentation), GroundingDINO + SAM ('Grounded SAM') for text-prompted detect-then-segment, YOLO-World for real-time open-vocab detection.
Generative vision: latent diffusion (Stable Diffusion 3.5, FLUX) and Diffusion Transformers (DiT) dominate image synthesis; ControlNet/IP-Adapter add structural/style conditioning; rectified-flow and few-step distillation (LCM, SDXL-Turbo) speed sampling.
Vision-Language Models (VLMs): GPT-4o, Claude, Gemini, and open models (Qwen2.5-VL, Llama 3.2 Vision, InternVL, Molmo) do captioning, VQA, OCR, grounding, and document understanding; most use a vision encoder + projector feeding an LLM.
Video & 3D: video diffusion (Sora-style) and 3D/novel-view methods (NeRF, 3D Gaussian Splatting) are mainstream for content and reconstruction.
Efficiency: FlashAttention and quantized/distilled backbones make ViTs and VLMs deployable; parameter-efficient fine-tuning (LoRA/adapters) is the norm for adapting large vision/VL models.

Common pitfalls

Data leakage from preprocessing: fitting normalization stats or making resize/augmentation choices using val/test data inflates scores.
Mismatched normalization between training and inference (or wrong mean/std for a pretrained backbone) silently tanks accuracy.
Confusing convolution with cross-correlation (deep-learning libs implement cross-correlation) and miscomputing output sizes/padding.
Judging detectors/segmenters by overall accuracy instead of mAP/mIoU; reporting only mAP@0.5 and ignoring the stricter COCO 0.5:0.95.
Applying label-breaking augmentations (flipping text/digits, heavy color shifts on color-critical tasks) or accidentally augmenting the validation set.
Using BatchNorm with tiny batch sizes (detection/segmentation/distributed) where its statistics are unreliable - prefer GroupNorm/SyncBN.
Forgetting model.eval() at inference (leaving dropout active and BN updating running stats), causing nondeterministic/worse predictions.
Expecting ViTs to match CNNs when trained from scratch on small datasets without heavy augmentation or large-scale pretraining.
Ignoring class imbalance: optimizing plain cross-entropy and reporting accuracy where the majority class dominates.
Channel/layout bugs: feeding NHWC to an NCHW model, BGR vs RGB (OpenCV loads BGR), or passing uint8 where float is expected.

Interview tips

Compute conv output dimensions and parameter counts on the spot, and explain stride/padding/dilation and receptive field.
Explain why residual connections enable deep networks (gradient flow / identity mapping) and what BatchNorm normalizes over and why it helps.
Contrast CNNs vs. ViTs: inductive biases, data efficiency, attention cost O(N^2), and when you'd pick each (and hybrids like Swin/ConvNeXt).
Walk through detection end-to-end: anchors vs anchor-free, RPN, NMS, and why DETR removes NMS via bipartite matching - plus how mAP is computed.
Define IoU, mAP, and mIoU precisely; expect a whiteboard ask to code IoU or Dice.
Discuss small/imbalanced datasets: transfer learning, augmentation (Mixup/CutMix), focal loss, and self-supervised pretraining (DINOv2/MAE).
Show systems awareness: quantization, ONNX/TensorRT export, latency vs accuracy trade-offs, and edge deployment with MobileNet/EfficientNet.
Know the 2025-2026 landscape: SAM 2, open-vocabulary detection (GroundingDINO/YOLO-World), diffusion (SD3.5/FLUX/DiT), CLIP/SigLIP, and VLMs (Qwen2.5-VL, Llama 3.2 Vision, GPT-4o).

Read the full lesson