Reference Notes

Use bagging (e.g., random forests) when your base learners are high-variance and you want to reduce variance without much tuning — it parallelizes well and is robust to noise and overfitting. Use boosting (e.g., gradient boosting, XGBoost, LightGBM) when you want to squeeze maximum accuracy by reducing bias, are willing to tune (learning rate, depth, regularization), and your data is fairly clean. Rule of thumb: start with a well-tuned gradient-boosted trees model for structured/tabular data; fall back to bagging when you see boosting overfit noisy labels.

Fit all preprocessing (scaling, imputation, encoding, feature selection) inside the cross-validation fold, never on the full dataset. Confirm no target-derived features (e.g., aggregates that include the row's own label) leak in. For time series, split chronologically and never let future rows inform past ones. Watch for identifiers or timestamps that proxy the target. If a single feature gives near-perfect accuracy, treat it as leakage until proven otherwise. Finally, keep a truly held-out test set touched only once at the very end.

A shadow deployment sends real production traffic to the new model in parallel with the current one, but the new model's predictions are logged rather than served to users. This lets you compare latency, resource use, and prediction distribution against the live model on real inputs, with zero user risk. You catch training-serving skew, feature-pipeline mismatches, and performance regressions before they affect anyone. Once shadow metrics look healthy, graduate to a canary (small % of live traffic) and then a full rollout.