CI/CD for machine learning quiz

Question 1

What is the key conceptual difference between traditional CI/CD and CI/CD for machine learning (often called CI/CD/CT)?

Accepted Answer

ML CI/CD adds versioning/validation of data and models plus Continuous Training (CT)

Answer

ML CI/CD only automates deployment, never testing

Answer

There is no difference; the same pipelines apply unchanged

Answer

ML CI/CD removes the need for unit tests entirely

Question 2

In Google's MLOps maturity model, what does 'MLOps level 0' describe?

Accepted Answer

A manual, script-driven process with no pipeline automation

Answer

Fully automated CI/CD with automatic retraining

Answer

Automated pipeline (CT) but manual deployment

Answer

Multi-region canary deployment with shadow testing

Question 3

What is training-serving skew and why must a CI/CD pipeline guard against it?

Accepted Answer

A mismatch between feature computation at training vs. serving time

Answer

A bias introduced only by class imbalance

Answer

Overfitting caused by too many training epochs

Answer

A network latency spike during model serving

Question 4

Why is data validation a mandatory stage in an ML CI/CD pipeline, and what does it typically check?

Accepted Answer

It checks schema, ranges, missingness, and distribution drift to fail fast on bad data

Answer

It is optional and only relevant for deep learning

Answer

It only verifies that the file format is CSV

Answer

It replaces the need for a test set

Question 5

What is a model registry and what role does it play in CI/CD for ML?

Accepted Answer

A versioned store of models with metadata and stage labels used as the deployment handoff

Answer

A database of raw training data only

Answer

A monitoring dashboard for GPU utilization

Answer

A container registry exclusively for Docker images

Question 6

What distinguishes Continuous Training (CT) triggers, and name three common ones.

Accepted Answer

Scheduled, data-arrival, and drift/performance-based triggers

Answer

Only manual triggers are possible in CT

Answer

Only compilation of source code triggers retraining

Answer

CT can only be triggered by code merges

Question 7

Why is pinning random seeds insufficient to guarantee full reproducibility in ML CI/CD, and what else is needed?

Accepted Answer

You also need versioned data/code, pinned dependencies/containers, and deterministic flags

Answer

Seeds alone fully guarantee reproducibility

Answer

Reproducibility is impossible and not worth pursuing

Answer

Only using the same dataset is required

Question 8

What is the purpose of a 'model validation' gate before promotion, and how does it differ from standard test-set evaluation?

Accepted Answer

It compares against production/baseline, checks slices and operational constraints before promotion

Answer

It is identical to computing accuracy on the test set

Answer

It only measures training loss

Answer

It validates the dataset schema, not the model

Question 9

In a GitHub Actions ML pipeline, why should large datasets and models NOT be committed directly to Git, and what is the standard alternative?

Accepted Answer

Use DVC/Git LFS pointers with artifacts in object storage

Answer

Commit everything to Git for simplicity

Answer

Email the datasets to teammates

Answer

Store data inside the Docker image layers only

Question 10

What is shadow (dark) deployment for ML models and what is its main benefit and limitation?

Accepted Answer

It logs new-model predictions on real traffic without serving them, eliminating user risk but not measuring behavioral impact

Answer

It serves new-model outputs to 50% of users immediately

Answer

It is only used for batch scoring

Answer

It permanently replaces the old model

Question 11

Compare canary deployment and blue-green deployment for model rollouts.

Accepted Answer

Blue-green switches all traffic at once; canary shifts traffic gradually with metric gating

Answer

They are identical strategies

Answer

Canary always switches 100% traffic instantly

Answer

Blue-green can only be used for databases

Question 12

What is the role of containerization (e.g., Docker) in ML CI/CD reproducibility?

Accepted Answer

Containers package code+runtime+deps into immutable images for consistent environments across stages

Answer

Containers only reduce image size

Answer

Containers replace the need for a model registry

Answer

Containers are only useful for training, not serving

Question 13

Why are unit tests for ML pipelines often split into 'tests on code' and 'tests on the trained model', and give an example of each.

Accepted Answer

Code tests verify deterministic logic; model tests verify learned/behavioral properties like invariance

Answer

Both test the same thing redundantly

Answer

Model tests only check file size

Answer

Code tests run only after deployment

Question 14

What is the 'CACE' principle and how does it affect CI/CD design for ML systems?

Accepted Answer

CACE = 'Changing Anything Changes Everything', so ML changes need holistic end-to-end validation

Answer

CACE means caching all computed examples

Answer

CACE is a container orchestration tool

Answer

CACE refers to a learning-rate schedule

Question 15

How does a feature store improve CI/CD and reduce training-serving skew?

Accepted Answer

It serves consistent, versioned features to training and inference, reducing skew and enabling point-in-time correctness

Answer

It stores only raw images

Answer

It is just a CSV cache

Answer

It replaces the model registry

Question 16

What is concept drift versus data (covariate) drift, and how do they relate to CT triggers?

Accepted Answer

Data drift changes P(X); concept drift changes P(Y|X), and concept drift usually needs fresh-label retraining

Answer

They are the same thing

Answer

Concept drift changes only the model weights

Answer

Data drift only occurs at training time

Question 17

In CML (Continuous Machine Learning) workflows, what is the purpose of posting metrics/plots as a comment on a pull request?

Accepted Answer

To surface model metrics/plots in code review so changes are evaluated before merge

Answer

To increase build time artificially

Answer

To replace the test suite

Answer

It has no practical purpose

Question 18

Why should secrets (API keys, cloud credentials, registry tokens) never be hardcoded in ML pipeline definitions, and what is the correct approach?

Accepted Answer

Inject them at runtime from a secrets manager / masked CI vars / OIDC with least privilege

Answer

Hardcode them so the pipeline is self-contained

Answer

Store them in plaintext in the README

Answer

Commit them in an encrypted-looking base64 string

Question 19

What is pipeline parameterization and why is it important for promoting ML pipelines from dev to prod?

Accepted Answer

It externalizes config as inputs so one pipeline runs across environments reproducibly

Answer

It hardcodes prod values for safety

Answer

It is only relevant to hyperparameter tuning

Answer

It disables logging in production

Question 20

What does it mean to test for 'data leakage' in a CI pipeline, and give one automated check.

Accepted Answer

Leakage is future/target info entering training; check via split-key overlap and point-in-time joins

Answer

Leakage means losing data rows during ETL

Answer

Leakage only affects deep learning

Answer

It is detected only after deployment

Question 21

Why is end-to-end pipeline integration testing necessary even when each component has unit tests in ML CI/CD?

Accepted Answer

They catch interface/serialization/serving failures that unit tests miss by running the full pipeline

Answer

Integration tests are redundant if unit tests pass

Answer

They only measure accuracy

Answer

They are only needed for the UI

Question 22

What is the difference between deploying a model-as-artifact versus deploying a model-as-service in CI/CD?

Accepted Answer

Model-as-artifact ships only the model to an existing runtime; model-as-service packages model+serving code as a deployable unit

Answer

They are identical

Answer

Model-as-service means no container is used

Answer

Model-as-artifact always requires retraining on deploy

Question 23

How can a CI pipeline keep model-training jobs fast and cheap while still gating quality on every commit?

Accepted Answer

Run smoke tests on small samples per commit, full training on merge/schedule, with caching

Answer

Always run full training on every commit

Answer

Disable testing to save time

Answer

Train only manually with no CI

Question 24

What is 'model lineage' and why is it critical for auditability in regulated ML deployments?

Accepted Answer

Lineage links a model to its exact code, data, params, and environment for reproducibility and audit

Answer

Lineage is the model's training loss curve

Answer

Lineage is only the model's accuracy history

Answer

Lineage is irrelevant once deployed

Question 25

When automating LLM/GenAI deployments in CI/CD, why are traditional accuracy gates insufficient and what replaces them?

Accepted Answer

Use eval suites (LLM-judge, groundedness, safety, regression) on curated prompts as gates

Answer

Accuracy alone is sufficient for LLMs

Answer

Skip evaluation since LLMs are pretrained

Answer

Only measure token latency

Question 26

What is the purpose of a 'rollback strategy' for models and what makes ML rollbacks harder than code rollbacks?

Accepted Answer

ML rollbacks must restore a model+data+feature combination and contend with feedback loops

Answer

ML rollbacks are trivially the same as code rollbacks

Answer

Rollbacks are impossible for ML

Answer

Rollback only means deleting the model

Question 27

In Kubeflow Pipelines or Vertex AI Pipelines, what is the benefit of caching step outputs by input hash?

Accepted Answer

Steps keyed by input hash reuse cached outputs when inputs/code are unchanged, saving time and cost

Answer

Caching randomly skips steps

Answer

Caching disables reproducibility

Answer

Caching stores only the final model

Question 28

What is the role of 'data contracts' in CI/CD for ML, and how are they enforced?

Accepted Answer

A versioned producer-consumer spec of schema/quality enforced by automated validation gates

Answer

A legal license for the dataset

Answer

A contract to buy compute

Answer

A naming convention with no enforcement

Question 29

Why is monitoring considered part of the CI/CD/CT loop rather than a separate afterthought in MLOps?

Accepted Answer

Monitoring signals feed CT/CD triggers, closing a self-correcting loop

Answer

Monitoring is unrelated to deployment

Answer

Monitoring only tracks server uptime

Answer

Monitoring replaces the need for testing

Question 30

What problem does delayed/partial ground-truth labeling create for performance monitoring in CI/CD, and how is it commonly addressed?

Accepted Answer

Without timely labels, use drift proxies, sampled labeling, and KPIs as early signals

Answer

Late labels have no effect on monitoring

Answer

Just wait years before deploying

Answer

Use training accuracy as the live metric

Test yourself