Docker and containers for ML quiz

Question 1

Why is a multi-stage Docker build particularly valuable for ML inference images, and what does it optimize?

Accepted Answer

It separates build-time dependencies from the runtime image to shrink size and attack surface

Answer

It compiles the model into native code for faster inference

Answer

It enables multi-GPU scheduling inside one container

Answer

It automatically quantizes model weights during the build

Question 2

What is the role of the NVIDIA Container Toolkit, and what does '--gpus all' actually do?

Accepted Answer

It mounts the host driver libraries and devices into the container at runtime

Answer

It installs the GPU driver inside the container image

Answer

It compiles CUDA kernels at container start

Answer

It virtualizes the GPU so multiple containers share one physical core

Question 3

Why should you pin base image tags by digest rather than relying on 'latest' or floating version tags in ML Dockerfiles?

Accepted Answer

Digests guarantee an immutable, reproducible base layer across builds

Answer

Digests make image pulls faster

Answer

Digests automatically scan for vulnerabilities

Answer

Digests reduce the number of layers in the image

Question 4

How does Docker layer caching interact with the order of COPY and RUN pip install instructions, and how should an ML Dockerfile be structured?

Accepted Answer

Copy requirements and pip install before copying source code to maximize cache hits

Answer

Copy all source first so pip can see every module

Answer

Run pip install last to ensure latest packages

Answer

Order does not affect caching, only image size

Question 5

What problem does BuildKit's '--mount=type=cache' solve for ML image builds, and why not just rely on layer caching?

Accepted Answer

It persists a download cache across builds without bloating the final image

Answer

It encrypts the build cache for security

Answer

It parallelizes RUN steps across multiple CPUs

Answer

It shares the cache between unrelated images automatically by default

Question 6

Why is it a best practice to run ML containers as a non-root user, and what is a subtle pitfall with mounted volumes?

Accepted Answer

Non-root reduces attack surface but can cause UID/GID mismatch issues on mounted volumes

Answer

Non-root containers run faster due to fewer syscalls

Answer

Non-root containers cannot access the GPU at all

Answer

Running as root is required for PyTorch to use shared memory

Question 7

PyTorch DataLoader with num_workers>0 often crashes in Docker with 'bus error' or 'DataLoader worker killed'. What is the root cause and fix?

Accepted Answer

The default /dev/shm is too small; increase --shm-size or use --ipc=host

Answer

GPU memory is exhausted; reduce batch size

Answer

The container lacks the CUDA driver; reinstall the toolkit

Answer

num_workers is unsupported in containers; set it to 0 always

Question 8

What are the trade-offs between baking model weights into the image versus loading them at runtime from object storage?

Accepted Answer

Baking gives reproducibility and fast start but bloats image; runtime loading keeps images lean but adds cold-start latency

Answer

Baking weights is always better because it is reproducible

Answer

Runtime loading is always better because it is smaller

Answer

There is no practical difference for models under 100GB

Question 9

Why does '.dockerignore' matter especially for ML projects, and what should it typically exclude?

Accepted Answer

It excludes large datasets, checkpoints, and secrets from the build context

Answer

It tells Docker which layers to cache

Answer

It specifies which GPU to use

Answer

It compresses the final image

Question 10

What is the difference between the 'runtime' and 'devel' variants of NVIDIA CUDA base images, and when do you need each?

Accepted Answer

'devel' includes nvcc and headers for compiling CUDA extensions; 'runtime' only ships execution libraries

Answer

'devel' is for development laptops, 'runtime' for servers

Answer

'runtime' has the driver, 'devel' does not

Answer

They are identical except for the tag name

Question 11

In a Kubernetes ML serving deployment, how do you request a GPU for a pod and what scheduling behavior results?

Accepted Answer

Request 'nvidia.com/gpu' under resources.limits; GPUs are scheduled as whole, non-overcommittable units

Answer

Set spec.gpu: true in the pod spec

Answer

Use a nodeSelector of gpu=yes; the kernel handles allocation

Answer

Add the --gpus all flag to the container args

Question 12

What is NVIDIA MIG (Multi-Instance GPU) and how does it help containerized ML workloads?

Accepted Answer

MIG partitions one GPU into isolated instances with dedicated memory and compute for sharing

Answer

MIG overclocks the GPU for faster training

Answer

MIG merges multiple GPUs into one logical device

Answer

MIG is a software scheduler that time-slices CUDA streams

Question 13

Why might an ML image built on a developer's Apple Silicon (arm64) Mac fail to run on an x86_64 GPU server, and how do you fix it?

Accepted Answer

The image architecture mismatches the host; build with buildx --platform linux/amd64

Answer

The Dockerfile syntax differs by CPU; rewrite it

Answer

Apple Silicon cannot build Docker images at all

Answer

The fix is to disable BuildKit

Question 14

How should secrets (e.g., a Hugging Face token or cloud credentials) be provided to an ML build without leaking them into image layers?

Accepted Answer

Use BuildKit --mount=type=secret so they never persist in image layers

Answer

Pass them as build ARGs for convenience

Answer

Bake them into an ENV and rotate later

Answer

COPY the creds file then RUN rm to remove it

Question 15

What causes large layer sizes when installing Python ML packages in separate RUN steps, and how do you minimize them?

Accepted Answer

Caches in separate RUN layers persist; combine installs and clean caches within the same RUN

Answer

Layers are compressed so size never matters

Answer

Use more RUN steps to let Docker deduplicate

Answer

Set MAXLAYERS to limit size

Question 16

In Docker Compose for an ML stack, how do you grant a service GPU access in modern Compose syntax?

Accepted Answer

Use deploy.resources.reservations.devices with driver nvidia and capabilities [gpu]

Answer

Add gpu: true under the service

Answer

Set environment NVIDIA_GPU=1

Answer

GPUs are not supported in Compose

Question 17

What is the purpose of the CUDA compatibility check between container and host, and what does NVIDIA_DRIVER_CAPABILITIES / driver compatibility govern?

Accepted Answer

The host driver must be new enough for the container's CUDA version, else 'driver version insufficient' errors

Answer

The container ships its own driver, so host version is irrelevant

Answer

CUDA version must exactly equal the driver version

Answer

Driver compatibility only affects training, not inference

Question 18

Why are distroless or slim base images attractive for ML inference, and what is a notable limitation?

Accepted Answer

They minimize size and CVEs but lack a shell, complicating in-container debugging

Answer

They include extra debugging tools by default

Answer

They are faster at GPU inference

Answer

They automatically include CUDA libraries

Question 19

How does the choice of container start command affect graceful shutdown of an inference server, and what is the PID 1 / signal issue?

Accepted Answer

Exec form or an init (tini) ensures SIGTERM reaches the app for graceful shutdown

Answer

Shell form is preferred so the shell forwards signals

Answer

PID 1 always receives signals regardless of form

Answer

Graceful shutdown is handled entirely by Kubernetes, not the container

Question 20

What is the benefit of using a dedicated, versioned base ML image (golden image) shared across teams?

Accepted Answer

It centralizes vetted CUDA/framework versions and patches for consistency and reuse

Answer

It forces every team to use the same model architecture

Answer

It eliminates the need for a registry

Answer

It guarantees faster inference latency

Question 21

When serving multiple models, what are the trade-offs between one-container-per-model versus a multi-model server in a single container?

Accepted Answer

Per-model gives isolation and independent scaling; multi-model server improves utilization but couples lifecycles

Answer

Per-model containers always use less GPU memory

Answer

Multi-model servers cannot use the GPU

Answer

There is no difference in resource usage

Question 22

Why can container image pull time dominate cold-start for large LLM serving images, and what techniques mitigate it?

Accepted Answer

Large images slow cold start; mitigate with lazy snapshotters, pre-pulling, and externalized weights

Answer

Pull time is negligible for any image size

Answer

Compression makes pull time irrelevant

Answer

Cold start is purely a GPU warmup issue

Question 23

What does setting CPU and memory requests/limits do for an ML training pod, and what is the risk of omitting memory limits?

Accepted Answer

Without a memory limit a job can exhaust node memory and trigger OOM-kills affecting other pods

Answer

Limits speed up training by reserving more RAM

Answer

Requests and limits only apply to GPUs

Answer

Omitting limits has no effect because Kubernetes auto-scales memory

Question 24

How do you achieve reproducible Python environments inside ML containers beyond just 'pip install -r requirements.txt'?

Accepted Answer

Use a hash-pinned lock file plus a digest-pinned base image

Answer

Use 'pip install --upgrade' to always get consistent versions

Answer

Reproducibility is impossible in containers

Answer

Set requirements with >= constraints

Question 25

What is an init container pattern in Kubernetes for ML, and a common use case?

Accepted Answer

A container that runs to completion first, e.g., to download model weights into a shared volume

Answer

A container that runs alongside the app for logging

Answer

A container that restarts the app on failure

Answer

A GPU warmup daemon

Question 26

Why is 'docker commit' on a manually-modified running container considered an anti-pattern for ML reproducibility?

Accepted Answer

It produces an opaque, non-reproducible image instead of a declarative Dockerfile

Answer

It is the recommended way to capture experiments

Answer

It cannot capture GPU state

Answer

It always increases image size beyond limits

Question 27

How can you profile and reduce the number of vulnerabilities (CVEs) in an ML container image?

Accepted Answer

Scan with Trivy/Grype in CI and shrink the base/remove build tools to cut CVEs

Answer

Disable scanning to avoid false positives

Answer

CVEs only matter for the host, not images

Answer

Use the largest base image for stability

Question 28

What is the difference between a bind mount and a named volume for ML workloads, and when do you use each?

Accepted Answer

Bind mount maps a host path (e.g., live code/data); named volume is Docker-managed persistent storage

Answer

Bind mounts are managed by Docker; named volumes map to host paths

Answer

They are identical except named volumes are read-only

Answer

Named volumes cannot persist data after container removal

Question 29

In distributed training across multiple containers/pods, why is the network configuration (e.g., NCCL, host networking, RDMA) critical?

Accepted Answer

Collective ops (NCCL) dominate scaling, so interfaces/ports/RDMA must be correctly exposed

Answer

Networking is irrelevant since GPUs talk over PCIe only

Answer

NCCL runs only over the public internet

Answer

Host networking always slows distributed training

Question 30

What is the purpose of a liveness vs. readiness probe for a containerized model server, and a common ML-specific gotcha?

Accepted Answer

Readiness gates traffic, liveness restarts; long model loads need a startupProbe to avoid crash loops

Answer

Liveness gates traffic, readiness restarts the pod

Answer

Both probes do the same thing

Answer

Probes are unnecessary for GPU pods

Test yourself