Privacy-preserving ML (federated, DP) quiz

Question 1

In the FedAvg algorithm, what is the primary role of the central server, and is client data private from the server in a standard (non-secure-aggregation) deployment?

Accepted Answer

The server averages locally-trained model updates; standard FedAvg exposes those weight-update vectors to the server — it is not private by default.

Answer

The server trains a global model on aggregated raw data from all clients.

Answer

The server uses homomorphic encryption so it never decrypts client updates.

Answer

The server only collects labels, never model weights, to ensure privacy.

Question 2

What does (ε, δ)-differential privacy guarantee, and what is the intuitive meaning of a small δ?

Accepted Answer

ε bounds the multiplicative privacy loss in the typical case; δ is the additive failure probability that the ε-bound does not hold.

Answer

ε is the probability of distinguishing two datasets; δ is the noise scale.

Answer

ε and δ together specify the clipping norm for gradient sanitization.

Answer

δ is the minimum noise standard deviation; ε is the number of training epochs allowed.

Question 3

Explain the Laplace mechanism and the Gaussian mechanism for DP. Under what condition would you prefer Gaussian over Laplace?

Accepted Answer

Gaussian is preferred when composing many mechanisms, as Rényi-DP / moments-accountant analysis gives tighter privacy accounting under repeated composition.

Answer

Gaussian is preferred when you need pure (ε,0)-DP with no failure probability.

Answer

Laplace is always preferred because it achieves strictly smaller noise variance than Gaussian for the same ε.

Answer

Gaussian requires no knowledge of query sensitivity, making it universally superior.

Question 4

Describe DP-SGD (Abadi et al. 2016). What are the two core per-step operations that make SGD differentially private?

Accepted Answer

Per-sample gradient clipping to bound sensitivity, followed by addition of calibrated Gaussian noise.

Answer

Gradient sparsification and secure multi-party aggregation.

Answer

Gradient quantization and shuffling of the training dataset before each epoch.

Answer

Replacing SGD with Adam and adding Laplace noise to the loss function.

Question 5

What is the 'moments accountant' (or Rényi DP accountant), and why does it yield tighter privacy guarantees under composition than both the basic and advanced composition theorems?

Accepted Answer

It tracks Rényi divergences at multiple orders; they compose exactly without slack, and optimizing α at conversion gives tighter (ε,δ)-DP than both basic (O(T·ε)) and advanced composition (also O(√T·ε) but with larger constants).

Answer

It replaces ε with the KL divergence, which is always smaller, giving trivially better bounds.

Answer

It uses the strong composition theorem that requires mechanisms to be independent across steps.

Answer

It computes exact DP via Fourier transforms and is only applicable to the Laplace mechanism.

Question 6

What is 'privacy amplification by subsampling', and how does it tighten the privacy guarantee of DP-SGD?

Accepted Answer

Subsampling amplifies privacy because excluded records do not affect the subsample's output at all, so each record is at risk only with probability q, reducing effective ε per step by ~q.

Answer

Subsampling has no effect on DP guarantees; privacy only depends on total noise added.

Answer

Privacy amplification applies only to the Laplace mechanism, not to Gaussian-based DP-SGD.

Answer

Amplification requires sampling with replacement; without-replacement sampling invalidates DP.

Question 7

What is Local Differential Privacy (LDP) and how does it differ from Central DP? Give a real-world deployment example of each.

Accepted Answer

In central DP a trusted server collects raw data and adds noise; in LDP each user randomizes locally, requiring no trust in the server but incurring higher noise for the same utility.

Answer

LDP and central DP are equivalent; the difference is only in where the noise is calibrated.

Answer

LDP always achieves tighter ε than central DP for the same utility because noise is added closer to the data.

Answer

Central DP is a special case of LDP where the noise multiplier is set to zero at the client side.

Question 8

Explain the PATE (Private Aggregation of Teacher Ensembles) framework. What DP mechanism does it use to privatize teacher votes?

Accepted Answer

PATE adds calibrated noise to teacher vote counts (noisy argmax) so the student label is DP, then trains a student on public data labeled this way.

Answer

PATE uses homomorphic encryption to aggregate teacher logits privately.

Answer

PATE applies DP-SGD to a single model trained on all partitions simultaneously.

Answer

PATE requires all teacher models to be trained with LDP before aggregation.

Question 9

What is Secure Aggregation (SecAgg) in federated learning, and what cryptographic primitive does it rely on?

Accepted Answer

SecAgg uses pairwise Diffie-Hellman masks that cancel in the aggregate, so the server sees only the sum of updates, not individual ones; dropout is handled via Shamir secret sharing.

Answer

SecAgg uses fully homomorphic encryption so the server decrypts the final sum with its private key.

Answer

SecAgg requires a trusted third party to decrypt and re-encrypt updates before forwarding them to the server.

Answer

SecAgg applies zero-knowledge proofs so each client proves its update is valid without revealing it.

Question 10

What is a membership inference attack (MIA) and how does it exploit overfitting to detect whether a record was in the training set?

Accepted Answer

MIA trains a meta-classifier on model confidence scores to predict whether a record was in the training set, exploiting the train–test confidence gap caused by overfitting.

Answer

MIA reconstructs the exact training record from model weights using gradient inversion.

Answer

MIA requires white-box access and cannot work against APIs returning only top-1 predictions.

Answer

MIA is only effective against models without dropout because dropout randomizes outputs.

Question 11

Describe the DLG (Deep Leakage from Gradients) attack. What setting makes it possible, and what are effective defenses?

Accepted Answer

DLG reconstructs training inputs by optimizing dummy inputs to match shared gradients; defenses include DP noise, SecAgg, and gradient compression.

Answer

DLG requires the attacker to have access to the test set to reconstruct training inputs.

Answer

DLG only works against convolutional networks; transformer gradients cannot be inverted.

Answer

DLG is defeated by FedAvg because averaging gradients from multiple users eliminates all individual signal.

Question 12

What is the 'shuffle model' of differential privacy, and how does it achieve amplification beyond what local DP provides alone?

Accepted Answer

Shuffling permutes LDP-randomized messages so the analyzer cannot link message to sender, yielding amplification to O(ε_L · e^{ε_L} · √(log(1/δ)/n)) central-DP equivalent.

Answer

The shuffle model achieves amplification by requiring the server to add central DP noise after shuffling, combining both noise sources.

Answer

The shuffle model reduces noise by allowing users to send raw data that the shuffler anonymizes using k-anonymity.

Answer

Shuffle DP requires homomorphic encryption and provides the same guarantee as pure LDP without any amplification.

Question 13

What is the composition theorem in DP, and how do the basic, advanced, and Rényi DP composition results differ in tightness for k sequential mechanisms?

Accepted Answer

Basic gives O(k·ε); advanced gives O(√k·ε) with larger constants; Rényi DP accountant also gives O(√k·ε) but with tighter constants via exact additive composition and optimization over Rényi order α.

Answer

All three give the same O(k·ε) bound; the difference is only in the constant factor.

Answer

Rényi DP gives a O(log k·ε) bound, which is strictly tighter than both basic and advanced composition.

Answer

Advanced composition applies only when all k mechanisms are identical; Rényi DP applies to arbitrary heterogeneous mechanisms.

Question 14

How does non-IID (heterogeneous) data distribution across clients harm FedAvg, and what algorithm variants address 'client drift'?

Accepted Answer

Non-IID data causes client models to diverge; FedProx (proximal regularization) and SCAFFOLD (control variates) are key mitigations.

Answer

Non-IID data only affects communication efficiency, not model accuracy; FedAvg handles it without modification.

Answer

Client drift is solved by increasing the server-side learning rate, which overrides local biases.

Answer

Non-IID distributions cause the Gaussian noise in DP-SGD to become non-symmetric, requiring Laplace noise instead.

Question 15

What is 'split learning', how does it differ from federated learning, and what privacy risks remain?

Accepted Answer

In split learning clients share intermediate activations (not full model weights) with the server; privacy risks remain because activations can be inverted to reconstruct raw inputs.

Answer

Split learning is completely private because the server only sees gradients, not raw data or model weights.

Answer

Split learning is identical to federated learning but uses a larger batch size on the server side.

Answer

Split learning requires homomorphic encryption to protect the activations sent to the server.

Question 16

What is DP-GAN (Differentially Private GAN), and what unique challenge does applying DP-SGD to GAN training introduce?

Accepted Answer

DP-SGD is applied to the discriminator (which touches real data); the added noise worsens GAN instability, requiring mitigations such as public pre-training of the discriminator.

Answer

DP-GAN applies DP noise to the generator's weights so the discriminator never sees private data.

Answer

DP-GAN is impossible because GANs require exact gradients and any noise causes complete training failure.

Answer

Both generator and discriminator are trained with DP-SGD, doubling the privacy budget consumption per step.

Question 17

Explain Federated Analytics (FA) vs Federated Learning (FL). What DP mechanisms are typically used in FA?

Accepted Answer

FA computes aggregate statistics over device data without model training; DP mechanisms like randomized response and Laplace are applied per query with sensitivity typically equal to 1.

Answer

Federated Analytics trains a model; Federated Learning only computes statistics — they are the reverse of what the names suggest.

Answer

FA uses SecAgg to encrypt model weights; FL uses LDP to protect statistics.

Answer

Federated Analytics requires a trusted third party because statistics have higher sensitivity than model gradients.

Question 18

What is the 'gradient clipping norm' C in DP-SGD, and how does its choice affect the privacy–utility tradeoff?

Accepted Answer

C is the per-sample L2 clipping norm; smaller C reduces noise magnitude but biases gradients; larger C preserves gradient directions but requires more noise at fixed (ε,δ).

Answer

C controls only the privacy budget ε and has no effect on gradient direction or utility.

Answer

C must always equal 1.0; any other value violates DP guarantees for the Gaussian mechanism.

Answer

Increasing C always improves utility because fewer gradients are clipped.

Question 19

What is 'Byzantine-robust' federated learning, and what attacks does it defend against?

Accepted Answer

Byzantine-robust FL replaces naive averaging with robust aggregators (e.g., Krum, coordinate-wise median) to defend against malicious clients sending arbitrary poisoned updates.

Answer

Byzantine robustness means the FL system tolerates network packet loss, ensuring gradient updates are never lost.

Answer

Byzantine-robust FL applies DP-SGD on the server side to neutralize adversarial gradients.

Answer

Byzantine faults are handled by SecAgg, which prevents the server from seeing individual updates and thus prevents poisoning.

Question 20

How is the privacy budget ε consumed and reported when fine-tuning a large language model (LLM) with DP-SGD, and what practical constraints arise?

Accepted Answer

Budget is consumed step-by-step via Rényi DP and reported at training end; LoRA reduces per-sample gradient memory; practical ε for useful LLM utility is typically 3–8.

Answer

LLM fine-tuning with DP-SGD is impractical; the only viable approach is PATE because LLMs cannot compute per-sample gradients.

Answer

DP-SGD for LLMs requires ε < 1 to ensure meaningful privacy, limiting fine-tuning to at most 100 steps.

Answer

The privacy budget is set before training and training automatically stops when exhausted, requiring no manual tracking.

Question 21

What is the 'Randomized Response' mechanism? Give the formula for ε-LDP and show how it is used in frequency estimation.

Accepted Answer

RR reports the true value with probability p = e^ε/(1+e^ε) and flips with 1−p, achieving ε-LDP; frequency is recovered by debiasing the noisy mean.

Answer

Randomized Response reports the true value always but encrypts it; the server decrypts only the aggregate.

Answer

RR adds Laplace noise to continuous values; it is not applicable to binary or categorical data.

Answer

RR requires all users to share a secret key so the server can unscramble responses; without the key, no unbiasing is possible.

Question 22

What is 'personalized federated learning' (PFL) and how do approaches like Ditto or Per-FedAvg balance global and local model objectives?

Accepted Answer

Ditto regularizes local objectives toward the global model; Per-FedAvg treats the global model as a MAML-style meta-initialization for fast local adaptation.

Answer

PFL trains a single global model that adapts to each client at inference time via attention over client IDs.

Answer

PFL requires DP noise to be added per client to ensure the personalized models are private with respect to each other.

Answer

Personalized FL is identical to transfer learning: the global model is pre-trained and clients only fine-tune the classification head.

Question 23

How does Homomorphic Encryption (HE) differ from Secure Aggregation in federated learning, and what computational bottleneck limits HE in practice?

Accepted Answer

HE allows arbitrary polynomial computation on encrypted data without decryption, enabling richer operations than SecAgg's sum, but is limited by enormous ciphertext size and compute overhead (1000×+ slower than plaintext).

Answer

HE and SecAgg are equivalent; HE is just a slower software implementation of the same masking approach.

Answer

HE is used for training; SecAgg is used only at inference time to protect predictions.

Answer

SecAgg requires FHE internally; the main difference is that SecAgg hides this complexity from the developer.

Question 24

What is a 'model inversion attack', and how does differential privacy provide a formal defense?

Accepted Answer

Model inversion optimizes inputs to maximize class confidence, reconstructing representative training samples; DP bounds the information any individual record contributes to the model.

Answer

Model inversion reconstructs model weights from training labels; DP prevents this by encrypting weights.

Answer

Model inversion only works against linear models; deep networks are immune due to non-linearity.

Answer

DP prevents model inversion by adding noise to model output logits at inference time.

Question 25

Describe the Google Gboard FL system's uplink bandwidth problem and three techniques used to reduce it.

Accepted Answer

Three techniques: structured updates (subspace restriction), gradient quantization, and sketched updates reduce uplink bandwidth while maintaining model accuracy.

Answer

Gboard uses 5G to avoid bandwidth constraints; no compression is needed in modern deployments.

Answer

Gboard transmits only label distributions rather than gradients to reduce bandwidth.

Answer

Communication overhead in FL is dominated by downlink (server to client); uplink bandwidth is negligible.

Question 26

What is the 'privacy odometer' concept, and how does it support adaptive composition in practice?

Accepted Answer

A privacy odometer tracks cumulative privacy loss online, enabling adaptive decisions about future queries without invalidating the DP guarantee.

Answer

A privacy odometer measures the time elapsed since the last DP noise was added to ensure regular re-randomization.

Answer

The privacy odometer is a hardware security module that enforces the ε budget by blocking queries once it is exhausted.

Answer

Privacy odometers only work under pure (ε,0)-DP; they are incompatible with (ε,δ)-DP.

Question 27

In the context of DP-SGD, what is 'per-sample gradient computation' and why is it more expensive than standard mini-batch gradient computation?

Accepted Answer

Standard mini-batch backprop averages before clipping, losing individual sample info; per-sample computation materializes each sample's gradient, requiring O(B) memory overhead for batch size B.

Answer

Per-sample gradients are cheaper because you only need one backward pass per epoch, not one per step.

Answer

Per-sample gradients require a separate model copy for each sample, making batch size > 1 impractical.

Answer

Per-sample computation is needed only when using the Laplace mechanism; Gaussian DP-SGD uses standard averaged gradients.

Question 28

What is 'differential privacy with public data' (semi-private learning) and why can it drastically reduce the required noise compared to fully private learning?

Accepted Answer

Public data allows learning representations at zero privacy cost; only private labeling or fine-tuning queries consume budget, drastically reducing required noise.

Answer

Semi-private learning is less private than standard DP because public data can be used to infer private labels via inversion.

Answer

Using public data alongside private data invalidates the DP guarantee because the public data introduces a side channel.

Answer

Semi-private learning requires all public data to have been collected under the same DP mechanism as the private data.

Question 29

How does the 'sensitivity' of a function affect DP noise calibration, and why is bounding sensitivity often the hardest part of DP system design?

Accepted Answer

Sensitivity Δf directly scales the required noise; bounding it is hard for complex functions (often unbounded without clipping), and tight bounds are critical for good utility at fixed (ε,δ).

Answer

Sensitivity only matters for the Laplace mechanism; the Gaussian mechanism is sensitivity-free.

Answer

Sensitivity is always 1.0 for classification tasks because outputs are probabilities bounded in [0,1].

Answer

Higher sensitivity means less noise is needed because the function already varies a lot naturally.

Question 30

What is 'zero-concentrated differential privacy' (zCDP) and how does it relate to (ε,δ)-DP and Rényi DP?

Accepted Answer

zCDP bounds the Rényi divergence linearly in α (≤ ρα), composes simply as Tρ under T-fold composition, and implies (ε,δ)-DP via a closed-form conversion; it lies strictly between pure ε-DP and approximate (ε,δ)-DP.

Answer

zCDP is weaker than (ε,δ)-DP and is used when utility demands override privacy.

Answer

zCDP is identical to Rényi DP at order α=2; other orders are not used in practice.

Answer

zCDP replaces the Gaussian mechanism with a zero-mean concentrated noise distribution that achieves better utility at the same ε.

Test yourself