Model Interpretability quiz

Question 1

What is model interpretability and why does it matter?

Accepted Answer

Understanding how/why a model makes predictions; needed for trust, fairness, debugging, regulation

Answer

Same as model accuracy

Answer

Only needed for complex models

Answer

Only required by law

Question 2

What is the difference between global and local interpretability?

Accepted Answer

Global: overall behavior; local: specific prediction explanation

Answer

They are the same

Answer

Global is always sufficient

Answer

Local explanations require model access always

Question 3

What is SHAP (SHapley Additive exPlanations)?

Accepted Answer

Average marginal contribution of each feature across all possible feature subsets

Answer

Simple feature importance from training

Answer

Same as gradient × input

Answer

Only for linear models

Question 4

What is GradCAM (Gradient-weighted Class Activation Mapping)?

Accepted Answer

Weights last conv feature maps by gradient-average of class score, upsampled as heatmap

Answer

Applies attention to features

Answer

Only for transformer models

Answer

Computes raw gradients at pixel level

Question 5

What is feature importance in tree-based models?

Accepted Answer

Impurity (fast but biased) / permutation (more reliable) / SHAP (theoretically grounded)

Answer

Only Gini importance is valid

Answer

Permutation importance requires labels

Answer

Feature importance is only for linear models

Question 6

What is a counterfactual explanation?

Accepted Answer

Minimal changes to input that would flip the prediction, providing actionable recourse

Answer

Explains what features matter globally

Answer

Computes gradient of output

Answer

Samples from counterfactual distribution

Question 7

What is a decision tree as an interpretable model?

Accepted Answer

Each prediction follows a human-readable if-then rule path, fully transparent for shallow trees

Answer

Decision trees are never interpretable

Answer

Deep decision trees are most interpretable

Answer

Requires SHAP for explanation

Question 8

What is ANCHOR (rule-based explanations)?

Accepted Answer

If-then rules that are sufficient for the prediction with high precision; crisp and actionable

Answer

Same as LIME coefficients

Answer

Requires gradient computation

Answer

Only for decision trees

Question 9

What is attention as explanation in Transformers?

Accepted Answer

Attention weights visualize token relations but are not always faithful explanations; combine with gradients

Answer

Attention weights are always faithful

Answer

Attention should replace all other methods

Answer

Only cross-attention is informative

Question 10

What is the difference between intrinsic and post-hoc interpretability?

Accepted Answer

Intrinsic: interpretable by design (trees, linear); post-hoc: explain black-box after training

Answer

They are the same

Answer

Post-hoc is always more faithful

Answer

Intrinsic models are always less accurate

Question 11

What is mechanistic interpretability?

Accepted Answer

Reverse-engineers neural computations into human-understandable circuits and algorithms

Answer

Same as feature importance

Answer

Measures model accuracy

Answer

Only for small models

Question 12

What is the logit lens?

Accepted Answer

Apply final unembedding to intermediate activations to read predictions at each layer

Answer

Only applicable to encoder models

Answer

Requires gradient computation

Answer

Same as probing

Question 13

What is TCAV vs SHAP for concept-level explanations?

Accepted Answer

SHAP: feature-level attribution; TCAV: human-concept attribution — TCAV is more interpretable for domain experts

Answer

They are identical

Answer

SHAP works only for concept explanations

Answer

TCAV requires no concept dataset

Question 14

What is the faithfulness of an explanation?

Accepted Answer

Explanation accurately reflects model computation; tested via deletion/insertion/ROAR

Answer

Always equal to human plausibility

Answer

Only measured by accuracy drop

Answer

Attention is always faithful

Question 15

What is model debugging with interpretability tools?

Accepted Answer

Interpretability reveals shortcuts, leakage, and distributional issues causing model errors

Answer

Only used for compliance reports

Answer

Requires labeled error examples

Answer

Only GradCAM works for debugging

Question 16

What is the problem of spurious correlations and how does interpretability detect them?

Accepted Answer

Models learn confounders correlated with labels in training; interpretability reveals what features are actually used

Answer

Spurious correlations improve generalization

Answer

Only occurs in computer vision

Answer

SHAP cannot detect spurious correlations

Question 17

What is the interpretability-accuracy trade-off?

Accepted Answer

Interpretable models are often less accurate; trade-off is smaller in low-dimensional/near-linear problems

Answer

Always a large trade-off

Answer

Interpretable models always have lower accuracy

Answer

No trade-off with modern methods

Question 18

What is the superimposition of explanations problem?

Accepted Answer

Each output class has its own attribution; SHAP/GradCAM computed per class; complex for multi-step reasoning

Answer

Single attribution shared across all outputs

Answer

Multi-output models cannot be explained

Answer

Only for image models

Question 19

What is activation maximization?

Accepted Answer

Gradient ascent on input to find what maximally activates a neuron; visualizes learned features

Answer

Backpropagates to the final layer only

Answer

Requires labeled data

Answer

Same as saliency maps

Question 20

What are model cards and datasheets for interpretability?

Accepted Answer

Documents describing model's intended use, evaluation by group, limitations, and biases for accountability

Answer

Same as SHAP summary plots

Answer

Only required for academic papers

Answer

Only needed during deployment

Question 21

What is the SHAP waterfall plot?

Accepted Answer

Shows how each feature's SHAP value pushes prediction from baseline to final output

Answer

Same as summary plot

Answer

Shows global feature importance only

Answer

Requires labeled explanations

Question 22

What is the difference between explanation faithfulness and plausibility?

Accepted Answer

Faithfulness: reflects actual computation; plausibility: aligns with human intuition — both needed, not always correlated

Answer

They are the same measure

Answer

Faithfulness = accuracy

Answer

Plausibility is always more important

Question 23

What is rationalization in NLP interpretability?

Accepted Answer

Selects text subset sufficient to reproduce prediction; ERASER evaluates comprehensiveness and sufficiency

Answer

Same as attention visualization

Answer

Generates random text

Answer

Only for sequence-to-sequence models

Question 24

What is the SHAP force plot?

Accepted Answer

Horizontal arrows from baseline showing each feature's signed SHAP contribution to the prediction

Answer

Bar chart of global importance

Answer

Only for tree models

Answer

Shows prediction over time

Question 25

What is influence functions in interpretability?

Accepted Answer

Estimate effect of removing a training example on test prediction via Hessian-based gradient formula

Answer

Gradient of test loss only

Answer

Attention over training examples

Answer

Same as leave-one-out cross-validation

Question 26

What is concept drift and how does interpretability detect it?

Accepted Answer

Monitor SHAP distributions over time; shifts in feature importance signal concept drift requiring retraining

Answer

Only accuracy monitoring needed

Answer

Concept drift cannot be detected

Answer

Retraining always fixes concept drift

Test yourself