Overfitting and regularization quiz

Question 1

What is overfitting in machine learning?

Accepted Answer

A model that learns noise in training data and fails to generalize to new data

Answer

A model that is too simple to capture the pattern

Answer

A model trained on too little data only

Answer

A model that runs too slowly during prediction

Question 2

What is underfitting, and how does it differ from overfitting?

Accepted Answer

Underfitting performs poorly on both training and test data; overfitting performs well on training but poorly on test

Answer

Underfitting means the model is too complex; overfitting too simple

Answer

They are the same thing

Answer

Underfitting only happens with neural networks

Question 3

What is the bias-variance tradeoff?

Accepted Answer

The balance between bias (underfitting) and variance (overfitting) error

Answer

The tradeoff between training speed and accuracy

Answer

The choice between linear and nonlinear models

Answer

The tradeoff between dataset size and number of features

Question 4

What is regularization and why is it used?

Accepted Answer

A technique that adds a complexity penalty to reduce overfitting

Answer

A technique to speed up training

Answer

A method to increase the number of features

Answer

A way to normalize input data only

Question 5

How does L2 regularization (Ridge) affect a model's weights?

Accepted Answer

It shrinks weights toward zero but rarely makes them exactly zero

Answer

It sets many weights exactly to zero

Answer

It only affects the bias term

Answer

It increases all weights equally

Question 6

How does L1 regularization (Lasso) differ from L2 in its effect on weights?

Accepted Answer

L1 can set some weights exactly to zero (sparsity); L2 shrinks them smoothly

Answer

L1 keeps all weights nonzero; L2 sets some to zero

Answer

L1 and L2 produce identical results

Answer

L1 only works on classification problems

Question 7

In the regularized loss L = Loss + lambda * penalty, what does the hyperparameter lambda control?

Accepted Answer

The strength of the regularization penalty

Answer

The learning rate of the optimizer

Answer

The number of training epochs

Answer

The size of the dataset

Question 8

What typically happens to training error and validation error as a model overfits during training?

Accepted Answer

Training error decreases while validation error starts to increase

Answer

Both training and validation error keep decreasing

Answer

Both errors increase together

Answer

Validation error decreases while training error increases

Question 9

What is early stopping and how does it prevent overfitting?

Accepted Answer

Stopping training when validation error stops improving to avoid overfitting

Answer

Stopping training after a fixed 100 epochs always

Answer

Removing early data points from the dataset

Answer

Starting training with a small learning rate

Question 10

What is dropout in neural networks?

Accepted Answer

Randomly deactivating neurons during training to reduce overfitting

Answer

Permanently removing neurons that perform poorly

Answer

Dropping the learning rate over time

Answer

Removing outlier samples from training data

Question 11

Why does collecting more training data often help reduce overfitting?

Accepted Answer

It gives a more representative sample so the model can't easily memorize noise

Answer

It makes training faster

Answer

It always increases bias

Answer

It removes the need for any regularization

Question 12

What role does a validation set play in detecting overfitting?

Accepted Answer

Held-out data used to detect overfitting and tune hyperparameters

Answer

Data used to train the model's weights

Answer

Data used only for the final reported accuracy

Answer

Data that is discarded before training

Question 13

What is k-fold cross-validation and how does it help assess generalization?

Accepted Answer

Splitting data into k folds, rotating which fold is validation, and averaging results

Answer

Training k separate models on the same data and picking the best

Answer

Using k percent of data for training

Answer

Repeating training k times with different learning rates

Question 14

How does increasing model complexity (e.g., polynomial degree) relate to overfitting?

Accepted Answer

Higher complexity lowers bias but raises variance, eventually causing overfitting

Answer

Higher complexity always improves test performance

Answer

Complexity has no effect on overfitting

Answer

Lower complexity always causes overfitting

Question 15

Why is a model that achieves 100% training accuracy potentially a concern?

Accepted Answer

It may indicate memorization and overfitting rather than true learning

Answer

It always means the model is excellent

Answer

It guarantees perfect test accuracy

Answer

It means the model is underfitting

Question 16

What is the connection between regularization and Occam's razor?

Accepted Answer

Regularization embodies Occam's razor by favoring simpler models

Answer

Regularization prefers complex models over simple ones

Answer

Occam's razor is unrelated to machine learning

Answer

Regularization always increases model complexity

Question 17

In scikit-learn, how do you create a Ridge regression model with regularization strength alpha=1.0?

Accepted Answer

Ridge(alpha=1.0)

Answer

Ridge(penalty=1.0)

Answer

Ridge(lambda=1.0)

Answer

Ridge(C=1.0)

Question 18

What is data augmentation and how does it help reduce overfitting?

Accepted Answer

Creating new training examples via label-preserving transformations

Answer

Deleting noisy samples from the dataset

Answer

Adding more layers to a neural network

Answer

Reducing the number of training examples

Question 19

What does it mean for a model to 'generalize well'?

Accepted Answer

It performs well on new, unseen data

Answer

It performs well only on training data

Answer

It memorizes the training set perfectly

Answer

It trains very quickly

Question 20

Why is the test set kept completely separate and used only once at the end?

Accepted Answer

To give an unbiased final estimate without leaking information

Answer

To make training faster

Answer

Because it is smaller than the training set

Answer

To tune hyperparameters repeatedly

Question 21

How does a high-variance model behave when trained on slightly different datasets?

Accepted Answer

It produces very different results with small data changes

Answer

It gives nearly identical results regardless of the data

Answer

It always underfits

Answer

Its predictions never change

Question 22

What is Elastic Net regularization?

Accepted Answer

A combination of L1 and L2 penalties

Answer

A method that uses only L1 penalty

Answer

A technique that removes regularization entirely

Answer

A type of dropout for linear models

Question 23

Why can adding irrelevant features increase the risk of overfitting?

Accepted Answer

They give the model more ways to fit noise and spurious patterns

Answer

Irrelevant features always improve accuracy

Answer

They reduce the number of parameters

Answer

They have no effect on overfitting

Question 24

What is the geometric reason L1 regularization tends to produce sparse solutions?

Accepted Answer

Because L1's diamond-shaped region has corners on the axes where weights are zero

Answer

Because L1's circular region pushes weights to zero

Answer

Because L1 squares the weights

Answer

Because L1 has no penalty effect

Question 25

How does increasing dataset size affect the bias-variance tradeoff for a fixed model?

Accepted Answer

It mainly reduces variance while bias stays roughly the same

Answer

It mainly reduces bias while variance increases

Answer

It increases both bias and variance

Answer

It has no effect on either

Question 26

In a decision tree, what is pruning and how does it relate to overfitting?

Accepted Answer

Removing branches to reduce complexity and overfitting

Answer

Adding more branches to fit data better

Answer

Increasing the tree depth without limit

Answer

Randomly selecting features at each split

Question 27

Why does limiting the maximum depth of a decision tree help prevent overfitting?

Accepted Answer

It restricts complexity so the tree captures broad patterns, not noise

Answer

It makes the tree fit noise more easily

Answer

It always reduces accuracy on all data

Answer

It increases the number of leaf nodes

Question 28

What is the difference between a parameter and a hyperparameter, with examples from regularization?

Accepted Answer

Parameters are learned from data; hyperparameters (like lambda) are set before training

Answer

Parameters are set manually; hyperparameters are learned

Answer

They are the same thing

Answer

Hyperparameters change during each gradient step

Question 29

How can you use a learning curve (error vs. training set size) to diagnose overfitting?

Accepted Answer

A large persistent gap (low train, high validation error) indicates overfitting

Answer

Both curves being high and close indicates overfitting

Answer

Overfitting cannot be diagnosed from learning curves

Answer

A gap means the model is underfitting

Question 30

Why should feature scaling be applied before L1 or L2 regularization?

Accepted Answer

Scaling ensures the penalty applies fairly across features of different magnitudes

Answer

Scaling is irrelevant to regularization

Answer

Scaling removes the need for regularization

Answer

Regularization only works on unscaled data

Test yourself