Regularization (L1, L2, dropout) quiz

Question 1

What is the main purpose of regularization in machine learning?

Accepted Answer

To reduce overfitting and improve generalization

Answer

To make training run faster

Answer

To increase the number of model parameters

Answer

To guarantee zero training error

Question 2

What does the L2 regularization term add to a loss function?

Accepted Answer

The sum of squared weights times lambda

Answer

The sum of absolute weights times lambda

Answer

The number of nonzero weights

Answer

The maximum weight value

Question 3

What does the L1 regularization term add to a loss function?

Accepted Answer

The sum of absolute weights times lambda

Answer

The sum of squared weights times lambda

Answer

The product of all weights

Answer

The average of the weights

Question 4

Why is L1 regularization said to produce sparse models?

Accepted Answer

It drives many weights to exactly zero, removing features

Answer

It makes all weights equal

Answer

It doubles the largest weights

Answer

It never changes the weights

Question 5

How does dropout work during training of a neural network?

Accepted Answer

It randomly zeroes out a fraction of neurons each pass

Answer

It deletes layers permanently

Answer

It increases all activations

Answer

It freezes the weights

Question 6

Is dropout applied during inference (test time)?

Accepted Answer

No, it is disabled at inference and all neurons stay active

Answer

Yes, with the same drop rate as training

Answer

Yes, but only on the last layer

Answer

No, but the network is retrained each time

Question 7

What is the role of the regularization strength lambda?

Accepted Answer

It controls the strength of the regularization penalty

Answer

It sets the learning rate

Answer

It counts the number of layers

Answer

It is the batch size

Question 8

What happens if lambda in L2 regularization is set extremely high?

Accepted Answer

The model underfits because weights shrink too much

Answer

The model overfits more strongly

Answer

Training error becomes zero

Answer

The model learns faster with no downside

Question 9

How does regularization relate to the bias-variance tradeoff?

Accepted Answer

It raises bias a bit but lowers variance

Answer

It lowers both bias and variance to zero

Answer

It raises variance and lowers bias

Answer

It has no effect on bias or variance

Question 10

Why is L2 regularization also called 'weight decay'?

Accepted Answer

Because each update shrinks weights toward zero

Answer

Because it deletes weights randomly

Answer

Because it decays the learning rate

Answer

Because it increases weights over time

Question 11

What is the key difference between L1 and L2 regularization in how they affect weights?

Accepted Answer

L1 zeroes weights (sparse); L2 shrinks them smoothly

Answer

Both make all weights exactly zero

Answer

L2 zeroes weights; L1 shrinks them

Answer

Neither changes the weights

Question 12

What is Elastic Net regularization?

Accepted Answer

A combination of L1 and L2 penalties

Answer

A penalty based only on dropout

Answer

A method that removes all weights

Answer

A type of learning rate schedule

Question 13

A model gets 99% accuracy on training data but only 70% on test data. What does this suggest, and how can regularization help?

Accepted Answer

Overfitting; regularization can shrink the train-test gap

Answer

Underfitting; remove regularization

Answer

The model is perfect; no action needed

Answer

Data leakage that regularization fixes directly

Question 14

Why should the bias (intercept) term usually NOT be regularized?

Accepted Answer

It only shifts output and penalizing it harms baseline fit

Answer

It is the most important weight to shrink

Answer

Regularizing it speeds up training

Answer

It must always be regularized first

Question 15

How can dropout be interpreted as training an ensemble of networks?

Accepted Answer

Each pass trains a random sub-network; test time averages them

Answer

It trains one fixed network repeatedly

Answer

It trains separate independent networks from scratch

Answer

It is unrelated to ensembles

Question 16

In scikit-learn, which regression model applies L2 regularization by default?

Accepted Answer

Ridge

Answer

Lasso

Answer

DecisionTree

Answer

KMeans

Question 17

In scikit-learn, which linear model uses L1 regularization to produce sparse coefficients?

Accepted Answer

Lasso

Answer

Ridge

Answer

LinearRegression

Answer

SVR

Question 18

What does a typical dropout rate of 0.5 mean?

Accepted Answer

Each neuron is dropped with 50% probability per pass

Answer

Half the layers are removed forever

Answer

Half the data is discarded

Answer

The learning rate is halved

Question 19

Why is it important to standardize/scale features before applying L1 or L2 regularization?

Accepted Answer

So the penalty treats all features on a comparable scale

Answer

Because scaling removes the need for a loss function

Answer

To make features integers

Answer

Scaling has no relation to regularization

Question 20

How does increasing the amount of training data act like a form of regularization?

Accepted Answer

It reduces overfitting by making noise harder to memorize

Answer

It increases overfitting

Answer

It replaces the need for a loss function

Answer

It only affects training speed

Question 21

What is early stopping and why is it considered a regularization technique?

Accepted Answer

Stopping training when validation stops improving to limit overfitting

Answer

Stopping at the first epoch always

Answer

Removing neurons during training

Answer

Increasing the number of epochs indefinitely

Question 22

Geometrically, why does the L1 constraint region (a diamond) encourage solutions at the axes?

Accepted Answer

Its sharp corners lie on the axes where coordinates are zero

Answer

Its smooth circular shape avoids axes

Answer

It has no corners

Answer

It always touches the center

Question 23

What is the regularization penalty term for L2 written mathematically?

Accepted Answer

lambda * sum(w_i^2)

Answer

lambda * sum(|w_i|)

Answer

lambda * count(w_i != 0)

Answer

lambda * max(w_i)

Question 24

Can dropout and L2 regularization be used together in the same network?

Accepted Answer

Yes, they are complementary and often combined

Answer

No, they cancel each other out

Answer

No, only one is allowed per model

Answer

Yes, but only in linear models

Question 25

What is 'co-adaptation' of neurons, and how does dropout reduce it?

Accepted Answer

Neurons over-relying on each other; dropout breaks that dependence

Answer

Neurons learning identical weights; dropout copies them

Answer

Neurons that never activate; dropout deletes them

Answer

A type of learning rate decay

Question 26

In PyTorch, how do you switch off dropout for evaluation?

Accepted Answer

Call model.eval()

Answer

Call model.dropout(0)

Answer

Delete the dropout layers

Answer

Set learning rate to zero

Question 27

How is weight decay typically specified for an optimizer in PyTorch?

Accepted Answer

Via the weight_decay argument of the optimizer

Answer

By calling loss.backward(decay=True)

Answer

Through the dataloader

Answer

By renaming the loss function

Question 28

Why does L2 regularization tend to handle correlated features more gracefully than L1?

Accepted Answer

L2 spreads weight across correlated features; L1 picks one arbitrarily

Answer

L1 spreads weight evenly; L2 picks one

Answer

Both ignore correlated features

Answer

Neither is affected by correlation

Question 29

From a Bayesian viewpoint, what prior does L2 regularization correspond to?

Accepted Answer

A zero-mean Gaussian prior on the weights

Answer

A uniform prior

Answer

A Laplace prior

Answer

No prior at all

Question 30

What practical problem can too much dropout (e.g., rate 0.9) cause?

Accepted Answer

Underfitting due to too little usable capacity

Answer

Faster convergence with no downside

Answer

Guaranteed overfitting

Answer

Exploding gradients only

Test yourself