Backpropagation explained quiz

Question 1

What is backpropagation in a neural network?

Accepted Answer

An algorithm that computes gradients of the loss w.r.t. all weights using the chain rule, moving backward through the network

Answer

A method to randomly initialize the network's weights before training

Answer

A technique to add more layers to a neural network automatically

Answer

A way to feed input data forward to produce predictions

Question 2

Why is the chain rule of calculus central to backpropagation?

Accepted Answer

Because the network is a composition of functions, so derivatives multiply along the chain from output to input

Answer

Because it speeds up the forward pass by skipping layers

Answer

Because it removes the need to compute any derivatives

Answer

Because it guarantees the loss reaches exactly zero

Question 3

What two passes make up one training step using backpropagation?

Accepted Answer

The forward pass (compute output and loss) and the backward pass (compute gradients)

Answer

The encoding pass and the decoding pass

Answer

The training pass and the validation pass

Answer

The pooling pass and the convolution pass

Question 4

Why must the forward pass store intermediate activations for backpropagation to work?

Accepted Answer

Because gradients depend on the layer inputs and local derivatives evaluated at the forward-pass values

Answer

Because activations are the final prediction of the network

Answer

Because it reduces the number of layers needed

Answer

Because the optimizer cannot run without storing the loss only

Question 5

What is the relationship between backpropagation and gradient descent?

Accepted Answer

Backpropagation computes the gradients; gradient descent uses them to update the weights

Answer

They are two names for the exact same operation

Answer

Gradient descent computes gradients and backpropagation updates weights

Answer

Neither involves gradients

Question 6

In the weight update rule w := w - η·(∂L/∂w), what does each symbol represent?

Accepted Answer

w is the weight, η the learning rate, ∂L/∂w the gradient supplied by backprop

Answer

w is the loss, η the gradient, ∂L/∂w the learning rate

Answer

η is the weight and w is the step size

Answer

∂L/∂w is the prediction error of the output neuron only

Question 7

What is the role of the loss function in backpropagation?

Accepted Answer

It quantifies prediction error and provides the initial gradient that backprop propagates backward

Answer

It selects which neurons to delete from the network

Answer

It replaces the need for a forward pass

Answer

It sets the random seed for initialization

Question 8

Why does backpropagation reuse intermediate gradient computations instead of recomputing them?

Accepted Answer

Shared upstream factors are computed once and reused, making it as cheap as a forward pass

Answer

Recomputing is impossible in software

Answer

Reuse changes the final gradient values to be approximate

Answer

It is only done to save disk space, not computation

Question 9

For a single neuron z = w·x + b with output a = σ(z), what is ∂L/∂w by the chain rule?

Accepted Answer

(∂L/∂a)·σ'(z)·x

Answer

(∂L/∂a)·σ'(z)·w

Answer

(∂L/∂a)·x·b

Answer

σ'(z) only

Question 10

What does the term 'local gradient' mean at a node during backpropagation?

Accepted Answer

The derivative of a node's output w.r.t. its own inputs, multiplied by the upstream gradient

Answer

The total gradient of the whole network

Answer

A randomly chosen scaling factor

Answer

The learning rate at that node

Question 11

What is a computational graph and why is it useful for backpropagation?

Accepted Answer

A graph of operations and data flow that is traversed in reverse to apply the chain rule

Answer

A bar chart of training loss over time

Answer

A diagram of the dataset's class distribution

Answer

A hardware diagram of the GPU

Question 12

At an addition node z = x + y, how does the upstream gradient distribute to x and y?

Accepted Answer

The gradient is copied unchanged to both x and y

Answer

The gradient is split in half between x and y

Answer

The gradient is multiplied by x for one input and y for the other

Answer

Only x receives the gradient

Question 13

At a multiplication node z = x·y, how is the upstream gradient routed to each input?

Accepted Answer

Each input's gradient is the upstream gradient times the other input's value

Answer

Each input gets the upstream gradient unchanged

Answer

Each input gets the gradient times its own value

Answer

The gradient becomes zero at a multiply node

Question 14

What is the derivative of the sigmoid function σ(z), and why is its form convenient?

Accepted Answer

σ'(z) = σ(z)(1 − σ(z)), reusing the forward output

Answer

σ'(z) = 1 − σ(z)²

Answer

σ'(z) = z·σ(z)

Answer

σ'(z) = σ(z) for all z

Question 15

What is the derivative of the ReLU activation, and how does it affect gradient flow?

Accepted Answer

1 for positive inputs and 0 for negative inputs, gating the gradient

Answer

Always 1, so gradients never change

Answer

σ(z)(1−σ(z)) like sigmoid

Answer

Always 0, blocking all gradients

Question 16

What is the vanishing gradient problem in the context of backpropagation?

Accepted Answer

Gradients shrink exponentially through layers when derivative factors are < 1, stalling early layers

Answer

Gradients grow without bound and crash training

Answer

The loss function disappears during training

Answer

The network runs out of memory storing activations

Question 17

What is the exploding gradient problem and one common way to address it?

Accepted Answer

Gradients grow exponentially large and unstable; gradient clipping is a common fix

Answer

Gradients vanish to zero; the fix is a larger learning rate

Answer

The loss becomes negative; the fix is more data

Answer

Activations overflow; the fix is removing the loss function

Question 18

Why can saturated sigmoid units cause learning to slow during backpropagation?

Accepted Answer

Saturated sigmoids have near-zero derivative, so the backward signal nearly disappears

Answer

Saturated sigmoids have a derivative greater than 1, exploding the gradient

Answer

Saturation increases the learning rate automatically

Answer

Saturation deletes the neuron's weights

Question 19

Why is backpropagation called a 'reverse-mode' automatic differentiation method?

Accepted Answer

It traverses the graph from outputs back to inputs, efficient for many inputs and one scalar output

Answer

It differentiates forward from inputs to outputs one weight at a time

Answer

It uses finite differences to estimate every derivative

Answer

It only works for networks with a single layer

Question 20

How does backpropagation handle a weight that influences the loss through multiple paths?

Accepted Answer

The gradients from all paths are summed at the shared node

Answer

Only the gradient from the shortest path is used

Answer

The gradients from different paths are multiplied together

Answer

The node is ignored to avoid double counting

Question 21

What is the 'delta' (error term) for the output layer when using mean squared error?

Accepted Answer

δ = (a − y)·σ'(z)

Answer

δ = (a − y) only

Answer

δ = σ'(z) only

Answer

δ = (y − a)²

Question 22

Why is weight initialization important for successful backpropagation?

Accepted Answer

Poor initialization causes symmetry or vanishing/exploding gradients; good schemes keep gradient variance stable

Answer

Initialization has no effect because backprop fixes everything

Answer

Weights must always start at exactly zero

Answer

Initialization only affects the forward pass, never gradients

Question 23

If all weights are initialized to zero in a fully connected layer, what goes wrong during backpropagation?

Accepted Answer

All neurons get identical gradients and never differentiate (broken symmetry)

Answer

Training becomes faster than with random weights

Answer

The gradients explode immediately

Answer

Nothing; zero initialization is ideal

Question 24

How does the learning rate interact with the gradients produced by backpropagation?

Accepted Answer

It scales the step size taken along the negative gradient without changing the gradient itself

Answer

It changes the computed gradient values

Answer

It determines the number of layers

Answer

It replaces the loss function

Question 25

What is the difference between batch, mini-batch, and stochastic gradient descent in terms of how often backprop updates weights?

Accepted Answer

Batch updates once per full dataset, SGD once per example, mini-batch once per small group

Answer

All three update exactly once per epoch

Answer

SGD uses the whole dataset and batch uses one example

Answer

They differ only in the activation function used

Question 26

Why does the choice of activation function matter for the gradients in backpropagation?

Accepted Answer

Its derivative is a multiplicative factor in the backward pass, controlling gradient flow

Answer

Activations only affect the forward output, not gradients

Answer

Activation choice only changes memory usage

Answer

All activations have identical derivatives

Question 27

What is gradient checking and how does it validate a backpropagation implementation?

Accepted Answer

It compares analytic backprop gradients to finite-difference estimates to verify correctness

Answer

It speeds up training by skipping the backward pass

Answer

It replaces backprop with random gradients

Answer

It checks whether the dataset is balanced

Question 28

Why is computing all gradients with backpropagation far cheaper than estimating them with finite differences?

Accepted Answer

Backprop gets all gradients in one backward pass, while finite differences need a pass per parameter

Answer

Finite differences are exact and backprop is only approximate

Answer

Both methods cost the same

Answer

Backprop needs a forward pass per weight too

Question 29

In a multi-layer network, how is the error signal (delta) at a hidden layer computed from the next layer?

Accepted Answer

δ_l = (W_{l+1}ᵀ · δ_{l+1}) ⊙ f'(z_l)

Answer

δ_l = W_{l+1} · δ_{l+1} + b

Answer

δ_l = f'(z_l) only

Answer

δ_l = δ_{l+1} unchanged

Question 30

Once the delta of a layer is known, how is the gradient for that layer's weights computed?

Accepted Answer

∂L/∂W_l = δ_l · a_{l-1}ᵀ (and bias gradient = δ_l)

Answer

∂L/∂W_l = δ_l · W_l

Answer

∂L/∂W_l = a_{l-1} only

Answer

∂L/∂W_l = δ_l²

Test yourself