Recurrent networks and sequences quiz

Question 1

What makes a Recurrent Neural Network (RNN) different from a standard feedforward neural network?

Accepted Answer

It feeds its hidden state back into itself across time steps, giving it memory

Answer

It has more layers than any feedforward network

Answer

It can only process images, not text

Answer

It never uses activation functions

Question 2

Why are RNNs well-suited for sequential data like text or time series?

Accepted Answer

Because they maintain a hidden state that carries context across the sequence

Answer

Because they require fixed-length inputs only

Answer

Because they ignore the order of inputs

Answer

Because they use convolutions over pixels

Question 3

What is the 'hidden state' in an RNN and what role does it play?

Accepted Answer

A vector summarizing past inputs, updated each step as the network's memory

Answer

A constant bias term that never changes

Answer

The final output label of the sequence

Answer

A list of all raw inputs stored unchanged

Question 4

Write the basic update equation for a vanilla RNN hidden state.

Accepted Answer

h_t = tanh(W_xh·x_t + W_hh·h_{t-1} + b)

Answer

h_t = W_xh·x_t only, ignoring the past

Answer

h_t = h_{t-1} - x_t

Answer

h_t = softmax(x_t) with no recurrence

Question 5

What is 'parameter sharing' in an RNN and why is it important?

Accepted Answer

The same weights are reused at every time step, enabling variable-length input

Answer

Each time step has unique weights learned separately

Answer

Parameters are shared with a separate CNN

Answer

Weights are frozen and never trained

Question 6

What is Backpropagation Through Time (BPTT)?

Accepted Answer

Unrolling the RNN over time and backpropagating gradients through all steps

Answer

Training only on the last time step and ignoring the rest

Answer

A method to skip gradient computation entirely

Answer

Forward propagation repeated many times

Question 7

What is the vanishing gradient problem in RNNs?

Accepted Answer

Gradients shrink exponentially over many steps, blocking long-range learning

Answer

Gradients become exactly zero only at the first step

Answer

The model has too many parameters to store

Answer

Inputs vanish from memory immediately every step

Question 8

What is the exploding gradient problem and one common way to handle it?

Accepted Answer

Gradients grow too large; gradient clipping caps their norm

Answer

Gradients shrink to zero; dropout fixes it

Answer

The loss is always negative; normalization fixes it

Answer

Inputs become too large; one-hot encoding fixes it

Question 9

What problem do LSTM networks solve compared to vanilla RNNs?

Accepted Answer

They preserve long-range dependencies that vanilla RNNs lose to vanishing gradients

Answer

They make the network smaller than an RNN

Answer

They remove the need for any training

Answer

They only work on images

Question 10

What are the three gates in an LSTM and what does each control?

Accepted Answer

Forget, input, and output gates

Answer

Read, write, and delete gates

Answer

Start, middle, and end gates

Answer

Encoder, decoder, and attention gates

Question 11

What is the role of the cell state in an LSTM?

Accepted Answer

A long-term memory channel that carries information across many steps

Answer

A temporary buffer cleared every step

Answer

The output label of the network

Answer

A random noise vector added for regularization

Question 12

How does a GRU differ from an LSTM?

Accepted Answer

GRU uses two gates and merges cell/hidden state, so it is simpler

Answer

GRU has more gates than an LSTM

Answer

GRU has no gates at all

Answer

GRU can only process numbers, not text

Question 13

What are the two gates in a GRU and what do they do?

Accepted Answer

Update gate and reset gate

Answer

Forget gate and output gate

Answer

Input gate and cell gate

Answer

Encoder gate and decoder gate

Question 14

What is a bidirectional RNN and when is it useful?

Accepted Answer

It processes the sequence in both directions to use past and future context

Answer

It runs the same RNN twice on the same direction

Answer

It only reads sequences backward

Answer

It is used only for real-time streaming prediction

Question 15

Explain the difference between a one-to-many, many-to-one, and many-to-many RNN architecture.

Accepted Answer

They describe how many inputs map to how many outputs (e.g., seq-to-one for sentiment)

Answer

They describe the number of hidden layers

Answer

They describe the learning rate schedule

Answer

They describe the batch size choices

Question 16

In sequence-to-sequence (seq2seq) models, what are the encoder and decoder responsible for?

Accepted Answer

Encoder compresses input into context; decoder generates the output sequence

Answer

Encoder generates output; decoder compresses input

Answer

Both only classify single tokens

Answer

Encoder and decoder do the same identical task

Question 17

What is the information bottleneck in a basic seq2seq model, and how does attention address it?

Accepted Answer

A single context vector limits long inputs; attention lets the decoder weigh all encoder states

Answer

Attention removes the decoder entirely

Answer

The bottleneck is too many gates; attention removes them

Answer

Attention makes the model ignore the input

Question 18

Why is the tanh activation commonly used in vanilla RNN cells rather than ReLU?

Accepted Answer

tanh keeps the recurrent hidden state bounded and centered, aiding stability

Answer

tanh is faster to compute than any other function

Answer

ReLU cannot be used in neural networks at all

Answer

tanh always gives higher accuracy on images

Question 19

What is 'teacher forcing' in training sequence models?

Accepted Answer

Feeding the true previous token as input during training instead of the prediction

Answer

Letting a human correct each output manually

Answer

Forcing the teacher to label all data

Answer

Skipping training on hard examples

Question 20

Why must variable-length sequences in a batch be padded, and what is a sequence mask used for?

Accepted Answer

Padding equalizes lengths; a mask tells the model to ignore padded positions

Answer

Padding adds noise on purpose to regularize

Answer

A mask hides the correct answer from the network forever

Answer

Padding shortens all sequences to length one

Question 21

What is word embedding and why is it preferred over one-hot encoding for RNN inputs?

Accepted Answer

Dense learned vectors capturing meaning, unlike sparse unrelated one-hot vectors

Answer

Random vectors that never change during training

Answer

A way to delete rare words

Answer

Identical to one-hot but slower

Question 22

In language modeling with an RNN, what does the network predict at each time step?

Accepted Answer

A probability distribution over the next possible token

Answer

The total length of the sentence

Answer

A single fixed number for the whole document

Answer

Whether the sentence is a question or not only

Question 23

What does 'unrolling' an RNN mean?

Accepted Answer

Expanding the recurrence into one cell copy per time step with shared weights

Answer

Removing all weights from the network

Answer

Running the network on a single input only

Answer

Converting the RNN into a CNN

Question 24

What is truncated backpropagation through time and why is it used?

Accepted Answer

Backpropagating over limited-length chunks to save memory and compute

Answer

Skipping backpropagation entirely for speed

Answer

Training only the first time step ever

Answer

Doubling the sequence length artificially

Question 25

How does stacking multiple RNN layers (a deep RNN) help?

Accepted Answer

Higher layers learn more abstract patterns from lower-layer outputs

Answer

It removes the need for any gates

Answer

It makes the sequence shorter automatically

Answer

It guarantees zero training error

Question 26

Why are Transformers often preferred over RNNs for long sequences today?

Accepted Answer

Self-attention is parallel and captures long-range dependencies better than recurrence

Answer

Transformers have no parameters to train

Answer

RNNs cannot process text at all

Answer

Transformers always use fewer than ten weights

Question 27

What is the purpose of the softmax function at the output of a language-model RNN?

Accepted Answer

It turns logits into a probability distribution over the vocabulary

Answer

It removes the recurrence from the network

Answer

It pads short sequences

Answer

It clips the gradient norm

Question 28

In sequence generation, what is the difference between greedy decoding and beam search?

Accepted Answer

Greedy takes the best token each step; beam search keeps top-k sequences

Answer

They are identical methods

Answer

Beam search always picks random tokens

Answer

Greedy decoding explores all possible sequences

Question 29

What does the forget gate output value being close to 1 versus 0 mean in an LSTM?

Accepted Answer

Near 1 keeps the memory; near 0 erases it

Answer

Near 1 erases the memory; near 0 keeps it

Answer

It always outputs exactly 0.5

Answer

It controls the learning rate

Question 30

Why does using the sigmoid function make sense for gate activations in LSTMs and GRUs?

Accepted Answer

Sigmoid maps to (0,1), acting as a soft on/off control for information flow

Answer

Sigmoid maps to (-1,1) which gates need

Answer

Sigmoid is the only function that exists

Answer

Sigmoid guarantees no overfitting

Test yourself