Fine-tuning and LoRA quiz

Question 1

What is the core mathematical idea behind LoRA (Low-Rank Adaptation)?

Accepted Answer

It learns a low-rank update ΔW=B·A added to frozen W0, training only A and B

Answer

It quantizes all weights to 4-bit and fine-tunes them directly

Answer

It prunes redundant weights then retrains the survivors

Answer

It distills the model into a smaller dense network

Question 2

In LoRA, how are matrices A and B typically initialized and why?

Accepted Answer

A random, B zeros, so ΔW starts at zero

Answer

Both A and B random Gaussian

Answer

Both A and B zeros

Answer

A zeros, B random Gaussian

Question 3

What role does the LoRA scaling factor alpha play, and how does it interact with rank r?

Accepted Answer

It scales the adapter output by α/r, decoupling update magnitude from rank

Answer

It sets the dropout rate inside the adapter

Answer

It controls the quantization bit-width

Answer

It is the L2 regularization coefficient on B·A

Question 4

What is QLoRA and what are its three key technical contributions?

Accepted Answer

NF4 quantization, double quantization, and paged optimizers

Answer

8-bit weights, gradient clipping, and FlashAttention

Answer

Weight pruning, knowledge distillation, and mixed precision

Answer

Rotary embeddings, RMSNorm, and SwiGLU

Question 5

In QLoRA, are the LoRA adapter weights stored in 4-bit like the base model?

Accepted Answer

No, adapters are trained in bf16/fp16 while only the frozen base is 4-bit

Answer

Yes, adapters are also stored and updated in 4-bit

Answer

Yes, both base and adapters are int8

Answer

No, adapters are stored in 1-bit binary form

Question 6

Why does LoRA add zero inference latency when merged, and what is the trade-off of merging?

Accepted Answer

ΔW can be folded into W0 yielding one matrix of identical shape; merging prevents adapter hot-swapping

Answer

It caches activations so no recomputation is needed

Answer

It uses a smaller vocabulary at inference

Answer

It skips the attention layers entirely

Question 7

Which Transformer modules are most commonly targeted by LoRA, and what does recent practice suggest?

Accepted Answer

Originally q/v attention; best practice now targets all linear layers including MLP

Answer

Only the embedding and LM head

Answer

Only LayerNorm parameters

Answer

Only the final output projection

Question 8

What problem does DoRA (Weight-Decomposed Low-Rank Adaptation) solve over vanilla LoRA?

Accepted Answer

It decomposes weights into magnitude and direction, applying LoRA to direction only

Answer

It adds 8-bit quantization to LoRA

Answer

It replaces matrix A with a convolution

Answer

It removes the scaling factor alpha

Question 9

How does the choice of rank r affect LoRA capacity and overfitting risk?

Accepted Answer

Higher r adds capacity but increases overfitting/memory; small r often suffices

Answer

Higher r always strictly improves test accuracy

Answer

r has no effect on parameter count

Answer

Lower r increases memory usage

Question 10

What is catastrophic forgetting in fine-tuning, and how does LoRA mitigate it?

Accepted Answer

Loss of prior capabilities; LoRA mitigates it by freezing W0 and learning a small additive update

Answer

A GPU memory error during training

Answer

When the optimizer diverges to NaN

Answer

Vocabulary mismatch between tokenizers

Question 11

Why is LoRA dropout applied, and where in the adapter is it placed?

Accepted Answer

Applied to the adapter input before A to regularize the low-rank path

Answer

Applied to the frozen base weights

Answer

Applied to the final logits only

Answer

Applied to the attention scores

Question 12

What is the memory advantage of LoRA/QLoRA specifically with respect to optimizer states?

Accepted Answer

Optimizer states (Adam moments) scale with tiny adapter size, not the full model

Answer

It eliminates the need for gradients entirely

Answer

It compresses activations to int4

Answer

It removes the need for a forward pass

Question 13

When should you prefer full fine-tuning over LoRA?

Accepted Answer

When data/compute are abundant and the needed weight change is high-rank or behavior-altering

Answer

Always, since full FT is universally cheaper

Answer

When you have only a few hundred examples

Answer

When you want zero-latency inference

Question 14

What is the difference between LoRA and prefix/prompt tuning as PEFT methods?

Accepted Answer

LoRA modifies weight matrices; prefix/prompt tuning prepends trainable soft-prompt vectors

Answer

They are identical techniques

Answer

LoRA only trains the tokenizer; prompt tuning trains all weights

Answer

Prompt tuning merges into weights; LoRA cannot

Question 15

What does 'intrinsic dimensionality' have to do with LoRA's motivation?

Accepted Answer

Fine-tuning updates lie in a low-dimensional subspace, so a low-rank ΔW suffices

Answer

It refers to the embedding dimension of the tokenizer

Answer

It is the number of attention heads

Answer

It measures dataset size requirements

Question 16

How do you serve many task-specific LoRA adapters efficiently from one base model?

Accepted Answer

Load base once and batch/hot-swap unmerged adapters (e.g., S-LoRA/Punica)

Answer

Merge all adapters into one giant model

Answer

Run a separate full model copy per task

Answer

Concatenate all adapters into the prompt

Question 17

Why might merging a LoRA adapter into a 4-bit quantized base degrade quality, and what is the fix?

Accepted Answer

Re-quantizing W0+ΔW adds rounding error; merge into a dequantized fp16 base instead

Answer

Merging changes the matrix shape, breaking the model

Answer

Adapters cannot mathematically be merged

Answer

The fix is to lower the rank to 1

Question 18

What is the parameter count of a LoRA adapter for a single d×k linear layer with rank r?

Accepted Answer

r·(d+k)

Answer

d·k

Answer

r·d·k

Answer

r²·(d+k)

Question 19

What is rsLoRA and what stability problem does it address?

Accepted Answer

It scales by α/√r so higher ranks remain stable and beneficial

Answer

It removes alpha and uses fixed scaling of 1

Answer

It applies LoRA only to odd-numbered layers

Answer

It quantizes the adapters to int4

Question 20

In instruction fine-tuning with LoRA, why is loss masking on the prompt/instruction tokens important?

Accepted Answer

Loss is computed only on response tokens so the model learns to answer, not echo prompts

Answer

Masking randomly drops 50% of all tokens

Answer

It masks attention to prevent lookahead

Answer

It is required to enable 4-bit quantization

Question 21

What is the typical learning rate regime difference between LoRA and full fine-tuning?

Accepted Answer

LoRA uses a higher LR (~1e-4–3e-4) than full FT (~1e-5–5e-5)

Answer

LoRA requires a much lower LR than full FT

Answer

They always use the same learning rate

Answer

LoRA cannot use a learning rate scheduler

Question 22

How does gradient checkpointing complement QLoRA, and what is its trade-off?

Accepted Answer

It trades extra recompute for lower activation memory, enabling larger models on small GPUs

Answer

It quantizes gradients to 4-bit

Answer

It removes the optimizer states

Answer

It speeds up training with no downside

Question 23

What is the difference between continued pretraining and supervised fine-tuning (SFT), and where does LoRA fit?

Accepted Answer

Continued pretraining injects knowledge via LM objective; SFT shapes behavior via labeled pairs; LoRA fits both but knowledge may need higher rank

Answer

They are the same thing

Answer

SFT uses unlabeled data; pretraining uses labeled data

Answer

LoRA can only be used for SFT, never pretraining

Question 24

Why is bf16 generally preferred over fp16 as the compute dtype in QLoRA/LoRA training?

Accepted Answer

bf16's wider exponent range avoids fp16 overflow/underflow and loss-scaling issues

Answer

fp16 has a wider range than bf16

Answer

bf16 uses fewer total bits than fp16

Answer

They are numerically identical

Question 25

What is NF4 (NormalFloat4) and why is it well-suited for quantizing LLM weights?

Accepted Answer

A 4-bit type with levels at the quantiles of a normal distribution, optimal for normally-distributed weights

Answer

A 4-bit uniform integer format

Answer

A floating-point format with 4 exponent bits

Answer

A lossless compression codec

Question 26

What is double quantization in QLoRA and how much does it save?

Accepted Answer

Quantizing the per-block quantization constants themselves, saving ~0.4 bits/param

Answer

Quantizing the model twice to int2

Answer

Running two forward passes for accuracy

Answer

Doubling the number of adapters

Question 27

How does LoRA interact with RLHF/DPO-style alignment training?

Accepted Answer

LoRA trains the policy as base+adapter while the frozen base serves as the reference, avoiding two full copies

Answer

LoRA cannot be used with DPO or PPO

Answer

LoRA replaces the reward model

Answer

DPO requires merging adapters before each step

Question 28

What is a practical sign that your LoRA rank is too low for the task?

Accepted Answer

Training loss plateaus high (underfitting) and target behavior isn't learned

Answer

Validation loss is far below training loss

Answer

Training is faster than expected

Answer

GPU memory usage is too low

Question 29

What is AdaLoRA and how does it differ from fixed-rank LoRA?

Accepted Answer

It adaptively allocates rank across layers via SVD-form pruning of singular values

Answer

It uses a fixed rank of 1 everywhere

Answer

It quantizes adapters adaptively

Answer

It is LoRA applied only to attention

Question 30

Why can stacking/combining multiple independently-trained LoRA adapters cause interference, and what mitigates it?

Accepted Answer

Updates overlap in subspace causing interference; TIES/DARE merging or routing mitigates it

Answer

Adapters cannot be combined at all under any method

Answer

Combining always improves all tasks

Answer

Interference only occurs in fp32

Test yourself