How LLMs Work quiz

Question 1

What is a large language model (LLM)?

Accepted Answer

An AI trained on huge amounts of text to understand and generate human language

Answer

A model that only processes images

Answer

A large physical robot that speaks

Answer

A very fast database system

Question 2

How does an LLM predict the next word?

Accepted Answer

It assigns probabilities to each possible next token and picks based on those probabilities

Answer

It always picks the most common word in the English language

Answer

It copies responses directly from its training data

Answer

It asks the user for help when it is unsure

Question 3

What is the 'transformer' architecture in AI?

Accepted Answer

A neural network architecture using attention to relate all words in a sequence

Answer

A type of AI robot that transforms its shape

Answer

A tool that converts images to text

Answer

A compression technique for shrinking AI models

Question 4

What does 'pre-training' mean for an LLM?

Accepted Answer

The phase where LLM reads massive text data and learns to predict next tokens

Answer

The final step before an LLM is released

Answer

The process of removing harmful content from the model

Answer

Testing the model on real user questions

Question 5

What is 'RLHF' (Reinforcement Learning from Human Feedback)?

Accepted Answer

Training an LLM to match human preferences by learning from human ratings of responses

Answer

Training an LLM using rewards from a video game

Answer

A technique for speeding up LLM inference

Answer

Using RL to make LLMs respond faster

Question 6

What is 'embedding' in the context of LLMs?

Accepted Answer

A list of numbers that represents the meaning of text mathematically

Answer

A technique for compressing video files

Answer

The process of inserting data into a database

Answer

A type of fixed font used in AI interfaces

Question 7

What is the difference between GPT-3 and GPT-4?

Accepted Answer

GPT-4 is more capable — better reasoning, multimodal, fewer errors — though more expensive

Answer

GPT-4 is identical to GPT-3 with a different name

Answer

GPT-3 is more powerful because it was trained longer

Answer

GPT-4 only works with voice, not text

Question 8

What is 'hallucination' and why do LLMs do it?

Accepted Answer

When an LLM generates confident-sounding but false information because it predicts plausible text

Answer

When an LLM refuses to answer a question

Answer

When an LLM's screen flickers during use

Answer

When an LLM runs too slowly to respond

Question 9

What is 'inference' in an LLM?

Accepted Answer

Using a trained model to generate outputs for real user inputs

Answer

The process of training the model on new data

Answer

A technique for checking model accuracy

Answer

Adding new parameters to an existing model

Question 10

Why do LLMs need so much computing power to train?

Accepted Answer

LLMs require massive parallel GPU/TPU computation because of billions of parameters and trillions of tokens

Answer

LLMs need special microphones for voice training

Answer

LLMs only need a normal laptop to train

Answer

LLMs need lots of power because they generate heat

Question 11

What is 'in-context learning' in LLMs?

Accepted Answer

Adapting an LLM to a new task by showing examples in the prompt without changing its weights

Answer

Teaching an LLM using examples during training

Answer

A type of coding done inside the model

Answer

In-context learning is only for very small models

Question 12

What is 'chain-of-thought' reasoning in LLMs?

Accepted Answer

A technique where the LLM writes out reasoning steps before giving a final answer

Answer

A type of LLM that chains multiple models together

Answer

A training method that uses chains of examples

Answer

A hardware technique for linking GPU clusters

Question 13

What is 'system prompt' vs 'user prompt' in an LLM API?

Accepted Answer

System prompts are developer-set hidden instructions; user prompts are what end users type

Answer

System prompts are visible; user prompts are hidden

Answer

They are both the same thing

Answer

System prompts are always in code; user prompts are in plain English

Question 14

What is 'sparse' vs 'dense' model architecture in LLMs?

Accepted Answer

In sparse models, only a subset of the network activates per input, making them more efficient at scale

Answer

Dense models have fewer layers; sparse models have more

Answer

Sparse models are always smaller than dense models

Answer

Dense and sparse refer only to image models, not LLMs

Question 15

What is 'tokenization' and why does it affect LLM output quality?

Accepted Answer

Splitting text into tokens for LLM processing, which affects how well it handles certain words and numbers

Answer

The process of adding security tokens to model queries

Answer

A method of reducing model training cost

Answer

A step in the model deployment pipeline

Question 16

What is 'RAG' (Retrieval-Augmented Generation) and why is it useful for LLMs?

Accepted Answer

A method that retrieves relevant documents before LLM generation to improve accuracy and reduce hallucinations

Answer

A technique for making LLMs generate text faster

Answer

A type of grammar rule followed by LLMs

Answer

A training technique using random augmented data

Question 17

What is 'prompt injection' and why is it a security concern for LLMs?

Accepted Answer

Malicious content that tries to override an LLM's original instructions

Answer

Adding medical injections to make the model faster

Answer

Inserting extra text into the prompt to make it longer

Answer

A technique developers use to protect the system prompt

Question 18

What are 'guardrails' in an LLM?

Accepted Answer

Safety mechanisms that prevent an LLM from generating harmful outputs

Answer

The physical casing around an AI server

Answer

A technique for improving LLM speed

Answer

The minimum computing requirements to run an LLM

Question 19

What is the difference between an LLM and a search engine?

Accepted Answer

Search engines retrieve existing pages; LLMs generate new text — they serve different purposes

Answer

They are exactly the same thing

Answer

Search engines are better at everything

Answer

LLMs are just faster search engines

Question 20

What is 'Anthropic' and why was it founded?

Accepted Answer

An AI safety company that created Claude and focuses on building safe AI systems

Answer

A European AI regulation agency

Answer

A hardware company making AI chips

Answer

An open-source AI research collective

Question 21

What is 'instruction following' in LLMs?

Accepted Answer

The ability to reliably do what the user asks — formatting, language, length — trained via fine-tuning

Answer

The ability to follow physical instructions in a robotic environment

Answer

A feature that lets the model read instruction manuals

Answer

An automatic summarization feature of LLMs

Question 22

What is 'model quantization'?

Accepted Answer

Compressing model weights to lower precision numbers to reduce size and speed up inference

Answer

Adding more layers to a model to improve accuracy

Answer

Converting the model to a quantum computing format

Answer

Splitting a large model into smaller parallel versions

Question 23

What is 'LLM benchmarking' and what are common benchmarks?

Accepted Answer

Testing LLMs on standardized tasks to compare performance across models

Answer

Measuring the physical speed of the computers running the LLM

Answer

Counting how many users a model can serve simultaneously

Answer

Checking the model's code for security vulnerabilities

Question 24

What is the 'attention head' in a transformer model?

Accepted Answer

Parallel attention mechanisms in each transformer layer, each capturing different types of relationships

Answer

The header section of an LLM's API documentation

Answer

A single neuron at the top of the attention layer

Answer

A monitoring tool for LLM performance

Question 25

What does 'greedy decoding' mean in LLM generation?

Accepted Answer

Always picking the most likely next token — fast but can produce repetitive text

Answer

Downloading an LLM model as quickly as possible

Answer

A technique for compressing LLM outputs

Answer

Generating text in batches for maximum efficiency

Question 26

What is 'model distillation'?

Accepted Answer

Training a small student model to mimic the outputs of a large teacher model

Answer

Refining a model by removing low-quality training data

Answer

A process of boiling down multiple models into one dataset

Answer

Removing parameters from a model to make it smaller

Test yourself