Prompt engineering quiz

Question 1

What is the core idea behind chain-of-thought (CoT) prompting, and why does it improve performance on multi-step reasoning tasks?

Accepted Answer

It generates intermediate reasoning steps, allocating more compute and conditioning each step on prior ones

Answer

It forces the model to output JSON so answers are parseable

Answer

It fine-tunes the model weights on reasoning data at inference time

Answer

It retrieves external documents to ground the answer

Question 2

How does zero-shot CoT differ from few-shot CoT, and when would you prefer each?

Accepted Answer

Zero-shot uses a trigger phrase with no exemplars; few-shot supplies worked reasoning examples for higher accuracy and format control

Answer

Zero-shot CoT requires fine-tuning while few-shot does not

Answer

Few-shot CoT only works with retrieval augmentation

Answer

They are identical except few-shot uses a lower temperature

Question 3

What is self-consistency decoding and how does it relate to chain-of-thought?

Accepted Answer

It samples multiple reasoning paths and majority-votes the final answers

Answer

It greedily decodes one path and trims low-probability tokens

Answer

It averages the logits of multiple models

Answer

It re-ranks answers using an external reward model only

Question 4

Describe the few-shot prompting failure mode where the model copies the format of exemplars but ignores their semantics. How do you mitigate it?

Accepted Answer

Balance/shuffle labels, diversify exemplars, add an explicit task instruction, and calibrate for label bias

Answer

Increase temperature to 2.0 to break the pattern

Answer

Remove all delimiters so the model reads continuously

Answer

Always place the correct answer last in every exemplar

Question 5

What is the difference between a system prompt and a user prompt in chat-style LLM APIs, and why does the distinction matter?

Accepted Answer

System prompts set durable, higher-priority behavior/constraints; user prompts carry per-turn requests

Answer

They are interchangeable; the API concatenates them identically

Answer

User prompts always override system prompts by design

Answer

System prompts are only used for token counting

Question 6

Explain the ReAct prompting pattern and what problem it addresses.

Accepted Answer

It interleaves Thought, Action, and Observation steps to combine reasoning with grounded tool use

Answer

It compresses prompts by removing reasoning tokens

Answer

It fine-tunes the model after each action

Answer

It runs multiple models in parallel and votes

Question 7

Why does the position of key instructions within a long-context prompt affect model performance, and what is the 'lost in the middle' effect?

Accepted Answer

Recall is U-shaped: start and end are favored, middle content is more often missed

Answer

Models read context strictly left-to-right with uniform recall

Answer

Only the middle of the context is attended to

Answer

Position has no measurable effect on modern models

Question 8

What is prompt injection, and how does it differ from jailbreaking?

Accepted Answer

Injection hijacks behavior via untrusted ingested content; jailbreaking specifically bypasses safety guardrails

Answer

They are the same thing with different names

Answer

Jailbreaking only affects open-source models

Answer

Injection requires API key theft

Question 9

Name three concrete defenses against indirect prompt injection in a RAG or agent system.

Accepted Answer

Delimit external content as data, enforce instruction hierarchy/least-privilege tools, and add output filtering with human-in-the-loop

Answer

Raise temperature, add more exemplars, and increase max tokens

Answer

Disable the system prompt and trust user input

Answer

Store secrets in the prompt so the model can validate them

Question 10

How do temperature and top-p (nucleus) sampling differ, and how should they be set for deterministic extraction tasks?

Accepted Answer

Temperature scales logit randomness; top-p truncates to a cumulative-probability nucleus; use temperature near 0 for deterministic extraction

Answer

Both must be set high for extraction tasks

Answer

Top-p increases randomness while temperature reduces it

Answer

They are aliases for the same parameter

Question 11

What is structured output prompting and what techniques improve schema adherence (e.g., valid JSON)?

Accepted Answer

Provide a schema/example, use constrained decoding or native structured-output mode, and validate with retries

Answer

Ask for JSON and hope; no tooling exists to enforce it

Answer

Always parse free-form prose with regex

Answer

Set temperature to 1.5 to encourage formatting

Question 12

Explain what a 'delimiter' is in prompt design and why it reduces ambiguity and injection risk.

Accepted Answer

An explicit marker separating prompt sections, reducing ambiguity and bounding untrusted content

Answer

A delimiter is a stop sequence that ends generation

Answer

A token that increases temperature locally

Answer

A required field in every API call

Question 13

What is the difference between few-shot in-context learning and fine-tuning, and what trade-offs guide the choice?

Accepted Answer

ICL conditions a frozen model via prompt examples; fine-tuning updates weights for cheaper prompts and stronger specialization

Answer

ICL updates weights; fine-tuning only edits prompts

Answer

They produce identical cost and latency

Answer

Fine-tuning is always preferable regardless of volume

Question 14

How can you reduce hallucination through prompting alone (no retrieval)?

Accepted Answer

Instruct it to answer only from context or say 'I don't know', request citations, and lower temperature

Answer

Raise temperature so the model explores more facts

Answer

Ask longer questions to confuse the model less

Answer

Disable the system prompt

Question 15

What is 'least-to-most' prompting and how does it differ from standard chain-of-thought?

Accepted Answer

It decomposes a problem into ordered subproblems and solves them sequentially, feeding results forward

Answer

It is identical to zero-shot CoT

Answer

It only works on arithmetic

Answer

It removes reasoning steps to save tokens

Question 16

In function/tool calling, why are the tool name, description, and parameter schema considered part of prompt engineering?

Accepted Answer

The tool name, description, and parameter schema are part of the model's context and drive correct selection/argument filling

Answer

Tool metadata is hidden from the model and only used by the runtime

Answer

Only the tool's return value matters for prompting

Answer

Tool descriptions are ignored unless fine-tuned

Question 17

What is the role of an 'output format' specification combined with a 'reasoning then answer' separation, and why separate them?

Accepted Answer

Reason first, then emit a fixed parseable answer—gaining reasoning benefits while keeping output extractable

Answer

Reasoning and answer must always be merged into one JSON field

Answer

Never let the model reason before answering

Answer

The format tag increases hallucination

Question 18

How does prompt caching (e.g., caching a long shared prefix) affect cost and latency, and what design changes maximize its benefit?

Accepted Answer

Cache the stable prefix (system, schema, examples, docs) first and put variable content after, reducing input cost and latency

Answer

Caching stores outputs only and ignores the prefix

Answer

Caching requires putting variable content first

Answer

Caching increases per-token cost

Question 19

What is 'meta-prompting' or 'prompt rewriting', and how is it used in production pipelines?

Accepted Answer

Using an LLM to generate/critique/optimize prompts, e.g., query rewriting and self-refinement, gated by evaluation

Answer

Manually typing prompts only

Answer

A caching strategy for prompts

Answer

A way to encrypt prompts

Question 20

Why might adding 'You are an expert...' role/persona framing help, and what are its limitations?

Accepted Answer

It mainly steers style/register and is task-dependent, with little reliable gain on objective accuracy

Answer

It guarantees factually correct answers

Answer

It replaces the need for instructions

Answer

It only works for coding tasks

Question 21

What is the difference between 'instruction-following' degradation with very long prompts and context-window truncation, and how do you diagnose each?

Accepted Answer

Truncation drops tokens past the window; degradation is softer adherence loss within the window—diagnose via token counts vs. length-controlled tests

Answer

Both are the same hard limit

Answer

Degradation only happens below 1k tokens

Answer

Truncation improves instruction following

Question 22

Explain 'self-refine' / 'reflection' prompting and a key caveat about its reliability.

Accepted Answer

Generate, critique, and revise—useful with external signals, but unreliable on pure reasoning without grounding

Answer

Self-critique is always reliable and improves all tasks

Answer

It requires retraining the model each iteration

Answer

It only works at temperature 0

Question 23

How do you write a robust classification prompt that avoids label leakage and order bias across many classes?

Accepted Answer

Define mutually exclusive labels, constrain output to one label, randomize/balance exemplars, and calibrate for prior bias

Answer

List labels once at the top and always order examples by class

Answer

Ask the model to invent new labels as needed

Answer

Use temperature 1.5 to spread predictions

Question 24

What is the purpose of 'negative' or 'guardrail' instructions, and why can they backfire?

Accepted Answer

They specify prohibitions but can prime the forbidden concept; prefer affirmative, concrete instructions with downstream validation

Answer

Negative instructions are always more effective than positive ones

Answer

They are ignored entirely by chat models

Answer

They only matter at high temperature

Question 25

In RAG, why is the prompt template around retrieved chunks critical, and what should it contain?

Accepted Answer

It should delimit context, require answering only from it with citations, handle missing/conflicting evidence, and order key chunks at the edges

Answer

The template is irrelevant once chunks are retrieved

Answer

It should hide the question from the model

Answer

It should always place chunks in the middle of the prompt

Question 26

What is the difference between greedy decoding and beam search for LLM generation, and why is beam search rarely used for open-ended generation?

Accepted Answer

Greedy takes the top token each step; beam keeps k sequences but yields repetitive/generic open-ended text, so sampling is preferred

Answer

Beam search adds randomness; greedy is stochastic

Answer

Beam search is always best for creative writing

Answer

They produce identical outputs

Question 27

How should you design few-shot exemplars when the task has a long-tail of edge cases?

Accepted Answer

Choose diverse exemplars including edge cases and the 'none/uncertain' output, balanced and consistently formatted, optionally selected per query

Answer

Use only the easiest, most typical examples

Answer

Use as many examples as fit regardless of relevance

Answer

Never demonstrate the fallback output

Question 28

What is 'contextual calibration' and what problem does it solve in few-shot prompting?

Accepted Answer

A method that probes the model's default bias with a content-free input and corrects output scores to neutralize prompt-induced prior bias

Answer

A way to set temperature automatically

Answer

A retrieval re-ranking algorithm

Answer

A fine-tuning loss function

Question 29

When prompting reasoning-optimized models that already 'think' internally, how should prompting strategy change versus standard models?

Accepted Answer

Keep prompts clear/goal-stating, specify output format, and avoid forcing reasoning style—control depth via reasoning settings

Answer

Add even more 'think step by step' scaffolding than usual

Answer

Always switch to zero-shot only

Answer

Use temperature 2.0 to trigger reasoning

Question 30

What is the difference between 'instruction' and 'demonstration' in a prompt, and which generalizes better as task complexity grows?

Accepted Answer

Instructions describe the task abstractly, demonstrations show examples; combining both generalizes best for complex tasks

Answer

Demonstrations alone always beat instructions

Answer

Instructions and demonstrations are the same

Answer

Instructions only work after fine-tuning

Test yourself