Prompt engineering

Chain-of-thought prompting instructs the model to generate intermediate reasoning steps before the final answer (e.g., 'Let's think step by step'). This works because it allocates more sequential compute/tokens to the problem and lets the model condition each step on prior explicit steps, reducing errors on arithmetic, logic, and multi-hop tasks. It is most beneficial for larger models and harder problems; for trivial tasks it adds latency without gains.

Zero-shot CoT elicits reasoning with a single trigger phrase (e.g., 'Let's think step by step') and no worked examples, making it cheap and easy to deploy. Few-shot CoT provides several exemplars that demonstrate the desired reasoning format and answer style, yielding higher accuracy and better format control at the cost of more tokens. Prefer zero-shot for quick prototyping or token budgets; prefer few-shot when you need consistent reasoning structure or domain-specific formats.

Self-consistency samples multiple independent CoT reasoning paths (with temperature > 0) for the same prompt, then takes a majority vote over the final answers rather than relying on a single greedy decode. It exploits the fact that correct answers tend to be reachable via diverse reasoning paths while errors are more idiosyncratic. It markedly improves accuracy on reasoning benchmarks but multiplies inference cost by the number of samples.

This is 'shortcut learning' or label/format copying: the model latches onto superficial patterns (label distribution, answer position, recency) instead of the input-output mapping. Mitigations include balancing and shuffling label order, diversifying exemplar content, using clear delimiters, adding an explicit instruction describing the task rather than relying on examples alone, and verifying with held-out probes. Calibration techniques (e.g., contextual calibration) can also offset majority-label bias.

The system prompt sets persistent role, behavior, constraints, and policy for the whole conversation, while user prompts carry the specific turn-by-turn requests. Models are trained to weight system instructions as higher-priority, durable context, so placing safety rules, persona, and output format there improves adherence and resistance to override. Mixing durable policy into user turns makes behavior less stable and more susceptible to later user-driven changes.

ReAct interleaves Reasoning traces ('Thought') with Actions (tool/API calls) and Observations (tool results) in a loop, so the model can plan, act, observe, and revise. It addresses the limitation of pure CoT that reasons without grounding by letting the model fetch external information and react to it, reducing hallucination and enabling multi-step tool use. It is the conceptual basis for many agent frameworks.

Thought: I need the current price.
Action: get_price("NIFTY")
Observation: 23150
Thought: Now compute the change.
Answer: ...

LLMs exhibit a U-shaped attention/recall bias: information at the very beginning and end of a long context is recalled more reliably than content buried in the middle, the 'lost in the middle' effect. Therefore, critical instructions, the question, and the most relevant retrieved passages should be placed at the start or end. For RAG, re-ranking so the most relevant chunks sit at the edges improves answer quality.

Prompt injection is an attack where untrusted input (e.g., a web page or document the model ingests) contains instructions that hijack the model's behavior, often to exfiltrate data or override the developer's instructions. Jailbreaking specifically aims to bypass safety/policy guardrails to elicit disallowed content. Injection is broader and especially dangerous in tool-using/RAG systems where external content is treated as instructions; defenses include input/output isolation, instruction hierarchy, and not granting the model authority over untrusted text.

(1) Treat retrieved/external content as data, not instructions, and clearly delimit/escape it so it cannot be confused with developer directives. (2) Enforce an instruction hierarchy and least-privilege tool access so untrusted text cannot trigger sensitive actions without confirmation. (3) Add output filtering, allowlists, and human-in-the-loop for high-risk actions, plus sandboxing and provenance tracking. Spotlighting/marking untrusted spans and using a separate model to screen content also help.

Temperature scales the logits before softmax, controlling overall randomness, while top-p restricts sampling to the smallest set of tokens whose cumulative probability exceeds p, truncating the long tail. For deterministic extraction/classification you want low randomness: set temperature near 0 (and optionally top-p low); for creative generation use higher temperature/top-p. Tuning both simultaneously is usually unnecessary—pick one primary knob.

Structured output prompting constrains the model to emit a machine-parseable format such as JSON matching a schema. Reliability improves by providing an explicit schema/example, instructing 'respond with JSON only,' and—most robustly—using constrained/grammar-based decoding or the provider's native structured-output/tool-calling mode that guarantees valid JSON. Post-validation with retries and a parser (e.g., Pydantic) catches residual errors.

{
  "sentiment": "positive",
  "confidence": 0.92
}

A delimiter is an explicit marker (e.g., triple backticks, XML-style tags, or unique tokens) that separates distinct sections of a prompt such as instructions, user data, and examples. It reduces ambiguity by telling the model where each segment begins and ends, improving parsing of the intended structure. It also mitigates injection by clearly bounding untrusted content so embedded instructions are treated as data within that block.

<user_input>
{{untrusted text}}
</user_input>

In-context learning conditions a frozen model via examples in the prompt at inference time—fast to iterate, no training, but limited by context window and per-call token cost. Fine-tuning updates model weights on a dataset, giving cheaper/shorter prompts, stronger task specialization, and better consistency, at the cost of data preparation, training infrastructure, and reduced flexibility. Choose ICL for prototyping, few examples, or rapidly changing tasks; fine-tune for high volume, stable tasks needing consistent behavior or shorter prompts.

Instruct the model to answer only from provided context or to say 'I don't know' when uncertain, discourage fabrication explicitly, and ask it to cite or quote supporting text. Adding step-by-step verification, requesting confidence, lowering temperature, and asking the model to first list what it does and doesn't know all help. However, prompting alone cannot fully eliminate hallucination—grounding via retrieval and verification is more robust.

Least-to-most prompting first decomposes a complex problem into an ordered list of simpler subproblems, then solves them sequentially, feeding earlier solutions into later ones. Unlike a single CoT pass that reasons in one continuous trace, it explicitly structures decomposition and solution into separate stages, improving generalization to harder problems than those seen in exemplars. It is especially helpful for compositional tasks.

The model decides whether and how to call a tool based almost entirely on the natural-language tool name, description, and the JSON parameter schema with field descriptions—these are injected into the model's context and effectively act as prompts. Clear, unambiguous descriptions, well-named parameters, and explicit constraints/examples dramatically improve correct tool selection and argument filling. Poorly described tools cause wrong calls, missing arguments, or unnecessary invocations.

You ask the model to reason first and then emit the final answer in a fixed, easily-parsed format (often on a final line or in a dedicated field/tag). Separation lets you capture the accuracy benefits of reasoning while keeping the extractable answer clean and machine-parseable. It also avoids the regression seen when forcing strict JSON around reasoning, which can suppress the reasoning itself.

Reasoning: ...
Final Answer: B

Prompt caching stores the processed key/value state of a stable prompt prefix so repeated requests reuse it, cutting input-token cost (often substantially) and time-to-first-token. To maximize hits, keep the large, reusable content—system instructions, schemas, few-shot examples, long documents—at the start as an unchanging prefix, and place variable user content after the cached boundary. Frequently changing the prefix invalidates the cache and negates savings.

Meta-prompting uses an LLM to generate, critique, or optimize prompts—e.g., expanding a terse user query into a fuller instruction, or having a model produce candidate prompts that are then evaluated. In production it powers automatic prompt optimization, query rewriting for retrieval, and self-refinement loops where one model proposes and another (or the same) critiques. It scales prompt iteration but requires evaluation harnesses to avoid drift and overfitting to a metric.

Persona framing can steer style, vocabulary, and the slice of distribution the model draws on, sometimes improving tone, domain register, and relevance. However, evidence shows it rarely improves factual accuracy on objective tasks and can even hurt; benefits are mostly stylistic and task-dependent. It should not be relied on as a correctness mechanism—clear instructions, constraints, and examples matter more.

Truncation is a hard limit: tokens beyond the context window are silently dropped or rejected, so check token counts against the model's max. Instruction-following degradation is softer: even within the window, models attend unevenly, dilute instructions among large context, and suffer 'lost in the middle', so adherence drops as prompts grow. Diagnose truncation by counting tokens and inspecting what fits; diagnose degradation via controlled tests that vary context length while holding the instruction fixed.

Self-refine prompts the model to generate an answer, then critique its own output against criteria and produce an improved revision, optionally iterating. It can improve quality on tasks with checkable structure (code, writing). The key caveat: without an external signal (tests, tools, ground truth), LLM self-critique is unreliable and may not detect its own reasoning errors—self-correction often fails or even degrades answers on pure reasoning unless grounded by external feedback.

Provide a clear, mutually exclusive label definition list, instruct the model to output exactly one label from the set (and a fallback like 'other/uncertain'), and constrain output to the label token only. Randomize example/label order across exemplars to avoid recency/majority bias, balance class representation in few-shots, and consider calibration (e.g., contextual/contextual-free calibration) to correct prior bias. Validate against a held-out confusion matrix.

Negative instructions tell the model what not to do (e.g., 'do not include personal opinions'). They can backfire because mentioning a concept can prime the model toward it, and long lists of prohibitions are harder to follow than positive specifications of desired behavior. Best practice is to state what to do affirmatively, keep constraints minimal and concrete, and reinforce critical guardrails in the high-priority system prompt with validation downstream.

The template determines how the model uses retrieved evidence: it should clearly delimit the context, instruct the model to answer strictly from it, require citations/quotes, and define behavior when evidence is missing or conflicting (e.g., 'say you don't know'). Ordering matters—place the most relevant chunks at the edges to counter 'lost in the middle.' A weak template leads to ignored context, ungrounded answers, or injection from poisoned documents.

Greedy decoding picks the single highest-probability token at each step; beam search maintains k partial sequences and expands the most probable overall. Beam search optimizes sequence likelihood but for open-ended text tends to produce repetitive, generic, and degenerate output (high-likelihood is not high-quality), so sampling methods (temperature/top-p) are preferred. Beam search remains useful for constrained tasks like translation where a single high-likelihood output is desirable.

Select exemplars that are diverse and representative of hard/edge cases rather than only easy, typical ones, since the model generalizes from the demonstrated distribution. Include at least one example of the tricky behavior (e.g., the 'uncertain/none' output) so the model knows it is allowed. Keep exemplars consistent in format, balanced across classes, and ordered to avoid recency bias; consider dynamic/k-NN exemplar selection per query for very heterogeneous inputs.

Few-shot LLMs have biases toward certain answers due to the prompt format, exemplar order, and label frequency (majority-label and recency bias). Contextual calibration estimates this bias by feeding a content-free input (e.g., 'N/A') and measuring the model's default class probabilities, then applies an affine correction to the output scores to neutralize the prior. This improves accuracy and stability without changing the model.

For models with built-in extended reasoning, explicit 'let's think step by step' and elaborate CoT scaffolding are often redundant and can even hurt by constraining the model's own process. Prompts should be clear and goal-stating, specify the desired output format, and avoid forcing a particular reasoning style; you control depth via reasoning-effort settings rather than prompt tricks. Few-shot with heavy reasoning traces may also be less necessary.

An instruction describes the task abstractly ('classify sentiment as positive/negative/neutral'); a demonstration shows concrete input-output examples. Instructions scale well and avoid token cost but can be ambiguous; demonstrations disambiguate format and edge behavior but consume context and risk format-copying. As complexity grows, combining a precise instruction with a few well-chosen demonstrations generalizes best—neither alone is optimal for nuanced tasks.

Models cannot count tokens internally and only loosely follow exact word/character limits, so soft guidance ('in 2-3 sentences', 'about 100 words') works better than rigid counts, and you should enforce hard limits via the API's max_tokens plus post-processing/truncation. Specifying structure (bullets, sentence count, sections) controls length more reliably than numeric targets. For strict limits, combine a length hint with max_tokens and validate.

Prompt chaining splits a task into a sequence of smaller prompts/LLM calls where each step's output feeds the next (e.g., extract -> summarize -> format). It is preferable when a single prompt would be overloaded, hard to debug, or error-prone, because each step is simpler, independently testable, and individually optimizable. Trade-offs are higher latency, more API calls/cost, and error propagation between steps, so use it when reliability and modularity outweigh overhead.

Prompt changes have non-obvious, sometimes regressive effects, so you need a fixed evaluation set with quantitative metrics to compare versions objectively and avoid anecdote-driven tuning and overfitting to a few examples. It should measure task accuracy/quality, format validity, robustness across input variations, cost/latency, and failure modes (e.g., hallucination rate). Automated graders (rules, LLM-as-judge with rubrics) plus a held-out set enable safe, reproducible iteration.

LLM-as-a-judge uses a strong model to score or compare outputs against a rubric, enabling scalable, automatic evaluation of open-ended generations. Known biases include position bias (favoring the first or second option), verbosity/length bias, self-preference (favoring its own or same-family outputs), and sensitivity to formatting. Mitigations: randomize and swap positions and average, use clear rubrics with reference answers, force pairwise plus tie options, and calibrate or validate against human labels.