Agentic Systems

An LLM agent is a system where an LLM acts as a reasoning engine that autonomously decides which actions to take, uses tools, observes results, and iterates toward a goal — rather than just responding to a single query. A chatbot generates a response given a conversation; an agent generates a plan, executes steps (tool calls, code execution, web search), integrates observations into its context, and decides when the goal is achieved. Agents have memory, tools, and the ability to plan across multiple steps.

ReAct (Yao et al., 2022) interleaves chain-of-thought reasoning (Thought) with action execution (Action → Observation) in a loop. At each step: the LLM generates a Thought explaining its reasoning, then an Action (tool call with parameters), the environment returns an Observation, and the cycle repeats until a Final Answer. ReAct improves over pure chain-of-thought (no grounding) and pure action-taking (no reasoning) by synergizing both. It is the foundation for most modern agent loops in LangChain and similar frameworks.

# ReAct format:
# Thought: I need to search for the current price.
# Action: search("AAPL stock price")
# Observation: $182.50
# Thought: Now I have the price...
# Final Answer: ...

Function calling allows the LLM to indicate that it wants to invoke an external function by generating a structured JSON output specifying the function name and arguments, rather than producing plain text. The host application parses this JSON, executes the function, and returns the result as a tool message to the LLM. The LLM's tool schema is provided in the system prompt or API parameters. OpenAI GPT-4, Claude, and Gemini all support native function calling with tool_use / function_call API fields. The model is fine-tuned to produce valid JSON tool calls reliably.

tools = [{
    'name': 'get_weather',
    'description': 'Get current weather',
    'input_schema': {'location': {'type': 'string'}}
}]

MCP (Anthropic, 2024) is an open protocol that standardizes how LLM applications connect to external data sources and tools. It defines a client-server architecture: MCP servers expose tools, resources (files, databases), and prompts; MCP clients (e.g., Claude Desktop, IDEs) connect to these servers. MCP replaces the need for bespoke tool integrations for each AI app by providing a universal interface — analogous to USB-C for peripherals. Servers can expose filesystem access, database queries, API calls, and custom functions in a standardized way.

Agent memory types: (1) In-context (working) memory — information in the current context window; limited to max_tokens but immediately accessible. (2) External (episodic/semantic) memory — stored in vector databases or knowledge graphs; retrieved via similarity search; unlimited capacity but retrieval adds latency and may miss relevant context. (3) Parametric memory — knowledge encoded in model weights during pretraining/fine-tuning; cannot be updated without retraining. Production agents combine all three: context for active task state, retrieval for knowledge, and parametric for reasoning.

RAG (Lewis et al., 2020) augments LLM generation with retrieved documents from an external knowledge base. For each query, a retriever (typically dense embedding search via FAISS or a vector DB) finds the K most relevant documents; these are injected into the LLM's context as grounding information. RAG extends agents by providing up-to-date, domain-specific, or private information beyond training data. Agents use RAG tools to retrieve knowledge mid-task. Advanced RAG uses rerankers, query decomposition, and iterative retrieval for multi-hop questions.

# RAG pipeline:
docs = retriever.retrieve(query, k=5)
context = '\n'.join([d.text for d in docs])
response = llm(f'Context: {context}\nQuestion: {query}')

Multi-agent orchestration coordinates multiple specialized AI agents to collaborate on complex tasks. Main patterns: (1) Sequential — agents pass outputs as inputs in a pipeline (e.g., planner → coder → tester); (2) Hierarchical — an orchestrator agent decomposes tasks and delegates to subagents; (3) Debate — multiple agents reason independently and argue to converge on answers; (4) Parallel fan-out — orchestrator spawns multiple agents simultaneously and aggregates results. Frameworks: LangGraph, AutoGen, CrewAI, and Anthropic's Claude agent SDK implement these patterns.

LangGraph (LangChain) models agent workflows as directed graphs (or DAGs) where nodes are LLM calls or function executions and edges define transitions. State is passed between nodes and can be conditionally routed based on node outputs. Cycles are supported for iterative refinement (ReAct loops). Checkpointing persists state to databases enabling pause/resume, fault tolerance, and human-in-the-loop interrupts. This graph structure makes complex agentic workflows (branching, looping, human approval gates) explicit and inspectable.

from langgraph.graph import StateGraph
graph = StateGraph(AgentState)
graph.add_node('agent', call_agent)
graph.add_node('tools', execute_tools)
graph.add_conditional_edges('agent', router)

Planning is the process of decomposing a complex goal into executable subgoals. Main approaches: (1) Plan-and-Execute — generate a full plan upfront, then execute each step; (2) ReAct-style — interleave planning and execution step-by-step; (3) Tree-of-Thought (ToT) — explore multiple reasoning branches, evaluate them, and backtrack; (4) MCTS (Monte Carlo Tree Search) — sample action sequences, evaluate returns, and expand high-value branches. LLM-based planners generate plans in natural language or code; symbolic planners use formal action models (PDDL).

In MCP, tools are callable functions that take parameters and perform actions or computations (e.g., run_sql_query, read_file, search_web, send_email). They are model-controlled — the LLM decides when to invoke them. Resources are data endpoints that expose content (files, database records, API responses) in a read-only, application-controlled manner — the host application decides when to expose resources, not the model. Resources are analogous to file system paths; tools are analogous to executable programs. Both are discoverable by MCP clients.

In agent architecture, the control plane handles task orchestration — which agent runs next, what tools are called, how results are routed, and when to stop. The data plane handles the actual computation — LLM inference, tool execution, database queries, file I/O. Separating them enables: independent scaling (control plane is lightweight; data plane is compute-heavy), better observability (log all control decisions independently), and cleaner fault isolation. Orchestration frameworks (LangGraph, Temporal.io for long-running workflows) implement the control plane; LLM inference servers handle the data plane.

A vector database stores high-dimensional embedding vectors and supports efficient approximate nearest neighbor (ANN) search for retrieval. Production choices: (1) Pinecone — managed cloud, scalable, easy to start; (2) Weaviate — open-source, multi-modal, hybrid search (dense + BM25); (3) Qdrant — open-source, Rust-based, high performance; (4) ChromaDB — Python-native, good for local development; (5) pgvector — PostgreSQL extension for vector search in existing Postgres databases; (6) Milvus — open-source, cloud-native, HNSW/IVF indexing. FAISS is the underlying ANN library used by many.

import chromadb
client = chromadb.Client()
collection = client.create_collection('docs')
collection.add(documents=texts, ids=ids, embeddings=embeddings)

HNSW builds a multi-layer graph where each node connects to its approximate nearest neighbors at multiple granularity levels — coarse at the top, fine at the bottom. Search starts at the top layer, greedily navigating to nearby nodes, then zooms in at each lower layer. HNSW achieves O(log N) search complexity with very high recall (>0.99) for modest ef (search depth) parameters. It outperforms IVF (inverted file index) in recall at the same latency and does not require retraining when new vectors are added. Used by Qdrant, Milvus, and pgvector.

Agent evaluation assesses task completion quality, efficiency (steps taken, tokens used), safety (avoiding harmful actions), and robustness (handling errors gracefully). Challenges: (1) ground truth is hard to define for open-ended tasks; (2) evaluation environments must be realistic and reproducible (WebArena, SWE-Bench, GAIA); (3) agents exhibit high variance — same agent may succeed or fail on the same task; (4) evaluating safety requires adversarial inputs; (5) human evaluation is expensive and slow. LLM-as-judge (GPT-4 scoring) and automated task-specific verifiers are common approaches.

Zero-shot tool use: the LLM uses tools based solely on tool descriptions in the system prompt, without examples. Few-shot: example tool calls and results are provided in the prompt to improve adherence to format and improve decision-making about when to use tools. Fine-tuned: the model is explicitly trained on tool use datasets to reliably generate valid tool call syntax, improving accuracy on complex tool schemas and multi-step tool chains. GPT-4 and Claude 3.7 are fine-tuned for tool use; smaller models often need few-shot prompting for reliable tool use.

Reflexion (Shinn et al., 2023) is an agent framework that uses verbal reinforcement: after each task attempt, the agent receives feedback (from an evaluator or self-reflection) and generates a textual reflection on what went wrong. This reflection is stored in episodic memory and prepended to the next attempt. Over multiple trials, the agent accumulates experience without gradient updates. Reflexion achieves near-SOTA on sequential decision-making benchmarks (HotpotQA, AlfWorld) by learning from failure in context rather than through backpropagation.

Computer use allows an agent to control a computer interface (mouse clicks, keyboard input, screen reading) to complete tasks on behalf of users — demonstrated by Claude Computer Use (Anthropic, 2024). The agent observes screenshots, infers UI state, and takes actions. Safety challenges: (1) prompt injection via on-screen content (a web page tells Claude to delete files); (2) irreversible actions (emptying recycle bin, sending emails); (3) credential access (agent may see and leak passwords); (4) scope creep (doing more than intended). Mitigations: sandboxing, minimal permissions, human approval for high-risk actions.

Synchronous agents execute steps sequentially — each tool call blocks until the result arrives before proceeding. This is simple but slow when tools have high latency (e.g., web search, API calls). Asynchronous agents issue multiple tool calls concurrently using async/await, gathering results in parallel when possible. For example, an agent researching multiple topics can fan out searches simultaneously. Python asyncio and frameworks like LangGraph support async node execution. Async execution significantly reduces wall-clock time for tool-heavy agents.

import asyncio
async def agent_step(state):
    results = await asyncio.gather(
        search_tool(state.query_1),
        search_tool(state.query_2)
    )

A critic agent reviews, challenges, or scores outputs from a proposer agent. In multi-agent debate (Du et al., 2023), multiple LLM instances independently answer a question, then each reads the others' answers and updates its own response — convergence produces more accurate final answers. The critic role can be formalized as a separate agent that identifies logical errors, missing evidence, or factual errors in the proposer's output. This improves accuracy on reasoning tasks where individual LLMs have high error rates.

Tool selection is the LLM's decision of which tool to invoke given the current state and goal. It relies on: (1) clear tool descriptions (name, purpose, input schema, examples); (2) the model's reasoning about task requirements; (3) few-shot examples of correct tool selection; (4) tool retrieval — for systems with hundreds of tools, a retriever first selects relevant tools to present to the LLM rather than including all tools in every prompt (reduces context length and confusion). Gorilla and ToolLLM are fine-tuned models specialized for large-scale tool selection.

SWE-Bench (Jimenez et al., 2023) is a benchmark evaluating LLM agents on real GitHub software engineering tasks: given a repository, an issue description, and a failing test, the agent must generate a code patch that resolves the issue and passes tests. Tasks span Python repositories (scikit-learn, Django, requests). SWE-Bench Verified filters to high-quality tasks with human-verified solutions. It measures end-to-end software engineering capability — understanding codebases, debugging, writing correct patches. GPT-4-based agents achieve 12-50% on various splits; top agents (Agentless, SWE-Agent) use search and edit tools.

The agent scratchpad is a section of the LLM prompt that accumulates the history of thoughts, tool calls, and observations from the current episode. As the agent takes actions, the Thought-Action-Observation triples are appended to the scratchpad, giving the LLM context on what has been done and what was observed. This enables coherent multi-step reasoning within a context window. The scratchpad is ephemeral (discarded when the episode ends) unless explicitly saved to external memory. Context length limits the number of steps before scratchpad truncation.

A sandbox is an isolated execution environment that restricts what code an agent can do — no network access (unless explicitly allowed), limited file system access, no persistent process spawning, and resource limits (CPU, memory, time). When an agent writes and runs code (Python REPL, Bash), the sandbox prevents malicious or accidental damage to the host system. Tools: gVisor, Docker containers with seccomp profiles, Firecracker microVMs, E2B (cloud code sandbox). Without sandboxing, an adversarially prompt-injected agent could delete files, exfiltrate data, or install malware.

Chain-of-thought prompting (Wei et al., 2022) elicits step-by-step reasoning by including examples showing intermediate reasoning steps before the answer, or by appending 'Let's think step by step.' CoT dramatically improves accuracy on arithmetic, symbolic reasoning, and multi-hop QA tasks. For agents, CoT enables the LLM to reason about which tool to use, what inputs to provide, and how to interpret results. Zero-shot CoT (just 'think step by step') works for many models; few-shot CoT provides structured examples for harder tasks.

# Zero-shot CoT:
prompt = f'{question}\nLet\'s think step by step.'

GAIA (Mialon et al., 2023) benchmarks real-world question answering requiring multi-step reasoning, web search, file analysis, and tool use — questions that a 'general-purpose AI assistant' should be able to answer. Tasks range from Level 1 (simple factual lookup) to Level 3 (multi-step, multi-tool tasks requiring sustained reasoning). GAIA evaluates whether agents can autonomously combine web browsing, calculator use, document reading, and reasoning. Top agent systems achieve ~30-70% on L1 but struggle on L3 tasks. It is closer to real-world utility than narrow academic benchmarks.

HITL allows human operators to approve, modify, or reject agent actions at designated checkpoints before they execute — essential for irreversible or high-risk actions (sending emails, deleting files, making purchases). Implementation: the agent pauses at a gate node, sends a notification with the proposed action and context to a human (via webhook, Slack, UI), waits for approval or correction, then continues or reverts. LangGraph supports HITL via interrupt_before and interrupt_after configuration on nodes. HITL is a key safety mechanism for production agents handling sensitive tasks.

graph.add_node('send_email', send_email_fn)
graph.compile(interrupt_before=['send_email'])  # pause for human approval

Task decomposition breaks a complex high-level goal into executable subtasks. Challenges: (1) subtask independence — subtasks may share state and must be ordered correctly; (2) under-decomposition — subtask still too complex for a single tool call; (3) over-decomposition — unnecessary steps that waste tokens and increase failure risk; (4) dependency handling — some subtasks must complete before others can start; (5) re-planning — if a subtask fails, the agent must adapt. Decomposition quality is a primary determinant of agent success on long-horizon tasks.

AutoGen is a Microsoft framework for building multi-agent applications where agents (LLM-backed or human-proxy) communicate through message passing. Key patterns: (1) two-agent chat — assistant and critic exchange messages until task completion; (2) group chat — orchestrator routes messages among multiple specialist agents; (3) nested chats — agents spawn sub-conversations for subtasks; (4) synchronous and asynchronous execution. AutoGen 0.4 introduced an actor model with event-driven communication, making it suitable for production distributed agent systems.

from autogen import ConversableAgent
assistant = ConversableAgent('assistant', llm_config=config)
critic = ConversableAgent('critic', llm_config=config)
chat_result = assistant.initiate_chat(critic, message=task)

An agent trajectory is the complete sequence of observations, thoughts, and actions taken during a task — the full audit trail from initial state to final answer. Trajectories are critical for evaluation: they reveal whether the agent reached the correct answer via the right reasoning process or through lucky shortcuts. Trajectory analysis identifies failure modes (wrong tool selection, incorrect intermediate conclusions, infinite loops). For training, high-quality trajectories can be used as supervised fine-tuning data to improve future agents. SWE-Bench and WebArena release full trajectory datasets for analysis.

Parametric knowledge is encoded in model weights during pretraining — fast (no retrieval latency), but static and potentially outdated. Retrieval-augmented knowledge is fetched at inference time from external databases — can be current, private, or domain-specific. Use parametric for general language understanding, common sense, and reasoning. Use RAG for: frequently-updated facts (news, stock prices), proprietary documents, long-tail specialized knowledge, and cases where citations are required. RAG can hallucinate by misinterpreting retrieved documents; parametric can hallucinate due to training data gaps.

A prompt template is a reusable pattern with variables (e.g., {context}, {query}) that generates a specific type of LLM call — it encapsulates a single prompt. A prompt chain sequences multiple LLM calls where outputs of earlier calls become inputs to later ones. For example: (1) extract key claims from document; (2) verify each claim via web search; (3) synthesize verified claims into final answer. LangChain's LCEL (LangChain Expression Language) and LangGraph both support prompt chains with conditional branching and parallel execution.

from langchain_core.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_template('Summarize: {document}')
chain = prompt | llm | StrOutputParser()

A well-designed agent system prompt includes: (1) role definition — who the agent is and its primary purpose; (2) tool descriptions — each tool's name, description, input schema, and usage examples; (3) behavioral guidelines — when to use tools vs parametric knowledge, how to handle uncertainty, escalation rules; (4) output format — structured response format if required; (5) safety constraints — what to avoid doing; (6) context about the user/task environment. The system prompt is the primary lever for controlling agent behavior without fine-tuning.

Self-consistency (Wang et al., 2022) generates multiple independent reasoning paths for the same question (with temperature > 0) and aggregates the final answers by majority vote. It improves accuracy on arithmetic and logical reasoning tasks by 5-20% over single-path chain-of-thought. In agents, it is applied at decision points where the best action is unclear — sample K reasoning paths, take the most common action. The cost is K× inference compute. It is most effective for closed-answer tasks; for open-ended generation, it requires a scoring mechanism beyond majority vote.

Structured output requires LLMs to generate syntactically valid JSON (or other formats) for tool calls — a single token error (extra comma, missing quote) breaks parsing. Approaches: (1) constrained decoding (grammar-based sampling with grammars like Lark) — only tokens forming valid JSON prefixes are sampled; (2) fine-tuning on tool call datasets improves native adherence; (3) JSON repair via post-processing; (4) JSON mode in OpenAI/Anthropic APIs forces valid JSON output. Constrained decoding via llama.cpp's grammar support or Outlines library is the most robust approach.

from outlines import generate
generator = generate.json(model, ToolCallSchema)
result = generator(prompt)

Lost-in-the-middle (Liu et al., 2023) is the empirical finding that LLMs have higher recall for information at the beginning and end of long contexts but systematically miss information placed in the middle. For agents with long scratchpads or retrieved documents, important tool results or instructions in the middle of context may be ignored. Mitigations: (1) reorder context to place most relevant information at beginning/end; (2) summarize middle sections; (3) use retrieval to pull only relevant sections; (4) use long-context models with better attention patterns.

Embedding models convert text chunks into dense vectors for similarity search. Quality depends on: (1) semantic representation — similar meanings should have similar vectors; (2) domain alignment — general models underperform on specialized domains (legal, medical, code). Top embedding models: text-embedding-3-large (OpenAI), e5-large-v2, BGE-M3 (multilingual, multi-granularity). Chunking strategy (fixed-size, sentence, semantic) significantly affects retrieval quality. Bi-encoder (fast approximate retrieval) is used for recall; cross-encoder reranker is used for precision on top-K results.

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('BAAI/bge-m3')
embeddings = model.encode(texts)

An agent loop must terminate when: (1) the goal is achieved (success); (2) a max_steps limit is reached (forced termination); (3) the agent explicitly signals completion. Common failure modes: (1) infinite loops — agent repeatedly calls the same tool with the same input; (2) goal drift — agent pursues a subtask and forgets the main objective; (3) premature termination — agent answers before gathering sufficient information; (4) tool failure cascade — a failed tool call causes subsequent decisions to be based on incorrect assumptions. Detection: tracking visited tool-call patterns and monitoring for repeated identical actions.

# Agent loop with max steps guard:
for step in range(max_steps):
    action = agent.step(state)
    if action.type == 'finish': break
    state = env.execute(action)

The OpenAI Assistants API (2023) provides managed agent infrastructure: Threads (persistent conversation context), Runs (individual agent execution), and Files (for RAG with built-in vector storage). It manages the agent loop automatically — persisting tool calls, waiting for results, and resuming execution. Built-in tools: Code Interpreter (sandboxed Python REPL), File Search (RAG over uploaded docs). This abstracts away state management, tool execution, and context window management, enabling rapid prototyping. Limitations: opaque internals, vendor lock-in, and higher cost vs self-hosted solutions.

In the subagent pattern, an orchestrator LLM decomposes a task and spawns specialized subagents for each component. The orchestrator maintains the high-level plan and monitors progress; subagents have narrow focus (e.g., web search agent, code writing agent, data analysis agent) with their own tools and prompts. Subagents return results to the orchestrator which integrates them. This enables parallel execution, specialization, and isolation of concerns. Anthropic's Claude agent SDK, AutoGen, and CrewAI implement this hierarchical pattern.

Context distillation (summarization of past agent context) is used when the agent's scratchpad approaches the model's context limit. The agent (or a separate summarization LLM) compresses old tool calls and observations into a concise summary, discarding verbatim content while preserving key facts. The summary replaces the detailed history, freeing context for new actions. Without distillation, agents on long tasks either fail (context overflow) or degrade (performance drops with very long contexts due to attention dilution). Strategies: rolling window summaries, hierarchical summarization, and task-specific extraction.

Production agent observability requires: (1) trace logging — every LLM call, tool invocation, input, and output logged with latency and token counts; (2) span/trace correlation — group all calls in one agent run under a trace ID; (3) cost tracking — token counts and model costs per run; (4) error monitoring — failed tool calls, LLM refusals, timeouts; (5) evaluation pipelines — automated scoring of agent outputs. Tools: LangSmith (LangChain), Arize Phoenix, Langfuse, Weights & Biases Traces, Helicone. OpenTelemetry (OTel) is becoming the standard instrumentation protocol.

Long-term memory stores knowledge across multiple agent sessions — unlike in-context memory which is lost when the conversation ends. Implementations: (1) entity memory — extracting and storing facts about users/entities in structured form (JSON, database rows); (2) summary memory — periodic summarization of past conversations stored in a database; (3) vector memory — embedding past interactions and retrieving relevant ones at the start of new sessions. MemGPT (Packer et al., 2023) implements hierarchical memory with in-context and external storage managed by the agent itself.

Minimal footprint means agents should request only necessary permissions, avoid storing sensitive information beyond immediate needs, prefer reversible over irreversible actions, and err on the side of doing less when uncertain about intended scope. This limits blast radius if the agent is compromised (prompt injection) or makes a mistake. Concrete examples: requesting read-only filesystem access when write is not needed; asking for confirmation before sending emails; writing to temporary directories rather than permanent locations. This is a core safety principle in Anthropic's model spec for agentic contexts.

After an agent calls a tool, the host application executes the tool and injects the result back into the conversation as a tool result message (in the Anthropic API: role='user' with type='tool_result'; in OpenAI: role='tool'). The LLM then receives the full updated context including the tool call and its result and generates the next action or final response. The injection must preserve the tool_use_id to correlate results with their corresponding tool calls. Malformed or truncated tool results can cause the agent to hallucinate or repeat tool calls.

messages.append({'role': 'user', 'content': [{
    'type': 'tool_result',
    'tool_use_id': tool_call_id,
    'content': result
}]})

Stateless agents treat each invocation independently — no memory of previous interactions; each call receives a full self-contained context. Simple and horizontally scalable (any server can handle any request). Stateful agents maintain state across multiple interactions — storing conversation history, task progress, and intermediate results. Stateful agents can handle multi-session tasks and personalize based on history, but require persistent storage and session routing. Most production systems use a hybrid: stateless LLM inference with externalized state (database, Redis) for scalability.

The system prompt is the primary mechanism for defining an agent's identity, scope, and constraints without fine-tuning. It specifies: the agent's name and role ('You are a customer support agent for Acme Corp'), its knowledge domain and limitations ('you only answer questions about our products'), tool descriptions and usage guidelines, output format requirements, and safety constraints ('never discuss competitor products'). Well-crafted system prompts significantly improve agent reliability and boundary compliance. They are considered operator-level instructions with authority above user messages.

WebArena (Zhou et al., 2023) evaluates autonomous web navigation agents on real-world tasks across functional web environments (e-commerce, GitLab, Reddit, Wikipedia). Tasks require multi-step actions: clicking, typing, form submission, search, navigation. Agents observe DOM/screenshots and take browser actions. Difficulty: low-level web interaction failures compound over multi-step tasks. Top agents achieve 10-40% task completion. WebArena evaluates the gap between LLM reasoning ability and reliable tool execution in realistic, noisy environments.

A bi-encoder retriever (e.g., dense embedding search) efficiently finds the top-K candidate documents but may rank by embedding similarity rather than true relevance to the query. Reranking applies a cross-encoder (e.g., Cohere Rerank, BGE-Reranker) that jointly encodes the query and each candidate document and scores their relevance more precisely. Cross-encoders are more accurate but slower (O(K) forward passes). Reranking pipelines: retrieve K=50 with bi-encoder, rerank to top-5 with cross-encoder. This significantly improves final answer quality at modest latency cost.

Toolformer (Schick et al., 2023) demonstrated that LLMs can be trained to self-supervisedly learn when and how to call APIs by: (1) generating candidate tool calls; (2) evaluating whether the tool result reduces perplexity on the following text; (3) fine-tuning on examples where tool use was beneficial. This produced a model that autonomously decides to use calculators, calendars, search engines, and translation APIs mid-text without explicit demonstrations. It showed that tool use can be learned with minimal human supervision through a bootstrapped data generation process.

Guardrails are safety and policy enforcement mechanisms that intercept agent inputs and outputs. Input guardrails scan user messages for jailbreak attempts, harmful intent, or PII before reaching the agent. Output guardrails scan agent responses for harmful content, hallucinations, policy violations, or sensitive data before returning to users. Implementations: NeMo Guardrails (NVIDIA) uses LLM-based classifiers and programmable rules; Guardrails AI uses validators with retry logic; custom rule-based filters check for specific patterns. Guardrails add latency but provide defense-in-depth beyond RLHF alignment.