Agentic AI and tool use

ReAct interleaves reasoning traces (thoughts) with concrete actions (tool calls) and observations from the environment, feeding tool results back into the context to ground subsequent reasoning. Pure chain-of-thought only produces internal reasoning without external grounding, so it cannot fetch fresh information or recover from factual gaps. This grounding loop reduces hallucination and lets the agent adapt its plan based on real observations.

A JSON Schema lets the model emit structured, machine-parseable arguments and lets the runtime validate them before execution, catching malformed or missing parameters deterministically. The typed schema plus per-field descriptions also steers the model toward correct argument values, reducing hallucinated or mistyped inputs. Free text alone gives no validation guarantees and makes reliable parsing brittle.

MCP is an open protocol that standardizes how LLM applications connect to external tools, data sources, and resources through a common client-server interface. It decouples agents from bespoke per-integration code, so any MCP-compatible client can use any MCP server's tools without custom glue. This promotes reuse and interoperability across the agent ecosystem.

After a failed or suboptimal attempt, the agent generates a verbal self-reflection summarizing what went wrong, and stores it in an episodic memory buffer. On the next attempt, this reflection is prepended to the context so the agent avoids repeating the mistake, effectively learning from feedback without updating model weights. It is a form of in-context, language-based reinforcement.

Use parallel (concurrent) tool calls: modern function-calling models can emit multiple tool invocations in a single turn, and the runtime executes the independent calls simultaneously. This cuts wall-clock latency and token round-trips versus sequential calling. Sequential chaining should be reserved for cases where one tool's output feeds another's input.

Combine multiple stop criteria: a maximum step/iteration budget, a token or cost ceiling, detection of a final-answer signal, and loop/repetition detection (e.g., identical tool calls). Relying solely on the model deciding it is 'done' risks infinite loops when it gets stuck. Hard budgets guarantee bounded execution even under degenerate behavior.

When an agent ingests untrusted external data (web pages, emails, documents) into its context, attacker-controlled text can issue instructions the model may follow, hijacking its tool use to exfiltrate data or take harmful actions. Because the agent has real capabilities (sending emails, calling APIs), a successful injection has real-world side effects, not just bad text output. Mitigations include treating tool outputs as data not instructions, least-privilege tools, and human approval for sensitive actions.

A supervisor centralizes routing, decomposition, and result aggregation, giving clearer control flow, easier debugging, and a single place to enforce budgets and stopping. Peer-to-peer meshes can suffer from emergent loops, ambiguous responsibility, and uncontrolled message amplification. The supervisor pattern trades some flexibility for predictability and observability.

Agentic RAG treats retrieval as a tool the agent can invoke iteratively: it can reformulate queries, decide which sources to search, retrieve multiple times, and critique/verify results before answering. Classic RAG retrieves once with the original query and stuffs results into the prompt. The agentic loop improves recall on multi-hop or ambiguous questions at the cost of more latency and tokens.

Return the structured error back to the model as a tool result so it can reason about recovery (retry, fix arguments, or choose another tool), while the runtime applies bounded retries with backoff for transient failures. Silently swallowing the error or crashing the loop prevents recovery; surfacing it as an observation keeps the ReAct loop self-correcting. Persistent failures should trigger a graceful fallback or human escalation.

After the model emits a `tool_use` block with an id and inputs, the application executes the tool and sends back a user-turn message containing a `tool_result` block referencing that same `tool_use_id` with the output (or an `is_error` flag). This closes the loop so the model can incorporate the result into its next turn. The id linkage lets the model match results to the correct call, including with parallel tool use.

messages.append({
  "role": "user",
  "content": [{
    "type": "tool_result",
    "tool_use_id": block.id,
    "content": str(result),
    "is_error": False
  }]
})

Prompt caching stores the processed representation of a stable prefix (system prompt, tool definitions, large context) so repeated requests reuse it, dramatically cutting input-token cost and latency on subsequent calls. In agents the tool schemas and instructions are reused every turn, making them ideal cache candidates. The dynamic suffix (new observations) stays uncached.

Large tool sets bloat the context, raise cost, and increase the chance the model selects the wrong tool due to confusable descriptions. Mitigations include dynamic tool filtering/retrieval (only expose tools relevant to the current query), hierarchical grouping, and a router/planner step that narrows the candidate set before the action LLM sees them. Clear, disjoint tool descriptions also reduce selection errors.

Plan-and-execute agents first generate an explicit multi-step plan, then execute steps (often dispatching to executors), optionally re-planning when reality diverges. Reactive agents decide the next single action each step with no upfront global plan. Planning improves coherence on long-horizon tasks and can reduce LLM calls, but rigid plans can fail when the environment is unpredictable, hence re-planning.

Semantic memory stores generalized facts and knowledge (e.g., user preferences, domain facts) independent of when they were learned, while episodic memory stores specific past experiences/interactions as events with context. Agents use semantic memory for stable knowledge retrieval and episodic memory to recall and learn from prior trajectories. Procedural memory, a third type, stores learned skills/instructions.

Arbitrary code execution can produce destructive side effects, infinite loops, or security breaches (file/network access). The standard mitigation is sandboxing: run code in an isolated, resource-limited, network-restricted environment (container/VM/microVM) with timeouts and no access to host secrets. Output and side effects are then captured and returned as observations.

Two agents can reach the same final answer via very different paths; trajectory metrics (tool-call correctness, number of steps, redundant actions, cost, adherence to constraints) reveal efficiency, safety, and failure modes that outcome accuracy hides. They also help diagnose where reasoning breaks down. This matters because production agents are judged on cost, latency, and safe behavior, not just correctness.

A hallucinated tool call is when the model invokes a tool that doesn't exist or supplies arguments that violate the schema (wrong types, fabricated enum values). Defenses include strict schema validation that rejects unknown tools/invalid args, returning a clear error observation so the model can correct itself, and using constrained/structured decoding to force schema-valid outputs. Logging these events also surfaces prompt or schema issues.

A state graph makes control flow explicit as nodes and conditional edges over a typed shared state, enabling features like checkpointing, human-in-the-loop interrupts, time-travel/resume, cycles, and deterministic branching. A bare while-loop hides this structure, making persistence, debugging, and interruption ad hoc. The graph also composes naturally for multi-agent subgraphs.

It pauses execution to require human confirmation before high-risk or irreversible actions (sending payments, deleting data, external communications), bounding the blast radius of model errors or prompt injection. Gates should sit at tool-execution boundaries for sensitive tools, ideally with the proposed action shown for review. Low-risk, reversible, read-only actions can proceed autonomously to preserve throughput.

Tool/retrieved outputs can contain attacker- or noise-controlled text; if the model treats them as instructions, it becomes vulnerable to indirect prompt injection that redirects its behavior. Framing them as data (clearly delimited, with system instructions taking precedence) reduces this risk. Combined with least-privilege tools and output filtering, it limits exploitation.

τ-bench evaluates an agent's ability to complete realistic, multi-turn customer-service tasks by interacting with a simulated user and tools/APIs while following domain policies (e.g., retail, airline). It measures task success and, critically, consistency/reliability across repeated trials (pass^k), exposing how often agents fail to reliably follow rules. It highlights the gap between single-attempt success and dependable real-world performance.

Compaction/summarization: periodically summarize older turns and tool outputs into a compact running summary, keep the most recent and most relevant items verbatim, and offload full detail to external memory retrievable on demand. This preserves salient state while staying within the window. Naively truncating the oldest tokens risks dropping critical earlier context.

A low temperature (often 0 or near 0) is preferred for tool-selection and argument generation to maximize determinism, reproducibility, and correct schema adherence. Decision steps benefit from consistency, whereas creative steps (e.g., brainstorming) may use higher temperature. Note that low temperature reduces but does not eliminate variance due to other nondeterminism sources, and some frontier reasoning models no longer expose a temperature knob at all.

GUI agents must visually parse pixel screenshots to locate elements, infer state, and emit precise coordinate-based actions (clicks, types), with no clean schema for the 'environment'. This adds grounding errors (mis-clicks), high latency per step, partial observability, and brittleness to UI changes. Text/API tool calling, by contrast, has structured, unambiguous interfaces.

A verifier independently checks the candidate output against the task, evidence, or constraints (e.g., does the answer cite retrieved sources, pass tests, satisfy a schema), and triggers revision if it fails. This self-checking catches errors the generator missed and is the basis of generate-verify and reflect-revise loops. Verification is often cheaper and more reliable than generation, improving overall correctness.

Agents make many interdependent, nondeterministic LLM and tool calls, so failures are emergent and hard to reproduce; full traces of inputs, outputs, tool args, latencies, and costs per step are needed to diagnose where a trajectory went wrong. Tracing also enables cost/latency attribution and regression detection across versions. Single-shot apps have a much smaller, more transparent surface.

A scratchpad lets the model externalize intermediate reasoning, plans, and tool-call deliberation without polluting the user-facing answer, improving multi-step reasoning quality. Many APIs expose this as explicit reasoning/thinking content that can be allotted a reasoning/effort budget. It separates 'how it thought' from 'what it concluded', aiding both performance and clean outputs.

The supervisor exposes each specialist agent as a callable 'tool' (handoff); when it decides to delegate, it 'calls' that agent with a scoped instruction, receives the result as an observation, and integrates it. This reuses the function-calling mechanism for routing and keeps interfaces uniform. The supervisor remains responsible for sequencing and final synthesis.

Each tool should expose only the minimum capability and data access needed for its purpose, with scoped credentials and narrow parameters, so a compromised or misused agent cannot perform broad damage. For example, prefer a 'get_order_status(order_id)' tool over a generic 'run_sql(query)' tool. This shrinks the attack surface from prompt injection and model errors.

Function calling decides whether and which tool to invoke and produces its arguments to be executed externally, whereas structured output forces the model's final response to conform to a schema you parse directly (no execution). Use function calling when the model should act on the environment; use structured output when you just need reliable, typed data extraction or a formatted final answer. They can be combined.

Unbounded parallelism can overwhelm downstream rate limits, spike cost, exhaust connection pools, and make failures harder to attribute and roll back. A concurrency cap (semaphore) balances latency gains against system stability and provider rate limits. It also bounds the blast radius if the model mistakenly fans out wrong calls.

Equip the agent with a retrieval/search tool (web search, internal knowledge base, or live API) and have it ground its answer in fetched results rather than relying on parametric memory. This keeps responses current and verifiable with citations. Fine-tuning is poorly suited for frequently changing facts because it is slow, costly, and quickly stale.

It targets agents that get stuck repeating the same (or alternating) tool calls and reasoning without progress, wasting cost and never terminating. Implementation hashes/compares recent actions and observations; on detecting repetition beyond a threshold, the runtime breaks out, injects a nudge, or escalates to human/fallback. This complements hard step/cost budgets by catching unproductive cycles early.