Set the process-wide default TracePolicy for agent middleware hook spans.
Call once at startup. A middleware's own trace_policy overrides this wholesale
(no field-level merge). Pass None to clear the default.
Decorator used to dynamically create a middleware with the after_agent hook.
Async version is aafter_agent.
Decorator used to dynamically create a middleware with the after_model hook.
Decorator used to dynamically create a middleware with the before_agent hook.
Decorator used to dynamically create a middleware with the before_model hook.
Decorator used to dynamically generate system prompts for the model.
This is a convenience decorator that creates middleware using wrap_model_call
specifically for dynamic prompt generation. The decorated function should return
a string that will be set as the system prompt for the model request.
Decorator to configure hook behavior in middleware methods.
Use this decorator on before_model or after_model methods in middleware classes
to configure their behavior. Currently supports specifying which destinations they
can jump to, which establishes conditional edges in the agent graph.
Create middleware with wrap_model_call hook from a function.
Converts a function with handler callback into middleware that can intercept model calls, implement retry logic, handle errors, and rewrite responses.
Create middleware with wrap_tool_call hook from a function.
Async version is awrap_tool_call.
Converts a function with handler callback into middleware that can intercept tool calls, implement retry logic, monitor execution, and modify responses.
Allow low-risk tool calls and block risky calls using a decision model.
The middleware intercepts explicitly configured tools immediately before execution
and asks a Noul question for the probability that each call is risky or
insufficiently authorized. Calls below 0.5 execute normally. Calls at or above it
return an error ToolMessage without invoking the tool handler. Tool names not
listed in tools bypass classification.
The classifier receives the proposed tool call, the tool description, and up to 30 recent messages. Classification failures propagate and the tool handler is not called. This middleware blocks calls; it does not request human approval.
This middleware is in beta. Its API may change without notice.
Configuration for clearing tool outputs when token limits are exceeded.
Automatically prune tool results to manage context size.
The middleware applies a sequence of edits when the total input token count exceeds configured thresholds.
Currently the ClearToolUsesEdit strategy is supported, aligning with Anthropic's
clear_tool_uses_20250919 behavior (read more).
Provides Glob and Grep search over filesystem files.
This middleware adds two tools that search through local filesystem:
Pause the agent so a person can review tool calls before they run.
Each tool in interrupt_on needs a decision before it runs: approve, edit, reject,
or respond. The pause is a LangGraph interrupt, so the agent needs a checkpointer,
and the run continues with Command(resume=...).
interrupt_mode sets how the agent pauses:
"batched" (default): one interrupt per model turn. Its value is a HITLRequest
listing every gated call, answered with a HITLResponse whose decisions match
the calls by position."per_call": one interrupt per gated tool call, as the call starts. Its value is
a ToolApprovalRequest and its response_schema lists the decisions that tool
allows. Each is answered with a single Decision keyed by interrupt ID, and
LangGraph checks the answer against the schema before saving it. "per_call"
is planned to become the default in the next major release.Configuration for an action requiring human in the loop.
This is the configuration format used in the HumanInTheLoopMiddleware.__init__
method.
Tracks model call counts and enforces limits.
This middleware monitors the number of model calls made during agent execution and can terminate the agent when specified limits are reached. It supports both thread-level and run-level call counting with configurable exit behaviors.
Thread-level: The middleware tracks the number of model calls and persists call count across multiple runs (invocations) of the agent.
Run-level: The middleware tracks the number of model calls made during a single run (invocation) of the agent.
Automatic fallback to alternative models on errors.
Retries failed model calls with alternative models in sequence until
success or all models exhausted. Primary model specified in create_agent.
Raise ModelRefusalError for responses containing a standard refusal block.
Add this middleware to create_agent to treat a refusal as an error instead of
a successful final answer. Model calls without this middleware continue to
return messages, including refusal content.
List it after ModelFallbackMiddleware so the fallback can catch the error.
Streaming content may already have been emitted before the refusal is known.
This middleware does not infer refusals from text or retry blocked requests.
Middleware that automatically retries failed model calls with configurable backoff.
Supports retrying on specific exceptions and exponential backoff.
A model available to the router and the criterion for selecting it.
Select an agent's model with a decision model's Choice answer.
The middleware classifies the latest human message once before an agent run,
stores the complete ChoiceAnswer in agent state, and uses its selected label for
every model call in the run. Keeping the complete answer makes probabilities and
confidence available in state and traces. Classifier failures propagate and
terminate the run rather than silently selecting a different model.
This middleware is in beta. Its API may change without notice.
Raised when configured to block on detected sensitive values.
Represents an individual match of sensitive data.
Detect and handle Personally Identifiable Information (PII) in conversations.
This middleware detects common PII types and applies configurable strategies to handle them. It can detect emails, credit cards, IP addresses, MAC addresses, and URLs in both user input and agent output.
Built-in PII types:
email: Email addressescredit_card: Credit card numbers (validated with Luhn algorithm)ip: IP addresses (validated with stdlib)mac_address: MAC addressesurl: URLs (both http/https and bare URLs)Strategies:
block: Raise an exception when PII is detectedredact: Replace PII with [REDACTED_TYPE] placeholdersmask: Partially mask PII (e.g., ****-****-****-1234 for credit card)hash: Replace PII with deterministic hash (e.g., <email_hash:a1b2c3d4>)Strategy Selection Guide:
| Strategy | Preserves Identity? | Best For |
|---|---|---|
block |
N/A | Avoid PII completely |
redact |
No | General compliance, log sanitization |
mask |
No | Human readability, customer service UIs |
hash |
Yes (pseudonymous) | Analytics, debugging |
Defer selected tools behind provider-native tool search.
Instead of sending every tool schema on every turn, this middleware marks
selected tools as deferred (via extras["defer_loading"]) and injects the
provider's server-side tool search tool. The provider then retrieves the
full schema of a deferred tool only when the model needs it, which keeps the
request payload small when many tools are bound.
A tool is deferred when its name (or instance) is passed in searchable_tools,
or when it already carries extras["defer_loading"] is True.
Only providers with server-side tool search are supported (currently Anthropic and OpenAI). The provider is inferred from the bound model.
This relies on provider-native tool search and only takes effect for
supported providers. If a tool is deferred but the model's provider
cannot be identified or does not support tool search, the model call
raises ValueError. When no tool is deferred, the middleware passes the
request through unchanged regardless of provider.
Launch the shell through the Codex CLI sandbox.
Ideal when you have the Codex CLI installed and want the additional syscall and
filesystem restrictions provided by Anthropic's Seatbelt (macOS) or Landlock/seccomp
(Linux) profiles. Commands still run on the host, but within the sandbox requested by
the CLI. If the Codex binary is unavailable or the runtime lacks the required
kernel features (e.g., Landlock inside some containers), process startup fails with a
RuntimeError.
Configure sandbox behavior via config_overrides to align with your Codex CLI
profile. This policy does not add its own resource limits; combine it with
host-level guards (cgroups, container resource limits) as needed.
Run the shell inside a dedicated Docker container.
Choose this policy when commands originate from untrusted users or you require
strong isolation between sessions. By default the workspace is bind-mounted only
when it refers to an existing non-temporary directory; ephemeral sessions run
without a mount to minimise host exposure. The container's network namespace is
disabled by default (--network none) and you can enable further hardening via
read_only_rootfs and user.
The security guarantees depend on your Docker daemon configuration. Run the agent on
a host where Docker is locked down (rootless mode, AppArmor/SELinux, etc.) and
review any additional volumes or capabilities passed through extra_run_args. The
default image is python:3.12-alpine3.19; supply a custom image if you need
preinstalled tooling.
Run the shell directly on the host process.
This policy is best suited for trusted or single-tenant environments (CI jobs, developer workstations, pre-sandboxed containers) where the agent must access the host filesystem and tooling without additional isolation. Enforces optional CPU and memory limits to prevent runaway commands but offers no filesystem or network sandboxing; commands can modify anything the process user can reach.
On Linux platforms resource limits are applied with resource.prlimit after the
shell starts. On macOS, where prlimit is unavailable, limits are set in a
preexec_fn before exec. In both cases the shell runs in its own process group
so timeouts can terminate the full subtree.
Configuration for handling a single PII type.
Middleware that registers a persistent shell tool for agents.
The middleware exposes a single long-lived shell session. Use the execution policy to match your deployment's security posture:
HostExecutionPolicy – full host access; best for trusted environments where the
agent already runs inside a container or VM that provides isolation.CodexSandboxExecutionPolicy – reuses the Codex CLI sandbox for additional
syscall/filesystem restrictions when the CLI is available.DockerExecutionPolicy – launches a separate Docker container for each agent run,
providing harder isolation, optional read-only root filesystems, and user
remapping.When no policy is provided the middleware defaults to HostExecutionPolicy.
Summarizes conversation history when token limits are approached.
This middleware monitors message token counts and automatically summarizes older messages when a threshold is reached, preserving recent messages and maintaining context continuity by ensuring AI/Tool message pairs remain together.
Summary-generation errors are retried in-process, up to 3 attempts total, via
Runnable.with_retry, unless the error is a ModelError marked non-retryable
(e.g. authentication or invalid request errors). If the summary prompt exceeds
the model's context window (ContextOverflowError), the call is retried once with
the middle of the history cut out. If a summary call still fails, the
underlying error propagates rather than fabricating a summary.
To generate summaries with something other than a chat model, pass a
summarizer. The middleware still decides when to summarize and which
messages to summarize; only the summary text comes from the summarizer.
Dictionary-based trigger specification for AND conditions.
All specified thresholds in a single TriggerClause must be met for the clause to
trigger summarization (AND semantics). When multiple clauses are provided in a list,
summarization triggers if any clause is met (OR semantics).
Middleware that provides todo list management capabilities to agents.
This middleware adds a write_todos tool that allows agents to create and manage
structured task lists for complex multi-step operations. It's designed to help
agents track progress, organize complex tasks, and provide users with visibility
into task completion status.
The middleware automatically injects system prompts that guide the agent on when
and how to use the todo functionality effectively. It also enforces that the
write_todos tool is called at most once per model turn, since the tool replaces
the entire todo list and parallel calls would create ambiguity about precedence.
Track tool call counts and enforces limits during agent execution.
This middleware monitors the number of tool calls made and can terminate or restrict execution when limits are exceeded. It supports both thread-level (persistent across runs) and run-level (per invocation) call counting.
Emulates specified tools using an LLM instead of executing them.
This middleware allows selective emulation of tools for testing purposes.
By default (when tools=None), all tools are emulated. You can specify which
tools to emulate by passing a list of tool names or BaseTool instances.
Return selected tool-execution exceptions to the model as error ToolMessages.
on_error is called for each exception raised by tool execution. Return content
(a str or a list of content blocks) to convert the exception into a
ToolMessage(status="error"); return None — or simply don't return — to let the
exception propagate (halting the run). Handling is therefore opt-in — exceptions you
do not return content for propagate unchanged, so arbitrary internal exceptions are
never serialized to the model or end user unless you choose to surface them.
Langgraph control-flow signals (interrupts, parent commands) always propagate and
never reach on_error.
Prefer returning content that names the exception type over the raw exception message, which may carry sensitive or internal detail.
Provide at least one of on_error or aon_error. aon_error handles errors on the
async execution path (falling back to on_error when omitted); the sync path only
ever calls on_error. For async-only usage, pass aon_error alone — running such a
middleware on the sync path raises, since the async handler cannot be awaited there.
This middleware does not retry. For retries, compose with ToolRetryMiddleware
placed inner and configured with on_failure="error" so exceptions reach this
middleware.
This middleware only sees exceptions raised by tool execution. Argument-binding
and validation errors are handled upstream by ToolNode (converted to an error
ToolMessage before the tool runs), so they do not reach on_error.
Middleware that automatically retries failed tool calls with configurable backoff.
Supports retrying on specific exceptions and exponential backoff.
Uses an LLM to select relevant tools before calling the main model.
When an agent has many tools available, this middleware filters them down to only the most relevant ones for the user's query. This reduces token usage and helps the main model focus on the right tools.
Base middleware class for an agent.
Subclass this and implement any of the defined methods to customize agent behavior between steps in the main agent loop.
State schema for the agent.
Model response with an optional 'Command' from 'wrap_model_call' middleware.
Use this to return a 'Command' alongside the model response from a 'wrap_model_call' handler. The command is applied as an additional state update after the model node completes, using the graph's reducers (e.g. 'add_messages' for the 'messages' key).
Because each 'Command' is applied through the reducer, messages in the command are added alongside the model response messages rather than replacing them. For non-reducer state fields, later commands overwrite earlier ones (outermost middleware wins over inner).
Input state schema for the agent.
Model request information for the agent.
Response from model execution including messages and optional structured output.
The result will usually contain a single AIMessage, but may include an additional
ToolMessage if the model used a tool for structured output.
Output state schema for the agent.
Return type for model call handlers.
Middleware can return either:
ModelResponse: Full response with messages and optional structured outputAIMessage: Simplified return for simple use casesExtendedModelResponse: Response with an optional Command for additional state updates
goto, resume, and graph are not yet supported on these commands.
A NotImplementedError will be raised if you try to use them.Tool error middleware for agents.
LLM-based tool selector middleware.
Opt-in error handling for explicit model refusals.
Tool-risk middleware driven by a decision model.
Tool retry middleware for agents.
Model-routing middleware driven by a decision model.
Summarization middleware.
Provider-side tool search middleware.
Middleware that exposes a persistent shell tool to agents.
Model retry middleware for agents.
Call tracking middleware for agents.
Context editing middleware.
Mirrors Anthropic's context editing capabilities by clearing older tool results once the conversation grows beyond a configurable token threshold.
The implementation is intentionally model-agnostic so it can be used with any LangChain chat model.
File search middleware for Anthropic text editor and memory tools.
This module provides Glob and Grep search tools that operate on files stored in state or filesystem.
Model fallback middleware for agents.
When an outer caching middleware modifies a request, those changes happen before the fallback loop runs. Provider-specific cache settings can cause API errors if the loop reuses them with a different provider. This middleware therefore strips unsupported Anthropic and Fireworks cache settings from each fallback attempt, while preserving settings accepted by the selected fallback.
This provider knowledge lives here because an outer caching middleware never re-runs during fallback and therefore cannot clean up after itself.
Tag and filter middleware-internal model calls.
Tag internal calls with internal_call_metadata() and declare
transformers = (InternalCallTransformer,) on the middleware class to keep
them out of run.messages. Both APIs are public for third-party middleware.
Human in the loop middleware.
Tool call limit middleware for agents.
PII detection and handling middleware for agents.
Planning and task management middleware for agents.
Types for middleware and agents.
Tool emulator middleware for testing.