Automatic fallback to alternative models on errors.
Retries failed model calls with alternative models in sequence until
success or all models exhausted. Primary model specified in create_agent.
ModelFallbackMiddleware(
self,
first_model: str | BaseChatModel,
*additional_models: str | BaseChatModel = ()
)Example:
from langchain.agents.middleware.model_fallback import ModelFallbackMiddleware
from langchain.agents import create_agent
fallback = ModelFallbackMiddleware(
"openai:gpt-4o-mini", # Try first on error
"anthropic:claude-sonnet-4-5-20250929", # Then this
)
agent = create_agent(
model="openai:gpt-4o", # Primary model
middleware=[fallback],
)
# If primary fails: tries gpt-4o-mini, then claude-sonnet-4-5-20250929
result = await agent.invoke({"messages": [HumanMessage("Hello")]})| Name | Type | Description |
|---|---|---|
first_model* | str | BaseChatModel | First fallback model (string name or instance). |
*additional_models | str | BaseChatModel | Default: ()Additional fallbacks in order. |
| Name | Type |
|---|---|
| first_model | str | BaseChatModel |
Logic to run before the agent execution starts.
Async logic to run before the agent execution starts.
Logic to run before the model is called.
Async logic to run before the model is called.
Logic to run after the model is called.
Async logic to run after the model is called.
Logic to run after the agent execution completes.
Async logic to run after the agent execution completes.
Intercept tool execution for retries, monitoring, or modification.
Intercept and control async tool execution via handler callback.