Model-routing middleware driven by a decision model.
Initialize a chat model from any supported provider using a unified interface.
Two main use cases:
config. Makes it easy to
switch between models/providers without changing your codeRequires the integration package for the chosen model provider to be installed.
See the model_provider parameter below for specific package names
(e.g., pip install langchain-openai).
Refer to the provider integration's API reference
for supported model parameters to use as **kwargs.
Base middleware class for an agent.
Subclass this and implement any of the defined methods to customize agent behavior between steps in the main agent loop.
State schema for the agent.
Model request information for the agent.
Response from model execution including messages and optional structured output.
The result will usually contain a single AIMessage, but may include an additional
ToolMessage if the model used a tool for structured output.
A model available to the router and the criterion for selecting it.
Select an agent's model with a decision model's Choice answer.
The middleware classifies the latest human message once before an agent run,
stores the complete ChoiceAnswer in agent state, and uses its selected label for
every model call in the run. Keeping the complete answer makes probabilities and
confidence available in state and traces. Classifier failures propagate and
terminate the run rather than silently selecting a different model.
This middleware is in beta. Its API may change without notice.