ConfigurableModelMiddleware

ConfigurableModelMiddleware(
  self,
  *,
  persist_model_state: bool = True,
  openai_prompt_cache_key: bool | None

Bases

AgentMiddleware

Constructors

Attributes

Methods

View source on GitHub

strict_model_resolution

Parameters

Name	Type	Description
`persist_model_state`	`bool`	Default:`True` Whether completed calls should write private resume metadata. Subagent instances disable this because they do not own the parent thread's resume state.
`openai_prompt_cache_key`	`bool \| None`	Default:`None` Whether to inject a per-thread OpenAI `prompt_cache_key`. Left as `None` (the default) it is resolved once here from `models.openai_prompt_cache_key` and cached, so no per-call read happens. The one-time `config.toml` read assumes current callers construct the middleware off the blockbuster-guarded server loop (the server path offloads `create_cli_agent` via `asyncio.to_thread`); if that assumption is ever broken the read would trip `BlockingError`, which `_resolve_openai_prompt_cache_key_enabled` re-raises rather than masks. Pass an explicit bool to bypass the config read (mainly for tests).
`cli_max_retries`	`int \| None`	Default:`None`
`strict_model_resolution`	`bool`	Default:`False`
`environ`	`Mapping[str, str] \| None`	Default:`None`
`model_result`	`ModelResult \| None`	Default:`None`

constructor

__init__

Name	Type
persist_model_state	bool
openai_prompt_cache_key	bool \| None
cli_max_retries	int \| None
strict_model_resolution	bool
environ	Mapping[str, str] \| None
model_result	ModelResult \| None

Swap the model or per-call settings from runtime.context.

Reads two optional keys from the runtime context dict:

'model' — a provider:model spec (e.g. "openai:gpt-5"). When present and different from the current model, the request is re-routed to the new model.
'model_params' — a dict of extra model settings (e.g. {"temperature": 0}) that are shallow-merged into the request's model_settings.

This middleware is typically the outermost layer so it intercepts every model call before provider-specific middleware (like AnthropicPromptCachingMiddleware) runs.

Explicit --max-retries value to retain across runtime model switches.

Whether invalid runtime model overrides should fail the call instead of falling back to the construction-time model.

Workspace environment retained for lazy model switches.

Construction-time workspace model metadata.