Prompt Caching Middleware.
Optimizes API usage by caching conversation prefixes for Anthropic models.
Requires both langchain and langchain-anthropic packages to be installed.
Learn more about Anthropic prompt caching here.
AnthropicPromptCachingMiddleware(
self,
type: Literal['ephemeral'] = 'ephemeral',
ttl: Literal['5m', '1h'] = '5m',
min_messages_to_cache: int = 0,
unsupported_model_behavior: Literal['ignore', 'warn', 'raise'] = 'warn'
)| Name | Type | Description |
|---|---|---|
type | Literal['ephemeral'] | Default: 'ephemeral'The type of cache to use, only |
ttl | Literal['5m', '1h'] | Default: '5m'The time to live for the cache, only |
min_messages_to_cache | int | Default: 0The minimum number of messages until the cache is used. |
unsupported_model_behavior | Literal['ignore', 'warn', 'raise'] | Default: 'warn'The behavior to take when an unsupported model is used.
|