prompt_cache

Configure the global prompt cache.

This should be called before any cache instances are created or used.

Configure the global prompt cache.

This should be called before any cache instances are created or used.

A single cache entry with metadata for TTL tracking.

Cache performance metrics.

Thread-safe LRU cache with background thread refresh.

For use with the synchronous Client.

Features:

• In-memory LRU cache with configurable max size
• Background thread for refreshing stale entries
• Stale-while-revalidate: returns stale data while refresh happens
• Thread-safe for concurrent access

Thread-safe LRU cache with asyncio task refresh.

For use with the asynchronous AsyncClient.

Features:

• In-memory LRU cache with configurable max size
• Asyncio task for refreshing stale entries
• Stale-while-revalidate: returns stale data while refresh happens
• Thread-safe for concurrent access

Deprecated alias for PromptCache. Use PromptCache instead.

Deprecated alias for AsyncPromptCache. Use AsyncPromptCache instead.