Class●Since v0.7

PromptCache

Thread-safe LRU cache with background thread refresh.

For use with the synchronous Client.

Features:

In-memory LRU cache with configurable max size
Background thread for refreshing stale entries
Stale-while-revalidate: returns stale data while refresh happens
Thread-safe for concurrent access

PromptCache(
  self,
  *,
  max_size: int = DEFAULT_PROMPT_CACHE_MAX_SIZE,
  ttl_seconds: Optional[float] = DEFAULT_PROMPT_CACHE_TTL_SECONDS,
  refresh_interval_seconds: float = DEFAULT_PROMPT_CACHE_REFRESH_INTERVAL_SECONDS
)

Bases

_BasePromptCache

Example:

def fetch_prompt(key: str) -> PromptCommit: ... return client._fetch_prompt_from_api(key) cache = PromptCache( ... max_size=100, ... ttl_seconds=3600, ... fetch_func=fetch_prompt, ... ) cache.set("my-prompt:latest", prompt_commit) cached = cache.get("my-prompt:latest") cache.shutdown()

Parameters

Name	Type	Description
`max_size`	`int`	Default:`DEFAULT_PROMPT_CACHE_MAX_SIZE` Maximum entries in cache (LRU eviction when exceeded).
`ttl_seconds`	`Optional[float]`	Default:`DEFAULT_PROMPT_CACHE_TTL_SECONDS` Time before entry is considered stale. Set to None for infinite TTL (offline mode - entries never expire). Default: 300 (5 minutes).
`refresh_interval_seconds`	`float`	Default:`DEFAULT_PROMPT_CACHE_REFRESH_INTERVAL_SECONDS` How often to check for stale entries.