LRU cache with background refresh for prompts.
Features:
class PromptCacheconst cache = new Cache({
maxSize: 100,
ttlSeconds: 3600,
});
// Use the cache
cache.set("my-prompt:latest", promptCommit);
const cached = cache.get("my-prompt:latest");
// Cleanup
cache.stop();Storage size (e.g., "1Gi", "10Gi").
Clear all cache entries.
Dump cache contents to a JSON file for offline use.
Get a value from cache.
Returns the cached value or undefined if not found. Stale entries are still returned (background refresh handles updates).
Remove a specific entry from cache.
Load cache contents from a JSON file.
Loaded entries get a fresh TTL starting from load time.
Reset all metrics to zero.
Set a value in the cache.
Start the background refresh loop.
Stop background refresh. Should be called when the client is being cleaned up.