Represents a specific implementation of a caching mechanism using Cloudflare KV
as the underlying storage system. It extends the BaseCache class and
overrides its methods to provide the Cloudflare KV-specific logic.
class CloudflareKVCacheRetrieves data from the cache. It constructs a cache key from the given
prompt and llmKey, and retrieves the corresponding value from the
Cloudflare KV namespace.
Sets a custom key encoder function for the cache. This function should take a prompt and an LLM key and return a string that will be used as the cache key.
Updates the cache with new data. It constructs a cache key from the
given prompt and llmKey, and stores the value in the Cloudflare KV
namespace.
// Example of using OpenAI with Cloudflare KV as cache in a Cloudflare Worker
const cache = new CloudflareKVCache(env.KV_NAMESPACE);
const model = new ChatAnthropic({
cache,
});
const response = await model.invoke("How are you today?");
return new Response(JSON.stringify(response), {
headers: { "content-type": "application/json" },
});