A cache that uses Upstash as the backing store. See https://docs.upstash.com/redis.
Lookup LLM generations in cache by prompt and associated LLM key.
Sets a custom key encoder function for the cache. This function should take a prompt and an LLM key and return a string that will be used as the cache key.
Update the cache with the given generations.
Note this overwrites any existing generations for the given prompt and LLM key.
const cache = new UpstashRedisCache({
config: {
url: "UPSTASH_REDIS_REST_URL",
token: "UPSTASH_REDIS_REST_TOKEN",
},
ttl: 3600, // Optional: Cache entries will expire after 1 hour
});
// Initialize the OpenAI model with Upstash Redis cache for caching responses
const model = new ChatOpenAI({
model: "gpt-4o-mini",
cache,
});
await model.invoke("How are you today?");
const cachedValues = await cache.lookup("How are you today?", "llmKey");