Represents a Semantic Cache that uses CosmosDB MongoDB backend as the underlying storage system.
class AzureCosmosDBMongoDBSemanticCachedeletes the semantic cache for a given llmKey
Retrieves data from the cache.
Sets a custom key encoder function for the cache. This function should take a prompt and an LLM key and return a string that will be used as the cache key.
Updates the cache with new data.
const embeddings = new OpenAIEmbeddings();
const cache = new AzureCosmosDBMongoDBSemanticCache(embeddings, {
client?: MongoClient
});
const model = new ChatOpenAI({ model: "gpt-4o-mini", cache });
// Invoke the model to perform an action
const response = await model.invoke("Do something random!");
console.log(response);