CassandraSemanticCache(
self,
session: Optional[CassandraSession] = None,
keyspace: Optional| Name | Type | Description |
|---|---|---|
session | Optional[CassandraSession] | Default: Nonean open Cassandra session. Leave unspecified to use the global cassio init (see below) |
keyspace | Optional[str] | Default: Nonethe keyspace to use for storing the cache. Leave unspecified to use the global cassio init (see below) |
embedding | Optional[Embeddings] | Default: None |
table_name | str | Default: CASSANDRA_SEMANTIC_CACHE_DEFAULT_TABLE_NAME |
distance_metric | Optional[str] | Default: None |
score_threshold | float | Default: CASSANDRA_SEMANTIC_CACHE_DEFAULT_SCORE_THRESHOLD |
ttl_seconds | Optional[int] | Default: CASSANDRA_SEMANTIC_CACHE_DEFAULT_TTL_SECONDS |
similarity_measure | str | Default: CASSANDRA_SEMANTIC_CACHE_DEFAULT_DISTANCE_METRIC |
setup_mode | CassandraSetupMode | Default: CassandraSetupMode.SYNC |
Cache that uses Cassandra as a vector-store backend for semantic (i.e. similarity-based) lookup.
Example:
.. code-block:: python
import cassio
from langchain_community.cache import CassandraSemanticCache
from langchain_core.globals import set_llm_cache
cassio.init(auto=True) # Requires env. variables, see CassIO docs
my_embedding = ...
set_llm_cache(CassandraSemanticCache(
embedding=my_embedding,
table_name="my_semantic_cache",
))
It uses a single (vector) Cassandra table and stores, in principle, cached values from several LLMs, so the LLM's llm_string is part of the rows' primary keys.
One can choose a similarity measure (default: "dot" for dot-product). Choosing another one ("cos", "l2") almost certainly requires threshold tuning. (which may be in order nevertheless, even if sticking to "dot").
Note:
The session and keyspace parameters, when left out (or passed as None), fall back to the globally-available cassio settings if any are available. In other words, if a previously-run 'cassio.init(...)' has been executed previously anywhere in the code, Cassandra-based objects need not specify the connection parameters at all.
Embedding provider for semantic encoding and search.
name of the Cassandra (vector) table to use as cache. There is a default for "simple" usage, but remember to explicitly specify different tables if several embedding models coexist in your app (they cannot share one cache table).
an alias for the 'similarity_measure' parameter (see below). As the "distance" terminology is misleading, please prefer 'similarity_measure' for clarity.
numeric value to use as cutoff for the similarity searches
time-to-live for cache entries (default: None, i.e. forever)
which measure to adopt for similarity searches. Note: this parameter is aliased by 'distance_metric' - however, it is suggested to use the "similarity" terminology since this value is in fact a similarity (i.e. higher means closer). Note that at most one of the two parameters 'distance_metric' and 'similarity_measure' can be provided.
a value in langchain_community.utilities.cassandra.SetupMode. Choose between SYNC, ASYNC and OFF - the latter if the Cassandra table is guaranteed to exist already, for a faster initialization.