Interface for a caching layer for LLMs and Chat models.
The cache interface consists of the following methods:
llm_string.llm_string.In addition, the cache interface provides an async version of each method.
The default implementation of the async methods is to run the synchronous method in an executor. It's recommended to override the async methods and provide async implementations to avoid unnecessary overhead.