globals

Set a new value for the verbose global setting.

Get the value of the verbose global setting.

Set a new value for the debug global setting.

Get the value of the debug global setting.

Set a new LLM cache, overwriting the previous value, if any.

Get the value of the llm_cache global setting.

Interface for a caching layer for LLMs and Chat models.

The cache interface consists of the following methods:

• lookup: Look up a value based on a prompt and llm_string.
• update: Update the cache based on a prompt and llm_string.
• clear: Clear the cache.

In addition, the cache interface provides an async version of each method.

The default implementation of the async methods is to run the synchronous method in an executor. It's recommended to override the async methods and provide async implementations to avoid unnecessary overhead.