Fake LLMs for testing purposes.
Async callback manager for LLM run.
Callback manager for LLM run.
Simple interface for implementing a custom LLM.
You should subclass this class and implement the following:
_call method: Run the LLM on the given prompt and input (used by invoke)._identifying_params property: Return a dictionary of the identifying parameters
This is critical for caching and tracing purposes. Identifying parameters
is a dict that identifies the LLM.
It should mostly include a model_name.Optional: Override the following methods to provide more optimizations:
_acall: Provide a native async version of the _call method.
If not provided, will delegate to the synchronous version using
run_in_executor. (Used by ainvoke)._stream: Stream the LLM on the given prompt and input.
stream will use _stream if provided, otherwise it
use _call and output will arrive in one chunk._astream: Override to provide a native async version of the _stream method.
astream will use _astream if provided, otherwise it will implement
a fallback behavior that will use _stream if _stream is implemented,
and use _acall if _stream is not implemented.Configuration for a Runnable.
Custom values
The TypedDict has total=False set intentionally to:
merge_configsvar_child_runnable_config (a ContextVar that automatically passes
config down the call stack without explicit parameter passing), where
configs are merged rather than replaced# Parent sets tags
chain.invoke(input, config={"tags": ["parent"]})
# Child automatically inherits and can add:
# ensure_config({"tags": ["child"]}) -> {"tags": ["parent", "child"]}Fake LLM for testing purposes.
Fake error for testing purposes.
Fake streaming list LLM for testing purposes.
An LLM that will return responses from a list in order.
This model also supports optionally sleeping between successive chunks in a streaming implementation.