Ask a question to get started
Enter to send•Shift+Enter new line
ChatLiteLLMRouter( self, *, router: Any, kwargs: Any = {}, )
ChatLiteLLM
Model name to use.
Run inference with this temperature. Must be in the closed
Holds any model parameters valid for API call not explicitly specified.
Decode using nucleus sampling: consider the smallest set of tokens whose
Decode using top-k sampling: consider the set of top_k most probable tokens.
Number of chat completions to generate for each prompt. Note that the API may
The maximum number of tokens to generate in the reply.
Context window size (e.g. for Ollama models).
Use tenacity to retry the completion call.
Use tenacity to retry the async completion call.
Validate api key, python package exists, temperature, top_p, and top_k.
Bind tool-like objects to this chat model.
Set the default model to use for completion calls.
Sets self.model to model_name if it is in the litellm router's (self.router) model list. This provides the default model to use for completion calls if no model kwarg is provided.
self.model
model_name
self.router
model
LiteLLM Router as LangChain Model.
