Build extra kwargs from additional params that were passed in.
Get a list of available models that work with ChatNVIDIA.
Bind tools to the model.
Bind a structured output schema to the model.
Configure the model to use thinking mode.
| Name | Type |
|---|---|
| model | Optional[str] |
| nvidia_api_key | Optional[str] |
| api_key | Optional[str] |
| base_url | Optional[str] |
| temperature | Optional[float] |
| max_completion_tokens | Optional[int] |
| top_p | Optional[float] |
| seed | Optional[int] |
| stop | Optional[Union[str, list[str]]] |
| default_headers | Optional[dict[str, str]] |
| osl | int |
| iat | int |
| latency_sensitivity | float |
| priority | int |
ChatNVIDIA subclass that injects nvext.agent_hints into requests
for Dynamo KV cache routing optimization.
A unique prefix_id is auto-generated for every request.
Example:
from langchain_nvidia_ai_endpoints import ChatNVIDIADynamo
llm = ChatNVIDIADynamo(model="meta/llama3-8b-instruct")
# override per-invocation:
llm.invoke("Hello", osl=2048, iat=50)