# ChatNVIDIADynamo

> **Class** in `langchain_nvidia_ai_endpoints`

📖 [View in docs](https://reference.langchain.com/python/langchain-nvidia-ai-endpoints/chat_models_dynamo/ChatNVIDIADynamo)

ChatNVIDIA subclass that injects `nvext.agent_hints` into requests
for Dynamo KV cache routing optimization.

A unique `prefix_id` is auto-generated for every request.

## Signature

```python
ChatNVIDIADynamo(
    self,
    *,
    model: Optional[str] = None,
    nvidia_api_key: Optional[str] = None,
    api_key: Optional[str] = None,
    base_url: Optional[str] = None,
    temperature: Optional[float] = None,
    max_completion_tokens: Optional[int] = None,
    top_p: Optional[float] = None,
    seed: Optional[int] = None,
    stop: Optional[Union[str, list[str]]] = None,
    default_headers: Optional[dict[str, str]] = None,
    osl: int = 512,
    iat: int = 250,
    latency_sensitivity: float = 1.0,
    priority: int = 1,
    **kwargs: Any = {},
)
```

## Description

**Example:**

```python
from langchain_nvidia_ai_endpoints import ChatNVIDIADynamo

llm = ChatNVIDIADynamo(model="meta/llama3-8b-instruct")
# override per-invocation:
llm.invoke("Hello", osl=2048, iat=50)
```

## Extends

- `ChatNVIDIA`

## Constructors

```python
__init__(
    self,
    *,
    model: Optional[str] = None,
    nvidia_api_key: Optional[str] = None,
    api_key: Optional[str] = None,
    base_url: Optional[str] = None,
    temperature: Optional[float] = None,
    max_completion_tokens: Optional[int] = None,
    top_p: Optional[float] = None,
    seed: Optional[int] = None,
    stop: Optional[Union[str, list[str]]] = None,
    default_headers: Optional[dict[str, str]] = None,
    osl: int = 512,
    iat: int = 250,
    latency_sensitivity: float = 1.0,
    priority: int = 1,
    **kwargs: Any = {},
) -> None
```

| Name | Type |
|------|------|
| `model` | `Optional[str]` |
| `nvidia_api_key` | `Optional[str]` |
| `api_key` | `Optional[str]` |
| `base_url` | `Optional[str]` |
| `temperature` | `Optional[float]` |
| `max_completion_tokens` | `Optional[int]` |
| `top_p` | `Optional[float]` |
| `seed` | `Optional[int]` |
| `stop` | `Optional[Union[str, list[str]]]` |
| `default_headers` | `Optional[dict[str, str]]` |
| `osl` | `int` |
| `iat` | `int` |
| `latency_sensitivity` | `float` |
| `priority` | `int` |


## Properties

- `osl`
- `iat`
- `latency_sensitivity`
- `priority`

---

[View source on GitHub](https://github.com/langchain-ai/langchain-nvidia/blob/025f99f43bf1d1dadc37798e6654d7d132c040cd/libs/ai-endpoints/langchain_nvidia_ai_endpoints/chat_models_dynamo.py#L14)