# ChatTogether

> **Class** in `langchain_together`

📖 [View in docs](https://reference.langchain.com/python/langchain-together/chat_models/ChatTogether)

ChatTogether chat model.

## Signature

```python
ChatTogether()
```

## Description

**Setup:**

Install ``langchain-together`` and set environment variable ``TOGETHER_API_KEY``.

.. code-block:: bash

    pip install -U langchain-together
    export TOGETHER_API_KEY="your-api-key"

Key init args — completion params:
    model: str
        Name of model to use.
    temperature: float
        Sampling temperature.
    max_tokens: Optional[int]
        Max number of tokens to generate.
    logprobs: Optional[bool]
        Whether to return logprobs.

Key init args — client params:
    timeout: Union[float, Tuple[float, float], Any, None]
        Timeout for requests.
    max_retries: int
        Max number of retries.
    api_key: Optional[str]
        Together API key. If not passed in will be read from env var OPENAI_API_KEY.

**Instantiate:**

.. code-block:: python

from langchain_together import ChatTogether

llm = ChatTogether(
    model="meta-llama/Llama-3-70b-chat-hf",
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2,
    # api_key="...",
    # other params...
)

**Invoke:**

.. code-block:: python

    messages = [
        (
            "system",
            "You are a helpful translator. Translate the user sentence to French.",
        ),
        ("human", "I love programming."),
    ]
    llm.invoke(messages)

.. code-block:: python

    AIMessage(
        content="J'adore la programmation.",
        response_metadata={
            'token_usage': {'completion_tokens': 9, 'prompt_tokens': 32, 'total_tokens': 41},
            'model_name': 'meta-llama/Llama-3-70b-chat-hf',
            'system_fingerprint': None,
            'finish_reason': 'stop',
            'logprobs': None
        },
        id='run-168dceca-3b8b-4283-94e3-4c739dbc1525-0',
        usage_metadata={'input_tokens': 32, 'output_tokens': 9, 'total_tokens': 41})

**Stream:**

.. code-block:: python

    for chunk in llm.stream(messages):
        print(chunk)

.. code-block:: python

    content='J' id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'
    content="'" id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'
    content='ad' id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'
    content='ore' id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'
    content=' la' id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'
    content=' programm' id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'
    content='ation' id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'
    content='.' id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'
    content='' response_metadata={'finish_reason': 'stop', 'model_name': 'meta-llama/Llama-3-70b-chat-hf'} id='run-1bc996b5-293f-4114-96a1-e0f755c05eb9'

**Async:**

.. code-block:: python

    await llm.ainvoke(messages)

    # stream:
    # async for chunk in (await llm.astream(messages))

    # batch:
    # await llm.abatch([messages])

.. code-block:: python

    AIMessage(
        content="J'adore la programmation.",
        response_metadata={
            'token_usage': {'completion_tokens': 9, 'prompt_tokens': 32, 'total_tokens': 41},
            'model_name': 'meta-llama/Llama-3-70b-chat-hf',
            'system_fingerprint': None,
            'finish_reason': 'stop',
            'logprobs': None
        },
        id='run-09371a11-7f72-4c53-8e7c-9de5c238b34c-0',
        usage_metadata={'input_tokens': 32, 'output_tokens': 9, 'total_tokens': 41})

**Tool calling:**

.. code-block:: python

    from pydantic import BaseModel, Field

    # Only certain models support tool calling, check the together website to confirm compatibility
    llm = ChatTogether(model="mistralai/Mixtral-8x7B-Instruct-v0.1")

    class GetWeather(BaseModel):
        '''Get the current weather in a given location'''

        location: str = Field(
            ..., description="The city and state, e.g. San Francisco, CA"
        )

    class GetPopulation(BaseModel):
        '''Get the current population in a given location'''

        location: str = Field(
            ..., description="The city and state, e.g. San Francisco, CA"
        )

    llm_with_tools = llm.bind_tools([GetWeather, GetPopulation])
    ai_msg = llm_with_tools.invoke(
        "Which city is bigger: LA or NY?"
    )
    ai_msg.tool_calls

.. code-block:: python

    [
        {
            'name': 'GetPopulation',
            'args': {'location': 'NY'},
            'id': 'call_m5tstyn2004pre9bfuxvom8x',
            'type': 'tool_call'
        },
        {
            'name': 'GetPopulation',
            'args': {'location': 'LA'},
            'id': 'call_0vjgq455gq1av5sp9eb1pw6a',
            'type': 'tool_call'
        }
    ]

**Structured output:**

.. code-block:: python

    from typing import Optional

    from pydantic import BaseModel, Field

    class Joke(BaseModel):
        '''Joke to tell user.'''

        setup: str = Field(description="The setup of the joke")
        punchline: str = Field(description="The punchline to the joke")
        rating: Optional[int] = Field(description="How funny the joke is, from 1 to 10")

    structured_llm = llm.with_structured_output(Joke)
    structured_llm.invoke("Tell me a joke about cats")

.. code-block:: python

    Joke(
        setup='Why was the cat sitting on the computer?',
        punchline='To keep an eye on the mouse!',
        rating=7
    )

**JSON mode:**

.. code-block:: python

    json_llm = llm.bind(response_format={"type": "json_object"})
    ai_msg = json_llm.invoke(
        "Return a JSON object with key 'random_ints' and a value of 10 random ints in [0-99]"
    )
    ai_msg.content

.. code-block:: python

    ' {\\n"random_ints": [\\n13,\\n54,\\n78,\\n45,\\n67,\\n90,\\n11,\\n29,\\n84,\\n33\\n]\\n}'

**Token usage:**

.. code-block:: python

    ai_msg = llm.invoke(messages)
    ai_msg.usage_metadata

.. code-block:: python

    {'input_tokens': 37, 'output_tokens': 6, 'total_tokens': 43}

**Logprobs:**

.. code-block:: python

    logprobs_llm = llm.bind(logprobs=True)
    messages=[("human","Say Hello World! Do not return anything else.")]
    ai_msg = logprobs_llm.invoke(messages)
    ai_msg.response_metadata["logprobs"]

.. code-block:: python

    {
        'content': None,
        'token_ids': [22557, 3304, 28808, 2],
        'tokens': [' Hello', ' World', '!', '</s>'],
        'token_logprobs': [-4.7683716e-06, -5.9604645e-07, 0, -0.057373047]
    }

Response metadata
.. code-block:: python

    ai_msg = llm.invoke(messages)
    ai_msg.response_metadata

.. code-block:: python

    {
        'token_usage': {
            'completion_tokens': 4,
            'prompt_tokens': 19,
            'total_tokens': 23
            },
        'model_name': 'mistralai/Mixtral-8x7B-Instruct-v0.1',
        'system_fingerprint': None,
        'finish_reason': 'eos',
        'logprobs': None
    }

## Extends

- `BaseChatOpenAI`

## Properties

- `lc_secrets`
- `lc_attributes`
- `model_name`
- `together_api_key`
- `together_api_base`
- `model_config`

## Methods

- [`get_lc_namespace()`](https://reference.langchain.com/python/langchain-together/chat_models/ChatTogether/get_lc_namespace)
- [`validate_environment()`](https://reference.langchain.com/python/langchain-together/chat_models/ChatTogether/validate_environment)

---

[View source on GitHub](https://github.com/langchain-ai/langchain-together/blob/1a6545d022497fbef3b414258a985d96b02a1963/libs/together/langchain_together/chat_models.py#L18)