ChatParallelWeb

Setup:

Install langchain-parallel and set environment variable PARALLEL_API_KEY.

pip install -U langchain-parallel
export PARALLEL_API_KEY="your-api-key"

Key init args — completion params: model: str Name of Parallel Web model to use. Defaults to "speed". temperature: Optional[float] Sampling temperature (ignored by Parallel). max_tokens: Optional[int] Max number of tokens to generate (ignored by Parallel).

Key init args — client params: timeout: Optional[float] Timeout for requests. max_retries: int Max number of retries. api_key: Optional[str] Parallel API key. If not passed in will be read from env var PARALLEL_API_KEY. base_url: str Base URL for Parallel API. Defaults to "https://api.parallel.ai".

Instantiate:

from langchain_parallel import ChatParallelWeb

llm = ChatParallelWeb(
    model="speed",
    temperature=0.7,
    max_tokens=None,
    timeout=None,
    max_retries=2,
    # api_key="...",
    # other params...
)

Invoke:

messages = [
    (
        "system",
        "You are a helpful assistant with access to real-time web "
        "information."
    ),
    ("human", "What are the latest developments in AI?"),
]
llm.invoke(messages)

Stream:

for chunk in llm.stream(messages):
    print(chunk.content, end="")

Async:

await llm.ainvoke(messages)

# stream:
async for chunk in llm.astream(messages):
    print(chunk.content, end="")

# batch:
await llm.abatch([messages])

Token usage:

ai_msg = llm.invoke(messages)
ai_msg.usage_metadata

Response metadata:

ai_msg = llm.invoke(messages)
ai_msg.response_metadata