`ChatWatsonx`¶

Reference docs

This page contains reference documentation for ChatWatsonx. See the docs for conceptual guides, tutorials, and examples on using ChatWatsonx.

langchain_ibm.chat_models.ChatWatsonx ¶

Bases: BaseChatModel

IBM watsonx.ai chat models integration.

Setup

To use, you should have langchain_ibm python package installed, and the environment variable WATSONX_API_KEY set with your API key, or pass it as a named parameter api_key to the constructor.

pip install -U langchain-ibm

# or using uv
uv add langchain-ibm

export WATSONX_API_KEY="your-api-key"

Deprecated

apikey and WATSONX_APIKEY are deprecated and will be removed in version 2.0.0. Use api_key and WATSONX_API_KEY instead.

Instantiate

Create a model instance with desired params. For example:

from langchain_ibm import ChatWatsonx
from ibm_watsonx_ai.foundation_models.schema import TextChatParameters

parameters = TextChatParameters(
    top_p=1, temperature=0.5, max_completion_tokens=None
)

model = ChatWatsonx(
    model_id="meta-llama/llama-3-3-70b-instruct",
    url="https://us-south.ml.cloud.ibm.com",
    project_id="*****",
    params=parameters,
    # api_key="*****"
)

Invoke

Generate a response from the model:

messages = [
    (
        "system",
        "You are a helpful translator. Translate the user sentence to French.",
    ),
    ("human", "I love programming."),
]
model.invoke(messages)

Results in an AIMessage response:

AIMessage(
    content="J'adore programmer.",
    additional_kwargs={},
    response_metadata={
        "token_usage": {
            "completion_tokens": 7,
            "prompt_tokens": 30,
            "total_tokens": 37,
        },
        "model_name": "ibm/granite-3-3-8b-instruct",
        "system_fingerprint": "",
        "finish_reason": "stop",
    },
    id="chatcmpl-529352c4-93ba-4801-8f1d-a3b4e3935194---daed91fb74d0405f200db1e63da9a48a---7a3ef799-4413-47e4-b24c-85d267e37fa2",
    usage_metadata={"input_tokens": 30, "output_tokens": 7, "total_tokens": 37},
)

Stream

Stream a response from the model:

for chunk in model.stream(messages):
    print(chunk.text)

Results in a sequence of AIMessageChunk objects with partial content:

AIMessageChunk(content="", id="run--e48a38d3-1500-4b5e-870c-6313e8cff775")
AIMessageChunk(content="J", id="run--e48a38d3-1500-4b5e-870c-6313e8cff775")
AIMessageChunk(content="'", id="run--e48a38d3-1500-4b5e-870c-6313e8cff775")
AIMessageChunk(content="ad", id="run--e48a38d3-1500-4b5e-870c-6313e8cff775")
AIMessageChunk(content="or", id="run--e48a38d3-1500-4b5e-870c-6313e8cff775")
AIMessageChunk(
    content=" programmer", id="run--e48a38d3-1500-4b5e-870c-6313e8cff775"
)
AIMessageChunk(content=".", id="run--e48a38d3-1500-4b5e-870c-6313e8cff775")
AIMessageChunk(
    content="",
    response_metadata={
        "finish_reason": "stop",
        "model_name": "ibm/granite-3-3-8b-instruct",
    },
    id="run--e48a38d3-1500-4b5e-870c-6313e8cff775",
)
AIMessageChunk(
    content="",
    id="run--e48a38d3-1500-4b5e-870c-6313e8cff775",
    usage_metadata={"input_tokens": 30, "output_tokens": 7, "total_tokens": 37},
)

To collect the full message, you can concatenate the chunks:

stream = model.stream(messages)
full = next(stream)
for chunk in stream:
    full += chunk

full

AIMessageChunk(
    content="J'adore programmer.",
    response_metadata={
        "finish_reason": "stop",
        "model_name": "ibm/granite-3-3-8b-instruct",
    },
    id="chatcmpl-88a48b71-c149-4a0c-9c02-d6b97ca5dc6c---b7ba15879a8c5283b1e8a3b8db0229f0---0037ca4f-8a74-4f84-a46c-ab3fd1294f24",
    usage_metadata={"input_tokens": 30, "output_tokens": 7, "total_tokens": 37},
)

Async

Asynchronous equivalents of invoke, stream, and batch are also available:

# Invoke
await model.ainvoke(messages)

# Stream
async for chunk in model.astream(messages):
    print(chunk.text)

# Batch
await model.abatch([messages])

Results in an AIMessage response:

AIMessage(
    content="J'adore programmer.",
    additional_kwargs={},
    response_metadata={
        "token_usage": {
            "completion_tokens": 7,
            "prompt_tokens": 30,
            "total_tokens": 37,
        },
        "model_name": "ibm/granite-3-3-8b-instruct",
        "system_fingerprint": "",
        "finish_reason": "stop",
    },
    id="chatcmpl-5bef2d81-ef56-463b-a8fa-c2bcc2a3c348---821e7750d18925f2b36226db66667e26---6396c786-9da9-4468-883e-11ed90a05937",
    usage_metadata={"input_tokens": 30, "output_tokens": 7, "total_tokens": 37},
)

For batched calls, results in a list[AIMessage].

Tool calling

from pydantic import BaseModel, Field


class GetWeather(BaseModel):
    '''Get the current weather in a given location'''

    location: str = Field(
        ..., description="The city and state, e.g. San Francisco, CA"
    )


class GetPopulation(BaseModel):
    '''Get the current population in a given location'''

    location: str = Field(
        ..., description="The city and state, e.g. San Francisco, CA"
    )


model_with_tools = model.bind_tools(
    [GetWeather, GetPopulation]
    # strict = True  # Enforce tool args schema is respected
)
ai_msg = model_with_tools.invoke(
    "Which city is hotter today and which is bigger: LA or NY?"
)
ai_msg.tool_calls

[
    {
        "name": "GetWeather",
        "args": {"location": "Los Angeles, CA"},
        "id": "chatcmpl-tool-59632abcee8f48a18a5f3a81422b917b",
        "type": "tool_call",
    },
    {
        "name": "GetWeather",
        "args": {"location": "New York, NY"},
        "id": "chatcmpl-tool-c6f3b033b4594918bb53f656525b0979",
        "type": "tool_call",
    },
    {
        "name": "GetPopulation",
        "args": {"location": "Los Angeles, CA"},
        "id": "chatcmpl-tool-175a23281e4747ea81cbe472b8e47012",
        "type": "tool_call",
    },
    {
        "name": "GetPopulation",
        "args": {"location": "New York, NY"},
        "id": "chatcmpl-tool-e1ccc534835945aebab708eb5e685bf7",
        "type": "tool_call",
    },
]

Reasoning output

from langchain_ibm import ChatWatsonx
from ibm_watsonx_ai.foundation_models.schema import TextChatParameters

parameters = TextChatParameters(
    include_reasoning=True, reasoning_effort="medium"
)

model = ChatWatsonx(
    model_id="openai/gpt-oss-120b",
    url="https://us-south.ml.cloud.ibm.com",
    project_id="*****",
    params=parameters,
    # api_key="*****"
)

response = model.invoke("What is 3^3?")

# Response text
print(f"Output: {response.content}")

# Reasoning summaries
print(f"Reasoning: {response.additional_kwargs['reasoning_content']}")

Output: 3^3 = 27
Reasoning: The user asks "What is 3^3?" That's 27. Provide answer.

Added in version 0.3.19: Updated AIMessage format

langchain-ibm >= 0.3.19 allows users to set Reasoning output parameters and will format output from reasoning summaries into additional_kwargs field.

Structured output

from pydantic import BaseModel, Field


class Joke(BaseModel):
    '''Joke to tell user.'''

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: int | None = Field(description="How funny the joke is, 1 to 10")


structured_model = model.with_structured_output(Joke)
structured_model.invoke("Tell me a joke about cats")

Joke(
    setup="Why was the cat sitting on the computer?",
    punchline="To keep an eye on the mouse!",
    rating=None,
)

See with_structured_output for more info.

JSON mode

json_model = model.bind(response_format={"type": "json_object"})
ai_msg = json_model.invoke(
    “Return JSON with 'random_ints': an array of 10 random integers from 0-99.”
)
ai_msg.content

'{\n  "random_ints": [12, 34, 56, 78, 10, 22, 44, 66, 88, 99]\n}'

Image input

import base64
import httpx
from langchain.messages import HumanMessage

image_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
image_data = base64.b64encode(httpx.get(image_url).content).decode("utf-8")
message = HumanMessage(
    content=[
        {"type": "text", "text": "describe the weather in this image"},
        {
            "type": "image_url",
            "image_url": {"url": f"data:image/jpeg;base64,{image_data}"},
        },
    ]
)

ai_msg = model.invoke([message])
ai_msg.content

"The weather in the image presents a clear, sunny day with good visibility
and no immediate signs of rain or strong winds. The vibrant blue sky with
scattered white clouds gives the impression of a tranquil, pleasant day
conducive to outdoor activities."

Token usage

ai_msg = model.invoke(messages)
ai_msg.usage_metadata

{'input_tokens': 30, 'output_tokens': 7, 'total_tokens': 37}

stream = model.stream(messages)
full = next(stream)
for chunk in stream:
    full += chunk
full.usage_metadata

{'input_tokens': 30, 'output_tokens': 7, 'total_tokens': 37}

Logprobs

logprobs_model = model.bind(logprobs=True)
ai_msg = logprobs_model.invoke(messages)
ai_msg.response_metadata["logprobs"]

{
    'content': [
        {
            'token': 'J',
            'logprob': -0.0017940393
        },
        {
            'token': "'",
            'logprob': -1.7523613e-05
        },
        {
            'token': 'ad',
            'logprob': -0.16112353
        },
        {
            'token': 'ore',
            'logprob': -0.0003091811
        },
        {
            'token': ' programmer',
            'logprob': -0.24849245
        },
        {
            'token': '.',
            'logprob': -2.5033638e-05
        },
        {
            'token': '<|end_of_text|>',
            'logprob': -7.080781e-05
        }
    ]
}

Response metadata

ai_msg = model.invoke(messages)
ai_msg.response_metadata

{
    'token_usage': {
        'completion_tokens': 7,
        'prompt_tokens': 30,
        'total_tokens': 37
    },
    'model_name': 'ibm/granite-3-3-8b-instruct',
    'system_fingerprint': '',
    'finish_reason': 'stop'
}

METHOD	DESCRIPTION
`is_lc_serializable`	Is lc serializable.
`validate_environment`	Validate that credentials and python package exists in environment.
`bind_tools`	Bind tool-like objects to this chat model.
`with_structured_output`	Model wrapper that returns outputs formatted to match the given schema.
`get_name`	Get the name of the `Runnable`.
`get_input_schema`	Get a Pydantic model that can be used to validate input to the `Runnable`.
`get_input_jsonschema`	Get a JSON schema that represents the input to the `Runnable`.
`get_output_schema`	Get a Pydantic model that can be used to validate output to the `Runnable`.
`get_output_jsonschema`	Get a JSON schema that represents the output of the `Runnable`.
`config_schema`	The type of config this `Runnable` accepts specified as a Pydantic model.
`get_config_jsonschema`	Get a JSON schema that represents the config of the `Runnable`.
`get_graph`	Return a graph representation of this `Runnable`.
`get_prompts`	Return a list of prompts used by this `Runnable`.
`__or__`	Runnable "or" operator.
`__ror__`	Runnable "reverse-or" operator.
`pipe`	Pipe `Runnable` objects.
`pick`	Pick keys from the output `dict` of this `Runnable`.
`assign`	Assigns new fields to the `dict` output of this `Runnable`.
`invoke`	Transform a single input into an output.
`ainvoke`	Transform a single input into an output.
`batch`	Default implementation runs invoke in parallel using a thread pool executor.
`batch_as_completed`	Run `invoke` in parallel on a list of inputs.
`abatch`	Default implementation runs `ainvoke` in parallel using `asyncio.gather`.
`abatch_as_completed`	Run `ainvoke` in parallel on a list of inputs.
`stream`	Default implementation of `stream`, which calls `invoke`.
`astream`	Default implementation of `astream`, which calls `ainvoke`.
`astream_log`	Stream all output from a `Runnable`, as reported to the callback system.
`astream_events`	Generate a stream of events.
`transform`	Transform inputs to outputs.
`atransform`	Transform inputs to outputs.
`bind`	Bind arguments to a `Runnable`, returning a new `Runnable`.
`with_config`	Bind config to a `Runnable`, returning a new `Runnable`.
`with_listeners`	Bind lifecycle listeners to a `Runnable`, returning a new `Runnable`.
`with_alisteners`	Bind async lifecycle listeners to a `Runnable`.
`with_types`	Bind input and output types to a `Runnable`, returning a new `Runnable`.
`with_retry`	Create a new `Runnable` that retries the original `Runnable` on exceptions.
`map`	Return a new `Runnable` that maps a list of inputs to a list of outputs.
`with_fallbacks`	Add fallbacks to a `Runnable`, returning a new `Runnable`.
`as_tool`	Create a `BaseTool` from a `Runnable`.
`__init__`
`get_lc_namespace`	Get the namespace of the LangChain object.
`lc_id`	Return a unique identifier for this class for serialization purposes.
`to_json`	Serialize the `Runnable` to JSON.
`to_json_not_implemented`	Serialize a "not implemented" object.
`configurable_fields`	Configure particular `Runnable` fields at runtime.
`configurable_alternatives`	Configure alternatives for `Runnable` objects that can be set at runtime.
`set_verbose`	If verbose is `None`, set it.
`generate_prompt`	Pass a sequence of prompts to the model and return model generations.
`agenerate_prompt`	Asynchronously pass a sequence of prompts and return model generations.
`get_token_ids`	Return the ordered IDs of the tokens in a text.
`get_num_tokens`	Get the number of tokens present in the text.
`get_num_tokens_from_messages`	Get the number of tokens in the messages.
`generate`	Pass a sequence of prompts to the model and return model generations.
`agenerate`	Asynchronously pass a sequence of prompts to a model and return generations.
`dict`	Return a dictionary of the LLM.

model_id `class-attribute` `instance-attribute` ¶

model_id: str | None = None

Type of model to use.

model `class-attribute` `instance-attribute` ¶

model: str | None = None

Name or alias of the foundation model to use. When using IBM's watsonx.ai Model Gateway (public preview), you can specify any supported third-party model—OpenAI, Anthropic, NVIDIA, Cerebras, or IBM's own Granite series—via a single, OpenAI-compatible interface. Models must be explicitly provisioned (opt-in) through the Gateway to ensure secure, vendor-agnostic access and easy switch-over without reconfiguration.

For more details on configuration and usage, see IBM watsonx Model Gateway docs

deployment_id `class-attribute` `instance-attribute` ¶

deployment_id: str | None = None

Type of deployed model to use.

project_id `class-attribute` `instance-attribute` ¶

project_id: str | None = None

ID of the Watson Studio project.

space_id `class-attribute` `instance-attribute` ¶

space_id: str | None = None

ID of the Watson Studio space.

url `class-attribute` `instance-attribute` ¶

url: SecretStr = Field(
    alias="url", default_factory=secret_from_env("WATSONX_URL", default=None)
)

URL to the Watson Machine Learning or CPD instance.

api_key `class-attribute` `instance-attribute` ¶

api_key: SecretStr | None = Field(
    default_factory=secret_from_env_multi(
        names_priority=["WATSONX_API_KEY", "WATSONX_APIKEY"],
        deprecated={"WATSONX_APIKEY"},
    ),
    serialization_alias="api_key",
    validation_alias=AliasChoices("api_key", "apikey"),
    description="API key to the Watson Machine Learning or CPD instance.",
)

API key to the Watson Machine Learning or CPD instance.

token `class-attribute` `instance-attribute` ¶

token: SecretStr | None = Field(
    alias="token", default_factory=secret_from_env("WATSONX_TOKEN", default=None)
)

Token to the CPD instance.

password `class-attribute` `instance-attribute` ¶

password: SecretStr | None = Field(
    alias="password", default_factory=secret_from_env("WATSONX_PASSWORD", default=None)
)

Password to the CPD instance.

username `class-attribute` `instance-attribute` ¶

username: SecretStr | None = Field(
    alias="username", default_factory=secret_from_env("WATSONX_USERNAME", default=None)
)

Username to the CPD instance.

instance_id `class-attribute` `instance-attribute` ¶

instance_id: SecretStr | None = Field(
    alias="instance_id",
    default_factory=secret_from_env("WATSONX_INSTANCE_ID", default=None),
)

Instance_id of the CPD instance.

version `class-attribute` `instance-attribute` ¶

version: SecretStr | None = None

Version of the CPD instance.

params `class-attribute` `instance-attribute` ¶

params: dict | TextChatParameters | None = None

Model parameters to use during request generation.

Note

ValueError is raised if the same Chat generation parameter is provided within the params attribute and as keyword argument.

frequency_penalty `class-attribute` `instance-attribute` ¶

frequency_penalty: float | None = None

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

logprobs `class-attribute` `instance-attribute` ¶

logprobs: bool | None = None

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

top_logprobs `class-attribute` `instance-attribute` ¶

top_logprobs: int | None = None

An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option logprobs must be set to true if this parameter is used.

max_tokens `class-attribute` `instance-attribute` ¶

max_tokens: int | None = None

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. This value is now deprecated in favor of 'max_completion_tokens' parameter.

max_completion_tokens `class-attribute` `instance-attribute` ¶

max_completion_tokens: int | None = None

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.

n `class-attribute` `instance-attribute` ¶

n: int | None = None

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

presence_penalty `class-attribute` `instance-attribute` ¶

presence_penalty: float | None = None

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

temperature `class-attribute` `instance-attribute` ¶

temperature: float | None = None

What sampling temperature to use. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

We generally recommend altering this or top_p but not both.

response_format `class-attribute` `instance-attribute` ¶

response_format: dict | None = None

The chat response format parameters.

top_p `class-attribute` `instance-attribute` ¶

top_p: float | None = None

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or temperature but not both.

time_limit `class-attribute` `instance-attribute` ¶

time_limit: int | None = None

Time limit in milliseconds - if not completed within this time, generation will stop.

logit_bias `class-attribute` `instance-attribute` ¶

logit_bias: dict | None = None

Increasing or decreasing probability of tokens being selected during generation.

seed `class-attribute` `instance-attribute` ¶

seed: int | None = None

Random number generator seed to use in sampling mode for experimental repeatability.

stop `class-attribute` `instance-attribute` ¶

stop: list[str] | None = None

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output.

chat_template_kwargs `class-attribute` `instance-attribute` ¶

chat_template_kwargs: dict | None = None

Additional chat template parameters.

verify `class-attribute` `instance-attribute` ¶

verify: str | bool | None = None

You can pass one of following as verify: * the path to a CA_BUNDLE file * the path of directory with certificates of trusted CAs * True - default path to truststore will be taken * False - no verification will be made

validate_model `class-attribute` `instance-attribute` ¶

validate_model: bool = True

Model ID validation.

streaming `class-attribute` `instance-attribute` ¶

streaming: bool = False

Whether to stream the results or not.

lc_secrets `property` ¶

lc_secrets: dict[str, str]

Mapping of secret environment variables.

name `class-attribute` `instance-attribute` ¶

name: str | None = None

The name of the Runnable. Used for debugging and tracing.

InputType `property` ¶

InputType: TypeAlias

Get the input type for this Runnable.

OutputType `property` ¶

OutputType: Any

Get the output type for this Runnable.

input_schema `property` ¶

input_schema: type[BaseModel]

The type of input this Runnable accepts specified as a Pydantic model.

output_schema `property` ¶

output_schema: type[BaseModel]

Output schema.

The type of output this Runnable produces specified as a Pydantic model.

config_specs `property` ¶

config_specs: list[ConfigurableFieldSpec]

List configurable fields for this Runnable.

lc_attributes `property` ¶

lc_attributes: dict

List of attribute names that should be included in the serialized kwargs.

These attributes must be accepted by the constructor.

Default is an empty dictionary.

cache `class-attribute` `instance-attribute` ¶

cache: BaseCache | bool | None = Field(default=None, exclude=True)

Whether to cache the response.

If True, will use the global cache.
If False, will not use a cache
If None, will use the global cache if it's set, otherwise no cache.
If instance of BaseCache, will use the provided cache.

Caching is not currently supported for streaming methods of models.

verbose `class-attribute` `instance-attribute` ¶

verbose: bool = Field(default_factory=_get_verbosity, exclude=True, repr=False)

Whether to print out response text.

callbacks `class-attribute` `instance-attribute` ¶

callbacks: Callbacks = Field(default=None, exclude=True)

Callbacks to add to the run trace.

tags `class-attribute` `instance-attribute` ¶

tags: list[str] | None = Field(default=None, exclude=True)

Tags to add to the run trace.

metadata `class-attribute` `instance-attribute` ¶

metadata: dict[str, Any] | None = Field(default=None, exclude=True)

Metadata to add to the run trace.

custom_get_token_ids `class-attribute` `instance-attribute` ¶

custom_get_token_ids: Callable[[str], list[int]] | None = Field(
    default=None, exclude=True
)

Optional encoder to use for counting tokens.

rate_limiter `class-attribute` `instance-attribute` ¶

rate_limiter: BaseRateLimiter | None = Field(default=None, exclude=True)

An optional rate limiter to use for limiting the number of requests.

disable_streaming `class-attribute` `instance-attribute` ¶

disable_streaming: bool | Literal['tool_calling'] = False

Whether to disable streaming for this model.

If streaming is bypassed, then stream/astream/astream_events will defer to invoke/ainvoke.

If True, will always bypass streaming case.
If 'tool_calling', will bypass streaming case only when the model is called with a tools keyword argument. In other words, LangChain will automatically switch to non-streaming behavior (invoke) only when the tools argument is provided. This offers the best of both worlds.
If False (Default), will always use streaming case if available.

The main reason for this flag is that code might be written using stream and a user may want to swap out a given model for another model whose the implementation does not properly support streaming.

output_version `class-attribute` `instance-attribute` ¶

output_version: str | None = Field(
    default_factory=from_env("LC_OUTPUT_VERSION", default=None)
)

Version of AIMessage output format to store in message content.

AIMessage.content_blocks will lazily parse the contents of content into a standard format. This flag can be used to additionally store the standard format in message content, e.g., for serialization purposes.

Supported values:

'v0': provider-specific format in content (can lazily-parse with content_blocks)
'v1': standardized format in content (consistent with content_blocks)

Partner packages (e.g., langchain-openai) can also use this field to roll out new content formats in a backward-compatible way.

Added in langchain-core 1.0

profile `property` ¶

profile: ModelProfile

Return profiling information for the model.

This property relies on the langchain-model-profiles package to retrieve chat model capabilities, such as context window sizes and supported features.

RAISES	DESCRIPTION
`ImportError`	If `langchain-model-profiles` is not installed.

RETURNS	DESCRIPTION
`ModelProfile`	A `ModelProfile` object containing profiling information for the model.

is_lc_serializable `classmethod` ¶

is_lc_serializable() -> bool

Is lc serializable.

validate_environment ¶

validate_environment() -> Self

Validate that credentials and python package exists in environment.

bind_tools ¶

bind_tools(
    tools: Sequence[dict[str, Any] | type | Callable | BaseTool],
    *,
    tool_choice: dict | str | bool | None = None,
    strict: bool | None = None,
    **kwargs: Any,
) -> Runnable[LanguageModelInput, AIMessage]

Bind tool-like objects to this chat model.

PARAMETER	DESCRIPTION
`tools`	A list of tool definitions to bind to this chat model. Can be a dictionary, pydantic model, callable, or BaseTool. Pydantic models, callables, and BaseTools will be automatically converted to their schema dictionary representation. TYPE: `Sequence[dict[str, Any] \| type \| Callable \| BaseTool]`
`tool_choice`	Which tool to require the model to call. Options are: `str` of the form `'<<tool_name>>'`: calls `<<tool_name>>` tool. `'auto'`: automatically selects a tool (including no tool). `'none'`: does not call a tool. `'any'` or `'required'` or `True`: force at least one tool to be called. `dict` of the form `{"type": "function", "function": {"name": <<tool_name>>}}`: calls `<<tool_name>>` tool. `False` or `None`: no effect, default OpenAI behavior. TYPE: `dict \| str \| bool \| None` DEFAULT: `None`
`strict`	If `True`, model output is guaranteed to exactly match the JSON Schema provided in the tool definition. The input schema will also be validated according to the supported schemas. If `False`, input schema will not be validated and model output will not be validated. If `None`, `strict` argument will not be passed to the model. TYPE: `bool \| None` DEFAULT: `None`
`kwargs`	Any additional parameters are passed directly to `bind`. TYPE: `Any` DEFAULT: `{}`

with_structured_output ¶

with_structured_output(
    schema: dict | type | None = None,
    *,
    method: Literal[
        "function_calling", "json_mode", "json_schema"
    ] = "function_calling",
    include_raw: bool = False,
    strict: bool | None = None,
    **kwargs: Any,
) -> Runnable[LanguageModelInput, dict | BaseModel]

Model wrapper that returns outputs formatted to match the given schema.

PARAMETER	DESCRIPTION
`schema`	The output schema. Can be passed in as: a JSON Schema, a TypedDict class, or a Pydantic class, If `schema` is a Pydantic class then the model output will be a Pydantic instance of that class, and the model-generated fields will be validated by the Pydantic class. Otherwise the model output will be a dict and will not be validated. TYPE: `dict \| type \| None` DEFAULT: `None`
`method`	The method for steering model generation, one of: `'function_calling'`: uses tool-calling features. `'json_schema'`: uses dedicated structured output features. `'json_mode'`: uses JSON mode. TYPE: `Literal['function_calling', 'json_mode', 'json_schema']` DEFAULT: `'function_calling'`
`include_raw`	If False then only the parsed structured output is returned. If an error occurs during model output parsing it will be raised. If True then both the raw model response (a BaseMessage) and the parsed model response will be returned. If an error occurs during output parsing it will be caught and returned as well. The final output is always a dict with keys `'raw'`, `'parsed'`, and `'parsing_error'`. TYPE: `bool` DEFAULT: `False`
`strict`	`True`: Model output is guaranteed to exactly match the schema. The input schema will also be validated according to the supported schemas. `False`: Input schema will not be validated and model output will not be validated. `None`: `strict` argument will not be passed to the model. TYPE: `bool \| None` DEFAULT: `None`
`kwargs`	Additional keyword args TYPE: `Any` DEFAULT: `{}`

RETURNS	DESCRIPTION
`Runnable[LanguageModelInput, dict \| BaseModel]`	A Runnable that takes same inputs as a `langchain_core.language_models.chat.BaseChatModel`.
`Runnable[LanguageModelInput, dict \| BaseModel]`	If `include_raw` is True, then Runnable outputs a dict with keys:
`Runnable[LanguageModelInput, dict \| BaseModel]`	`'raw'`: BaseMessage
`Runnable[LanguageModelInput, dict \| BaseModel]`	`'parsed'`: None if there was a parsing error, otherwise the type depends on the `schema` as described above.
`Runnable[LanguageModelInput, dict \| BaseModel]`	`'parsing_error'`: Optional[BaseException]

Example: schema=Pydantic class, method='function_calling', include_raw=True

from langchain_ibm import ChatWatsonx
from pydantic import BaseModel


class AnswerWithJustification(BaseModel):
    '''An answer to the user question along with justification for the answer.'''

    answer: str
    justification: str


model = ChatWatsonx(...)
structured_model = model.with_structured_output(
    AnswerWithJustification, include_raw=True
)

structured_model.invoke(
    "What weighs more a pound of bricks or a pound of feathers"
)

{
    "raw": AIMessage(
        content="",
        additional_kwargs={
            "tool_calls": [
                {
                    "id": "call_Ao02pnFYXD6GN1yzc0uXPsvF",
                    "function": {
                        "arguments": '{"answer":"They weigh the same.","justification":"Both a pound of bricks and a pound of feathers weigh one pound. The weight is the same, but the volume or density of the objects may differ."}',
                        "name": "AnswerWithJustification",
                    },
                    "type": "function",
                }
            ]
        },
    ),
    "parsed": AnswerWithJustification(
        answer="They weigh the same.",
        justification="Both a pound of bricks and a pound of feathers weigh one pound. The weight is the same, but the volume or density of the objects may differ.",
    ),
    "parsing_error": None,
}

Example: schema=JSON schema, method='function_calling', include_raw=False

from langchain_ibm import ChatWatsonx
from pydantic import BaseModel
from langchain_core.utils.function_calling import convert_to_openai_tool


class AnswerWithJustification(BaseModel):
    '''An answer to the user question along with justification for the answer.'''

    answer: str
    justification: str


dict_schema = convert_to_openai_tool(AnswerWithJustification)
model = ChatWatsonx(...)
structured_model = model.with_structured_output(dict_schema)

structured_model.invoke(
    "What weighs more a pound of bricks or a pound of feathers"
)

{
    "answer": "They weigh the same",
    "justification": "Both a pound of bricks and a pound of feathers weigh one pound. The weight is the same, but the volume and density of the two substances differ.",
}

Example: schema=Pydantic class, method='json_schema', include_raw=True

from langchain_ibm import ChatWatsonx
from pydantic import BaseModel


class AnswerWithJustification(BaseModel):
    '''An answer to the user question along with justification for the answer.'''

    answer: str
    justification: str


model = ChatWatsonx(...)
structured_model = model.with_structured_output(
    AnswerWithJustification, method="json_schema", include_raw=True
)

structured_model.invoke(
    "What weighs more a pound of bricks or a pound of feathers"
)

{
    "raw": AIMessage(
        content="",
        additional_kwargs={
            "tool_calls": [
                {
                    "id": "chatcmpl-tool-bfbd6f6dd33b438990c5ddf277485971",
                    "type": "function",
                    "function": {
                        "name": "AnswerWithJustification",
                        "arguments": '{"answer": "They weigh the same", "justification": "A pound is a unit of weight or mass, so both a pound of bricks and a pound of feathers weigh the same amount, one pound."}',
                    },
                }
            ]
        },
        response_metadata={
            "token_usage": {
                "completion_tokens": 45,
                "prompt_tokens": 275,
                "total_tokens": 320,
            },
            "model_name": "meta-llama/llama-3-3-70b-instruct",
            "system_fingerprint": "",
            "finish_reason": "stop",
        },
        id="chatcmpl-461ca5bd-1982-412c-b886-017c483bf481---8c18b06eead65ae4691364798787bda7---71896588-efa5-439f-a25f-d1abfe289f5a",
        tool_calls=[
            {
                "name": "AnswerWithJustification",
                "args": {
                    "answer": "They weigh the same",
                    "justification": "A pound is a unit of weight or mass, so both a pound of bricks and a pound of feathers weigh the same amount, one pound.",
                },
                "id": "chatcmpl-tool-bfbd6f6dd33b438990c5ddf277485971",
                "type": "tool_call",
            }
        ],
        usage_metadata={
            "input_tokens": 275,
            "output_tokens": 45,
            "total_tokens": 320,
        },
    ),
    "parsed": AnswerWithJustification(
        answer="They weigh the same",
        justification="A pound is a unit of weight or mass, so both a pound of bricks and a pound of feathers weigh the same amount, one pound.",
    ),
    "parsing_error": None,
}

Example: schema=function schema, method='json_schema', include_raw=False

from langchain_ibm import ChatWatsonx
from pydantic import BaseModel

function__schema = {
    "name": "AnswerWithJustification",
    "description": "An answer to the user question along with justification for the answer.",
    "parameters": {
        "type": "object",
        "properties": {
            "answer": {"type": "string"},
            "justification": {
                "description": "A justification for the answer.",
                "type": "string",
            },
        },
        "required": ["answer"],
    },
}

model = ChatWatsonx(...)
structured_model = model.with_structured_output(
    function_schema, method="json_schema", include_raw=False
)

structured_model.invoke(
    "What weighs more a pound of bricks or a pound of feathers"
)

{
    "answer": "They weigh the same",
    "justification": "Both a pound of bricks and a pound of feathers weigh one pound. The weight is the same, but the volume and density of the two substances differ.",
}

Example: schema=Pydantic class, method='json_mode', include_raw=True

from langchain_ibm import ChatWatsonx
from pydantic import BaseModel


class AnswerWithJustification(BaseModel):
    answer: str
    justification: str


model = ChatWatsonx(...)
structured_model = model.with_structured_output(
    AnswerWithJustification, method="json_mode", include_raw=True
)

structured_model.invoke(
    "Answer the following question. "
    "Make sure to return a JSON blob with keys 'answer' and 'justification'.\n\n"
    "What's heavier a pound of bricks or a pound of feathers?"
)

{
    "raw": AIMessage(
        content='{\n    "answer": "They are both the same weight.",\n    "justification": "Both a pound of bricks and a pound of feathers weigh one pound. The difference lies in the volume and density of the materials, not the weight." \n}'
    ),
    "parsed": AnswerWithJustification(
        answer="They are both the same weight.",
        justification="Both a pound of bricks and a pound of feathers weigh one pound. The difference lies in the volume and density of the materials, not the weight.",
    ),
    "parsing_error": None,
}

Example: schema=None, method='json_mode', include_raw=True

from langchain_ibm import ChatWatsonx

model = ChatWatsonx(...)
structured_model = model.with_structured_output(
    method="json_mode", include_raw=True
)

structured_model.invoke(
    "Answer the following question. "
    "Make sure to return a JSON blob with keys 'answer' and 'justification'.\n\n"
    "What's heavier a pound of bricks or a pound of feathers?"
)

{
    "raw": AIMessage(
        content='{\n    "answer": "They are both the same weight.",\n    "justification": "Both a pound of bricks and a pound of feathers weigh one pound. The difference lies in the volume and density of the materials, not the weight." \n}'
    ),
    "parsed": {
        "answer": "They are both the same weight.",
        "justification": "Both a pound of bricks and a pound of feathers weigh one pound. The difference lies in the volume and density of the materials, not the weight.",
    },
    "parsing_error": None,
}

get_name ¶

get_name(suffix: str | None = None, *, name: str | None = None) -> str

Get the name of the Runnable.

PARAMETER	DESCRIPTION
`suffix`	An optional suffix to append to the name. TYPE: `str \| None` DEFAULT: `None`
`name`	An optional name to use instead of the `Runnable`'s name. TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`str`	The name of the `Runnable`.

get_input_schema ¶

get_input_schema(config: RunnableConfig | None = None) -> type[BaseModel]

Get a Pydantic model that can be used to validate input to the Runnable.

Runnable objects that leverage the configurable_fields and configurable_alternatives methods will have a dynamic input schema that depends on which configuration the Runnable is invoked with.

This method allows to get an input schema for a specific configuration.

PARAMETER	DESCRIPTION
`config`	A config to use when generating the schema. TYPE: `RunnableConfig \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`type[BaseModel]`	A Pydantic model that can be used to validate input.

get_input_jsonschema ¶

get_input_jsonschema(config: RunnableConfig | None = None) -> dict[str, Any]

Get a JSON schema that represents the input to the Runnable.

PARAMETER	DESCRIPTION
`config`	A config to use when generating the schema. TYPE: `RunnableConfig \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`dict[str, Any]`	A JSON schema that represents the input to the `Runnable`.

Example

from langchain_core.runnables import RunnableLambda


def add_one(x: int) -> int:
    return x + 1


runnable = RunnableLambda(add_one)

print(runnable.get_input_jsonschema())

Added in langchain-core 0.3.0

get_output_schema ¶

get_output_schema(config: RunnableConfig | None = None) -> type[BaseModel]

Get a Pydantic model that can be used to validate output to the Runnable.

Runnable objects that leverage the configurable_fields and configurable_alternatives methods will have a dynamic output schema that depends on which configuration the Runnable is invoked with.

This method allows to get an output schema for a specific configuration.

PARAMETER	DESCRIPTION
`config`	A config to use when generating the schema. TYPE: `RunnableConfig \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`type[BaseModel]`	A Pydantic model that can be used to validate output.

get_output_jsonschema ¶

get_output_jsonschema(config: RunnableConfig | None = None) -> dict[str, Any]

Get a JSON schema that represents the output of the Runnable.

PARAMETER	DESCRIPTION
`config`	A config to use when generating the schema. TYPE: `RunnableConfig \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`dict[str, Any]`	A JSON schema that represents the output of the `Runnable`.

Example

from langchain_core.runnables import RunnableLambda


def add_one(x: int) -> int:
    return x + 1


runnable = RunnableLambda(add_one)

print(runnable.get_output_jsonschema())

Added in langchain-core 0.3.0

config_schema ¶

config_schema(*, include: Sequence[str] | None = None) -> type[BaseModel]

The type of config this Runnable accepts specified as a Pydantic model.

To mark a field as configurable, see the configurable_fields and configurable_alternatives methods.

PARAMETER	DESCRIPTION
`include`	A list of fields to include in the config schema. TYPE: `Sequence[str] \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`type[BaseModel]`	A Pydantic model that can be used to validate config.

get_config_jsonschema ¶

get_config_jsonschema(*, include: Sequence[str] | None = None) -> dict[str, Any]

Get a JSON schema that represents the config of the Runnable.

PARAMETER	DESCRIPTION
`include`	A list of fields to include in the config schema. TYPE: `Sequence[str] \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`dict[str, Any]`	A JSON schema that represents the config of the `Runnable`.

Added in langchain-core 0.3.0

get_graph ¶

get_graph(config: RunnableConfig | None = None) -> Graph

Return a graph representation of this Runnable.

get_prompts ¶

get_prompts(config: RunnableConfig | None = None) -> list[BasePromptTemplate]

Return a list of prompts used by this Runnable.

or ¶

__or__(
    other: Runnable[Any, Other]
    | Callable[[Iterator[Any]], Iterator[Other]]
    | Callable[[AsyncIterator[Any]], AsyncIterator[Other]]
    | Callable[[Any], Other]
    | Mapping[str, Runnable[Any, Other] | Callable[[Any], Other] | Any],
) -> RunnableSerializable[Input, Other]

Runnable "or" operator.

Compose this Runnable with another object to create a RunnableSequence.

PARAMETER	DESCRIPTION
`other`	Another `Runnable` or a `Runnable`-like object. TYPE: `Runnable[Any, Other] \| Callable[[Iterator[Any]], Iterator[Other]] \| Callable[[AsyncIterator[Any]], AsyncIterator[Other]] \| Callable[[Any], Other] \| Mapping[str, Runnable[Any, Other] \| Callable[[Any], Other] \| Any]`

RETURNS	DESCRIPTION
`RunnableSerializable[Input, Other]`	A new `Runnable`.

ror ¶

__ror__(
    other: Runnable[Other, Any]
    | Callable[[Iterator[Other]], Iterator[Any]]
    | Callable[[AsyncIterator[Other]], AsyncIterator[Any]]
    | Callable[[Other], Any]
    | Mapping[str, Runnable[Other, Any] | Callable[[Other], Any] | Any],
) -> RunnableSerializable[Other, Output]

Runnable "reverse-or" operator.

Compose this Runnable with another object to create a RunnableSequence.

PARAMETER	DESCRIPTION
`other`	Another `Runnable` or a `Runnable`-like object. TYPE: `Runnable[Other, Any] \| Callable[[Iterator[Other]], Iterator[Any]] \| Callable[[AsyncIterator[Other]], AsyncIterator[Any]] \| Callable[[Other], Any] \| Mapping[str, Runnable[Other, Any] \| Callable[[Other], Any] \| Any]`

RETURNS	DESCRIPTION
`RunnableSerializable[Other, Output]`	A new `Runnable`.

pipe ¶

pipe(
    *others: Runnable[Any, Other] | Callable[[Any], Other], name: str | None = None
) -> RunnableSerializable[Input, Other]

Pipe Runnable objects.

Compose this Runnable with Runnable-like objects to make a RunnableSequence.

Equivalent to RunnableSequence(self, *others) or self | others[0] | ...

Example

from langchain_core.runnables import RunnableLambda


def add_one(x: int) -> int:
    return x + 1


def mul_two(x: int) -> int:
    return x * 2


runnable_1 = RunnableLambda(add_one)
runnable_2 = RunnableLambda(mul_two)
sequence = runnable_1.pipe(runnable_2)
# Or equivalently:
# sequence = runnable_1 | runnable_2
# sequence = RunnableSequence(first=runnable_1, last=runnable_2)
sequence.invoke(1)
await sequence.ainvoke(1)
# -> 4

sequence.batch([1, 2, 3])
await sequence.abatch([1, 2, 3])
# -> [4, 6, 8]

PARAMETER	DESCRIPTION
`*others`	Other `Runnable` or `Runnable`-like objects to compose TYPE: `Runnable[Any, Other] \| Callable[[Any], Other]` DEFAULT: `()`
`name`	An optional name for the resulting `RunnableSequence`. TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`RunnableSerializable[Input, Other]`	A new `Runnable`.

pick ¶

pick(keys: str | list[str]) -> RunnableSerializable[Any, Any]

Pick keys from the output dict of this Runnable.

Pick a single key:

import json

from langchain_core.runnables import RunnableLambda, RunnableMap

as_str = RunnableLambda(str)
as_json = RunnableLambda(json.loads)
chain = RunnableMap(str=as_str, json=as_json)

chain.invoke("[1, 2, 3]")
# -> {"str": "[1, 2, 3]", "json": [1, 2, 3]}

json_only_chain = chain.pick("json")
json_only_chain.invoke("[1, 2, 3]")
# -> [1, 2, 3]

Pick a list of keys:

from typing import Any

import json

from langchain_core.runnables import RunnableLambda, RunnableMap

as_str = RunnableLambda(str)
as_json = RunnableLambda(json.loads)


def as_bytes(x: Any) -> bytes:
    return bytes(x, "utf-8")


chain = RunnableMap(str=as_str, json=as_json, bytes=RunnableLambda(as_bytes))

chain.invoke("[1, 2, 3]")
# -> {"str": "[1, 2, 3]", "json": [1, 2, 3], "bytes": b"[1, 2, 3]"}

json_and_bytes_chain = chain.pick(["json", "bytes"])
json_and_bytes_chain.invoke("[1, 2, 3]")
# -> {"json": [1, 2, 3], "bytes": b"[1, 2, 3]"}

PARAMETER	DESCRIPTION
`keys`	A key or list of keys to pick from the output dict. TYPE: `str \| list[str]`

RETURNS	DESCRIPTION
`RunnableSerializable[Any, Any]`	a new `Runnable`.

assign ¶

assign(
    **kwargs: Runnable[dict[str, Any], Any]
    | Callable[[dict[str, Any]], Any]
    | Mapping[str, Runnable[dict[str, Any], Any] | Callable[[dict[str, Any]], Any]],
) -> RunnableSerializable[Any, Any]

Assigns new fields to the dict output of this Runnable.

from langchain_core.language_models.fake import FakeStreamingListLLM
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import SystemMessagePromptTemplate
from langchain_core.runnables import Runnable
from operator import itemgetter

prompt = (
    SystemMessagePromptTemplate.from_template("You are a nice assistant.")
    + "{question}"
)
model = FakeStreamingListLLM(responses=["foo-lish"])

chain: Runnable = prompt | model | {"str": StrOutputParser()}

chain_with_assign = chain.assign(hello=itemgetter("str") | model)

print(chain_with_assign.input_schema.model_json_schema())
# {'title': 'PromptInput', 'type': 'object', 'properties':
{'question': {'title': 'Question', 'type': 'string'}}}
print(chain_with_assign.output_schema.model_json_schema())
# {'title': 'RunnableSequenceOutput', 'type': 'object', 'properties':
{'str': {'title': 'Str',
'type': 'string'}, 'hello': {'title': 'Hello', 'type': 'string'}}}

PARAMETER	DESCRIPTION
`**kwargs`	A mapping of keys to `Runnable` or `Runnable`-like objects that will be invoked with the entire output dict of this `Runnable`. TYPE: `Runnable[dict[str, Any], Any] \| Callable[[dict[str, Any]], Any] \| Mapping[str, Runnable[dict[str, Any], Any] \| Callable[[dict[str, Any]], Any]]` DEFAULT: `{}`

RETURNS	DESCRIPTION
`RunnableSerializable[Any, Any]`	A new `Runnable`.

invoke ¶

invoke(
    input: LanguageModelInput,
    config: RunnableConfig | None = None,
    *,
    stop: list[str] | None = None,
    **kwargs: Any,
) -> AIMessage

Transform a single input into an output.

PARAMETER	DESCRIPTION
`input`	The input to the `Runnable`. TYPE: `Input`
`config`	A config to use when invoking the `Runnable`. The config supports standard keys like `'tags'`, `'metadata'` for tracing purposes, `'max_concurrency'` for controlling how much work to do in parallel, and other keys. Please refer to `RunnableConfig` for more details. TYPE: `RunnableConfig \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Output`	The output of the `Runnable`.

ainvoke `async` ¶

ainvoke(
    input: LanguageModelInput,
    config: RunnableConfig | None = None,
    *,
    stop: list[str] | None = None,
    **kwargs: Any,
) -> AIMessage

Transform a single input into an output.

PARAMETER	DESCRIPTION
`input`	The input to the `Runnable`. TYPE: `Input`
`config`	A config to use when invoking the `Runnable`. The config supports standard keys like `'tags'`, `'metadata'` for tracing purposes, `'max_concurrency'` for controlling how much work to do in parallel, and other keys. Please refer to `RunnableConfig` for more details. TYPE: `RunnableConfig \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Output`	The output of the `Runnable`.

batch ¶

batch(
    inputs: list[Input],
    config: RunnableConfig | list[RunnableConfig] | None = None,
    *,
    return_exceptions: bool = False,
    **kwargs: Any | None,
) -> list[Output]

Default implementation runs invoke in parallel using a thread pool executor.

The default implementation of batch works well for IO bound runnables.

Subclasses must override this method if they can batch more efficiently; e.g., if the underlying Runnable uses an API which supports a batch mode.

PARAMETER	DESCRIPTION
`inputs`	A list of inputs to the `Runnable`. TYPE: `list[Input]`
`config`	A config to use when invoking the `Runnable`. The config supports standard keys like `'tags'`, `'metadata'` for tracing purposes, `'max_concurrency'` for controlling how much work to do in parallel, and other keys. Please refer to `RunnableConfig` for more details. TYPE: `RunnableConfig \| list[RunnableConfig] \| None` DEFAULT: `None`
`return_exceptions`	Whether to return exceptions instead of raising them. TYPE: `bool` DEFAULT: `False`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any \| None` DEFAULT: `{}`

RETURNS	DESCRIPTION
`list[Output]`	A list of outputs from the `Runnable`.

batch_as_completed ¶

batch_as_completed(
    inputs: Sequence[Input],
    config: RunnableConfig | Sequence[RunnableConfig] | None = None,
    *,
    return_exceptions: bool = False,
    **kwargs: Any | None,
) -> Iterator[tuple[int, Output | Exception]]

Run invoke in parallel on a list of inputs.

Yields results as they complete.

PARAMETER	DESCRIPTION
`inputs`	A list of inputs to the `Runnable`. TYPE: `Sequence[Input]`
`config`	A config to use when invoking the `Runnable`. The config supports standard keys like `'tags'`, `'metadata'` for tracing purposes, `'max_concurrency'` for controlling how much work to do in parallel, and other keys. Please refer to `RunnableConfig` for more details. TYPE: `RunnableConfig \| Sequence[RunnableConfig] \| None` DEFAULT: `None`
`return_exceptions`	Whether to return exceptions instead of raising them. TYPE: `bool` DEFAULT: `False`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any \| None` DEFAULT: `{}`

YIELDS	DESCRIPTION
`tuple[int, Output \| Exception]`	Tuples of the index of the input and the output from the `Runnable`.

abatch `async` ¶

abatch(
    inputs: list[Input],
    config: RunnableConfig | list[RunnableConfig] | None = None,
    *,
    return_exceptions: bool = False,
    **kwargs: Any | None,
) -> list[Output]

Default implementation runs ainvoke in parallel using asyncio.gather.

The default implementation of batch works well for IO bound runnables.

Subclasses must override this method if they can batch more efficiently; e.g., if the underlying Runnable uses an API which supports a batch mode.

PARAMETER	DESCRIPTION
`inputs`	A list of inputs to the `Runnable`. TYPE: `list[Input]`
`config`	A config to use when invoking the `Runnable`. The config supports standard keys like `'tags'`, `'metadata'` for tracing purposes, `'max_concurrency'` for controlling how much work to do in parallel, and other keys. Please refer to `RunnableConfig` for more details. TYPE: `RunnableConfig \| list[RunnableConfig] \| None` DEFAULT: `None`
`return_exceptions`	Whether to return exceptions instead of raising them. TYPE: `bool` DEFAULT: `False`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any \| None` DEFAULT: `{}`

RETURNS	DESCRIPTION
`list[Output]`	A list of outputs from the `Runnable`.

abatch_as_completed `async` ¶

abatch_as_completed(
    inputs: Sequence[Input],
    config: RunnableConfig | Sequence[RunnableConfig] | None = None,
    *,
    return_exceptions: bool = False,
    **kwargs: Any | None,
) -> AsyncIterator[tuple[int, Output | Exception]]

Run ainvoke in parallel on a list of inputs.

Yields results as they complete.

PARAMETER	DESCRIPTION
`inputs`	A list of inputs to the `Runnable`. TYPE: `Sequence[Input]`
`config`	A config to use when invoking the `Runnable`. The config supports standard keys like `'tags'`, `'metadata'` for tracing purposes, `'max_concurrency'` for controlling how much work to do in parallel, and other keys. Please refer to `RunnableConfig` for more details. TYPE: `RunnableConfig \| Sequence[RunnableConfig] \| None` DEFAULT: `None`
`return_exceptions`	Whether to return exceptions instead of raising them. TYPE: `bool` DEFAULT: `False`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any \| None` DEFAULT: `{}`

YIELDS	DESCRIPTION
`AsyncIterator[tuple[int, Output \| Exception]]`	A tuple of the index of the input and the output from the `Runnable`.

stream ¶

stream(
    input: LanguageModelInput,
    config: RunnableConfig | None = None,
    *,
    stop: list[str] | None = None,
    **kwargs: Any,
) -> Iterator[AIMessageChunk]

Default implementation of stream, which calls invoke.

Subclasses must override this method if they support streaming output.

PARAMETER	DESCRIPTION
`input`	The input to the `Runnable`. TYPE: `Input`
`config`	The config to use for the `Runnable`. TYPE: `RunnableConfig \| None` DEFAULT: `None`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any \| None` DEFAULT: `{}`

YIELDS	DESCRIPTION
`Output`	The output of the `Runnable`.

astream `async` ¶

astream(
    input: LanguageModelInput,
    config: RunnableConfig | None = None,
    *,
    stop: list[str] | None = None,
    **kwargs: Any,
) -> AsyncIterator[AIMessageChunk]

Default implementation of astream, which calls ainvoke.

Subclasses must override this method if they support streaming output.

PARAMETER	DESCRIPTION
`input`	The input to the `Runnable`. TYPE: `Input`
`config`	The config to use for the `Runnable`. TYPE: `RunnableConfig \| None` DEFAULT: `None`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any \| None` DEFAULT: `{}`

YIELDS	DESCRIPTION
`AsyncIterator[Output]`	The output of the `Runnable`.

astream_log `async` ¶

astream_log(
    input: Any,
    config: RunnableConfig | None = None,
    *,
    diff: bool = True,
    with_streamed_output_list: bool = True,
    include_names: Sequence[str] | None = None,
    include_types: Sequence[str] | None = None,
    include_tags: Sequence[str] | None = None,
    exclude_names: Sequence[str] | None = None,
    exclude_types: Sequence[str] | None = None,
    exclude_tags: Sequence[str] | None = None,
    **kwargs: Any,
) -> AsyncIterator[RunLogPatch] | AsyncIterator[RunLog]

Stream all output from a Runnable, as reported to the callback system.

This includes all inner runs of LLMs, Retrievers, Tools, etc.

Output is streamed as Log objects, which include a list of Jsonpatch ops that describe how the state of the run has changed in each step, and the final state of the run.

The Jsonpatch ops can be applied in order to construct state.

PARAMETER	DESCRIPTION
`input`	The input to the `Runnable`. TYPE: `Any`
`config`	The config to use for the `Runnable`. TYPE: `RunnableConfig \| None` DEFAULT: `None`
`diff`	Whether to yield diffs between each step or the current state. TYPE: `bool` DEFAULT: `True`
`with_streamed_output_list`	Whether to yield the `streamed_output` list. TYPE: `bool` DEFAULT: `True`
`include_names`	Only include logs with these names. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`include_types`	Only include logs with these types. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`include_tags`	Only include logs with these tags. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`exclude_names`	Exclude logs with these names. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`exclude_types`	Exclude logs with these types. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`exclude_tags`	Exclude logs with these tags. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any` DEFAULT: `{}`

YIELDS	DESCRIPTION
`AsyncIterator[RunLogPatch] \| AsyncIterator[RunLog]`	A `RunLogPatch` or `RunLog` object.

astream_events `async` ¶

astream_events(
    input: Any,
    config: RunnableConfig | None = None,
    *,
    version: Literal["v1", "v2"] = "v2",
    include_names: Sequence[str] | None = None,
    include_types: Sequence[str] | None = None,
    include_tags: Sequence[str] | None = None,
    exclude_names: Sequence[str] | None = None,
    exclude_types: Sequence[str] | None = None,
    exclude_tags: Sequence[str] | None = None,
    **kwargs: Any,
) -> AsyncIterator[StreamEvent]

Generate a stream of events.

Use to create an iterator over StreamEvent that provide real-time information about the progress of the Runnable, including StreamEvent from intermediate results.

A StreamEvent is a dictionary with the following schema:

event: Event names are of the format: on_[runnable_type]_(start|stream|end).
name: The name of the Runnable that generated the event.
run_id: Randomly generated ID associated with the given execution of the Runnable that emitted the event. A child Runnable that gets invoked as part of the execution of a parent Runnable is assigned its own unique ID.
parent_ids: The IDs of the parent runnables that generated the event. The root Runnable will have an empty list. The order of the parent IDs is from the root to the immediate parent. Only available for v2 version of the API. The v1 version of the API will return an empty list.
tags: The tags of the Runnable that generated the event.
metadata: The metadata of the Runnable that generated the event.
data: The data associated with the event. The contents of this field depend on the type of event. See the table below for more details.

Below is a table that illustrates some events that might be emitted by various chains. Metadata fields have been omitted from the table for brevity. Chain definitions have been included after the table.

Note

This reference table is for the v2 version of the schema.

event	name	chunk	input	output
`on_chat_model_start`	`'[model name]'`		`{"messages": [[SystemMessage, HumanMessage]]}`
`on_chat_model_stream`	`'[model name]'`	`AIMessageChunk(content="hello")`
`on_chat_model_end`	`'[model name]'`		`{"messages": [[SystemMessage, HumanMessage]]}`	`AIMessageChunk(content="hello world")`
`on_llm_start`	`'[model name]'`		`{'input': 'hello'}`
`on_llm_stream`	`'[model name]'`	`'Hello'`
`on_llm_end`	`'[model name]'`		`'Hello human!'`
`on_chain_start`	`'format_docs'`
`on_chain_stream`	`'format_docs'`	`'hello world!, goodbye world!'`
`on_chain_end`	`'format_docs'`		`[Document(...)]`	`'hello world!, goodbye world!'`
`on_tool_start`	`'some_tool'`		`{"x": 1, "y": "2"}`
`on_tool_end`	`'some_tool'`			`{"x": 1, "y": "2"}`
`on_retriever_start`	`'[retriever name]'`		`{"query": "hello"}`
`on_retriever_end`	`'[retriever name]'`		`{"query": "hello"}`	`[Document(...), ..]`
`on_prompt_start`	`'[template_name]'`		`{"question": "hello"}`
`on_prompt_end`	`'[template_name]'`		`{"question": "hello"}`	`ChatPromptValue(messages: [SystemMessage, ...])`

In addition to the standard events, users can also dispatch custom events (see example below).

Custom events will be only be surfaced with in the v2 version of the API!

A custom event has following format:

Attribute	Type	Description
`name`	`str`	A user defined name for the event.
`data`	`Any`	The data associated with the event. This can be anything, though we suggest making it JSON serializable.

Here are declarations associated with the standard events shown above:

format_docs:

def format_docs(docs: list[Document]) -> str:
    '''Format the docs.'''
    return ", ".join([doc.page_content for doc in docs])


format_docs = RunnableLambda(format_docs)

some_tool:

@tool
def some_tool(x: int, y: str) -> dict:
    '''Some_tool.'''
    return {"x": x, "y": y}

prompt:

template = ChatPromptTemplate.from_messages(
    [
        ("system", "You are Cat Agent 007"),
        ("human", "{question}"),
    ]
).with_config({"run_name": "my_template", "tags": ["my_template"]})

For instance:

from langchain_core.runnables import RunnableLambda


async def reverse(s: str) -> str:
    return s[::-1]


chain = RunnableLambda(func=reverse)

events = [event async for event in chain.astream_events("hello", version="v2")]

# Will produce the following events
# (run_id, and parent_ids has been omitted for brevity):
[
    {
        "data": {"input": "hello"},
        "event": "on_chain_start",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
    {
        "data": {"chunk": "olleh"},
        "event": "on_chain_stream",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
    {
        "data": {"output": "olleh"},
        "event": "on_chain_end",
        "metadata": {},
        "name": "reverse",
        "tags": [],
    },
]

Example: Dispatch Custom Event

from langchain_core.callbacks.manager import (
    adispatch_custom_event,
)
from langchain_core.runnables import RunnableLambda, RunnableConfig
import asyncio


async def slow_thing(some_input: str, config: RunnableConfig) -> str:
    """Do something that takes a long time."""
    await asyncio.sleep(1) # Placeholder for some slow operation
    await adispatch_custom_event(
        "progress_event",
        {"message": "Finished step 1 of 3"},
        config=config # Must be included for python < 3.10
    )
    await asyncio.sleep(1) # Placeholder for some slow operation
    await adispatch_custom_event(
        "progress_event",
        {"message": "Finished step 2 of 3"},
        config=config # Must be included for python < 3.10
    )
    await asyncio.sleep(1) # Placeholder for some slow operation
    return "Done"

slow_thing = RunnableLambda(slow_thing)

async for event in slow_thing.astream_events("some_input", version="v2"):
    print(event)

PARAMETER	DESCRIPTION
`input`	The input to the `Runnable`. TYPE: `Any`
`config`	The config to use for the `Runnable`. TYPE: `RunnableConfig \| None` DEFAULT: `None`
`version`	The version of the schema to use either `'v2'` or `'v1'`. Users should use `'v2'`. `'v1'` is for backwards compatibility and will be deprecated in `0.4.0`. No default will be assigned until the API is stabilized. custom events will only be surfaced in `'v2'`. TYPE: `Literal['v1', 'v2']` DEFAULT: `'v2'`
`include_names`	Only include events from `Runnable` objects with matching names. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`include_types`	Only include events from `Runnable` objects with matching types. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`include_tags`	Only include events from `Runnable` objects with matching tags. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`exclude_names`	Exclude events from `Runnable` objects with matching names. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`exclude_types`	Exclude events from `Runnable` objects with matching types. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`exclude_tags`	Exclude events from `Runnable` objects with matching tags. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. These will be passed to `astream_log` as this implementation of `astream_events` is built on top of `astream_log`. TYPE: `Any` DEFAULT: `{}`

YIELDS	DESCRIPTION
`AsyncIterator[StreamEvent]`	An async stream of `StreamEvent`.

RAISES	DESCRIPTION
`NotImplementedError`	If the version is not `'v1'` or `'v2'`.

transform ¶

transform(
    input: Iterator[Input], config: RunnableConfig | None = None, **kwargs: Any | None
) -> Iterator[Output]

Transform inputs to outputs.

Default implementation of transform, which buffers input and calls astream.

Subclasses must override this method if they can start producing output while input is still being generated.

PARAMETER	DESCRIPTION
`input`	An iterator of inputs to the `Runnable`. TYPE: `Iterator[Input]`
`config`	The config to use for the `Runnable`. TYPE: `RunnableConfig \| None` DEFAULT: `None`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any \| None` DEFAULT: `{}`

YIELDS	DESCRIPTION
`Output`	The output of the `Runnable`.

atransform `async` ¶

atransform(
    input: AsyncIterator[Input],
    config: RunnableConfig | None = None,
    **kwargs: Any | None,
) -> AsyncIterator[Output]

Transform inputs to outputs.

Default implementation of atransform, which buffers input and calls astream.

Subclasses must override this method if they can start producing output while input is still being generated.

PARAMETER	DESCRIPTION
`input`	An async iterator of inputs to the `Runnable`. TYPE: `AsyncIterator[Input]`
`config`	The config to use for the `Runnable`. TYPE: `RunnableConfig \| None` DEFAULT: `None`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any \| None` DEFAULT: `{}`

YIELDS	DESCRIPTION
`AsyncIterator[Output]`	The output of the `Runnable`.

bind ¶

bind(**kwargs: Any) -> Runnable[Input, Output]

Bind arguments to a Runnable, returning a new Runnable.

Useful when a Runnable in a chain requires an argument that is not in the output of the previous Runnable or included in the user input.

PARAMETER	DESCRIPTION
`**kwargs`	The arguments to bind to the `Runnable`. TYPE: `Any` DEFAULT: `{}`

RETURNS	DESCRIPTION
`Runnable[Input, Output]`	A new `Runnable` with the arguments bound.

Example

from langchain_ollama import ChatOllama
from langchain_core.output_parsers import StrOutputParser

model = ChatOllama(model="llama3.1")

# Without bind
chain = model | StrOutputParser()

chain.invoke("Repeat quoted words exactly: 'One two three four five.'")
# Output is 'One two three four five.'

# With bind
chain = model.bind(stop=["three"]) | StrOutputParser()

chain.invoke("Repeat quoted words exactly: 'One two three four five.'")
# Output is 'One two'

with_config ¶

with_config(
    config: RunnableConfig | None = None, **kwargs: Any
) -> Runnable[Input, Output]

Bind config to a Runnable, returning a new Runnable.

PARAMETER	DESCRIPTION
`config`	The config to bind to the `Runnable`. TYPE: `RunnableConfig \| None` DEFAULT: `None`
`**kwargs`	Additional keyword arguments to pass to the `Runnable`. TYPE: `Any` DEFAULT: `{}`

RETURNS	DESCRIPTION
`Runnable[Input, Output]`	A new `Runnable` with the config bound.

with_listeners ¶

with_listeners(
    *,
    on_start: Callable[[Run], None]
    | Callable[[Run, RunnableConfig], None]
    | None = None,
    on_end: Callable[[Run], None] | Callable[[Run, RunnableConfig], None] | None = None,
    on_error: Callable[[Run], None]
    | Callable[[Run, RunnableConfig], None]
    | None = None,
) -> Runnable[Input, Output]

Bind lifecycle listeners to a Runnable, returning a new Runnable.

The Run object contains information about the run, including its id, type, input, output, error, start_time, end_time, and any tags or metadata added to the run.

PARAMETER	DESCRIPTION
`on_start`	Called before the `Runnable` starts running, with the `Run` object. TYPE: `Callable[[Run], None] \| Callable[[Run, RunnableConfig], None] \| None` DEFAULT: `None`
`on_end`	Called after the `Runnable` finishes running, with the `Run` object. TYPE: `Callable[[Run], None] \| Callable[[Run, RunnableConfig], None] \| None` DEFAULT: `None`
`on_error`	Called if the `Runnable` throws an error, with the `Run` object. TYPE: `Callable[[Run], None] \| Callable[[Run, RunnableConfig], None] \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Runnable[Input, Output]`	A new `Runnable` with the listeners bound.

Example

from langchain_core.runnables import RunnableLambda
from langchain_core.tracers.schemas import Run

import time


def test_runnable(time_to_sleep: int):
    time.sleep(time_to_sleep)


def fn_start(run_obj: Run):
    print("start_time:", run_obj.start_time)


def fn_end(run_obj: Run):
    print("end_time:", run_obj.end_time)


chain = RunnableLambda(test_runnable).with_listeners(
    on_start=fn_start, on_end=fn_end
)
chain.invoke(2)

with_alisteners ¶

with_alisteners(
    *,
    on_start: AsyncListener | None = None,
    on_end: AsyncListener | None = None,
    on_error: AsyncListener | None = None,
) -> Runnable[Input, Output]

Bind async lifecycle listeners to a Runnable.

Returns a new Runnable.

The Run object contains information about the run, including its id, type, input, output, error, start_time, end_time, and any tags or metadata added to the run.

PARAMETER	DESCRIPTION
`on_start`	Called asynchronously before the `Runnable` starts running, with the `Run` object. TYPE: `AsyncListener \| None` DEFAULT: `None`
`on_end`	Called asynchronously after the `Runnable` finishes running, with the `Run` object. TYPE: `AsyncListener \| None` DEFAULT: `None`
`on_error`	Called asynchronously if the `Runnable` throws an error, with the `Run` object. TYPE: `AsyncListener \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Runnable[Input, Output]`	A new `Runnable` with the listeners bound.

Example

from langchain_core.runnables import RunnableLambda, Runnable
from datetime import datetime, timezone
import time
import asyncio


def format_t(timestamp: float) -> str:
    return datetime.fromtimestamp(timestamp, tz=timezone.utc).isoformat()


async def test_runnable(time_to_sleep: int):
    print(f"Runnable[{time_to_sleep}s]: starts at {format_t(time.time())}")
    await asyncio.sleep(time_to_sleep)
    print(f"Runnable[{time_to_sleep}s]: ends at {format_t(time.time())}")


async def fn_start(run_obj: Runnable):
    print(f"on start callback starts at {format_t(time.time())}")
    await asyncio.sleep(3)
    print(f"on start callback ends at {format_t(time.time())}")


async def fn_end(run_obj: Runnable):
    print(f"on end callback starts at {format_t(time.time())}")
    await asyncio.sleep(2)
    print(f"on end callback ends at {format_t(time.time())}")


runnable = RunnableLambda(test_runnable).with_alisteners(
    on_start=fn_start, on_end=fn_end
)


async def concurrent_runs():
    await asyncio.gather(runnable.ainvoke(2), runnable.ainvoke(3))


asyncio.run(concurrent_runs())
# Result:
# on start callback starts at 2025-03-01T07:05:22.875378+00:00
# on start callback starts at 2025-03-01T07:05:22.875495+00:00
# on start callback ends at 2025-03-01T07:05:25.878862+00:00
# on start callback ends at 2025-03-01T07:05:25.878947+00:00
# Runnable[2s]: starts at 2025-03-01T07:05:25.879392+00:00
# Runnable[3s]: starts at 2025-03-01T07:05:25.879804+00:00
# Runnable[2s]: ends at 2025-03-01T07:05:27.881998+00:00
# on end callback starts at 2025-03-01T07:05:27.882360+00:00
# Runnable[3s]: ends at 2025-03-01T07:05:28.881737+00:00
# on end callback starts at 2025-03-01T07:05:28.882428+00:00
# on end callback ends at 2025-03-01T07:05:29.883893+00:00
# on end callback ends at 2025-03-01T07:05:30.884831+00:00

with_types ¶

with_types(
    *, input_type: type[Input] | None = None, output_type: type[Output] | None = None
) -> Runnable[Input, Output]

Bind input and output types to a Runnable, returning a new Runnable.

PARAMETER	DESCRIPTION
`input_type`	The input type to bind to the `Runnable`. TYPE: `type[Input] \| None` DEFAULT: `None`
`output_type`	The output type to bind to the `Runnable`. TYPE: `type[Output] \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Runnable[Input, Output]`	A new `Runnable` with the types bound.

with_retry ¶

with_retry(
    *,
    retry_if_exception_type: tuple[type[BaseException], ...] = (Exception,),
    wait_exponential_jitter: bool = True,
    exponential_jitter_params: ExponentialJitterParams | None = None,
    stop_after_attempt: int = 3,
) -> Runnable[Input, Output]

Create a new Runnable that retries the original Runnable on exceptions.

PARAMETER	DESCRIPTION
`retry_if_exception_type`	A tuple of exception types to retry on. TYPE: `tuple[type[BaseException], ...]` DEFAULT: `(Exception,)`
`wait_exponential_jitter`	Whether to add jitter to the wait time between retries. TYPE: `bool` DEFAULT: `True`
`stop_after_attempt`	The maximum number of attempts to make before giving up. TYPE: `int` DEFAULT: `3`
`exponential_jitter_params`	Parameters for `tenacity.wait_exponential_jitter`. Namely: `initial`, `max`, `exp_base`, and `jitter` (all `float` values). TYPE: `ExponentialJitterParams \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`Runnable[Input, Output]`	A new `Runnable` that retries the original `Runnable` on exceptions.

Example

from langchain_core.runnables import RunnableLambda

count = 0


def _lambda(x: int) -> None:
    global count
    count = count + 1
    if x == 1:
        raise ValueError("x is 1")
    else:
        pass


runnable = RunnableLambda(_lambda)
try:
    runnable.with_retry(
        stop_after_attempt=2,
        retry_if_exception_type=(ValueError,),
    ).invoke(1)
except ValueError:
    pass

assert count == 2

map ¶

map() -> Runnable[list[Input], list[Output]]

Return a new Runnable that maps a list of inputs to a list of outputs.

Calls invoke with each input.

RETURNS	DESCRIPTION
`Runnable[list[Input], list[Output]]`	A new `Runnable` that maps a list of inputs to a list of outputs.

Example

from langchain_core.runnables import RunnableLambda


def _lambda(x: int) -> int:
    return x + 1


runnable = RunnableLambda(_lambda)
print(runnable.map().invoke([1, 2, 3]))  # [2, 3, 4]

with_fallbacks ¶

with_fallbacks(
    fallbacks: Sequence[Runnable[Input, Output]],
    *,
    exceptions_to_handle: tuple[type[BaseException], ...] = (Exception,),
    exception_key: str | None = None,
) -> RunnableWithFallbacks[Input, Output]

Add fallbacks to a Runnable, returning a new Runnable.

The new Runnable will try the original Runnable, and then each fallback in order, upon failures.

PARAMETER	DESCRIPTION
`fallbacks`	A sequence of runnables to try if the original `Runnable` fails. TYPE: `Sequence[Runnable[Input, Output]]`
`exceptions_to_handle`	A tuple of exception types to handle. TYPE: `tuple[type[BaseException], ...]` DEFAULT: `(Exception,)`
`exception_key`	If `string` is specified then handled exceptions will be passed to fallbacks as part of the input under the specified key. If `None`, exceptions will not be passed to fallbacks. If used, the base `Runnable` and its fallbacks must accept a dictionary as input. TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`RunnableWithFallbacks[Input, Output]`	A new `Runnable` that will try the original `Runnable`, and then each Fallback in order, upon failures.

Example

from typing import Iterator

from langchain_core.runnables import RunnableGenerator


def _generate_immediate_error(input: Iterator) -> Iterator[str]:
    raise ValueError()
    yield ""


def _generate(input: Iterator) -> Iterator[str]:
    yield from "foo bar"


runnable = RunnableGenerator(_generate_immediate_error).with_fallbacks(
    [RunnableGenerator(_generate)]
)
print("".join(runnable.stream({})))  # foo bar

PARAMETER	DESCRIPTION
`fallbacks`	A sequence of runnables to try if the original `Runnable` fails. TYPE: `Sequence[Runnable[Input, Output]]`
`exceptions_to_handle`	A tuple of exception types to handle. TYPE: `tuple[type[BaseException], ...]` DEFAULT: `(Exception,)`
`exception_key`	If `string` is specified then handled exceptions will be passed to fallbacks as part of the input under the specified key. If `None`, exceptions will not be passed to fallbacks. If used, the base `Runnable` and its fallbacks must accept a dictionary as input. TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`RunnableWithFallbacks[Input, Output]`	A new `Runnable` that will try the original `Runnable`, and then each Fallback in order, upon failures.

as_tool ¶

as_tool(
    args_schema: type[BaseModel] | None = None,
    *,
    name: str | None = None,
    description: str | None = None,
    arg_types: dict[str, type] | None = None,
) -> BaseTool

Create a BaseTool from a Runnable.

as_tool will instantiate a BaseTool with a name, description, and args_schema from a Runnable. Where possible, schemas are inferred from runnable.get_input_schema.

Alternatively (e.g., if the Runnable takes a dict as input and the specific dict keys are not typed), the schema can be specified directly with args_schema.

You can also pass arg_types to just specify the required arguments and their types.

PARAMETER	DESCRIPTION
`args_schema`	The schema for the tool. TYPE: `type[BaseModel] \| None` DEFAULT: `None`
`name`	The name of the tool. TYPE: `str \| None` DEFAULT: `None`
`description`	The description of the tool. TYPE: `str \| None` DEFAULT: `None`
`arg_types`	A dictionary of argument names to types. TYPE: `dict[str, type] \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`BaseTool`	A `BaseTool` instance.

Typed dict input:

from typing_extensions import TypedDict
from langchain_core.runnables import RunnableLambda


class Args(TypedDict):
    a: int
    b: list[int]


def f(x: Args) -> str:
    return str(x["a"] * max(x["b"]))


runnable = RunnableLambda(f)
as_tool = runnable.as_tool()
as_tool.invoke({"a": 3, "b": [1, 2]})

dict input, specifying schema via args_schema:

from typing import Any
from pydantic import BaseModel, Field
from langchain_core.runnables import RunnableLambda

def f(x: dict[str, Any]) -> str:
    return str(x["a"] * max(x["b"]))

class FSchema(BaseModel):
    """Apply a function to an integer and list of integers."""

    a: int = Field(..., description="Integer")
    b: list[int] = Field(..., description="List of ints")

runnable = RunnableLambda(f)
as_tool = runnable.as_tool(FSchema)
as_tool.invoke({"a": 3, "b": [1, 2]})

dict input, specifying schema via arg_types:

from typing import Any
from langchain_core.runnables import RunnableLambda


def f(x: dict[str, Any]) -> str:
    return str(x["a"] * max(x["b"]))


runnable = RunnableLambda(f)
as_tool = runnable.as_tool(arg_types={"a": int, "b": list[int]})
as_tool.invoke({"a": 3, "b": [1, 2]})

str input:

from langchain_core.runnables import RunnableLambda


def f(x: str) -> str:
    return x + "a"


def g(x: str) -> str:
    return x + "z"


runnable = RunnableLambda(f) | g
as_tool = runnable.as_tool()
as_tool.invoke("b")

init ¶

__init__(*args: Any, **kwargs: Any) -> None

get_lc_namespace `classmethod` ¶

get_lc_namespace() -> list[str]

Get the namespace of the LangChain object.

For example, if the class is langchain.llms.openai.OpenAI, then the namespace is ["langchain", "llms", "openai"]

RETURNS	DESCRIPTION
`list[str]`	The namespace.

lc_id `classmethod` ¶

lc_id() -> list[str]

Return a unique identifier for this class for serialization purposes.

The unique identifier is a list of strings that describes the path to the object.

For example, for the class langchain.llms.openai.OpenAI, the id is ["langchain", "llms", "openai", "OpenAI"].

to_json ¶

to_json() -> SerializedConstructor | SerializedNotImplemented

Serialize the Runnable to JSON.

RETURNS	DESCRIPTION
`SerializedConstructor \| SerializedNotImplemented`	A JSON-serializable representation of the `Runnable`.

to_json_not_implemented ¶

to_json_not_implemented() -> SerializedNotImplemented

Serialize a "not implemented" object.

RETURNS	DESCRIPTION
`SerializedNotImplemented`	`SerializedNotImplemented`.

configurable_fields ¶

configurable_fields(
    **kwargs: AnyConfigurableField,
) -> RunnableSerializable[Input, Output]

Configure particular Runnable fields at runtime.

PARAMETER	DESCRIPTION
`**kwargs`	A dictionary of `ConfigurableField` instances to configure. TYPE: `AnyConfigurableField` DEFAULT: `{}`

RAISES	DESCRIPTION
`ValueError`	If a configuration key is not found in the `Runnable`.

RETURNS	DESCRIPTION
`RunnableSerializable[Input, Output]`	A new `Runnable` with the fields configured.

from langchain_core.runnables import ConfigurableField
from langchain_openai import ChatOpenAI

model = ChatOpenAI(max_tokens=20).configurable_fields(
    max_tokens=ConfigurableField(
        id="output_token_number",
        name="Max tokens in the output",
        description="The maximum number of tokens in the output",
    )
)

# max_tokens = 20
print("max_tokens_20: ", model.invoke("tell me something about chess").content)

# max_tokens = 200
print(
    "max_tokens_200: ",
    model.with_config(configurable={"output_token_number": 200})
    .invoke("tell me something about chess")
    .content,
)

configurable_alternatives ¶

configurable_alternatives(
    which: ConfigurableField,
    *,
    default_key: str = "default",
    prefix_keys: bool = False,
    **kwargs: Runnable[Input, Output] | Callable[[], Runnable[Input, Output]],
) -> RunnableSerializable[Input, Output]

Configure alternatives for Runnable objects that can be set at runtime.

PARAMETER	DESCRIPTION
`which`	The `ConfigurableField` instance that will be used to select the alternative. TYPE: `ConfigurableField`
`default_key`	The default key to use if no alternative is selected. TYPE: `str` DEFAULT: `'default'`
`prefix_keys`	Whether to prefix the keys with the `ConfigurableField` id. TYPE: `bool` DEFAULT: `False`
`**kwargs`	A dictionary of keys to `Runnable` instances or callables that return `Runnable` instances. TYPE: `Runnable[Input, Output] \| Callable[[], Runnable[Input, Output]]` DEFAULT: `{}`

RETURNS	DESCRIPTION
`RunnableSerializable[Input, Output]`	A new `Runnable` with the alternatives configured.

from langchain_anthropic import ChatAnthropic
from langchain_core.runnables.utils import ConfigurableField
from langchain_openai import ChatOpenAI

model = ChatAnthropic(
    model_name="claude-sonnet-4-5-20250929"
).configurable_alternatives(
    ConfigurableField(id="llm"),
    default_key="anthropic",
    openai=ChatOpenAI(),
)

# uses the default model ChatAnthropic
print(model.invoke("which organization created you?").content)

# uses ChatOpenAI
print(
    model.with_config(configurable={"llm": "openai"})
    .invoke("which organization created you?")
    .content
)

set_verbose ¶

set_verbose(verbose: bool | None) -> bool

If verbose is None, set it.

This allows users to pass in None as verbose to access the global setting.

PARAMETER	DESCRIPTION
`verbose`	The verbosity setting to use. TYPE: `bool \| None`

RETURNS	DESCRIPTION
`bool`	The verbosity setting to use.

generate_prompt ¶

generate_prompt(
    prompts: list[PromptValue],
    stop: list[str] | None = None,
    callbacks: Callbacks = None,
    **kwargs: Any,
) -> LLMResult

Pass a sequence of prompts to the model and return model generations.

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to:

Take advantage of batched calls,
Need more output from the model than just the top generated value,
Are building chains that are agnostic to the underlying language model type (e.g., pure text completion models vs chat models).

PARAMETER	DESCRIPTION
`prompts`	List of `PromptValue` objects. A `PromptValue` is an object that can be converted to match the format of any language model (string for pure text generation models and `BaseMessage` objects for chat models). TYPE: `list[PromptValue]`
`stop`	Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings. TYPE: `list[str] \| None` DEFAULT: `None`
`callbacks`	`Callbacks` to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation. TYPE: `Callbacks` DEFAULT: `None`
`**kwargs`	Arbitrary additional keyword arguments. These are usually passed to the model provider API call. TYPE: `Any` DEFAULT: `{}`

RETURNS	DESCRIPTION
`LLMResult`	An `LLMResult`, which contains a list of candidate `Generation` objects for each input prompt and additional model provider-specific output.

agenerate_prompt `async` ¶

agenerate_prompt(
    prompts: list[PromptValue],
    stop: list[str] | None = None,
    callbacks: Callbacks = None,
    **kwargs: Any,
) -> LLMResult

Asynchronously pass a sequence of prompts and return model generations.

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to:

Take advantage of batched calls,
Need more output from the model than just the top generated value,
Are building chains that are agnostic to the underlying language model type (e.g., pure text completion models vs chat models).

PARAMETER	DESCRIPTION
`prompts`	List of `PromptValue` objects. A `PromptValue` is an object that can be converted to match the format of any language model (string for pure text generation models and `BaseMessage` objects for chat models). TYPE: `list[PromptValue]`
`stop`	Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings. TYPE: `list[str] \| None` DEFAULT: `None`
`callbacks`	`Callbacks` to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation. TYPE: `Callbacks` DEFAULT: `None`
`**kwargs`	Arbitrary additional keyword arguments. These are usually passed to the model provider API call. TYPE: `Any` DEFAULT: `{}`

RETURNS	DESCRIPTION
`LLMResult`	An `LLMResult`, which contains a list of candidate `Generation` objects for each input prompt and additional model provider-specific output.

get_token_ids ¶

get_token_ids(text: str) -> list[int]

Return the ordered IDs of the tokens in a text.

PARAMETER	DESCRIPTION
`text`	The string input to tokenize. TYPE: `str`

RETURNS	DESCRIPTION
`list[int]`	A list of IDs corresponding to the tokens in the text, in order they occur in the text.

get_num_tokens ¶

get_num_tokens(text: str) -> int

Get the number of tokens present in the text.

Useful for checking if an input fits in a model's context window.

PARAMETER	DESCRIPTION
`text`	The string input to tokenize. TYPE: `str`

RETURNS	DESCRIPTION
`int`	The integer number of tokens in the text.

get_num_tokens_from_messages ¶

get_num_tokens_from_messages(
    messages: list[BaseMessage], tools: Sequence | None = None
) -> int

Get the number of tokens in the messages.

Useful for checking if an input fits in a model's context window.

Note

The base implementation of get_num_tokens_from_messages ignores tool schemas.

PARAMETER	DESCRIPTION
`messages`	The message inputs to tokenize. TYPE: `list[BaseMessage]`
`tools`	If provided, sequence of dict, `BaseModel`, function, or `BaseTool` objects to be converted to tool schemas. TYPE: `Sequence \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`int`	The sum of the number of tokens across the messages.

generate ¶

generate(
    messages: list[list[BaseMessage]],
    stop: list[str] | None = None,
    callbacks: Callbacks = None,
    *,
    tags: list[str] | None = None,
    metadata: dict[str, Any] | None = None,
    run_name: str | None = None,
    run_id: UUID | None = None,
    **kwargs: Any,
) -> LLMResult

Pass a sequence of prompts to the model and return model generations.

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to:

Take advantage of batched calls,
Need more output from the model than just the top generated value,
Are building chains that are agnostic to the underlying language model type (e.g., pure text completion models vs chat models).

PARAMETER	DESCRIPTION
`messages`	List of list of messages. TYPE: `list[list[BaseMessage]]`
`stop`	Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings. TYPE: `list[str] \| None` DEFAULT: `None`
`callbacks`	`Callbacks` to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation. TYPE: `Callbacks` DEFAULT: `None`
`tags`	The tags to apply. TYPE: `list[str] \| None` DEFAULT: `None`
`metadata`	The metadata to apply. TYPE: `dict[str, Any] \| None` DEFAULT: `None`
`run_name`	The name of the run. TYPE: `str \| None` DEFAULT: `None`
`run_id`	The ID of the run. TYPE: `UUID \| None` DEFAULT: `None`
`**kwargs`	Arbitrary additional keyword arguments. These are usually passed to the model provider API call. TYPE: `Any` DEFAULT: `{}`

RETURNS	DESCRIPTION
`LLMResult`	An `LLMResult`, which contains a list of candidate `Generations` for each input prompt and additional model provider-specific output.

agenerate `async` ¶

agenerate(
    messages: list[list[BaseMessage]],
    stop: list[str] | None = None,
    callbacks: Callbacks = None,
    *,
    tags: list[str] | None = None,
    metadata: dict[str, Any] | None = None,
    run_name: str | None = None,
    run_id: UUID | None = None,
    **kwargs: Any,
) -> LLMResult

Asynchronously pass a sequence of prompts to a model and return generations.

This method should make use of batched calls for models that expose a batched API.

Use this method when you want to:

Take advantage of batched calls,
Need more output from the model than just the top generated value,
Are building chains that are agnostic to the underlying language model type (e.g., pure text completion models vs chat models).

PARAMETER	DESCRIPTION
`messages`	List of list of messages. TYPE: `list[list[BaseMessage]]`
`stop`	Stop words to use when generating. Model output is cut off at the first occurrence of any of these substrings. TYPE: `list[str] \| None` DEFAULT: `None`
`callbacks`	`Callbacks` to pass through. Used for executing additional functionality, such as logging or streaming, throughout generation. TYPE: `Callbacks` DEFAULT: `None`
`tags`	The tags to apply. TYPE: `list[str] \| None` DEFAULT: `None`
`metadata`	The metadata to apply. TYPE: `dict[str, Any] \| None` DEFAULT: `None`
`run_name`	The name of the run. TYPE: `str \| None` DEFAULT: `None`
`run_id`	The ID of the run. TYPE: `UUID \| None` DEFAULT: `None`
`**kwargs`	Arbitrary additional keyword arguments. These are usually passed to the model provider API call. TYPE: `Any` DEFAULT: `{}`

RETURNS	DESCRIPTION
`LLMResult`	An `LLMResult`, which contains a list of candidate `Generations` for each input prompt and additional model provider-specific output.

dict ¶

dict(**kwargs: Any) -> dict

Return a dictionary of the LLM.

ChatWatsonx¶

langchain_ibm.chat_models.ChatWatsonx ¶

model_id class-attribute instance-attribute ¶

model class-attribute instance-attribute ¶

deployment_id class-attribute instance-attribute ¶

project_id class-attribute instance-attribute ¶

space_id class-attribute instance-attribute ¶

url class-attribute instance-attribute ¶

api_key class-attribute instance-attribute ¶

token class-attribute instance-attribute ¶

password class-attribute instance-attribute ¶

username class-attribute instance-attribute ¶

instance_id class-attribute instance-attribute ¶

version class-attribute instance-attribute ¶

params class-attribute instance-attribute ¶

frequency_penalty class-attribute instance-attribute ¶

logprobs class-attribute instance-attribute ¶

top_logprobs class-attribute instance-attribute ¶

max_tokens class-attribute instance-attribute ¶

max_completion_tokens class-attribute instance-attribute ¶

n class-attribute instance-attribute ¶

presence_penalty class-attribute instance-attribute ¶

temperature class-attribute instance-attribute ¶

response_format class-attribute instance-attribute ¶

top_p class-attribute instance-attribute ¶

time_limit class-attribute instance-attribute ¶

logit_bias class-attribute instance-attribute ¶

seed class-attribute instance-attribute ¶

stop class-attribute instance-attribute ¶

chat_template_kwargs class-attribute instance-attribute ¶

verify class-attribute instance-attribute ¶

validate_model class-attribute instance-attribute ¶

streaming class-attribute instance-attribute ¶

lc_secrets property ¶

name class-attribute instance-attribute ¶

InputType property ¶

OutputType property ¶

input_schema property ¶

output_schema property ¶

config_specs property ¶

lc_attributes property ¶

cache class-attribute instance-attribute ¶

verbose class-attribute instance-attribute ¶

callbacks class-attribute instance-attribute ¶

tags class-attribute instance-attribute ¶

metadata class-attribute instance-attribute ¶

custom_get_token_ids class-attribute instance-attribute ¶

rate_limiter class-attribute instance-attribute ¶

disable_streaming class-attribute instance-attribute ¶

output_version class-attribute instance-attribute ¶

profile property ¶

is_lc_serializable classmethod ¶

validate_environment ¶

bind_tools ¶

with_structured_output ¶

get_name ¶

get_input_schema ¶

get_input_jsonschema ¶

get_output_schema ¶

get_output_jsonschema ¶

config_schema ¶

get_config_jsonschema ¶

get_graph ¶

get_prompts ¶

__or__ ¶

__ror__ ¶

pipe ¶

pick ¶

assign ¶

invoke ¶

ainvoke async ¶

batch ¶

batch_as_completed ¶

abatch async ¶

abatch_as_completed async ¶

stream ¶

astream async ¶

astream_log async ¶

astream_events async ¶

transform ¶

`ChatWatsonx`¶

model_id `class-attribute` `instance-attribute` ¶

model `class-attribute` `instance-attribute` ¶

deployment_id `class-attribute` `instance-attribute` ¶

project_id `class-attribute` `instance-attribute` ¶

space_id `class-attribute` `instance-attribute` ¶

url `class-attribute` `instance-attribute` ¶

api_key `class-attribute` `instance-attribute` ¶

token `class-attribute` `instance-attribute` ¶

password `class-attribute` `instance-attribute` ¶

username `class-attribute` `instance-attribute` ¶

instance_id `class-attribute` `instance-attribute` ¶

version `class-attribute` `instance-attribute` ¶

params `class-attribute` `instance-attribute` ¶

frequency_penalty `class-attribute` `instance-attribute` ¶

logprobs `class-attribute` `instance-attribute` ¶

top_logprobs `class-attribute` `instance-attribute` ¶

max_tokens `class-attribute` `instance-attribute` ¶

max_completion_tokens `class-attribute` `instance-attribute` ¶

n `class-attribute` `instance-attribute` ¶

presence_penalty `class-attribute` `instance-attribute` ¶

temperature `class-attribute` `instance-attribute` ¶

response_format `class-attribute` `instance-attribute` ¶

top_p `class-attribute` `instance-attribute` ¶

time_limit `class-attribute` `instance-attribute` ¶

logit_bias `class-attribute` `instance-attribute` ¶

seed `class-attribute` `instance-attribute` ¶

stop `class-attribute` `instance-attribute` ¶

chat_template_kwargs `class-attribute` `instance-attribute` ¶

verify `class-attribute` `instance-attribute` ¶

validate_model `class-attribute` `instance-attribute` ¶

streaming `class-attribute` `instance-attribute` ¶

lc_secrets `property` ¶

name `class-attribute` `instance-attribute` ¶

InputType `property` ¶

OutputType `property` ¶

input_schema `property` ¶

output_schema `property` ¶

config_specs `property` ¶

lc_attributes `property` ¶

cache `class-attribute` `instance-attribute` ¶

verbose `class-attribute` `instance-attribute` ¶

callbacks `class-attribute` `instance-attribute` ¶

tags `class-attribute` `instance-attribute` ¶

metadata `class-attribute` `instance-attribute` ¶

custom_get_token_ids `class-attribute` `instance-attribute` ¶

rate_limiter `class-attribute` `instance-attribute` ¶

disable_streaming `class-attribute` `instance-attribute` ¶

output_version `class-attribute` `instance-attribute` ¶

profile `property` ¶

is_lc_serializable `classmethod` ¶

or ¶

ror ¶

ainvoke `async` ¶

abatch `async` ¶

abatch_as_completed `async` ¶

astream `async` ¶

astream_log `async` ¶

astream_events `async` ¶

atransform `async` ¶

init ¶

get_lc_namespace `classmethod` ¶

lc_id `classmethod` ¶

agenerate_prompt `async` ¶

agenerate `async` ¶