Skip to content

Base tests

ChatModelTests

Bases: BaseStandardTests

Base class for chat model tests.

METHOD DESCRIPTION
model

Model fixture.
my_adder_tool

Adder tool fixture.
test_no_overrides_DO_NOT_OVERRIDE

Test that no standard tests are overridden.

chat_model_class abstractmethod property 

chat_model_class: type[BaseChatModel]

The chat model class to test, e.g., ChatParrotLink.

chat_model_params property 

chat_model_params: dict

Initialization parameters for the chat model.

standard_chat_model_params property 

standard_chat_model_params: dict

Standard chat model parameters.

has_tool_calling property 

has_tool_calling: bool

Whether the model supports tool calling.

tool_choice_value property 

tool_choice_value: str | None

(None or str) to use for tool choice when used in tests.

has_tool_choice property 

has_tool_choice: bool

Whether the model supports tool calling.

has_structured_output property 

has_structured_output: bool

Whether the chat model supports structured output.

structured_output_kwargs property 

structured_output_kwargs: dict

Additional kwargs to pass to with_structured_output() in tests.

Override this property to customize how structured output is generated for your model. The most common use case is specifying the method parameter, which controls the mechanism used to enforce structured output:

  • 'function_calling': Uses tool/function calling to enforce the schema.
  • 'json_mode': Uses the model's JSON mode.
  • 'json_schema': Uses native JSON schema support (e.g., OpenAI's structured outputs).
RETURNS DESCRIPTION
dict

A dict of kwargs passed to with_structured_output().
Example 
@property
def structured_output_kwargs(self) -> dict:
    return {"method": "json_schema"}

supports_json_mode property 

supports_json_mode: bool

Whether the chat model supports JSON mode.

supports_image_inputs property 

supports_image_inputs: bool

Supports image inputs.

Whether the chat model supports image inputs, defaults to False.

supports_image_urls property 

supports_image_urls: bool

Supports image inputs from URLs.

Whether the chat model supports image inputs from URLs, defaults to False.

supports_pdf_inputs property 

supports_pdf_inputs: bool

Whether the chat model supports PDF inputs, defaults to False.

supports_audio_inputs property 

supports_audio_inputs: bool

Supports audio inputs.

Whether the chat model supports audio inputs, defaults to False.

supports_video_inputs property 

supports_video_inputs: bool

Supports video inputs.

Whether the chat model supports video inputs, defaults to False.

No current tests are written for this feature.

returns_usage_metadata property 

returns_usage_metadata: bool

Returns usage metadata.

Whether the chat model returns usage metadata on invoke and streaming responses.

supports_anthropic_inputs property 

supports_anthropic_inputs: bool

Whether the chat model supports Anthropic-style inputs.

supports_image_tool_message property 

supports_image_tool_message: bool

Supports image ToolMessage objects.

Whether the chat model supports ToolMessage objects that include image content.

supports_pdf_tool_message property 

supports_pdf_tool_message: bool

Supports PDF ToolMessage objects.

Whether the chat model supports ToolMessage objects that include PDF content.

enable_vcr_tests property 

enable_vcr_tests: bool

Whether to enable VCR tests for the chat model.

Warning

See enable_vcr_tests dropdown above <ChatModelTests> for more information.

supported_usage_metadata_details property 

supported_usage_metadata_details: dict[
    Literal["invoke", "stream"],
    list[
        Literal[
            "audio_input",
            "audio_output",
            "reasoning_output",
            "cache_read_input",
            "cache_creation_input",
        ]
    ],
]

Supported usage metadata details.

What usage metadata details are emitted in invoke and stream. Only needs to be overridden if these details are returned by the model.

model 

model(request: Any) -> BaseChatModel

Model fixture.

my_adder_tool 

my_adder_tool() -> BaseTool

Adder tool fixture.

test_no_overrides_DO_NOT_OVERRIDE 

test_no_overrides_DO_NOT_OVERRIDE() -> None

Test that no standard tests are overridden.

EmbeddingsTests

Bases: BaseStandardTests

Embeddings tests base class.

METHOD DESCRIPTION
model

Embeddings model fixture.
test_no_overrides_DO_NOT_OVERRIDE

Test that no standard tests are overridden.

embeddings_class abstractmethod property 

embeddings_class: type[Embeddings]

Embeddings class.

embedding_model_params property 

embedding_model_params: dict

Embeddings model parameters.

model 

model() -> Embeddings

Embeddings model fixture.

test_no_overrides_DO_NOT_OVERRIDE 

test_no_overrides_DO_NOT_OVERRIDE() -> None

Test that no standard tests are overridden.

ToolsTests

Bases: BaseStandardTests

Base class for testing tools.

This won't show in the documentation, but the docstrings will be inherited by subclasses.

METHOD DESCRIPTION
tool

Tool fixture.
test_no_overrides_DO_NOT_OVERRIDE

Test that no standard tests are overridden.

tool_constructor abstractmethod property 

tool_constructor: type[BaseTool] | BaseTool

Returns a class or instance of a tool to be tested.

tool_constructor_params property 

tool_constructor_params: dict

Returns a dictionary of parameters to pass to the tool constructor.

tool_invoke_params_example property 

tool_invoke_params_example: dict

Returns a dictionary representing the "args" of an example tool call.

This should NOT be a ToolCall dict - it should not have {"name", "id", "args"} keys.

tool 

tool() -> BaseTool

Tool fixture.

test_no_overrides_DO_NOT_OVERRIDE 

test_no_overrides_DO_NOT_OVERRIDE() -> None

Test that no standard tests are overridden.

BaseStandardTests

Base class for standard tests.

METHOD DESCRIPTION
test_no_overrides_DO_NOT_OVERRIDE

Test that no standard tests are overridden.

test_no_overrides_DO_NOT_OVERRIDE 

test_no_overrides_DO_NOT_OVERRIDE() -> None

Test that no standard tests are overridden.