LangChain Reference home pageLangChain ReferenceLangChain Reference
  • GitHub
  • Main Docs
Deep Agents
LangChain
LangGraph
Integrations
LangSmith
  • Overview
  • Client
  • AsyncClient
  • Run Helpers
  • Run Trees
  • Evaluation
  • Schemas
  • Utilities
  • Wrappers
  • Anonymizer
  • Testing
  • Expect API
  • Middleware
  • Pytest Plugin
  • Deployment SDK
⌘I

LangChain Assistant

Ask a question to get started

Enter to send•Shift+Enter new line

Menu

OverviewClientAsyncClientRun HelpersRun TreesEvaluationSchemasUtilitiesWrappersAnonymizerTestingExpect APIMiddlewarePytest PluginDeployment SDK
Language
Theme
PythonlangsmithevaluationevaluatorDynamicRunEvaluator
Classā—Since v0.0

DynamicRunEvaluator

Copy
DynamicRunEvaluator(
  self,
  func: Callable[[Run, Optional[Example]], Union[_RUNNABLE_OUTPUT, Awaitable[

Bases

RunEvaluator

Constructors

Attributes

Methods

View source on GitHub
_RUNNABLE_OUTPUT
]
]
]
,
afunc
:
Optional
[
Callable
[
[
Run
,
Optional
[
Example
]
]
,
Awaitable
[
_RUNNABLE_OUTPUT
]
]
]
=
None
)

Parameters

NameTypeDescription
func*Callable

A function that takes a Run and an optional Example as

constructor
__init__
NameType
funcCallable[[Run, Optional[Example]], Union[_RUNNABLE_OUTPUT, Awaitable[_RUNNABLE_OUTPUT]]]
afuncOptional[Callable[[Run, Optional[Example]], Awaitable[_RUNNABLE_OUTPUT]]]
attribute
afunc
attribute
func
attribute
is_async: bool

Check if the evaluator function is asynchronous.

method
evaluate_run

Evaluate a run using the wrapped function.

This method directly invokes the wrapped function with the provided arguments.

method
aevaluate_run

Evaluate a run asynchronously using the wrapped async function.

This method directly invokes the wrapped async function with the provided arguments.

A dynamic evaluator that wraps a function and transforms it into a RunEvaluator.

This class is designed to be used with the @run_evaluator decorator, allowing functions that take a Run and an optional Example as arguments, and return an EvaluationResult or EvaluationResults, to be used as instances of RunEvaluator.