A dynamic evaluator that wraps a function and transforms it into a RunEvaluator.
This class is designed to be used with the @run_evaluator decorator, allowing
functions that take a Run and an optional Example as arguments, and return
an EvaluationResult or EvaluationResults, to be used as instances of RunEvaluator.
DynamicRunEvaluator(
self,
func: Callable[[Run, Optional[Example]], Union[_RUNNABLE_OUTPUT, Awaitable[_RUNNABLE_OUTPUT]]],
afunc: Optional[Callable[[Run, Optional[Example]], Awaitable[_RUNNABLE_OUTPUT]]] = None
)| Name | Type | Description |
|---|---|---|
func* | Callable | A function that takes a |
Evaluate a run using the wrapped function.
This method directly invokes the wrapped function with the provided arguments.
Evaluate a run asynchronously using the wrapped async function.
This method directly invokes the wrapped async function with the provided arguments.