evaluator

Create a run evaluator from a function.

Decorator that transforms a function into a RunEvaluator.

Create a comaprison evaluator from a function.

Example model.

Run schema when loading from the DB.

A category for categorical feedback.

Configuration to define a type of feedback.

Applied on on the first creation of a feedback_key.

Evaluation result.

Batch evaluation results.

This makes it easy for your evaluator to return multiple metrics at once.

Evaluator interface class.

Feedback scores for the results of comparative evaluations.

These are generated by functions that compare two or more runs, returning a ranking or other feedback.

A dynamic evaluator that wraps a function and transforms it into a RunEvaluator.

This class is designed to be used with the @run_evaluator decorator, allowing functions that take a Run and an optional Example as arguments, and return an EvaluationResult or EvaluationResults, to be used as instances of RunEvaluator.

Compare predictions (as traces) from 2 or more runs.

Decorator for creating a run tree from functions.

Schemas for the LangSmith API.