OptionalagentOptionalchainThe options to pass to the evaluator chain.
OptionalcriteriaThe criteria to use for the evaluator.
OptionaldistanceThe distance metric to use for comparing the embeddings.
OptionalembeddingThe embedding objects to vectorize the outputs.
The name of the evaluator to use. Example: labeled_criteria, criteria, etc.
OptionalfeedbackThe feedback (or metric) name to use for the logged evaluation results. If none provided, we default to the evaluationName.
Convert the evaluation data into formats that can be used by the evaluator. This should most commonly be a string. Parameters are the raw input from the run, the raw output, raw reference output, and the raw run.
// Chain input: { input: "some string" }
// Chain output: { output: "some output" }
// Reference example output format: { output: "some reference output" }
const formatEvaluatorInputs = ({
rawInput,
rawPrediction,
rawReferenceOutput,
}) => {
return {
input: rawInput.input,
prediction: rawPrediction.output,
reference: rawReferenceOutput.output,
};
};
The language model to use for the evaluator.
A list of tools available to the agent, for TrajectoryEvalChain.