OptionalclientOptionalevaluationOptionalevaluatorsEvaluators to apply to a dataset run. You can optionally specify these by name, or by configuring them with an EvalConfig object.
OptionalformatConvert the evaluation data into formats that can be used by the evaluator. This should most commonly be a string. Parameters are the raw input from the run, the raw output, raw reference output, and the raw run.
// Chain input: { input: "some string" }
// Chain output: { output: "some output" }
// Reference example output format: { output: "some reference output" }
const formatEvaluatorInputs = ({
rawInput,
rawPrediction,
rawReferenceOutput,
}) => {
return {
input: rawInput.input,
prediction: rawPrediction.output,
reference: rawReferenceOutput.output,
};
};
OptionalmaxMaximum concurrency level for dataset processing.
OptionalprojectAdditional metadata for the project.
OptionalprojectName of the project for logging and tracking.
Client instance for LangSmith service interaction.