Type Alias Criteria

Criteria: EvalConfig & {
    criteria?: Criteria | Record<string, string>;
    evaluatorType: "criteria";
    llm?: Toolkit;
}

Configuration to load a "CriteriaEvalChain" evaluator, which prompts an LLM to determine whether the model's prediction complies with the provided criteria.

Type Declaration

Optionalcriteria?: Criteria | Record<string, string>
The "criteria" to insert into the prompt template used for evaluation. See the prompt at https://smith.langchain.com/hub/langchain-ai/criteria-evaluator for more information.
evaluatorType: "criteria"
Optionalllm?: Toolkit
The language model to use as the evaluator, defaults to GPT-4

Param: criteria

The criteria to use for the evaluator.

Param: llm

The language model to use for the evaluator.

Returns

The configuration for the evaluator.

Example

const evalConfig = {
  evaluators: [Criteria("helpfulness")],
};
@example
```ts
const evalConfig = {
  evaluators: [
    Criteria({
      "isCompliant": "Does the submission comply with the requirements of XYZ"
    })
  ],

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "criteria",
    criteria: "helpfulness"
    formatEvaluatorInputs: ...
  }]
};

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "criteria",
    criteria: { "isCompliant": "Does the submission comply with the requirements of XYZ" },
    formatEvaluatorInputs: ...
  }]

Type Alias Criteria

Type Declaration

`Optional`criteria?: Criteria | Record<string, string>

evaluatorType: "criteria"

`Optional`llm?: Toolkit

Param: criteria

Param: llm

Returns

Example

Example

Example

Settings

Type Alias Criteria

Type Declaration

Optionalcriteria?: Criteria | Record<string, string>

evaluatorType: "criteria"

Optionalllm?: Toolkit

Param: criteria

Param: llm

Returns

Example

Example

Example

Settings

`Optional`criteria?: Criteria | Record<string, string>

`Optional`llm?: Toolkit