LabeledCriteriaEvalChain

Whether the evaluation requires a reference text.

Create a LabeledCriteriaEvalChain instance from an llm and criteria.

Parameters

llm : BaseLanguageModel The language model to use for evaluation. criteria : CRITERIA_TYPE - default=None for "helpfulness" The criteria to evaluate the runs against. It can be: - a mapping of a criterion name to its description - a single criterion name present in one of the default criteria - a single ConstitutionalPrinciple instance prompt : Optional[BasePromptTemplate], default=None The prompt template to use for generating prompts. If not provided, a default prompt will be used. **kwargs : Any Additional keyword arguments to pass to the LLMChain constructor.

Returns:

LabeledCriteriaEvalChain An instance of the LabeledCriteriaEvalChain class.

Examples:

from langchain_openai import OpenAI from langchain_classic.evaluation.criteria import LabeledCriteriaEvalChain model = OpenAI() criteria = { "hallucination": ( "Does this submission contain information" " not present in the input or reference?" ), } chain = LabeledCriteriaEvalChain.from_llm( llm=model, criteria=criteria, )