Create a LabeledCriteriaEvalChain instance from an llm and criteria.
Parameters
llm : BaseLanguageModel
The language model to use for evaluation.
criteria : CRITERIA_TYPE - default=None for "helpfulness"
The criteria to evaluate the runs against. It can be:
- a mapping of a criterion name to its description
- a single criterion name present in one of the default criteria
- a single ConstitutionalPrinciple instance
prompt : Optional[BasePromptTemplate], default=None
The prompt template to use for generating prompts. If not provided,
a default prompt will be used.
**kwargs : Any
Additional keyword arguments to pass to the LLMChain
constructor.
Returns:
LabeledCriteriaEvalChain
An instance of the LabeledCriteriaEvalChain class.
Examples:
from langchain_openai import OpenAI
from langchain_classic.evaluation.criteria import LabeledCriteriaEvalChain
model = OpenAI()
criteria = {
"hallucination": (
"Does this submission contain information"
" not present in the input or reference?"
),
}
chain = LabeledCriteriaEvalChain.from_llm(
llm=model,
criteria=criteria,
)
