| Name | Type | Description |
|---|---|---|
prompt_template* | Union[str, List[Tuple[str, str]] | The prompt template to use for the evaluation. If a string is provided, it is assumed to be a human / user message. |
score_config* | Union[CategoricalScoreConfig, ContinuousScoreConfig] | |
map_variables | Optional[Callable[[Run, Example], dict]] | Default: None |
model_name | Optional[str] | Default: 'gpt-4o' |
model_provider | Optional[str] | Default: 'openai' |
A class for building LLM-as-a-judge evaluators.
.. deprecated:: 0.5.0
LLMEvaluator is deprecated. Use openevals instead: https://github.com/langchain-ai/openevals
The configuration for the score, either categorical or continuous.
A function that maps the run and example to the variables in the prompt.
If None, it is assumed that the prompt only requires 'input',
'output', and 'expected'.
The model to use for the evaluation.
The model provider to use for the evaluation.
