| Name | Type | Description |
|---|---|---|
model* | BaseChatModel | The chat model instance to use for the evaluation. |
prompt_template* | Union[str, List[Tuple[str, str]] | |
score_config* | Union[CategoricalScoreConfig, ContinuousScoreConfig] | |
map_variables | Optional[Callable[[Run, Example]], dict]] | Default: None |
Create an LLMEvaluator instance from a BaseChatModel instance.
The prompt template to use for the evaluation. If a string is provided, it is assumed to be a system message.
The configuration for the score, either categorical or continuous.
A function that maps the run and example to the variables in the prompt.
If None, it is assumed that the prompt only requires 'input',
'output', and 'expected'.
