interface EvaluatorCreateDecisionModelParamsCall this endpoint directly instead of through the LangSmith gateway, with the workspace's TYPESAFE_API_KEY or OPENAI_API_KEY secret. Required for typesafe_compatible.
The dataset whose experiments to evaluate.
The evaluator's name.
Only evaluate runs that match this filter.
How many times to retry a failed model call. openai_decisions only.
Defaults to jev-latest for typesafe and gpt-6-luna for openai_decisions. Use semif-qwen3.5-4b for SemIf.
Headers sent with each model call. They are saved in the evaluator's prompt as given, so do not put API keys here.
A saved model configuration to start from: its model, endpoint, key and settings apply unless set here.
The prompt the model reads, as (role, template) pairs.
typesafe for Jev and SemIf, typesafe_compatible for a TypeSafe-compatible endpoint, or openai_decisions.
The questions, keyed by feedback key, in the order to ask them.
The fraction of runs to evaluate.
The tracing project to evaluate.
Seconds to wait for each model call. openai_decisions only.
Maps the prompt's variables to run fields.