| Name | Type | Description |
|---|---|---|
prompt* | Iterable[Iterable[object]] | The prompt the model reads, as (role, template) pairs. |
questions* | Dict[str, evaluator_create_decision_model_params.Questions] | The questions, keyed by feedback key, in the order to ask them. |
base_url | Optional[str] | Omit | Default: omit |
dataset_id | Optional[str] | Omit | Default: omit |
filter | Optional[str] | Omit | Default: omit |
max_retries | Optional[int] | Omit | Default: omit |
model | Optional[str] | Omit | Default: omit |
model_headers | Optional[Dict[str, str]] | Omit | Default: omit |
playground_settings_id | Optional[str] | Omit | Default: omit |
provider | Literal['typesafe', 'typesafe_compatible', 'openai_decisions'] | Omit | Default: omit |
sampling_rate | float | Omit | Default: omit |
session_id | Optional[str] | Omit | Default: omit |
timeout_seconds | Optional[int] | Omit | Default: omit |
variable_mapping | Optional[Dict[str, str]] | Omit | Default: omit |
extra_headers | Headers | None | Default: None |
extra_query | Query | None | Default: None |
extra_body | Body | None | Default: None |
timeout | float | httpx.Timeout | None | NotGiven | Default: not_given |
Create a decision-model evaluator (Jev, SemIf or OpenAI Decisions) from its prompt and questions, and attach it to a project or dataset. Each question's key becomes a feedback key.
Call this endpoint directly instead of through the LangSmith gateway, with the workspace's TYPESAFE_API_KEY or OPENAI_API_KEY secret. Required for typesafe_compatible.
The dataset whose experiments to evaluate.
Only evaluate runs that match this filter.
How many times to retry a failed model call. openai_decisions only.
Defaults to jev-latest for typesafe and gpt-6-luna for openai_decisions. Use semif-qwen3.5-4b for SemIf.
Headers sent with each model call. They are saved in the evaluator's prompt as given, so do not put API keys here.
A saved model configuration to start from: its model, endpoint, key and settings apply unless set here.
typesafe for Jev and SemIf, typesafe_compatible for a TypeSafe-compatible endpoint, or openai_decisions.
The fraction of runs to evaluate.
The tracing project to evaluate.
Seconds to wait for each model call. openai_decisions only.
Maps the prompt's variables to run fields.
Send extra headers
Add additional query parameters to the request
Add additional JSON properties to the request
Override the client-level default timeout for this request, in seconds
