AsyncOnlineEvaluatorsResource

This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content.

For more information, see https://www.github.com/langchain-ai/langsmith-python#accessing-raw-response-data-eg-headers

An alternative to .with_raw_response that doesn't eagerly read the response body.

For more information, see https://www.github.com/langchain-ai/langsmith-python#with_streaming_response

Create a new LLM or code evaluator for the current workspace.

Retrieve a single evaluator by its ID.

Update an existing evaluator's name, LLM configuration, or code configuration. Returns 409 when a code evaluator build is ENQUEUED or BUILDING.

List evaluators for the current workspace, with optional filtering by type, name, tag, feedback key, or resource ID.

Delete an evaluator.

Returns 409 when a code evaluator build is ENQUEUED or BUILDING, or when run rules still reference the evaluator and delete_run_rules is false. When delete_run_rules is true, all run rules referencing this evaluator are deleted first (same tenant) if the build is not in flight. Associated llm_evaluators and code_evaluators rows are removed by foreign-key cascade when the evaluator row is deleted.

Delete multiple evaluators by their IDs.

Returns per-item success/failure.

Returns per-day LLM evaluator spend for the requested 7-day period, grouped by evaluator, resource, or run rule. Exactly one of group_by, evaluator_id, session_id, or dataset_id is required. resource_id, type, feedback_key, tag_value_id, and agent_id may be supplied with group_by to narrow listing aggregations.