LangChain Reference home pageLangChain ReferenceLangChain Reference
  • GitHub
  • Main Docs
Deep Agents
LangChain
LangGraph
Integrations
LangSmith
  • Overview
  • Client
  • AsyncClient
  • Run Helpers
  • Run Trees
  • Evaluation
  • Schemas
  • Utilities
  • Wrappers
  • Anonymizer
  • Testing
  • Expect API
  • Middleware
  • Pytest Plugin
  • Deployment SDK
⌘I

Chat LangChain

What can I help with?

Enter to send·Shift+Enter new line

Menu

OverviewClientAsyncClientRun HelpersRun TreesEvaluationSchemasUtilitiesWrappersAnonymizerTestingExpect APIMiddlewarePytest PluginDeployment SDK
Language
Theme
PythonlangsmithtypesevaluatorEvaluator
Class●Since v0.14

Evaluator

Copy
Evaluator()

Bases

BaseModel

Attributes

Inherited fromBaseModel

Attributes

Amodel_fields_set: set[str]Amodel_config: ConfigDictAmodel_construct: construct

Methods

Mto_dict
—

Recursively generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

View source on GitHub
Mto_json
—

Generates a JSON string representing this model as it would be received from or sent to the API (but with indentation).

Mconstruct
Mmodel_dump
—

Usage docs: https://docs.pydantic.dev/2.4/concepts/serialization/#modelmodel_dump

Mmodel_dump_json
—

Usage docs: https://docs.pydantic.dev/2.4/concepts/serialization/#modelmodel_dump_json

Classes

CConfig
attribute
id: str
attribute
created_at: datetime
attribute
display_name: str
attribute
evaluator_version: int
attribute
sampling_rate: float
attribute
tenant_id: str
attribute
updated_at: datetime
attribute
webhooks: Optional[List[EvaluatorWebhook]]
attribute
add_to_annotation_queue_id: Optional[str]
attribute
add_to_annotation_queue_name: Optional[str]
attribute
add_to_dataset_id: Optional[str]
attribute
add_to_dataset_name: Optional[str]
attribute
add_to_dataset_prefer_correction: Optional[bool]
attribute
alerts: Optional[List[EvaluatorPagerdutyAlert]]
attribute
alignment_annotation_queue_id: Optional[str]
attribute
backfill_completed_at: Optional[datetime]
attribute
backfill_error: Optional[str]
attribute
backfill_from: Optional[datetime]
attribute
backfill_id: Optional[str]
attribute
backfill_progress: Optional[float]
attribute
backfill_status: Optional[str]
attribute
code_evaluators: Optional[List[CodeEvaluatorTopLevel]]
attribute
corrections_dataset_id: Optional[str]
attribute
dataset_id: Optional[str]
attribute
dataset_name: Optional[str]
attribute
evaluator_id: Optional[str]
attribute
evaluator_name: Optional[str]
attribute
evaluator_type: Optional[str]
attribute
evaluators: Optional[List[EvaluatorTopLevel]]
attribute
extend_annotation_queue_trace_retention: Optional[bool]
attribute
extend_dataset_trace_retention: Optional[bool]
attribute
extend_evaluator_trace_retention: Optional[bool]
attribute
extend_only: Optional[bool]
attribute
extend_webhook_trace_retention: Optional[bool]
attribute
filter: Optional[str]
attribute
group_by: Optional[Literal['thread_id', 'trajectory_id']]
attribute
include_extended_stats: Optional[bool]
attribute
is_enabled: Optional[bool]
attribute
is_managed_evaluator: Optional[bool]
attribute
is_tracing_disabled: Optional[bool]
attribute
num_few_shot_examples: Optional[int]
attribute
session_id: Optional[str]
attribute
session_name: Optional[str]
attribute
spend_limit: Optional[SpendLimit]
attribute
spend_usd: Optional[float]
attribute
trace_count: Optional[int]
attribute
trace_filter: Optional[str]
attribute
transient: Optional[bool]
attribute
tree_filter: Optional[str]
attribute
use_corrections_dataset: Optional[bool]

Run rules schema.