| Name | Type | Description |
|---|---|---|
key | str | Default: 'unnamed'The name of the feedback metric. |
score | Optional[Union[float, int, bool]] | Default: None |
value | Optional[Union[float, int, bool, str, dict]] | Default: None |
run_id | Optional[Union[UUID, str]] | Default: None |
trace_id | Optional[Union[UUID, str]] | Default: None |
correction | Optional[dict] | Default: None |
comment | Optional[str] | Default: None |
source_info | Optional[Dict[str, Any]] | Default: None |
feedback_source_type | Union[FeedbackSourceType, str] | Default: ls_schemas.FeedbackSourceType.API |
source_run_id | Optional[Union[UUID, str]] | Default: None |
feedback_id | Optional[Union[UUID, str]] | Default: None |
feedback_config | Optional[FeedbackConfig] | Default: None |
stop_after_attempt | int, default=10 | Default: 10 |
project_id | Optional[Union[UUID, str]] | Default: None |
comparative_experiment_id | Optional[Union[UUID, str]] | Default: None |
feedback_group_id | Optional[Union[UUID, str]] | Default: None |
extra | Optional[Dict] | Default: None |
session_id | Optional[Union[UUID, str]] | Default: None |
start_time | Optional[datetime] | Default: None |
**kwargs | Any | Default: {} |
Create feedback for a run.
To enable feedback to be batch uploaded in the background you must
specify trace_id. We highly encourage this for latency-sensitive environments.
Example:
from langsmith import trace, traceable, Client
@traceable
def foo(x):
return {"y": x * 2}
@traceable
def bar(y):
return {"z": y - 1}
client = Client()
inputs = {"x": 1}
with trace(name="foobar", inputs=inputs) as root_run:
result = foo(**inputs)
result = bar(**result)
root_run.outputs = result
trace_id = root_run.id
child_runs = root_run.child_runs
# Provide feedback for a trace (a.k.a. a root run)
client.create_feedback(
key="user_feedback",
score=1,
trace_id=trace_id,
)
# Provide feedback for a child run
foo_run_id = [run for run in child_runs if run.name == "foo"][0].id
client.create_feedback(
key="correctness",
score=0,
run_id=foo_run_id,
# trace_id= is optional but recommended to enable batched and backgrounded
# feedback ingestion.
trace_id=trace_id,
)The score to rate this run on the metric or aspect.
The display value or non-numeric value for this feedback.
The ID of the run to provide feedback for. At least one of run_id, trace_id, or project_id must be specified.
The ID of the trace (i.e. root parent run) of the run to provide feedback for (specified by run_id). If run_id and trace_id are the same, only trace_id needs to be specified. NOTE: trace_id is required feedback ingestion to be batched and backgrounded.
The proper ground truth for this run.
A comment about this feedback, such as a justification for the score or chain-of-thought trajectory for an LLM judge.
Information about the source of this feedback.
The type of feedback source, such as model (for model-generated feedback) or API.
The ID of the run that generated this feedback, if a "model" type.
The ID of the feedback to create. If not provided, a random UUID will be generated.
The configuration specifying how to interpret feedback with this key. Examples include continuous (with min/max bounds), categorical, or freeform.
The number of times to retry the request before giving up.
The ID of the project (or experiment) to provide feedback on. This is used for creating summary metrics for experiments. Cannot specify run_id or trace_id if project_id is specified, and vice versa.
If this feedback was logged as a part of a comparative experiment, this associates the feedback with that experiment.
When logging preferences, ranking runs, or other comparative feedback, this is used to group feedback together.
Metadata for the feedback.
The session (project) ID of the run this feedback is for. Used to optimize feedback ingestion by avoiding server-side lookups.
The start time of the run this feedback is for. Used to optimize feedback ingestion by avoiding server-side lookups.
Additional keyword arguments.