# evaluate_comparative

> **Function** in `langsmith`

📖 [View in docs](https://reference.langchain.com/python/langsmith/evaluation/_runner/evaluate_comparative)

Evaluate existing experiment runs against each other.

This lets you use pairwise preference scoring to generate more
reliable feedback in your experiments.

## Signature

```python
evaluate_comparative(
    experiments: tuple[EXPERIMENT_T, EXPERIMENT_T],
    /,
    evaluators: Sequence[COMPARATIVE_EVALUATOR_T],
    experiment_prefix: Optional[str] = None,
    description: Optional[str] = None,
    max_concurrency: int = 5,
    client: Optional[langsmith.Client] = None,
    metadata: Optional[dict] = None,
    load_nested: bool = False,
    randomize_order: bool = False,
) -> ComparativeExperimentResults
```

## Parameters

| Name | Type | Required | Description |
|------|------|----------|-------------|
| `experiments` | `Tuple[Union[str, uuid.UUID], Union[str, uuid.UUID]]` | Yes |  The identifiers of the experiments to compare. |
| `evaluators` | `Sequence[COMPARATIVE_EVALUATOR_T]` | Yes |  A list of evaluators to run on each example. |
| `experiment_prefix` | `Optional[str]` | No | A prefix to provide for your experiment name. (default: `None`) |
| `description` | `Optional[str]` | No | A free-form text description for the experiment. (default: `None`) |
| `max_concurrency` | `int` | No | The maximum number of concurrent evaluations to run. (default: `5`) |
| `client` | `Optional[langsmith.Client]` | No | The LangSmith client to use. (default: `None`) |
| `metadata` | `Optional[dict]` | No | Metadata to attach to the experiment. (default: `None`) |
| `load_nested` | `bool` | No | Whether to load all child runs for the experiment.  Default is to only load the top-level root runs. (default: `False`) |
| `randomize_order` | `bool` | No | Whether to randomize the order of the outputs for each evaluation. (default: `False`) |

## Returns

`ComparativeExperimentResults`

The results of the comparative evaluation.

---

[View source on GitHub](https://github.com/langchain-ai/langsmith-sdk/blob/6a74bf5af9e542d8065af8edca54b2448f430916/python/langsmith/evaluation/_runner.py#L673)