Start Eval Run

Create an eval run and dispatch background execution. An eval run re-runs a candidate task revision against the same inputs as a set of reference ("baseline") task runs, then uses an LLM judge to compare each candidate output against its corresponding reference output. **Choosing the reference runs** (provide exactly one of): - `eval_set_id` — use the task runs stored in a previously created eval set - `task_run_ids` — use a specific, ad-hoc list of task run IDs (max 20) - `last_n` — use the task's `n` most recent successful runs (1-20) **Candidate revision:** - `candidate_revision_id` — the task revision to re-run and judge. Must differ from the revision that produced the reference runs; the run is rejected with a 400 if the candidate revision matches all reference runs. **Judge model:** - `judge_model_id` — optional; a platform default judge model is used if omitted. The run starts in `pending` status and transitions to `running`, then `completed` or `failed`, as it executes in the background. Poll Get Eval Run for the per-comparison verdicts once it completes.

Authentication

AuthorizationBearer

API key or access token, sent as Authorization: Bearer <token>. The simplest option: create an API key in your dashboard under Settings, then paste it here.

OR
AuthorizationBearer

API key or access token, sent as Authorization: Bearer <token>. The simplest option: create an API key in your dashboard under Settings, then paste it here.

Path parameters

org_idstringRequiredformat: "uuid"
The organization ID
project_idstringRequiredformat: "uuid"
The project ID
task_idstringRequiredformat: "uuid"
The task ID

Request

This endpoint expects an object.
candidate_revision_idstring or nullOptionalformat: "uuid"
eval_set_idstring or nullOptionalformat: "uuid"
task_run_idslist of strings or nullOptional
last_ninteger or nullOptional1-20
judge_model_idstring or nullOptionalformat: "uuid"

Response

Eval run created successfully
idstringformat: "uuid"
task_idstringformat: "uuid"
project_idstringformat: "uuid"
candidate_revision_idstringformat: "uuid"
task_run_idslist of strings
statusenum
pass_countinteger
fail_countinteger
error_countinteger
createdstring
modifiedstring
pass_ratedouble or nullRead-only
eval_set_idstring or nullOptionalformat: "uuid"
started_atstring or nullOptional
completed_atstring or nullOptional
error_messagestring or nullOptional
created_by_user_idstring or nullOptionalformat: "uuid"

Errors

400
Bad Request Error
401
Unauthorized Error
403
Forbidden Error
404
Not Found Error
422
Unprocessable Entity Error