Start TaskAgent Eval Run

POST

/org/:org_id/project/:project_id/task-agent/:task_agent_id/eval/run

POST

/api/v1/org/:org_id/project/:project_id/task-agent/:task_agent_id/eval/run

$ curl -X POST https://app.rightbrain.ai/api/v1/org/org_id/project/project_id/task-agent/task_agent_id/eval/run \
>      -H "Authorization: Bearer <token>" \
>      -H "Content-Type: application/json" \
>      -d '{
>   "candidate_revision_id": "019929cf-f316-196f-3bf9-118b22da0c3c",
>   "eval_set_id": "0190d234-1111-2222-3333-444444444444",
>   "judge_model_id": "ec217e75-72ea-4281-a1b8-cb7bd0ef9f41"
> }'

1 {
2   "id": "0190e234-1111-2222-3333-444444444444",
3   "task_agent_id": "019df849-e106-3301-5894-291b4c9c6cab",
4   "project_id": "0190a234-9876-5432-10ab-cdef01234567",
5   "candidate_revision_id": "019929cf-f316-196f-3bf9-118b22da0c3c",
6   "status": "pending",
7   "pass_count": 0,
8   "fail_count": 0,
9   "error_count": 0,
10   "created": "2026-07-27T18:54:27.451807Z",
11   "modified": "2026-07-27T18:54:28.185742Z",
12   "task_agent_run_ids": [
13     "0190b234-1111-2222-3333-444444444444",
14     "0190b234-5555-6666-7777-888888888888"
15   ],
16   "pass_rate": null,
17   "eval_set_id": "0190d234-1111-2222-3333-444444444444",
18   "created_by_user_id": "0190c234-1111-2222-3333-444444444444"
19 }

Create a TaskAgent eval run and enqueue background execution. Callers are responsible for understanding candidate run side effects, external tool or integration mutations, and quota usage before starting an eval. An optional judge_config supplies one objective and an ordered rubric for the complete run. The reference remains a known-good comparator, and every rubric criterion must pass. Omitting judge_config uses the standard functional-equivalence rubric. The resolved configuration is returned by run detail. Objective and rubric text is sent to the selected external judge model provider and is visible to callers authorized to view TaskAgent Eval run detail. The candidate revision must differ from every reference run’s revision — evaluating a candidate against itself is rejected.

Create a TaskAgent eval run and enqueue background execution. Callers are responsible for understanding candidate run side effects, external tool or integration mutations, and quota usage before starting an eval. An optional judge_config supplies one objective and an ordered rubric for the complete run. The reference remains a known-good comparator, and every rubric criterion must pass. Omitting judge_config uses the standard functional-equivalence rubric. The resolved configuration is returned by run detail. Objective and rubric text is sent to the selected external judge model provider and is visible to callers authorized to view TaskAgent Eval run detail. The candidate revision must differ from every reference run's revision — evaluating a candidate against itself is rejected.

Authentication

AuthorizationBearer

API key or access token, sent as Authorization: Bearer <token>. The simplest option: create an API key in your dashboard under Settings, then paste it here.

AuthorizationBearer

API key or access token, sent as Authorization: Bearer <token>. The simplest option: create an API key in your dashboard under Settings, then paste it here.

Path parameters

org_idstringRequiredformat: "uuid"

The organization ID

project_idstringRequiredformat: "uuid"

The project ID

task_agent_idstringRequiredformat: "uuid"

The task agent ID

Request

This endpoint expects an object.

candidate_revision_idstring or nullOptionalformat: "uuid"

eval_set_idstring or nullOptionalformat: "uuid"

task_agent_run_idslist of strings or nullOptional

last_ninteger or nullOptional1-20

judge_model_idstring or nullOptionalformat: "uuid"

judge_configobject or nullOptional

Response

TaskAgent eval run created and enqueued

idstringformat: "uuid"

task_agent_idstringformat: "uuid"

project_idstringformat: "uuid"

candidate_revision_idstringformat: "uuid"

statusenum

Status of a TaskAgent eval run.

pass_countinteger

fail_countinteger

error_countinteger

createdstring

modifiedstring

task_agent_run_idslist of stringsRead-only

pass_ratedouble or nullRead-only

eval_set_idstring or nullOptionalformat: "uuid"

started_atstring or nullOptional

completed_atstring or nullOptional

last_activity_atstring or nullOptional

error_messagestring or nullOptional

created_by_user_idstring or nullOptionalformat: "uuid"

Errors

400

Bad Request Error

401

Unauthorized Error

403

Forbidden Error

404

Not Found Error

422

Unprocessable Entity Error

500

Internal Server Error

503

Service Unavailable Error