Start TaskAgent Eval Run

POST

/org/:org_id/project/:project_id/task-agent/:task_agent_id/eval/run

POST

/api/v1/org/:org_id/project/:project_id/task-agent/:task_agent_id/eval/run

$ curl -X POST https://app.rightbrain.ai/api/v1/org/org_id/project/project_id/task-agent/task_agent_id/eval/run \
>      -H "Authorization: Bearer <token>" \
>      -H "Content-Type: application/json" \
>      -d '{
>   "candidate_revision_id": "019929cf-f316-196f-3bf9-118b22da0c3c",
>   "last_n": 5,
>   "judge_model_id": "ec217e75-72ea-4281-a1b8-cb7bd0ef9f41"
> }'

1 {
2   "id": "string",
3   "task_agent_id": "string",
4   "project_id": "string",
5   "candidate_revision_id": "string",
6   "status": "pending",
7   "pass_count": 1,
8   "fail_count": 1,
9   "error_count": 1,
10   "created": "string",
11   "modified": "string",
12   "task_agent_run_ids": [
13     "string"
14   ],
15   "pass_rate": 1.1,
16   "eval_set_id": "string",
17   "started_at": "string",
18   "completed_at": "string",
19   "last_activity_at": "string",
20   "error_message": "string",
21   "created_by_user_id": "string"
22 }

Create a TaskAgent eval run and enqueue background execution. Callers are responsible for understanding candidate run side effects, external tool or integration mutations, and quota usage before starting an eval.

Authentication

AuthorizationBearer

Bearer token authentication using JWT tokens from Ory Kratos/Hydra

Path parameters

org_idstringRequiredformat: "uuid"

The organization ID

project_idstringRequiredformat: "uuid"

The project ID

task_agent_idstringRequiredformat: "uuid"

The task agent ID

Request

This endpoint expects an object.

candidate_revision_idstring or nullOptionalformat: "uuid"

eval_set_idstring or nullOptionalformat: "uuid"

task_agent_run_idslist of strings or nullOptional

last_ninteger or nullOptional1-20

judge_model_idstring or nullOptionalformat: "uuid"

Response

Successful Response

idstringformat: "uuid"

task_agent_idstringformat: "uuid"

project_idstringformat: "uuid"

candidate_revision_idstringformat: "uuid"

statusenum

Status of a TaskAgent eval run.

pass_countinteger

fail_countinteger

error_countinteger

createdstring

modifiedstring

task_agent_run_idslist of stringsRead-only

pass_ratedouble or nullRead-only

eval_set_idstring or nullformat: "uuid"

started_atstring or null

completed_atstring or null

last_activity_atstring or null

error_messagestring or null

created_by_user_idstring or nullformat: "uuid"

Errors

400

Bad Request Error

403

Forbidden Error

404

Not Found Error

422

Unprocessable Entity Error

503

Service Unavailable Error