Get Eval Run

GET

/org/:org_id/project/:project_id/task/:task_id/eval/run/:eval_run_id

GET

/api/v1/org/:org_id/project/:project_id/task/:task_id/eval/run/:eval_run_id

$ curl https://app.rightbrain.ai/api/v1/org/org_id/project/project_id/task/task_id/eval/run/eval_run_id \
>      -H "Authorization: Bearer <token>"

1 {
2   "id": "string",
3   "task_id": "string",
4   "project_id": "string",
5   "candidate_revision_id": "string",
6   "task_run_ids": [
7     "string"
8   ],
9   "status": "pending",
10   "pass_count": 1,
11   "fail_count": 1,
12   "error_count": 1,
13   "created": "string",
14   "modified": "string",
15   "judge_model": {
16     "name": "string",
17     "provider": "string"
18   },
19   "pass_rate": 1.1,
20   "eval_set_id": "string",
21   "started_at": "string",
22   "completed_at": "string",
23   "error_message": "string",
24   "created_by_user_id": "string",
25   "metrics": {
26     "token_cost": {
27       "reference_avg": 1.1,
28       "candidate_avg": 1.1,
29       "delta_pct": 1.1
30     },
31     "charged_credits": {
32       "reference_avg": 1.1,
33       "candidate_avg": 1.1,
34       "delta_pct": 1.1
35     },
36     "llm_call_timing": {
37       "reference_avg": 1.1,
38       "candidate_avg": 1.1,
39       "delta_pct": 1.1
40     }
41   },
42   "results": [
43     {
44       "id": "string",
45       "eval_run_id": "string",
46       "reference_run_id": "string",
47       "judge_input_tokens": 1,
48       "judge_output_tokens": 1,
49       "is_error": true,
50       "created": "string",
51       "candidate_run_id": "string",
52       "verdict": "pass",
53       "reasoning": "string",
54       "error_message": "string",
55       "metrics": {
56         "reference": {
57           "total_tokens": 1,
58           "charged_credits": "string",
59           "llm_call_timing": 1.1
60         },
61         "candidate": {
62           "total_tokens": 1,
63           "charged_credits": "string",
64           "llm_call_timing": 1.1
65         }
66       }
67     }
68   ]
69 }

Get eval run details with results and aggregate metrics.

Authentication

AuthorizationBearer

Bearer token authentication using JWT tokens from Ory Kratos/Hydra

Path parameters

eval_run_idstringRequiredformat: "uuid"

org_idstringRequiredformat: "uuid"

The organization ID

project_idstringRequiredformat: "uuid"

The project ID

task_idstringRequiredformat: "uuid"

The task ID

Response

Successful Response

idstringformat: "uuid"

task_idstringformat: "uuid"

project_idstringformat: "uuid"

candidate_revision_idstringformat: "uuid"

task_run_idslist of strings

statusenum

pass_countinteger

fail_countinteger

error_countinteger

createdstring

modifiedstring

judge_modelobject

pass_ratedouble or nullRead-only

eval_set_idstring or nullformat: "uuid"

started_atstring or null

completed_atstring or null

error_messagestring or null

created_by_user_idstring or nullformat: "uuid"

metricsobject or null

resultslist of objects

Errors

400

Bad Request Error

403

Forbidden Error

422

Unprocessable Entity Error