Observability & audit

You can’t operate what you can’t see. Rightbrain records every run, breaks down what each one used and cost, and keeps a tamper-evident audit trail of who did what.

Run records

Every Task and agent run is captured and available through list and detail endpoints. Each record identifies the trigger that started it, including API, webhook, schedule, and inbox sources, and carries failure metadata when execution does not complete.

A waiting_for_human run is paused on an approval; a failed run carries a reason drawn from the run’s error events.

Four levels of detail

Agent telemetry nests, so you can zoom from a whole conversation down to a single tool call:

1. Session — the multi-turn conversation, with its aggregate credit total across every run in it.
2. Run — one turn: duration, tokens, credits, which tools and skills it invoked, and whether it fell back or hit an error.
3. Events — the ordered stream of what happened inside the run (model output, tool calls, tool results).
4. TaskRun — the execution of an individual Task tool the run called, with its own timing split across input processing, the LLM call, and any MCP tool time.

Start at the run to triage, then drop into events or a specific TaskRun to find the cause.

Run actor and execution source

Task and agent run records separate the user attributed to a run from the authority that caused it:

Anonymous or unresolved execution has no trusted actor_user_id. Actor identity is also used when an integration binding has an invoker credential policy; an eligible actor’s user-scoped connection is selected for the call.

Run detail and events

Open a run to see its full record: inputs, output, token counts, timing, credits used, files and their provenance, and per-tool invocation records (task_tools_invoked, skills_activated, mcp_tools_invoked, integration_tools_invoked). phase_timings breaks wall-clock time into model, tool, Skill, and memory-compaction phases.

context_usage attributes estimated input to instructions, messages, history, files, Task tools, MCP tools, Integrations, Skills, and registered tools. Provider totals remain the usage authority; the category breakdown is an estimate accumulated across the run’s model requests, not a measure of remaining context-window capacity. See Memory & context.

Triage a run

Inspect evidence in this order:

1. Completeness — confirm the expected sources were called and returned data. A completed formatter can still produce a partial result after an optional upstream source failed.
2. Failures and retries — check failure flags, repeated error signatures, fallback use, approval outcomes, and retry counts.
3. Arguments and results — use ordered events and child Task runs to verify that each tool received the intended input.
4. Timing — inspect phases to separate model inference, external-tool waiting, Task execution, and memory compaction.
5. Context and cost — compare attributed context, provider token totals, and credits before adding more tools, history, or turns.

Support requests

Use POST …/task-agent/{agent_id}/support-request to request Rightbrain support for one visible agent run. The request accepts exactly one ID in run_ids and an optional annotation. Rightbrain stores a diagnostic snapshot for the run, redacts secret-shaped fields and values, and returns metadata rather than the snapshot payload. The snapshot receives an expiry timestamp 30 days after creation.

Agent runs stream Server-Sent Events as they execute — text, tool calls, tool results, approval pauses, errors, and a terminal done (the full event reference is in Run agents via API). Those events are retained, so you can read a completed run’s events after the fact:

curl https://app.rightbrain.ai/api/v1/org/$ORG_ID/project/$PROJECT_ID/task-agent/$AGENT_ID/run/$RUN_ID/events \
  -H "Authorization: Bearer $RB_API_KEY"

Debugging a failed run: a worked example

The invocation records are what turn “the run looked wrong” into a root cause. Take a repo-evaluation agent whose report came back thin. Its tool records showed two MCP-bound Task tools failing with a tool_call_id sequence mismatch, an auto-retry hitting the same error, and the agent producing a degraded report from the tool calls that did succeed rather than failing outright:

More work, less output — and the failure reason sitting right there in the record. Without per-tool telemetry you’d see only a mediocre answer; with it you see exactly which tool broke and that the agent’s fallback path saved the run.

Usage reports

Rightbrain aggregates run telemetry into four report types at several levels. Each is a GET returning aggregated figures:

Agent usage reports include the per-model split — primary vs fallback, and reasoning vs memory_compaction — so you can see how often runs fall back and what agent memory compaction is costing you. See Fallbacks & reliability.

curl https://app.rightbrain.ai/api/v1/org/$ORG_ID/project/$PROJECT_ID/task-agent/$AGENT_ID/usage_report \
  -H "Authorization: Bearer $RB_API_KEY"

Reporting groups

Reporting groups let you tag runs so you can attribute usage — for example, per customer or per feature. Manage them under …/reporting-groups, then pass reporting_group as a query parameter when you run a task to file that run under the group. Reports can then be read with visibility controls per group.

Audit log

Beyond run telemetry, Rightbrain keeps an audit log of actions taken in a project. Audit events are read and exported through the API:

Cryptographic integrity

The audit log is tamper-evident: changing or removing a retained event breaks the chain. The integrity status endpoint checks the persisted chain-head state. The verification endpoint scans the retained event chain and reports the first detected failure.

curl https://app.rightbrain.ai/api/v1/org/$ORG_ID/project/$PROJECT_ID/audit_event/integrity \
  -H "Authorization: Bearer $RB_API_KEY"

Related