Observability & audit
Observability & audit
It’s easy to build a compelling demo. It’s incredibly hard to keep AI running in production — and you can’t operate what you can’t see. Rightbrain records every run, breaks down what each one used and cost, and keeps a tamper-evident audit trail of who did what.
Logs and the Observe page
Every task and agent run is captured. In the dashboard:
- Logs shows task and agent runs across the project.
- Observe (on an agent) is a runs table built for operating an agent in production. It includes a Trigger column showing what started each run — an API call, webhook, schedule, or Gmail inbox message — server-side history filters, and surfaced failure reasons for runs that didn’t complete, including primary-model failures and integration failures.
Run statuses are running, waiting_for_human, completed, and failed. A waiting_for_human run is paused on an approval; a failed run carries a reason drawn from the run’s error events.
Four levels of detail
Agent telemetry nests, so you can zoom from a whole conversation down to a single tool call:
- Session — the multi-turn conversation, with its aggregate credit total across every run in it.
- Run — one turn: duration, tokens, credits, which tools and skills it invoked, and whether it fell back or hit an error.
- Events — the ordered stream of what happened inside the run (model output, tool calls, tool results).
- TaskRun — the execution of an individual Task tool the run called, with its own timing split across input processing, the LLM call, and any MCP tool time.
Start at the run to triage, then drop into events or a specific TaskRun to find the cause.
Run detail and events
Open a run to see its full record: inputs, output, token counts, timing, credits used, any files it produced, and per-tool invocation records (task_tools_invoked, skills_activated, mcp_tools_invoked, integration_tools_invoked). phase_timings breaks the wall-clock time into phases, and context_usage attributes the run’s input tokens by source — instructions, conversation history, task tools, MCP tools, integrations, skills — so you can see exactly what filled the context window. On tool-heavy agents that source attribution is often the surprise: a single set of integration tool declarations can account for the majority of a run’s input tokens.
Agent runs stream Server-Sent Events as they execute, and those events are retained. The stream carries:
Read a completed agent run’s events after the fact:
For consuming the stream live during a run, see Run agents via API.
Debugging a failed run: a worked example
The invocation records are what turn “the run looked wrong” into a root cause. Take a repo-evaluation agent whose report came back thin. The run’s tool records showed six tool invocations with four errors: two MCP-bound Task tools had failed with a tool_call_id sequence mismatch, the agent auto-retried them (the retry ran faster, ~5s versus the initial ~10s), hit the same error, and then produced a degraded report from the tool calls that did succeed rather than failing outright. Compared against a healthy run of the same agent — 4 tool invocations, 0 errors, ~4,121 tokens, ~47s — the degraded run showed 6 invocations, 4 errors, ~4,845 tokens, ~39s: more work, less output, and the failure reason sitting right there in the record. Without per-tool telemetry you’d see only a mediocre answer; with it you see exactly which tool broke and that the agent’s fallback path saved the run.
Usage reports
Rightbrain aggregates run telemetry into four report types at several levels. Each is a GET returning aggregated figures:
Agent usage reports include the per-model split — primary vs fallback, and reasoning vs memory_compaction — so you can see how often runs fall back and what agent memory compaction is costing you. See Fallbacks & reliability.
Reporting groups
Reporting groups let you tag runs so you can attribute usage — for example, per customer or per feature. Manage them under …/reporting-groups, then pass reporting_group as a query parameter when you run a task to file that run under the group. Reports can then be read with visibility controls per group.
Audit log
Beyond run telemetry, Rightbrain keeps an audit log of actions taken in a project. Audit events are read and exported through the API:
Cryptographic integrity
The audit log is tamper-evident: events are chained so that altering or removing one breaks the chain. You can ask Rightbrain to verify the chain and prove the record is intact.
Verification is what makes the audit log defensible in a compliance review: it demonstrates the record hasn’t been altered since it was written, not merely that a log exists.