Analytics
Enterprise only
Flightdeck's dashboards, reports and SIEM feeds are not included in DVARA Open Source. The open gateway still publishes Prometheus metrics, OpenTelemetry traces and structured logs.
Every request through DVARA emits cost, latency, token, policy-decision, audit, and trace telemetry. Flightdeck shows that data according to the signed-in user's role and active workspace. Machine-readable feeds—Prometheus, OpenTelemetry, structured logs, and SIEM streams—carry the same evidence into your observability stack.
This page is a survey of what's available and where to find it. For deep dives, follow the links into the per-feature documentation.
The three lenses
| Lens | Audience | What it gives you |
|---|---|---|
| Platform-wide Flightdeck pages | Platform operators, security admins, finance | Cross-workspace operational view with live auto-refreshing panels (3–10s), role-based access controls, and click-through detail. |
| Workspace-scoped Flightdeck pages | Workspace users — application developers, team leads, auditors | The same evidence narrowed to the active workspace. Self-service for workspace admin, developer, and viewer roles. |
| Raw feeds (Prometheus scrape, OTLP push, stdout JSON logs, Kafka/Splunk/CloudWatch SIEM) | Your own observability stack — Grafana, Datadog, Tempo, Jaeger, Splunk, ELK, … | Machine-readable telemetry meant to feed your existing dashboards, alerts, and SIEM. DVARA does not try to replace your observability platform; it emits the standard formats so you don't have to. |
Use the dashboards for day-to-day operations and the raw feeds for long-term retention, custom dashboards, and alerting. Reports (chargeback, compliance) are the one-shot PDF / CSV artifacts you hand to finance or auditors.
Quick orientation by audience
| You are a … | Start here |
|---|---|
| Platform operator running the gateway | Dashboard for live health; Traffic → Latency for routing decisions; Prometheus scrape for alerting |
| Security / compliance lead | Governance → Audit for the event stream; Governance → Compliance for SOC2 / HIPAA / GDPR / RBI / SEBI rollups; SIEM stream for long-term retention |
| Finance / FinOps | Cost → Cost Dashboard for live spend; Cost → Chargeback Reports for monthly workspace reports; gateway_cost_dollars_total for time-series budget burn |
| Workspace developer or team lead | Dashboard for workspace request counts, tokens, and cost; Usage for per-model breakdowns; Audit for the workspace audit trail |
| Agent platform team | MCP → Analytics for session outcomes, approval funnels, and tool-call heatmaps; MCP → Sessions for per-session timelines |
The full surface
Every analytic the platform exposes, what it shows, where to view it, and the underlying endpoint.
Live dashboards
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Operational dashboard | Req/s, P95 latency, error rate, tokens/min, provider health, cache stats, priority-admission stats | Dashboard | GET /actuator/prometheus + GET /actuator/gateway-status |
| Per-workspace dashboard | Request count, token usage, 30-day cost, active API keys, recent audit events — scoped to caller's workspace | Workspace Dashboard | Flightdeck page scoped to the active workspace |
| Cost dashboard | Total cost, Cost-by-Provider donut, Cost-by-Model bar, budget status, forecast cards, anomaly alerts, raw records table | Cost → Cost Dashboard | Flightdeck only — no API |
| Latency dashboard | EWMA latency per (provider, model) — sample count, raw last sample, freshness badge | Traffic → Latency (owner, policy-admin) | Flightdeck only — no API |
| Priority admission stats | Concurrent / max concurrent / load %, per-tier admitting vs throttling | Dashboard priority card (when dvara.llm-gateway.routing.priority.enabled=true) | Flightdeck only — no API |
| Response cache | Whether the workspace's cache is on, its limits, and the calls and cost it saved. There is no entry count: entries live in each Gateway's memory. | Workspace Cost → Cache (workspace admin); the platform owner sets the same values in the workspace form's Response cache section | Hit rate, average similarity and lookup counts are per-Gateway-process counters. Scrape gateway_semantic_cache_{hits,misses}_total{workspace} from /actuator/prometheus |
| Licence status | Posture, licensee, type, expiry, days remaining, runtime status badge | Settings → License in the sidebar footer and the status banner | Enterprise GET /actuator/license |
Telemetry & event streams
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Token usage telemetry | owner and billing-admin see per-request tokens and key metadata; workspace roles see workspace totals and model charts. | owner and billing-admin: from Cost → Cost Dashboard. policy-admin does not have it. Workspace roles: Cost → Usage, or Cost → Token Usage for the workspace admin | Flightdeck only — no API |
| Audit event stream | HMAC-signed, hash-chained events—policy decisions, auth, config imports, licence transitions—with detail and export | Governance → Audit | Flightdeck only — no API |
| Budget cap usage | Current period spend vs limit, color-coded progress bar, period boundaries, dollars remaining | Cost → Budget Caps | Flightdeck only — no API |
| MCP tool-call telemetry | Per-call records and aggregate counts | MCP → Tool Calls | Flightdeck only — no API |
| Agent session timeline | Session summaries, tool use, per-call timeline, and JSON export | MCP → Sessions | Flightdeck only — no API |
| Approval queue stats | Pending decisions, history, status badges, and navigation count | Governance → Approvals (platform roles) or Agents → Approvals (workspace roles) | Flightdeck only — no API |
Comparison & experimentation analytics
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Canary A/B report | Per-variant metrics on a canary-routed route; reset + live split-percentage update | Flightdeck /routes/{id}/canary | Flightdeck only — no API |
| Shadow policy report | Divergence stats (1h / 24h / 7d) + per-event diff between active and SHADOW policy | Flightdeck /policies/{id}/shadow | Flightdeck only — no API |
| Shadow routing report | Provider-vs-shadow comparison on a route (latency, status, response delta) | Flightdeck /routes/{id}/shadow | Flightdeck only — no API |
| Prompt experiment metrics | Per-variant A/B metrics snapshot for a prompt experiment | Prompts → Experiments | Flightdeck only — no API |
| Agent behaviour analytics | Session-outcomes donut, approval-funnel bar, top-20 agent leaderboard, tool-call heatmap (server × tool), policy firing frequency | Flightdeck /analytics | Flightdeck page (composed from MCP + audit data) |
| Model fingerprint drift | Drift report per golden prompt — when an upstream model's response semantics shifted | Flightdeck eval/fingerprint surface | Flightdeck only — no API |
| Eval pipeline reports | Per-model eval suite results vs golden corpus, drift detection, time-ranged report queries | Flightdeck /eval/reports | Flightdeck only — no API |
Downloadable reports
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Chargeback reports | Per-workspace monthly rollup: workspace summary, model/provider breakdown, daily cost trend, forecasts, anomalies — PDF + CSV | Cost → Chargeback Reports | Flightdeck only — no API |
| Compliance reports | SOC2 / HIPAA / GDPR / RBI / SEBI rollups aggregated from audit + token usage + workspace + policy data — PDF | Governance → Compliance | Flightdeck only — no API |
Raw feeds — point your own stack at these
| Analytic | What it shows | Where to view | Endpoint |
|---|---|---|---|
| Prometheus metrics | Request, latency, token, cost, guardrail, budget, MCP tool-call, and email-delivery series when their corresponding component is running | Your Grafana / Datadog / Mimir | GET /actuator/prometheus on the DVARA Gateway and Flightdeck — Bearer $DVARA_ACTUATOR_METRICS_API_KEY. MCP and A2A series use the Gateway endpoint; they do not have separate scrape endpoints. |
| OpenTelemetry traces | Distributed spans — gateway.provider.chat, …stream, …embed; mcp.filter.*, mcp.server.call — with traceparent propagation | Your OTel collector → Jaeger / Tempo / Datadog | OTLP push to OTEL_EXPORTER_OTLP_ENDPOINT. There is no default: when it is unset, no traces are exported |
| Structured access log | One JSON line per request: trace_id, method, path, status, latency_ms, model, provider, cache_status, opaque api_key ID (or anonymous), workspace_id, token counts, error_code | Your log aggregator | stdout (structured JSON) |
| SIEM stream | All audit events forwarded to Splunk HEC, CloudWatch Logs, Kafka, or a log line (siem.log) | Your SIEM | dvara.llm-gateway.siem.{splunk|cloudwatch|kafka|log}.* |
The opaque API-key ID is the stable join key across usage, cost, budgets,
structured logs, audit events, analytics, and exports. Flightdeck turns it into
a recognizable key name and public prefix where a person needs to read it. The
plaintext bearer credential is not written to these surfaces; a request made
without an authenticated key is attributed to anonymous.
Per-workspace vs platform scoping
Almost every analytic in the tables above respects DVARA's multi-tenant scoping rules:
- Platform-role callers (
owner/policy-admin/billing-admin) see cross-workspace data by default. Most Flightdeck list pages accept an optional?workspace_id=filter to narrow. - Workspace-role callers (
admin/developer/viewer) only ever see their own workspace's rows. Passing another workspace's id returns HTTP 403.
Flightdeck enforces the same rule at the request boundary. Changing a URL cannot move a workspace user outside the workspaces their team membership allows. See Multi-Tenancy for the full isolation model.
What's deliberately not in the box
A few things the platform deliberately leaves to your existing tools:
- Long-term metric retention beyond the Prometheus scrape window — point a Mimir / Thanos / Datadog at the scrape endpoint.
- Custom dashboards beyond the built-in Flightdeck views — Grafana on top of the Prometheus and OTLP feeds covers the long tail.
- Alerting — Prometheus AlertManager or Grafana alerts on the same metrics; the platform emits the data, not the alert routing.
- Cross-system correlation (joining DVARA spans against your service's app spans) — handled by your APM via the
traceparentpropagation DVARA already emits.
Where to go next
- DVARA Flight Deck — the full Flightdeck tour, including every dashboard listed above with screenshots.
- Cost Management — cost dashboard, forecasts, anomaly alerts, budget caps, chargeback reports.
- Observability — audit event taxonomy, Prometheus metrics list, OpenTelemetry span names, SIEM configuration.
- Multi-Tenancy — the scoping rules that govern who sees what data.