Appendix I: Dashboard and Monitoring Templates

A dashboard for an MCP server has to answer four questions quickly: is the service healthy, what is failing or slowing down, who is affected, and what changed?

Core Panels

Panel	Why it matters
Requests per minute by tool	Shows whether the issue is broad or isolated
Error rate by `outcome`	Separates validation, auth, policy, dependency, and timeout failures
p95 and p99 latency by tool	Highlights which operations are actually unusable in conversation
Top tenants or workspaces by traffic	Exposes noisy-neighbor and abuse patterns
Dependency health and latency	Shows whether the server is the problem or an upstream system is
Long-running task backlog and age	Stops asynchronous work from becoming an invisible outage
Selection share by `catalog_version`	Catches a description change that shifted model behavior with no errors
Recent rollouts, flags, and client versions	Connects incidents to changes quickly

Alert Rules

Alert	Example threshold	Immediate question to ask
High error rate	More than 5 percent for 5 minutes on a core tool family	Is this auth, schema drift, or dependency failure?
Tail-latency regression	p95 or p99 exceeds its budget for 10 minutes	Which dependency or workflow changed?
Auth failure spike	Sudden jump in `401` or `403`	Did issuer, audience, scopes, or tenant mapping change?
Dependency outage	Critical dependency unavailable or timing out	Which tool families should degrade or be disabled?
Task backlog growth	Queue age or backlog size above its normal band	Are callbacks, workers, or downstream systems stuck?
Model loop	4+ identical calls (same `args_hash`) in 5 minutes	Is an error telling the model to retry when nothing will change?
Cost spike	Sudden jump in usage, retries, or token estimates	Is a workflow looping or a tenant misconfigured?

Metric Names and Labels

Derive the metrics from the one event per tool call in Chapter 23 §One Event Per Tool Call. Use the same field names, so a dashboard label and a log field never disagree:

mcp_tool_calls_total{tool, outcome, tenant, client_name, catalog_version, approval}
mcp_tool_duration_seconds{tool, outcome}               # histogram
mcp_auth_failures_total{status, token_aud}              # 401 / 403
mcp_tool_retries_total{tool, client_name}               # calls with retry_count > 0

outcome is the closed set from Chapter 23 (success, empty, validation_error, auth_error, policy_denied, dependency_error, timeout). approval is required or none, because approval-gated paths have different latency and failure profiles. Keep principal and args_hash in logs, not in metric labels. Their cardinality is unbounded.

Error rate by outcome, per tool, over five minutes:

sum by (tool, outcome) (rate(mcp_tool_calls_total{outcome!~"success|empty"}[5m]))
  / ignoring(outcome) group_left
sum by (tool) (rate(mcp_tool_calls_total[5m]))

Trace Context

2026-07-28 reserves traceparent, tracestate and baggage in a request’s _meta for W3C trace context. Continue the client’s trace from _meta.traceparent when it is well formed, fall back to the HTTP traceparent header, and otherwise start a new trace. Put the trace_id on the event, so one click goes from a dashboard spike to the calls behind it. Never use baggage for authorization. → ch23 §Tracing Crosses the Client Hop

Starting SLOs

availability: 99.9 percent for core read paths
latency: p95 under 2 seconds for common read-oriented tools
turn-level latency: every tool call the model makes to answer one user message, end to end, within 10 seconds at p95. The user waits for the whole chain, not for one call
error rate: under 1 percent for high-volume stable tools
task freshness: backlog age below your agreed user-visible threshold

Write-heavy or approval-heavy paths deserve their own targets. Don’t pretend all tool families are equivalent.

Drill Questions

Can an on-call engineer identify the failing tool family within two minutes?
Can they see whether one tenant or all tenants are affected?
Can they tell whether auth, dependency health, or schema drift is the likeliest cause?
Can they see what changed recently, including catalog_version and client versions, without opening a separate system?

Keyboard shortcuts