Instrumentation: tracing spans + /metrics endpoint #14

New issue

Open

opened 2026-04-27 07:31:37 +00:00 by zaelgohary · 0 comments

zaelgohary commented

2026-04-27 07:31:37 +00:00

Member

Symptom

We can't answer:

How many editor sessions per day?
Save latency p50/p95?
Upload size distribution?
How often does the cache need to be evicted?
Which RPCs error out?

Cause

No instrumentation beyond tracing::info! log lines. Logs are unstructured; aggregation is grep.

Proposed fix

tracing spans on the 4 hot RPCs (list_documents, create_editor_session, get_document, save_document) + the upload_document / delete_document we just added.
A /metrics endpoint on hero_office_ui (Prometheus text format) or a metrics JSON-RPC method on hero_office_server. Counters: rpc_calls_total, rpc_errors_total. Histograms: rpc_duration_ms, save_size_bytes.
No external dependency: just metrics crate + metrics-exporter-prometheus.

Acceptance

Hitting the metrics endpoint from a fresh box returns scrapable text.
Save latency p95 is observable under load.
We can answer the bullets above without reading logs.

## Symptom We can't answer: - How many editor sessions per day? - Save latency p50/p95? - Upload size distribution? - How often does the cache need to be evicted? - Which RPCs error out? ## Cause No instrumentation beyond `tracing::info!` log lines. Logs are unstructured; aggregation is grep. ## Proposed fix - `tracing` spans on the 4 hot RPCs (`list_documents`, `create_editor_session`, `get_document`, `save_document`) + the `upload_document` / `delete_document` we just added. - A `/metrics` endpoint on hero_office_ui (Prometheus text format) or a `metrics` JSON-RPC method on hero_office_server. Counters: rpc_calls_total, rpc_errors_total. Histograms: rpc_duration_ms, save_size_bytes. - No external dependency: just `metrics` crate + `metrics-exporter-prometheus`. ## Acceptance - Hitting the metrics endpoint from a fresh box returns scrapable text. - Save latency p95 is observable under load. - We can answer the bullets above without reading logs.