[bug] hero_rpc OpenRpcTransport UDS drops X-Hero-Context/X-Hero-Claims headers — latent cross-service breakage #233

Closed
opened 2026-05-11 12:28:54 +00:00 by casper-stevens · 2 comments
Member

Summary

In hero_rpc, the OpenRpcTransport::post_raw_json_with_headers method silently drops all extra headers (e.g. X-Hero-Context, X-Hero-Claims) when the transport is a Unix Domain Socket. Only the HTTP transport branch forwards them.

Tracked in: lhumina_code/hero_rpc#42

Root cause

http_post_unix (transport.rs:354) has no extra_headers parameter. The UnixSocket branch of post_raw_json_with_headers (line 248) calls it without passing the headers:

// headers silently dropped:
let resp_bytes = self.http_post_unix(path, rpc_path, &body_bytes).await?;

This was a known gap when the function was introduced in commit ed6e7eb (April 13) — the comment reads "Unix-socket transport currently drops the headers — raw socket framing has no place to carry them" — but this is incorrect: we use HTTP-over-UDS, so headers can and must be carried in the HTTP framing.

Current impact

No repos are broken today because the only real caller of post_raw_json_with_headers is OsisClient (in openrpc_http_client_lib), which uses OpenRpcTransport::http (via hero_router), not UDS. So X-Hero-Context reaches the service correctly through that path.

When it becomes a real problem

The moment any service makes a direct server-to-server OSIS call over UDS (bypassing hero_router) while passing a context — e.g. a future OsisClient::new_unix() constructor — the context header will be silently dropped. The service will return default-context data without any error, making the failure invisible and hard to diagnose.

Fix (small, isolated to one file)

All changes are in hero_rpc/crates/openrpc/src/transport.rs:

  1. Add extra_headers: &[(&str, &str)] parameter to http_post_unix
  2. Apply them in the hyper::Request::builder() chain inside http_post_unix
  3. Pass extra_headers through in the UnixSocket branch of post_raw_json_with_headers
  4. Pass &[] from call_unix_http (no headers needed there)

No public API changes. No other files need to change.

## Summary In `hero_rpc`, the `OpenRpcTransport::post_raw_json_with_headers` method silently drops all extra headers (e.g. `X-Hero-Context`, `X-Hero-Claims`) when the transport is a Unix Domain Socket. Only the HTTP transport branch forwards them. **Tracked in:** [lhumina_code/hero_rpc#42](https://forge.ourworld.tf/lhumina_code/hero_rpc/issues/42) ## Root cause `http_post_unix` (transport.rs:354) has no `extra_headers` parameter. The `UnixSocket` branch of `post_raw_json_with_headers` (line 248) calls it without passing the headers: ```rust // headers silently dropped: let resp_bytes = self.http_post_unix(path, rpc_path, &body_bytes).await?; ``` This was a known gap when the function was introduced in commit [ed6e7eb](https://forge.ourworld.tf/lhumina_code/hero_rpc/commit/ed6e7eb3c0aca6e102d0c15b3cfbb6ba838462de) (April 13) — the comment reads _"Unix-socket transport currently drops the headers — raw socket framing has no place to carry them"_ — but this is incorrect: we use HTTP-over-UDS, so headers can and must be carried in the HTTP framing. ## Current impact No repos are broken **today** because the only real caller of `post_raw_json_with_headers` is `OsisClient` (in `openrpc_http_client_lib`), which uses `OpenRpcTransport::http` (via hero_router), not UDS. So `X-Hero-Context` reaches the service correctly through that path. ## When it becomes a real problem The moment any service makes a **direct server-to-server OSIS call over UDS** (bypassing hero_router) while passing a context — e.g. a future `OsisClient::new_unix()` constructor — the context header will be silently dropped. The service will return default-context data without any error, making the failure invisible and hard to diagnose. ## Fix (small, isolated to one file) All changes are in [`hero_rpc/crates/openrpc/src/transport.rs`](https://forge.ourworld.tf/lhumina_code/hero_rpc/src/branch/development/crates/openrpc/src/transport.rs): 1. Add `extra_headers: &[(&str, &str)]` parameter to `http_post_unix` 2. Apply them in the `hyper::Request::builder()` chain inside `http_post_unix` 3. Pass `extra_headers` through in the `UnixSocket` branch of `post_raw_json_with_headers` 4. Pass `&[]` from `call_unix_http` (no headers needed there) No public API changes. No other files need to change.
Author
Member

@despiegk @timur — question before this gets fixed:

If the architectural rule is that all calls go through hero_router, then the scenario described above (a service calling OSIS directly over UDS with X-Hero-Context) should never happen in the first place. OsisClient::new_unix() would be an architectural violation, and the broken header forwarding would actually enforce the constraint correctly — you get silent wrong behavior if you bypass the router, which signals you are doing it wrong.

In that case, hero_rpc#42 does not need to be fixed, and the right resolution is to close it as by design and document that all context-aware calls must route through hero_router.

Is that the intent, or are there legitimate cases where a service should call another service directly over UDS while carrying a context?

@despiegk @timur — question before this gets fixed: If the architectural rule is that **all calls go through hero_router**, then the scenario described above (a service calling OSIS directly over UDS with `X-Hero-Context`) should never happen in the first place. `OsisClient::new_unix()` would be an architectural violation, and the broken header forwarding would actually enforce the constraint correctly — you get silent wrong behavior if you bypass the router, which signals you are doing it wrong. In that case, `hero_rpc#42` does not need to be fixed, and the right resolution is to close it as by design and document that all context-aware calls must route through hero_router. Is that the intent, or are there legitimate cases where a service should call another service directly over UDS while carrying a context?
Author
Member

Root cause found — and it is different from all previous analyses

After tracing the full call chain from hero_biz through hero_router down to the OSIS dispatch layer, the actual bug was found in hero_rpc/crates/osis/src/rpc/dispatch.rs.

What was actually broken

dispatch_jsonrpc_auto_context read the context name exclusively from params._context in the JSON body, defaulting to "root" if absent:

// Before fix — ignores X-Hero-Context header entirely
let context_name = req.params
    .as_ref()
    .and_then(|p| p.get("_context"))
    .and_then(|v| v.as_str())
    .unwrap_or("root")
    .to_string();

OsisClient sends context via the X-Hero-Context HTTP header, not in the body. RequestContext::from_headers correctly parses the header into hero_context_name, but dispatch_jsonrpc_auto_context never consulted it — so every call silently fell back to "root".

Fix applied

dispatch_jsonrpc_auto_context now prefers request_context.hero_context_name (from the header) and falls back to params._context for backwards compatibility:

let context_name = request_context
    .hero_context_name
    .clone()
    .or_else(|| {
        req.params
            .as_ref()
            .and_then(|p| p.get("_context"))
            .and_then(|v| v.as_str())
            .map(|s| s.to_string())
    })
    .unwrap_or_else(|| "root".to_string());

Why the previous analyses went off track

  1. hero_rpc#42 / home#233 — focused on whether UDS transport drops X-Hero-Context headers. The transport was a red herring: the header was being forwarded correctly all the way to the server. The server just never used it for context selection.

  2. The UDS header fix was architecturally wrong anyway — per the herolib_openrpc_authorize and hero_context skills, all context-aware calls go through hero_router, which injects headers. Direct UDS calls are trusted internal calls with no context header — the UDS header-dropping behaviour is correct by design.

  3. The Cargo.lock / endpoint format analyses — also red herrings; OsisClient was already using the correct endpoint and the correct header. The problem was one layer deeper in how the server processed that header.

Summary

The bug was always in the dispatch layer, not in the transport. Fixed in hero_rpc crates/osis/src/rpc/dispatch.rs.

## Root cause found — and it is different from all previous analyses After tracing the full call chain from hero_biz through hero_router down to the OSIS dispatch layer, the actual bug was found in `hero_rpc/crates/osis/src/rpc/dispatch.rs`. ### What was actually broken `dispatch_jsonrpc_auto_context` read the context name exclusively from `params._context` in the JSON body, defaulting to `"root"` if absent: ```rust // Before fix — ignores X-Hero-Context header entirely let context_name = req.params .as_ref() .and_then(|p| p.get("_context")) .and_then(|v| v.as_str()) .unwrap_or("root") .to_string(); ``` `OsisClient` sends context via the `X-Hero-Context` HTTP header, not in the body. `RequestContext::from_headers` correctly parses the header into `hero_context_name`, but `dispatch_jsonrpc_auto_context` never consulted it — so every call silently fell back to `"root"`. ### Fix applied `dispatch_jsonrpc_auto_context` now prefers `request_context.hero_context_name` (from the header) and falls back to `params._context` for backwards compatibility: ```rust let context_name = request_context .hero_context_name .clone() .or_else(|| { req.params .as_ref() .and_then(|p| p.get("_context")) .and_then(|v| v.as_str()) .map(|s| s.to_string()) }) .unwrap_or_else(|| "root".to_string()); ``` ### Why the previous analyses went off track 1. **hero_rpc#42 / home#233** — focused on whether UDS transport drops `X-Hero-Context` headers. The transport was a red herring: the header was being forwarded correctly all the way to the server. The server just never used it for context selection. 2. **The UDS header fix was architecturally wrong anyway** — per the `herolib_openrpc_authorize` and `hero_context` skills, all context-aware calls go through `hero_router`, which injects headers. Direct UDS calls are trusted internal calls with no context header — the UDS header-dropping behaviour is correct by design. 3. **The Cargo.lock / endpoint format analyses** — also red herrings; `OsisClient` was already using the correct endpoint and the correct header. The problem was one layer deeper in how the server processed that header. ### Summary The bug was always in the dispatch layer, not in the transport. Fixed in `hero_rpc` `crates/osis/src/rpc/dispatch.rs`.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/home#233
No description provided.