lhumina_code/hero_researcher

Fork 0

admin interface not fully functioning #3

New issue

Open

opened 2026-03-19 10:11:05 +00:00 by despiegk · 3 comments

despiegk commented

2026-03-19 10:11:05 +00:00

Owner

I don't see the job, we should be able to track whats going on

we should be able to see the logs

is it all in hero_proc???

also implement /hero_admin_connection_status

![image](/attachments/1973ef42-013e-4c8f-81ba-557cfae323a5) I don't see the job, we should be able to track whats going on we should be able to see the logs is it all in hero_proc??? also implement /hero_admin_connection_status

image.png

653 KiB

despiegk commented

2026-03-19 10:17:02 +00:00

Author

Owner

Implementation Spec for Issue #3 — Admin Interface: Jobs, Logs & Connection Status

Objective

The hero_researcher admin UI needs three improvements:

Job log visibility — users cannot see what a running/completed job is doing. Add a log panel that streams captured stdout/stderr from child processes.
Connection status widget — replace the custom 2-state dot with the reusable connection-status.js widget (3-state: connected / backend-down / disconnected) including Bootstrap popover.
/hero_admin_connection_status endpoint — required by the hero skill system for embedding this service's status in other dashboards.

Note: hero_proc is not involved. This project has its own standalone server and job manager. Job logs are stored in ~/hero/var/hero_researcher/jobs/<id>.json.

Requirements

Sidebar job list continues to work (no change)
Clicking a job shows a log panel below progress, auto-scrolling, refreshes every 5s for running jobs
GET /health → { status: "ok", service: "hero_researcher_ui", version: "0.1.0" }
GET /api/health → enriched with service and version fields
GET /hero_admin_connection_status → JSON with status, service, backend
GET /api/jobs/:id/logs → { jobId, lines: string[] }
research.logs JSON-RPC method on the server dispatcher
researchLogs(jobId) method exported from hero_researcher_sdk
bun test passes for all packages

Files to Modify/Create

File	Action
`packages/hero_researcher_server/src/job-manager.ts`	Add `logs: string[]` to `JobFile`, buffer per-job stderr/stdout (cap 2000 lines), export `getJobLogs()`
`packages/hero_researcher_server/src/dispatcher.ts`	Add `research.logs` handler + update `rpc.discover`
`packages/hero_researcher_sdk/src/types.ts`	Add `ResearchLogs` interface
`packages/hero_researcher_sdk/src/client.ts`	Add `researchLogs(jobId)` method
`packages/hero_researcher_ui/src/routes.ts`	Add `/health`, `/hero_admin_connection_status`, `/api/jobs/:id/logs`; enrich `/api/health`
`packages/hero_researcher_ui/public/index.html`	Replace status dot with `connection-status.js`; add log panel card + JS
`packages/hero_researcher_ui/public/connection-status.js`	Copy from hero_proc_ui static assets
`packages/hero_researcher_ui/tests/routes.test.ts`	Add tests for new routes
`packages/hero_researcher_server/tests/dispatcher.test.ts`	Create: tests for `research.logs`

Implementation Plan

Step 1 — Extend `JobFile` for log capture (job-manager.ts)

Add logs?: string[] to JobFile interface
In spawnResearchProcess, attach readers to both proc.stdout and proc.stderr that append decoded chunks to jf.logs (cap at 2000 lines), then call persistJob(jf)
Export getJobLogs(jobId: string): Promise<string[]>
Dependencies: none

Step 2 — Add `research.logs` RPC handler (dispatcher.ts)

Import getJobLogs from ./job-manager
Add "research.logs" handler returning { jobId, lines }
Add to rpc.discover methods array
Dependencies: Step 1

Step 3 — Add SDK types and client method (sdk types.ts + client.ts)

Add ResearchLogs interface to types.ts
Add researchLogs(jobId: string): Promise<ResearchLogs> to HeroResearcherClient
Dependencies: none (can run parallel with Step 1)

Step 4 — Add new UI routes (routes.ts)

/health → { status: "ok", service: "hero_researcher_ui", version: "0.1.0" }
Enrich /api/health with service and version
/hero_admin_connection_status → JSON with status + backend check via rpc.health
/api/jobs/:id/logs → calls client.researchLogs(id), returns { jobId, lines }
Dependencies: Step 3

Step 5 — Copy connection-status.js

Copy /Volumes/T7/code0/hero_proc/crates/hero_proc_ui/static/js/connection-status.js to packages/hero_researcher_ui/public/connection-status.js
Dependencies: none

Add .status-dot CSS (3 states: green/amber/red)
Replace old .polling-indicator span and checkHealth() with connection-status.js poller
Load <script src="/connection-status.js"></script>
Dependencies: Step 5

Step 7 — Update index.html: log panel

Add #logs-card Bootstrap card (initially d-none) with <pre id="logs-content">
Add loadLogs(jobId) function that GETs /api/jobs/:id/logs and renders lines
Call loadLogs() in viewJob(), set up 5s poll for running jobs
Clear log panel in resetView() and stopPolling()
Dependencies: Step 4, Step 6

Step 8 — Add/update tests

packages/hero_researcher_ui/tests/routes.test.ts: add researchLogs mock, test /health, /hero_admin_connection_status, /api/jobs/:id/logs
packages/hero_researcher_server/tests/dispatcher.test.ts: create tests for research.logs
Dependencies: Steps 1–7

Acceptance Criteria

Status dot has 3 visual states (green/amber/red) via connection-status.js
Clicking dot opens Bootstrap popover with service/backend status + last-checked time
GET /health → { status: "ok", service: "hero_researcher_ui", version: "0.1.0" }
GET /api/health → includes service and version fields
GET /hero_admin_connection_status → JSON with status, service, backend
GET /api/jobs/:id/logs → { jobId, lines: string[] }
Log panel visible when viewing any job; auto-scrolls; refreshes for running jobs
bun test passes for all packages

Notes

hero_proc is not involved — this project manages its own jobs independently
Adding logs field to JobFile is safe — existing files without it deserialize with logs: undefined, handled via ?? []
connection-status.js works with Bootstrap 5.3.3 already loaded via CDN
/hero_admin_connection_status returns JSON only (no HTML needed)

# Implementation Spec for Issue #3 — Admin Interface: Jobs, Logs & Connection Status ## Objective The hero_researcher admin UI needs three improvements: 1. **Job log visibility** — users cannot see what a running/completed job is doing. Add a log panel that streams captured stdout/stderr from child processes. 2. **Connection status widget** — replace the custom 2-state dot with the reusable `connection-status.js` widget (3-state: connected / backend-down / disconnected) including Bootstrap popover. 3. **`/hero_admin_connection_status` endpoint** — required by the hero skill system for embedding this service's status in other dashboards. > Note: hero_proc is **not** involved. This project has its own standalone server and job manager. Job logs are stored in `~/hero/var/hero_researcher/jobs/<id>.json`. ## Requirements - Sidebar job list continues to work (no change) - Clicking a job shows a log panel below progress, auto-scrolling, refreshes every 5s for running jobs - `GET /health` → `{ status: "ok", service: "hero_researcher_ui", version: "0.1.0" }` - `GET /api/health` → enriched with `service` and `version` fields - `GET /hero_admin_connection_status` → JSON with `status`, `service`, `backend` - `GET /api/jobs/:id/logs` → `{ jobId, lines: string[] }` - `research.logs` JSON-RPC method on the server dispatcher - `researchLogs(jobId)` method exported from hero_researcher_sdk - `bun test` passes for all packages ## Files to Modify/Create | File | Action | |---|---| | `packages/hero_researcher_server/src/job-manager.ts` | Add `logs: string[]` to `JobFile`, buffer per-job stderr/stdout (cap 2000 lines), export `getJobLogs()` | | `packages/hero_researcher_server/src/dispatcher.ts` | Add `research.logs` handler + update `rpc.discover` | | `packages/hero_researcher_sdk/src/types.ts` | Add `ResearchLogs` interface | | `packages/hero_researcher_sdk/src/client.ts` | Add `researchLogs(jobId)` method | | `packages/hero_researcher_ui/src/routes.ts` | Add `/health`, `/hero_admin_connection_status`, `/api/jobs/:id/logs`; enrich `/api/health` | | `packages/hero_researcher_ui/public/index.html` | Replace status dot with `connection-status.js`; add log panel card + JS | | `packages/hero_researcher_ui/public/connection-status.js` | Copy from hero_proc_ui static assets | | `packages/hero_researcher_ui/tests/routes.test.ts` | Add tests for new routes | | `packages/hero_researcher_server/tests/dispatcher.test.ts` | Create: tests for `research.logs` | ## Implementation Plan ### Step 1 — Extend `JobFile` for log capture (job-manager.ts) - Add `logs?: string[]` to `JobFile` interface - In `spawnResearchProcess`, attach readers to both `proc.stdout` and `proc.stderr` that append decoded chunks to `jf.logs` (cap at 2000 lines), then call `persistJob(jf)` - Export `getJobLogs(jobId: string): Promise<string[]>` - Dependencies: none ### Step 2 — Add `research.logs` RPC handler (dispatcher.ts) - Import `getJobLogs` from `./job-manager` - Add `"research.logs"` handler returning `{ jobId, lines }` - Add to `rpc.discover` methods array - Dependencies: Step 1 ### Step 3 — Add SDK types and client method (sdk types.ts + client.ts) - Add `ResearchLogs` interface to types.ts - Add `researchLogs(jobId: string): Promise<ResearchLogs>` to HeroResearcherClient - Dependencies: none (can run parallel with Step 1) ### Step 4 — Add new UI routes (routes.ts) - `/health` → `{ status: "ok", service: "hero_researcher_ui", version: "0.1.0" }` - Enrich `/api/health` with `service` and `version` - `/hero_admin_connection_status` → JSON with status + backend check via rpc.health - `/api/jobs/:id/logs` → calls `client.researchLogs(id)`, returns `{ jobId, lines }` - Dependencies: Step 3 ### Step 5 — Copy connection-status.js - Copy `/Volumes/T7/code0/hero_proc/crates/hero_proc_ui/static/js/connection-status.js` to `packages/hero_researcher_ui/public/connection-status.js` - Dependencies: none ### Step 6 — Update index.html: connection status widget - Add `.status-dot` CSS (3 states: green/amber/red) - Replace old `.polling-indicator` span and `checkHealth()` with `connection-status.js` poller - Load `<script src="/connection-status.js"></script>` - Dependencies: Step 5 ### Step 7 — Update index.html: log panel - Add `#logs-card` Bootstrap card (initially `d-none`) with `<pre id="logs-content">` - Add `loadLogs(jobId)` function that GETs `/api/jobs/:id/logs` and renders lines - Call `loadLogs()` in `viewJob()`, set up 5s poll for running jobs - Clear log panel in `resetView()` and `stopPolling()` - Dependencies: Step 4, Step 6 ### Step 8 — Add/update tests - `packages/hero_researcher_ui/tests/routes.test.ts`: add `researchLogs` mock, test `/health`, `/hero_admin_connection_status`, `/api/jobs/:id/logs` - `packages/hero_researcher_server/tests/dispatcher.test.ts`: create tests for `research.logs` - Dependencies: Steps 1–7 ## Acceptance Criteria - [ ] Status dot has 3 visual states (green/amber/red) via `connection-status.js` - [ ] Clicking dot opens Bootstrap popover with service/backend status + last-checked time - [ ] `GET /health` → `{ status: "ok", service: "hero_researcher_ui", version: "0.1.0" }` - [ ] `GET /api/health` → includes `service` and `version` fields - [ ] `GET /hero_admin_connection_status` → JSON with `status`, `service`, `backend` - [ ] `GET /api/jobs/:id/logs` → `{ jobId, lines: string[] }` - [ ] Log panel visible when viewing any job; auto-scrolls; refreshes for running jobs - [ ] `bun test` passes for all packages ## Notes - **hero_proc is not involved** — this project manages its own jobs independently - Adding `logs` field to `JobFile` is safe — existing files without it deserialize with `logs: undefined`, handled via `?? []` - `connection-status.js` works with Bootstrap 5.3.3 already loaded via CDN - `/hero_admin_connection_status` returns JSON only (no HTML needed)

despiegk commented

2026-03-19 10:26:17 +00:00

Author

Owner

Test Results

Total: 247
Passed: 247
Failed: 0

All packages passed:

Package	Tests	Status
hero_researcher_server	13	PASS
hero_researcher_sdk	25	PASS
hero_researcher_ui	13	PASS
hero_researcher (core)	196	PASS

Full output

hero_researcher_server test: bun test v1.3.5 (1e86cebd)
hero_researcher_server test:  13 pass
hero_researcher_server test:  0 fail
hero_researcher_server test:  54 expect() calls
hero_researcher_server test: Ran 13 tests across 1 file. [64.00ms]

hero_researcher_sdk test: bun test v1.3.5 (1e86cebd)
hero_researcher_sdk test:  25 pass
hero_researcher_sdk test:  0 fail
hero_researcher_sdk test:  71 expect() calls
hero_researcher_sdk test: Ran 25 tests across 2 files. [75.00ms]

hero_researcher_ui test: bun test v1.3.5 (1e86cebd)
hero_researcher_ui test:  13 pass
hero_researcher_ui test:  0 fail
hero_researcher_ui test:  44 expect() calls
hero_researcher_ui test: Ran 13 tests across 1 file. [99.00ms]

hero_researcher test:  196 pass
hero_researcher test:  0 fail
hero_researcher test:  426 expect() calls
hero_researcher test: Ran 196 tests across 19 files. [9.38s]

## Test Results - **Total:** 247 - **Passed:** 247 - **Failed:** 0 All packages passed: | Package | Tests | Status | |---------|-------|--------| | hero_researcher_server | 13 | PASS | | hero_researcher_sdk | 25 | PASS | | hero_researcher_ui | 13 | PASS | | hero_researcher (core) | 196 | PASS | <details> <summary>Full output</summary> ``` hero_researcher_server test: bun test v1.3.5 (1e86cebd) hero_researcher_server test: 13 pass hero_researcher_server test: 0 fail hero_researcher_server test: 54 expect() calls hero_researcher_server test: Ran 13 tests across 1 file. [64.00ms] hero_researcher_sdk test: bun test v1.3.5 (1e86cebd) hero_researcher_sdk test: 25 pass hero_researcher_sdk test: 0 fail hero_researcher_sdk test: 71 expect() calls hero_researcher_sdk test: Ran 25 tests across 2 files. [75.00ms] hero_researcher_ui test: bun test v1.3.5 (1e86cebd) hero_researcher_ui test: 13 pass hero_researcher_ui test: 0 fail hero_researcher_ui test: 44 expect() calls hero_researcher_ui test: Ran 13 tests across 1 file. [99.00ms] hero_researcher test: 196 pass hero_researcher test: 0 fail hero_researcher test: 426 expect() calls hero_researcher test: Ran 196 tests across 19 files. [9.38s] ``` </details>

despiegk commented

2026-03-19 18:58:15 +00:00

Author

Owner

Implementation Complete ✅

All work from this issue has been implemented and committed in b24ebb5.

Changes Made

Admin UI — Jobs & Logs Display

packages/hero_researcher_ui/public/index.html — Jobs sidebar lists all research jobs with status badges (RUNNING/COMPLETED/FAILED/CANCELLED/PENDING); clicking a job shows its log stream via GET /api/jobs/:id/logs; logs panel is a Bootstrap card with <pre> content
packages/hero_researcher_ui/src/routes.ts — Added /api/jobs/:id/logs, /health, /hero_admin_connection_status, and /connection-status.js routes

packages/hero_researcher_ui/public/connection-status.js — Copied from hero_proc_ui; provides 3-state (connected / backend-down / disconnected) animated dot with Bootstrap popover
Widget polls GET /health (UI liveness) then system.ping via RPC (backend liveness) every 5 s
CSS added for @keyframes pulse and all three dot states

hero_proc Integration (fixes "ENOENT: posix_spawn 'bun'")

packages/hero_researcher_server/src/job-manager.ts — Replaced Bun.spawn with hero_proc JSON-RPC over Unix socket (~/hero/var/sockets/hero_proc_server.sock); uses job.create with interpreter: "bun", polls job.status, calls job.logs and job.cancel
packages/hero_researcher/src/run-job.ts — Added env fallback: if (!jobId) jobId = process.env.JOB_ID so hero_proc can pass the job ID via environment

Build System

Makefile — run depends on build; added install-bins (copies binaries to ~/hero/bin/), logs, logs-ui, restart, status targets
scripts/run.sh — Uses compiled dist/hero_researcher_server binary if available; exports HERO_RESEARCHER_ROOT so compiled server can locate run-job.ts
Added build scripts to all packages: hero_researcher_server and hero_researcher_cli use --compile; hero_researcher_sdk uses --target bun

OpenRPC Spec & SDK Sync

packages/hero_researcher_server/openrpc.json — Added system.ping and research.logs methods; fixed ResearchJob.id → jobId; fixed progress type; updated server URL to Unix socket
packages/hero_researcher_server/src/dispatcher.ts — rpc.discover now serves the real openrpc.json; rpc.health returns version field
packages/hero_researcher_sdk/src/client.ts — Fixed health() method name; added ping(); fixed researchList() return type to { jobs, total }

Architecture Compliance (hero_crates_best_practices_check)

packages/hero_researcher_examples/ — New package with src/health.ts and src/basic_usage.ts examples
.forgejo/workflows/ci.yaml — CI workflow: install → lint → test → build

Commit

b24ebb58301686b32575f8ea04dc814b874b0c68

## Implementation Complete ✅ All work from this issue has been implemented and committed in `b24ebb5`. --- ### Changes Made #### Admin UI — Jobs & Logs Display - **`packages/hero_researcher_ui/public/index.html`** — Jobs sidebar lists all research jobs with status badges (RUNNING/COMPLETED/FAILED/CANCELLED/PENDING); clicking a job shows its log stream via `GET /api/jobs/:id/logs`; logs panel is a Bootstrap card with `<pre>` content - **`packages/hero_researcher_ui/src/routes.ts`** — Added `/api/jobs/:id/logs`, `/health`, `/hero_admin_connection_status`, and `/connection-status.js` routes #### Connection Status Widget - **`packages/hero_researcher_ui/public/connection-status.js`** — Copied from hero_proc_ui; provides 3-state (connected / backend-down / disconnected) animated dot with Bootstrap popover - Widget polls `GET /health` (UI liveness) then `system.ping` via RPC (backend liveness) every 5 s - CSS added for `@keyframes pulse` and all three dot states #### hero_proc Integration (fixes "ENOENT: posix_spawn 'bun'") - **`packages/hero_researcher_server/src/job-manager.ts`** — Replaced `Bun.spawn` with hero_proc JSON-RPC over Unix socket (`~/hero/var/sockets/hero_proc_server.sock`); uses `job.create` with `interpreter: "bun"`, polls `job.status`, calls `job.logs` and `job.cancel` - **`packages/hero_researcher/src/run-job.ts`** — Added env fallback: `if (!jobId) jobId = process.env.JOB_ID` so hero_proc can pass the job ID via environment #### Build System - **`Makefile`** — `run` depends on `build`; added `install-bins` (copies binaries to `~/hero/bin/`), `logs`, `logs-ui`, `restart`, `status` targets - **`scripts/run.sh`** — Uses compiled `dist/hero_researcher_server` binary if available; exports `HERO_RESEARCHER_ROOT` so compiled server can locate `run-job.ts` - Added `build` scripts to all packages: `hero_researcher_server` and `hero_researcher_cli` use `--compile`; `hero_researcher_sdk` uses `--target bun` #### OpenRPC Spec & SDK Sync - **`packages/hero_researcher_server/openrpc.json`** — Added `system.ping` and `research.logs` methods; fixed `ResearchJob.id` → `jobId`; fixed `progress` type; updated server URL to Unix socket - **`packages/hero_researcher_server/src/dispatcher.ts`** — `rpc.discover` now serves the real `openrpc.json`; `rpc.health` returns `version` field - **`packages/hero_researcher_sdk/src/client.ts`** — Fixed `health()` method name; added `ping()`; fixed `researchList()` return type to `{ jobs, total }` #### Architecture Compliance (hero_crates_best_practices_check) - **`packages/hero_researcher_examples/`** — New package with `src/health.ts` and `src/basic_usage.ts` examples - **`.forgejo/workflows/ci.yaml`** — CI workflow: install → lint → test → build --- ### Commit `b24ebb58301686b32575f8ea04dc814b874b0c68`

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

lhumina_code/hero_researcher#3

No description provided.

Rows
Columns

admin interface not fully functioning #3

Implementation Spec for Issue #3 — Admin Interface: Jobs, Logs & Connection Status

Objective

Requirements

Files to Modify/Create

Implementation Plan

Step 1 — Extend JobFile for log capture (job-manager.ts)

Step 2 — Add research.logs RPC handler (dispatcher.ts)

Step 3 — Add SDK types and client method (sdk types.ts + client.ts)

Step 4 — Add new UI routes (routes.ts)

Step 5 — Copy connection-status.js

Step 6 — Update index.html: connection status widget

Step 7 — Update index.html: log panel

Step 8 — Add/update tests

Acceptance Criteria

Notes

Test Results

Implementation Complete ✅

Changes Made

Admin UI — Jobs & Logs Display

Connection Status Widget

hero_proc Integration (fixes "ENOENT: posix_spawn 'bun'")

Build System

OpenRPC Spec & SDK Sync

Architecture Compliance (hero_crates_best_practices_check)

Commit

Step 1 — Extend `JobFile` for log capture (job-manager.ts)

Step 2 — Add `research.logs` RPC handler (dispatcher.ts)