Combined deploy — all PRs merged + Dockerfile.prod + TFGrid deployment #43

Closed

mik-tf wants to merge 102 commits from development_combined_deploy into development

pull from: development_combined_deploy

merge into: lhumina_code:development

lhumina_code:development

lhumina_code:development_base_image

lhumina_code:development_timur

lhumina_code:development_kristof

lhumina_code:main

Conversation 0 Commits 102 Files changed 54 +2172 -222

mik-tf commented 2026-02-27 14:26:44 +00:00

Owner

Copy link

Summary

Combines all deployment work from PRs #26, #31, #32, #34, #36, #40, #42 into a single branch for the first full Hero OS container deployment on TFGrid.

Production container (Dockerfile.prod)

• Two-stage build: Rust builder compiles ALL 20+ service binaries, runtime is debian:bookworm-slim (no Rust toolchain)
• Services built via docker/build-services.sh: hero_auth, hero_books, hero_embedder, hero_fossil, hero_indexer, hero_inspector, hero_os, hero_osis, hero_proxy, hero_redis, hero_voice + hero_indexer_ui
• ONNX Runtime bundled for hero_embedder (load-dynamic)
• WASM frontends built (hero_os_ui shell + hero_archipelagos islands)
• Service TOMLs stripped of [build]/[install] sections (binaries are pre-built)
• zinit TOMLs removed from user profile (zinit is infrastructure, started by entrypoint)

CI pipeline

• build-container.yaml switched to Dockerfile.prod
• Tag push produces :version + :dev (no :latest until production-ready)
• Manual dispatch produces :version only
• Removed broken build-macos.yaml (no runner) and redundant build-prod-container.yaml
• Disabled tag trigger on build-linux.yaml (missing build_lib.sh functions, manual only)

TFGrid deployment (deploy/single-vm/)

• OpenTofu config with grid_scheduler for node selection
• Multi-environment support: envs/dev/ and envs/prod/ with per-env tfvars
• setup.sh handles Docker install on zinit VMs (no systemd — uses dockerd directly)
• fuse-overlayfs storage driver, data-root on /data disk
• Web gateway (HTTPS) via TFGrid name proxy
• Makefile orchestration: make all ENV=prod for full deploy

Service fixes

• hero_indexer: corrected repo URL in TOML
• hero_books: switched to Unix socket (unix://__HERO_VAR__/sockets/hero_books.sock)
• hero_osis/hero_os: set HERO_CONTEXTS=default for auth
• Container CI: DinD checkout, SSH forwarding, proper build context

PRs included

• #26 fix: container CI pipeline
• #31 fix: hero_indexer repo URL
• #32 fix: container build (included in #34)
• #34 production container with pre-built binaries
• #36 switch hero_books to Unix socket
• #40 fix auth (HERO_CONTEXTS=default)
• #42 TFGrid deployment

Closes #25
Closes #27
Closes #29
Closes #30
Closes #35
Closes #41

Test plan

• CI green (build.yaml + build-container.yaml)
• Container image pushed as hero_zero:0.1.0 + hero_zero:dev
• TFGrid deploy: make all ENV=prod
• All 20 services start inside container
• Gateway responds at heroos.gent02.grid.tf

## Summary Combines all deployment work from PRs #26, #31, #32, #34, #36, #40, #42 into a single branch for the first full Hero OS container deployment on TFGrid. ### Production container (Dockerfile.prod) - Two-stage build: Rust builder compiles ALL 20+ service binaries, runtime is debian:bookworm-slim (no Rust toolchain) - Services built via `docker/build-services.sh`: hero_auth, hero_books, hero_embedder, hero_fossil, hero_indexer, hero_inspector, hero_os, hero_osis, hero_proxy, hero_redis, hero_voice + hero_indexer_ui - ONNX Runtime bundled for hero_embedder (load-dynamic) - WASM frontends built (hero_os_ui shell + hero_archipelagos islands) - Service TOMLs stripped of [build]/[install] sections (binaries are pre-built) - zinit TOMLs removed from user profile (zinit is infrastructure, started by entrypoint) ### CI pipeline - `build-container.yaml` switched to Dockerfile.prod - Tag push produces `:version` + `:dev` (no `:latest` until production-ready) - Manual dispatch produces `:version` only - Removed broken `build-macos.yaml` (no runner) and redundant `build-prod-container.yaml` - Disabled tag trigger on `build-linux.yaml` (missing build_lib.sh functions, manual only) ### TFGrid deployment (`deploy/single-vm/`) - OpenTofu config with grid_scheduler for node selection - Multi-environment support: `envs/dev/` and `envs/prod/` with per-env tfvars - `setup.sh` handles Docker install on zinit VMs (no systemd — uses dockerd directly) - fuse-overlayfs storage driver, data-root on /data disk - Web gateway (HTTPS) via TFGrid name proxy - Makefile orchestration: `make all ENV=prod` for full deploy ### Service fixes - hero_indexer: corrected repo URL in TOML - hero_books: switched to Unix socket (`unix://__HERO_VAR__/sockets/hero_books.sock`) - hero_osis/hero_os: set `HERO_CONTEXTS=default` for auth - Container CI: DinD checkout, SSH forwarding, proper build context ## PRs included - #26 fix: container CI pipeline - #31 fix: hero_indexer repo URL - #32 fix: container build (included in #34) - #34 production container with pre-built binaries - #36 switch hero_books to Unix socket - #40 fix auth (HERO_CONTEXTS=default) - #42 TFGrid deployment Closes #25 Closes #27 Closes #29 Closes #30 Closes #35 Closes #41 ## Test plan - [ ] CI green (build.yaml + build-container.yaml) - [ ] Container image pushed as hero_zero:0.1.0 + hero_zero:dev - [ ] TFGrid deploy: `make all ENV=prod` - [ ] All 20 services start inside container - [ ] Gateway responds at heroos.gent02.grid.tf

mik-tf added 30 commits 2026-02-27 14:26:44 +00:00

fix: replace actions/checkout with git clone in container build CI ... 6149d0c847

actions/checkout@v4 fails in docker:24-dind (alpine) due to
glibc/musl mismatch. Replace with manual git clone using
FORGEJO_TOKEN for auth. Also removes nodejs dependency since
checkout action is no longer used.

Fixes both build-container and create-release jobs.

Closes #25

Co-Authored-By: mik-tf <mik@threefold.io>

fix: pass SSH_PRIVATE_KEY via env block to preserve newlines ... 5158b1a887

Direct ${{ secrets }} interpolation in run blocks mangles multi-line
SSH keys. Pass via env: block instead, matching the pattern used in
build.yaml which works.

Co-Authored-By: mik-tf <mik@threefold.io>

fix: improve SSH setup — skip ssh-keyscan, add debug output ... 3ca3e2b267

ssh-keyscan may hang in DinD container. Use ssh config with
StrictHostKeyChecking instead. Add error output to identify
which step fails.

Co-Authored-By: mik-tf <mik@threefold.io>

fix: add nodejs to apk install — Docker actions require Node ... ff901bdb3b

docker/setup-buildx-action, docker/login-action, and
docker/build-push-action are JavaScript actions that need
Node.js in the runner. Without it, they fail with exit 127
("node: not found").

Co-Authored-By: mik-tf <mik@threefold.io>

fix: clone zinit in Dockerfile for zinit_sdk path dependency ... 2a4a2fd98b

The workspace Cargo.toml has a path dependency on
../zinit/crates/zinit_sdk. In the Docker build context this
resolves to /build/zinit/ which must be cloned before cargo
build can proceed.

Co-Authored-By: mik-tf <mik@threefold.io>

fix: update Dockerfile to build actual workspace binaries ... 2a02b76fc1

The Dockerfile referenced a hero_zero binary that doesn't exist
in this workspace. The workspace produces hero_services_server,
hero_services, and hero_services_ui. Updated to build and copy
the actual binaries.

Simplified the builder stage — removed aspirational hero_zero
install-service loop and zinit install steps that depend on
non-existent binaries.

Co-Authored-By: mik-tf <mik@threefold.io>

fix: correct hero_indexer repo URL from hero_index_server to hero_indexer ... 003141c3fb

The TOML referenced lhumina_code/hero_index_server which is an empty
repo. The actual repo is lhumina_code/hero_indexer.

Co-Authored-By: mik-tf <mik@threefold.io>

fix: correct Dockerfile binary names, CI pipeline, and add entrypoint ... 05b6c3ff98

- Dockerfile: fix binary names (hero_services_openrpc, not hero_zero),
  build zinit workspace, use rust:slim-bookworm runtime with g++ for
  services that need C++ at install time
- CI workflow: manual git clone (actions/checkout fails in alpine DinD),
  explicit dockerd startup, SSH key via env block to prevent multiline
  mangling, StrictHostKeyChecking accept-new
- Entrypoint: start zinit_openrpc, wait for socket, launch
  hero_services_openrpc with user profile. Generic SSH key permission
  fix for any mounted key type.

Co-Authored-By: mik-tf <mik@threefold.io>

fix: use flock to prevent race condition on shared repo installs ... a9b13c11a6

When multiple services share the same git repo (e.g. zinit_openrpc
and zinit_http both use geomind_code/zinit), their install oneshots
race on the same directory. The second install starts milliseconds
after the first finishes and fails with exit 128 (git lock conflict).

Wrap clone_or_update_sh in flock so concurrent installs serialize
their git operations on the same repo directory.

Fixes #33

Co-Authored-By: mik-tf <mik@threefold.io>

feat: add production container with pre-built service binaries ... 0f01466789

Production Dockerfile (Dockerfile.prod) compiles ALL hero service
binaries at build time, producing a slim debian:bookworm-slim image
with no Rust toolchain or SSH keys needed at runtime.

- docker/build-services.sh: clones and builds 12 service repos
- docker/strip-build-sections.sh: removes [build]/[install] TOML
  sections so orchestrator starts services without install oneshots
- build-prod-container.yaml: CI workflow for production image builds

Co-Authored-By: mik-tf <mik@threefold.io>

fix: preserve pre-built binaries in production mode ... e422ebe918

- stop_and_clean: skip binary deletion when no services have [build]
  sections (production containers with pre-baked binaries)
- build-services.sh: add build_cargo() for repos needing direct cargo
  builds (hero_os: skip WASM/Dioxus frontend), fix status tracking
- strip-build-sections.sh: also remove "install" from profile actions
  to prevent orchestrator from writing install oneshots

Tested: 19/21 services running in production container.
Remaining: hero_indexer (TOML naming mismatch) and hero_books (blocked
by hero_indexer dependency).

Co-Authored-By: mik-tf <mik@threefold.io>

fix: split hero_indexer into openrpc/http to match repo binary names ... d2f96f7aed

The hero_indexer repo now builds hero_indexer_openrpc + hero_indexer_http
(not a single hero_indexer binary). Split the TOML accordingly and update
all depends_on references in hero_books, hero_indexer_ui, hero_osis_openrpc.

Refs #29

Co-Authored-By: mik-tf <mik@threefold.io>

fix: remove hardcoded port from hero_books TOML ... 3bf8866d01

hero_books already listens on both TCP (default 8883) and Unix socket
(~/hero/var/sockets/hero_books_server.sock) automatically. Remove the
explicit --port 8883 and ports = [8883] so the binary uses its defaults
and hero_proxy can route via the existing Unix socket.

Closes #35

Co-Authored-By: mik-tf <mik@threefold.io>

fix: disable kill_others in production mode to prevent restart cascades ... 45ea909e95

When no [build] section exists (production container with pre-built
binaries), kill_others is unnecessary since there are no stale processes.
The flag causes _http services to kill each other's ports on simultaneous
startup, exhausting zinit's retry budget.

Refs #33

Co-Authored-By: mik-tf <mik@threefold.io>

fix: add ONNX Runtime, fix zinit double-start, fix hero_books embedder URL ... d6c8ff3eb3

Production container fixes for 20/20 services:
- Download ONNX Runtime 1.23.2 for hero_embedder (uses load-dynamic dlopen)
- Remove zinit TOMLs from production profile (infrastructure, not user service)
- Fix hero_books embedder URL to use Unix socket instead of broken HTTP URL

Co-Authored-By: mik-tf <mik@threefold.io>

docs: add production container section to README ... a4e6ee54d5

Document the production container image (pull, run, tags), 20 service
list, startup notes, key ports, CI build process, and architecture.
Rename existing Docker section to "Development Container".

Co-Authored-By: mik-tf <mik@threefold.io>

feat: add WASM frontend builds to production container ... 021ec9ca35

Add hero_os_ui Dioxus shell and hero_archipelagos standalone island
builds to the production Docker image. This enables the Hero OS desktop
environment with all island apps (settings, books, calendar, contacts,
etc.) to be served by hero_os_http.

Changes:
- Add wasm32-unknown-unknown target, wasm-pack, and dioxus-cli to builder
- New docker/build-wasm.sh script (builds shell + 37 islands)
- Copy WASM assets to runtime stage
- Set HERO_OS_ASSETS/HERO_OS_ISLANDS env vars for hero_os_http
- Pin zinit to 9d21ba5 (workaround rust-version inheritance bug)
- Expose port 8804 (hero_os_http)

Tested locally: 22 services running, shell served at /, 31/37 islands
built successfully.

Closes #37

Co-Authored-By: mik-tf <mik@threefold.io>

Merge pull request 'Add WASM frontend (hero_os_ui + archipelagos islands) to production container' (#38) from development_wasm_frontend into development_production_container b8d7c7325d

fix: set HERO_CONTEXTS=default for osis and os openrpc services ... 11408b636b

The frontend (hero_os_ui) uses context "default" when calling RPC
methods, but hero_osis_openrpc defaults to context "root". This
causes a socket path mismatch — the frontend calls /rpc/default but
the socket only exists at .../root/hero_osis_openrpc.sock.

Set HERO_CONTEXTS=default in both service TOMLs to align with the
frontend expectation.

Closes #39

add TFGrid single-VM deployment with web gateway ... 2bfdba787d

OpenTofu config with grid_name_proxy for HTTPS via TFGrid web gateway,
Docker-based setup/update scripts, multi-environment Makefile (prod/dev).

Closes #41

use env_secrets pattern: mnemonic via TF_VAR, FORGEJO_TOKEN via SSH ... 505ee583f0

- Mnemonic set as TF_VAR_mnemonic in ~/hero/cfg/env/env.sh, not in tfvars
- credentials.auto.tfvars now infra config only (node_id, cpu, memory, etc.)
- FORGEJO_TOKEN passed from local env to VM via Makefile SSH
- app.env reduced to non-secret config (image, port)
- Use canonical FORGEJO_TOKEN name per env_secrets skill

auto-source env vars in Makefile, clean up example files ... 86533f6ee0

Makefile auto-sources TF_VAR_mnemonic and FORGEJO_TOKEN from
~/hero/cfg/env/env.sh — no manual sourcing needed, just make all ENV=prod.
Example files trimmed to only what they contain, no secret references.

Merge remote-tracking branch 'origin/development_fix_container_ci' into development_combined_deploy f8d6de8757

Merge remote-tracking branch 'origin/development_fix_indexer_repo_url' into development_combined_deploy 7a76f4d48b

merge development_production_container (resolve conflicts: take #34 Dockerfile/CI, #31 indexer fix subsumed) d1dedd9c69

Merge remote-tracking branch 'origin/development_hero_books_socket' into development_combined_deploy 283d4524d6

Merge remote-tracking branch 'origin/development_fix_osis_context' into development_combined_deploy 1f693127f0

Merge remote-tracking branch 'origin/development_deploy_tfgrid' into development_combined_deploy 8c40313977

fix: setup.sh Docker on zinit VMs, per-env tfvars examples, Makefile help path 489da7460f

fix: switch CI to Dockerfile.prod (all binaries pre-built, no runtime compilation) e4368da245

mik-tf added 1 commit 2026-02-27 14:52:09 +00:00

fix: manual dispatch tags only :version, tag push adds :latest decdf1fe56

mik-tf referenced this pull request 2026-02-27 14:52:32 +00:00

fix: container CI pipeline — DinD checkout, SSH, and Dockerfile fixes #26

mik-tf referenced this pull request 2026-02-27 14:52:33 +00:00

Fix hero_indexer repo URL in service TOML #31

mik-tf referenced this pull request 2026-02-27 14:52:35 +00:00

Fix container build — correct binary names, CI pipeline, entrypoint #32

mik-tf referenced this pull request 2026-02-27 14:52:36 +00:00

Production container with pre-built service binaries #34

mik-tf referenced this pull request 2026-02-27 14:52:38 +00:00

Switch hero_books TOML to Unix socket #36

mik-tf referenced this pull request 2026-02-27 14:52:39 +00:00

Fix auth — set HERO_CONTEXTS=default for osis/os openrpc #40

mik-tf referenced this pull request 2026-02-27 14:52:40 +00:00

WIP: TFGrid single-VM deployment with web gateway (HTTPS) #42

mik-tf added 1 commit 2026-02-27 15:13:24 +00:00

fix: clean up CI workflows, use :dev tag instead of :latest ... 0104cff1d8

- Delete build-macos.yaml (no macOS runner available)
- Delete build-prod-container.yaml (redundant with build-container.yaml)
- Disable tag trigger on build-linux.yaml (missing build_lib.sh functions)
- Change build-container.yaml image tag from :latest to :dev

mik-tf added 1 commit 2026-02-27 15:16:30 +00:00

fix: use hero_zero:dev image tag across all deploy scripts 47d11a06da

mik-tf added 1 commit 2026-02-27 15:30:19 +00:00

fix: install fuse-overlayfs in CI for Docker daemon startup ... f3c7caf20c

Runner lacks overlay2 privileges — explicitly install fuse-overlayfs
and configure Docker to use it as storage driver.

mik-tf added 1 commit 2026-02-27 16:25:27 +00:00

fix: hero_os_http bind 0.0.0.0 + hero_books build skip UI crate ... 642c0f8da6

- hero_os_http: add --host 0.0.0.0 so port 8804 is reachable via
  Docker port forwarding (was 127.0.0.1, caused 502 from gateway)
- hero_books: switch from build_repo to build_cargo with -p hero_books
  to skip hero_books_ui WASM/Dioxus crate that fails in Docker build

mik-tf added 1 commit 2026-02-27 18:26:41 +00:00

fix: create-release CI job + gitignore tfstate ... 326adad31a

- Add alpine container image to create-release job (was missing, causing git/curl not found)
- Remove unnecessary checkout step (release only needs env vars)
- Add terraform.tfstate to deploy gitignore
- Fix comment: :latest → :dev

Co-Authored-By: mik-tf <mik@threefold.io>

mik-tf added 1 commit 2026-02-27 18:55:35 +00:00

fix: create-release JSON payload (use jq for proper escaping) ... d44503aaa9

Multiline BODY variable broke the JSON string literal in curl -d.
Use jq to construct the payload with proper newline escaping.

Co-Authored-By: mik-tf <mik@threefold.io>

mik-tf changed title from WIP: Combined deploy — all PRs merged + Dockerfile.prod + TFGrid deployment to Combined deploy — all PRs merged + Dockerfile.prod + TFGrid deployment 2026-02-27 20:45:46 +00:00

mik-tf added 1 commit 2026-02-28 02:11:36 +00:00

fix: route gateway through hero_proxy (port 8805) instead of hero_os_http (8804) ... 840a935870

The TFGrid gateway was pointing to hero_os_http (port 8804), which is a
static WASM file server (GET only). WASM auth RPCs use relative URLs
resolved against the page origin, so POST requests hit the static server
and got HTTP 405. Changing to hero_proxy_http (port 8805), which routes
all paths to the correct backend services via Unix sockets.

Co-Authored-By: mik-tf <mik@threefold.io>

mik-tf added 1 commit 2026-02-28 02:29:03 +00:00

fix: hero_os_http socket path to match hero_proxy routing ... e27243253a

hero_proxy routes /hero_os/* to ~/hero/var/sockets/hero_os.sock but
hero_os_http was defaulting to hero_os_http.sock (binary name). Add
explicit --bind to the service TOML so it creates the socket the
proxy expects. Without this, /hero_os/ returns 502.

Co-Authored-By: mik-tf <mik@threefold.io>

mik-tf referenced this pull request from mycelium/www_migrate_mycelium 2026-02-28 03:29:20 +00:00

Container CI workflow fails: no Docker on ubuntu-latest runner #25

mik-tf referenced this pull request from mycelium/www_migrate_mycelium 2026-02-28 03:29:21 +00:00

Fix container CI: use Docker-in-Docker + buildx + auto-release #26

mik-tf added 1 commit 2026-02-28 03:29:29 +00:00

fix: increase Docker daemon startup timeout to 60s in CI ... e4634efee6

DinD (Docker-in-Docker) with fuse-overlayfs can take longer than 30s
to initialize on shared CI runners. 60s matches industry standard and
our own deploy/setup.sh timeout.

Co-Authored-By: mik-tf <mik@threefold.io>

mik-tf added 1 commit 2026-02-28 04:00:24 +00:00

fix: revert hero_os_http socket override — use default hero_os_http.sock ... bc4ad5f1a1

The Dioxus frontend uses base_path="hero_os_http", so all asset paths
are absolute at /hero_os_http/assets/.... The proxy routes /hero_os_http/*
to hero_os_http.sock, which only works when the binary uses its default
socket name. The previous --bind override to hero_os.sock broke asset
loading (white screen).

Correct access URL: /hero_os_http/ (not /hero_os/)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-02-28 04:37:49 +00:00

fix: always configure Docker fuse-overlayfs on TFGrid VMs ... 133192be86

TFGrid VMs often have Docker pre-installed but configured with the
default overlayfs driver, which fails on virtiofs. The setup script
now checks the current storage driver and reconfigures to
fuse-overlayfs + /data when needed, regardless of whether Docker
was freshly installed or already present.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-02-28 12:51:21 +00:00

fix: add "root" context to hero_osis_openrpc for auth support ... 9366e88d8b

The Hero OS UI sends auth RPC calls (authservice.get_challenge, etc.)
to the "root" context via hero_osis. With only "default" configured,
hero_osis_openrpc never creates sockets/root/hero_osis_openrpc.sock,
causing 502 "Cannot connect to backend socket" on login.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-02-28 19:43:56 +00:00

fix: add admin seeding and per-repo branch overrides for Docker builds ... abb73618f2

- Add HERO_OSIS_SEED_DIR to hero_osis_openrpc service TOML for automatic
  admin user provisioning on first boot (idempotent)
- Add per-repo branch overrides in build-services.sh (HERO_OS_BRANCH,
  HERO_OSIS_BRANCH env vars) so Docker builds can use fix branches
  without waiting for PRs to merge
- Add seed data COPY step in Dockerfile.prod (copies from hero_osis
  data/seed/ directory into /root/hero/var/seed/)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-01 03:39:03 +00:00

fix: pin zinit to development_fix_herolib_core_dep branch ... d3c51ca4f4

Zinit's development branch is broken — commit 11a6bd4 added
herolib_core as a workspace dep in zinit_sdk but never defined it
in the root Cargo.toml. This breaks cargo resolution for all
downstream consumers.

Pin to the fix branch until zinit#25 is merged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-01 03:43:48 +00:00

fix: update zinit binary names after upstream restructure ... 28884afddd

zinit codebase was restructured: zinit_openrpc → zinit_server,
zinit_http → zinit_ui, --web-port → --http. Update Dockerfiles
and entrypoint to match.

mik-tf added 1 commit 2026-03-01 04:13:21 +00:00

fix: rename hero_redis service TOMLs to match upstream binary names ... 46828651ea

hero_redis renamed its binaries: hero_redis_openrpc → hero_redis_server,
hero_redis_http → hero_redis_ui. Update service TOMLs, dependency
references, and build script accordingly.

mik-tf added 1 commit 2026-03-01 16:28:43 +00:00

fix: add HERO_BOOKS_BRANCH build arg for per-repo branch override 32eb597913

mik-tf added 1 commit 2026-03-01 16:49:03 +00:00

fix: update service TOMLs and build script for upstream binary renames ... 703c16f7d2

hero_voice, hero_indexer repos renamed their binaries from *_openrpc/*_http
to *_server/*_ui convention. Update TOMLs and build-services.sh to match.
Also switch hero_inspector to build_cargo to bypass broken Makefile.

Affected:
- hero_voice_openrpc → hero_voice_server
- hero_voice_http → hero_voice_ui
- hero_indexer_openrpc → hero_indexer_server
- hero_indexer_http removed (merged into server)
- hero_inspector: build_repo → build_cargo

mik-tf added 1 commit 2026-03-01 17:04:08 +00:00

fix: update depends_on references for hero_indexer rename ... be75bce54d

hero_osis_openrpc and hero_books had depends_on referencing the old
hero_indexer_openrpc service name. Updated to hero_indexer_server.

mik-tf added 1 commit 2026-03-01 20:26:58 +00:00

fix: update hero_inspector TOMLs, add proxy default service, add HERO_PROXY_BRANCH ... a483c013c8

- Rename hero_inspector TOMLs to match upstream binary names (_server/_ui)
- Revert hero_inspector to build_repo (development already has Makefile fix)
- Add HERO_PROXY_DEFAULT_SERVICE=hero_os_http to proxy TOML for SPA routing
- Add HERO_PROXY_BRANCH build arg to Dockerfile.prod

mik-tf added 1 commit 2026-03-01 21:15:19 +00:00

fix: hero_inspector build — use build_cargo + symlink zinit for path deps ... 88b29c9894

hero_inspector's Makefile has build_lib.sh dependencies that fail in
Docker. Switch to build_cargo and symlink /build/zinit into SRC_DIR
so zinit_sdk relative path deps resolve correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 02:50:48 +00:00

fix: health checks use unix socket probing + branch overrides for all fixes ... fe082030b5

Health check generation now supports a `socket` field in service TOMLs.
When present, uses `curl --unix-socket` instead of TCP port probing.
Fixes 4 failing health checks (voice_ui, inspector_ui, redis_ui,
embedder_http) which bind Unix sockets but had TCP-only health probes.

Branch overrides updated:
- hero_os: development_ui_polish (system menu, maximize, tests)
- hero_osis: development_fix_missing_domains (communication, media, job)
- hero_proxy: development_default_service (SPA fallback routing)
- hero_inspector: development_fix_makefile_target (Docker build fix)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 03:15:22 +00:00

fix: correct hero_inspector binary and package names ... 80f2f84ad8

The hero_inspector crates produce binaries named hero_inspector_openrpc
and hero_inspector_http, not hero_inspector_server and hero_inspector_ui.
Fix service TOMLs and build script to use the actual binary names.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 03:33:40 +00:00

fix: correct hero_embedder_http socket path to match binary name ... 6a1ef6d86c

The hero_embedder_ui binary creates hero_embedder_ui.sock, not
hero_embedder_http.sock. Fix health check socket path to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 04:31:14 +00:00

fix: add fallback binary search in build scripts ... 8f88a1f0c5

Add fallback to check repo-local target/release/ when binary is not
found at CARGO_TARGET_DIR/release/. Handles edge cases where Docker
cache mounts cause cargo to use an alternate target directory.
Also switch hero_books from build_cargo to build_repo for Makefile install.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 05:12:19 +00:00

fix: update hero_books binary name to hero_books_server ... ccbca54856

The hero_books repo renamed its main binary from hero_books to
hero_books_server. Update build script and service TOML to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 12:55:53 +00:00

chore: update HERO_OS_BRANCH to development_consolidated ... 37d8dc1f9e

Point to the new consolidated branch (PR #19) which merges
UI polish, SPA routing, and projects island PRs (#11, #15, #18).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 14:05:24 +00:00

fix: set hero_proxy_http to listen on port 6666 (public-facing) ... c46cc86d5e

In Docker container, hero_proxy_http is the public-facing reverse proxy
and should listen on port 6666 (mapped externally). The hero_ports registry
assigns 8805 for single-machine use, but in container mode 6666 is correct.

Add HERO_PROXY_PORT=6666 env var so the binary uses the right port.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 15:51:44 +00:00

feat: add HERO_ARCHIPELAGOS_BRANCH support in WASM build ... 4c72bcf93d

- Dockerfile.prod: add ARG HERO_ARCHIPELAGOS_BRANCH=development_consolidated
  and pass to build-wasm.sh
- build-wasm.sh: use HERO_ARCHIPELAGOS_BRANCH (falls back to BRANCH)
  when cloning hero_archipelagos for standalone island builds

This ensures standalone book/other island WASMs are built from
development_consolidated which has the books iframe→views fix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf referenced this pull request 2026-03-02 17:25:09 +00:00

Production Hardening: Auth Security, UX, Tests, Docs #44

mik-tf added 1 commit 2026-03-02 17:39:37 +00:00

fix: map host APP_PORT to container PROXY_PORT (6666) in setup.sh ... 6c4ddea1b4

The hero_proxy_http service listens on container port 6666 (HERO_PROXY_PORT),
not 8805. The Docker run script was mapping 8805:8805 which left nothing
on the mapped port, causing 502s at the gateway.

Introduces PROXY_PORT variable (default 6666) so the mapping becomes
APP_PORT:PROXY_PORT (e.g. 8805:6666).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 18:51:19 +00:00

test: add gateway smoke tests for routing verification ... ec26c0cf50

Add tests/smoke_gateway.sh with 7 checks covering SPA routing (via
HERO_PROXY_DEFAULT_SERVICE), prefix routing, JSON-RPC proxy, and direct
osis prefix routing. Update deploy/single-vm/Makefile test target to
invoke the script with the deployed gateway URL.

Root path GET / correctly expects 200 (hero_proxy returns JSON service
listing), while /login and /hero_os_http/ assert text/html.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-02 20:38:11 +00:00

feat: update branch ARGs for Books WASM island migration ... 2afc5af2a8

- HERO_BOOKS_BRANCH: development → development_wasm_access
  (adds hero_books_http.sock + /admin/rpc for proxy routing)
- HERO_ARCHIPELAGOS_BRANCH: development_consolidated → development_books_wasm_views
  (replaces iframe with native Dioxus views, adds [package.metadata.island])

Previous HERO_ARCHIPELAGOS_BRANCH=development_consolidated did not exist
on remote, causing build-wasm.sh to silently skip all island builds.

mik-tf added 1 commit 2026-03-02 22:59:27 +00:00

fix: use socat bridge for hero_books routing + revert broken branch ... 0dbe1d3edb

- Revert HERO_BOOKS_BRANCH from development_wasm_access to development
  (development_wasm_access has broken lib.rs missing module files)
- Add socat to runtime image
- Add hero_books_socket_bridge zinit service: bridges hero_books.sock
  → hero_books_server.sock so hero_proxy /hero_books/ routes correctly
  (depends_on hero_books so it starts after the server is healthy)

mik-tf added 1 commit 2026-03-02 23:35:52 +00:00

chore: add CACHE_BUST ARG to force rebuild with updated git deps ... bf19b8d8cf

development_consolidated now has nav bar — bump CACHE_BUST=2 to
invalidate the Docker registry cache so hero_os_ui picks up the
new hero_archipelagos_books commit.

mik-tf added 1 commit 2026-03-03 00:43:03 +00:00

fix: add unlink-early to socat socket bridge to survive container restarts ... 2d18e59977

The hero_books_socket_bridge socat service failed on container restart
because the hero_books.sock file persisted in the data volume from the
previous run. Adding `unlink-early` causes socat to remove the stale
socket file before binding, preventing the EADDRINUSE failure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-03 04:13:00 +00:00

feat: point HERO_BOOKS_BRANCH to development_wasm_access for http socket + admin/rpc 117cbcdb1a

mik-tf added 1 commit 2026-03-03 05:08:26 +00:00

fix: revert HERO_BOOKS_BRANCH to development (wasm_access fails Rust 1.93 type inference) d0bf1c07a9

mik-tf added 1 commit 2026-03-03 13:09:39 +00:00

fix: pin builder to rust:1.92-bookworm per hero_ecosystem standard (Rust 1.93 breaks hero_books type inference) 551c16a1e1

mik-tf added 2 commits 2026-03-03 18:56:58 +00:00

feat: add HERO_ARCHIPELAGOS_BRANCH override, point to development_consolidated ... 067e12a934

- Dockerfile.prod: add HERO_ARCHIPELAGOS_BRANCH ARG (default: development_consolidated)
  and pass it to build-wasm.sh
- build-wasm.sh: use HERO_ARCHIPELAGOS_BRANCH env var (fallback to BRANCH) when
  cloning hero_archipelagos

development_consolidated consolidates all archipelagos improvements (CI, MDX,
PDF, islands) including the books iframe → Dioxus WASM view router migration
previously on development_books_wasm_views (PR#37, now closed and merged in).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Merge: set HERO_ARCHIPELAGOS_BRANCH=development_consolidated ... 3161ed64e8

Keeps all branch overrides from remote (HERO_PROXY_BRANCH, HERO_INSPECTOR_BRANCH)
and switches HERO_ARCHIPELAGOS_BRANCH from development_books_wasm_views to
development_consolidated — the single consolidated PR that now includes
the Books WASM view router migration (PR#37 merged in and closed).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-03 21:13:35 +00:00

fix: restore HERO_BOOKS_BRANCH=development_wasm_access + add Chromium ... 05a02c533b

- HERO_BOOKS_BRANCH=development_wasm_access: restores hero_books_http.sock
  creation + /admin/rpc endpoint. Previously reverted due to Rust 1.93 type
  inference issues, safe to re-enable now that builder is pinned to 1.92.
- Add chromium + dependencies to runtime stage for PDF generation
  (hero_books pdfservice requires headless Chrome via headless_chrome crate)
- Bump CACHE_BUST=3 to force rebuild with new branch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 02:07:29 +00:00

fix: set HERO_ARCHIPELAGOS_BRANCH=development_books_wasm_views, bump CACHE_BUST=4 ... 1bb60f47f2

development_consolidated didn't exist on hero_archipelagos remote.
Correct branch for Books WASM island migration is development_books_wasm_views.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 04:18:07 +00:00

fix: pre-install binaryen so dx build finds wasm-opt without downloading ... a8813096e5

dx build --release tries to download wasm-opt from GitHub at runtime,
which fails in the Docker build environment. Installing binaryen via apt
puts wasm-opt in PATH so dx uses it directly.

Also bumps CACHE_BUST=5 to force re-clone.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 04:26:14 +00:00

feat: add reliable/reliable-clean Makefile targets for prod image builds ... 986dd6834e

Two modes:
  make reliable TAG=reliable26       — with layer cache (normal, fast)
  make reliable-clean TAG=reliable26 — --no-cache (when Dockerfile changed)

Layer cache (~20-25 min saved) via BuildKit mount caches for cargo registry,
git deps, and target/. Use --no-cache only when apt deps or tool versions change.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 04:59:48 +00:00

chore: bump CACHE_BUST=6 to pick up hero_os default window fix ... e8b2422454

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 2 commits 2026-03-04 05:39:35 +00:00

fix: bump CACHE_BUST to pick up hero_proxy redirect fix ... cf7f996365

hero_proxy development_default_service now redirects /login and other
unprefixed paths to /hero_os_http/{path} instead of proxying transparently.
This fixes the /hero_os_http/hero_os_http double-prefix bug in the browser.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: resolve merge, bump CACHE_BUST to 7 for hero_proxy redirect fix ... 20d8315eef

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 06:05:21 +00:00

fix: bump CACHE_BUST=8 for loading overlay fix ... 8abdd41056

hero_os development_consolidated: add use_effect to dismiss #loading
overlay once WASM app mounts (was permanently blocking rendered app).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mik-tf referenced this pull request from lhumina_code/home 2026-03-04 17:54:03 +00:00

Demo readiness — services, inspector, deployment #3

mik-tf added 1 commit 2026-03-04 18:43:59 +00:00

chore: bump builder image to rust:1.93-bookworm ... 802e7d58f5

Per hero_ecosystem standard — Rust 1.93 is now the minimum.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 2 commits 2026-03-04 19:09:54 +00:00

fix: correct package names for hero_inspector and hero_fossil ... 0ab7e74b91

- hero_inspector: packages are hero_inspector_server/hero_inspector_ui
  (not hero_inspector_openrpc/hero_inspector_http)
- hero_fossil: use build_cargo to bypass Makefile build_lib.sh issues
  in container; binaries renamed to hero_foundry_*

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chore: bump CACHE_BUST=9 for hero_archipelagos webdav rename fix ... 0bdd57ab14

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 19:24:15 +00:00

chore: bump CACHE_BUST=10 for hero_os fix (archipelagos development_consolidated) ... df6a320196

Cherry-picked the herofossil_webdav_client rename into the
development_consolidated branch of hero_archipelagos, which
hero_os_ui references. hero_os workspace resolution now succeeds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 20:00:46 +00:00

fix: update service TOML exec paths for renamed binaries ... 10b124bca3

hero_osis: openrpc→server, http→ui
hero_inspector: openrpc→server, http→ui
hero_fossil: hero_fossil→hero_foundry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 20:13:48 +00:00

fix: use hero_foundry_server (not hero_foundry CLI) in fossil service TOML ... 3038091551

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 21:18:13 +00:00

feat: add hero_biz to container build pipeline ... 687ad73ee9

- Add build_repo entry in build-services.sh
- Create hero_biz.toml service definition (port 8881)
- Expose port 8881 in Dockerfile.prod
- CACHE_BUST=11

Note: hero_biz starts but cannot reach hero_osis backend
(no HTTP API bridge exists). See hero_biz#6.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 21:27:09 +00:00

fix: use inspector branch with HTTP fallback and MCP fixes ... 17247831f6

Switch HERO_INSPECTOR_BRANCH to development_fix_openrpc_http_fallback
which includes HTTP fallback for _server.sock probing and optional
ConnectInfo for MCP handler behind Unix socket proxy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 21:51:12 +00:00

fix: use build_cargo for hero_biz to bypass Makefile in Docker ... dacba097bd

hero_biz Makefile depends on build_lib.sh which isn't available in
the Docker builder. Switch to build_cargo for direct cargo build.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-04 23:57:56 +00:00

refactor: rename service TOMLs to _server/_ui naming convention ... d50308d91b

Rename all service TOML files and internal names from legacy
_openrpc/_http suffixes to _server/_ui per Hero ecosystem skills.
Update all depends_on, run_after references, socket paths, and
default service names. Bump CACHE_BUST to 14.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 00:00:22 +00:00

fix: add embedder/osis basepath branches, bump CACHE_BUST to 15 ... 4be1715259

Pick up development_fix_basepath_regex branches for hero_embedder
and hero_osis which fix the BASE_PATH JS regex (_http -> _ui).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 00:39:50 +00:00

fix: add fossil branch, use merged inspector, bump CACHE_BUST to 16 ... 788d79d378

- HERO_INSPECTOR_BRANCH back to development (PR #4 merged)
- HERO_FOSSIL_BRANCH=development_fix_openrpc_json (adds /openrpc.json)
- CACHE_BUST=16

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 01:23:47 +00:00

fix: use inspector PR branch with 405 fix, bump CACHE_BUST to 18 ... e3df89b44e

HERO_INSPECTOR_BRANCH back to development_fix_openrpc_http_fallback
(PR #5) which includes the HTTP 405 fallback fix needed for
hero_os_http /openrpc.json discovery.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 02:02:34 +00:00

Add comprehensive smoke tests + fix RPC proxy routing ... 58025f9124

- smoke_container.sh: 57 tests covering zinit status, sockets, health,
  RPC proxy, OpenRPC spec, auth flow, and inspector discovery
- smoke_gateway.sh: updated for _ui naming, added auth/inspector tests
- Makefile: added smoke-container and smoke-gateway targets
- Dockerfile.prod: CACHE_BUST=20 for hero_os RPC proxy fix

Auth dispatch via osis is a known skip (spec aggregated but not dispatched).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 02:06:27 +00:00

Fix smoke tests: remove raw JSON-RPC health checks, fix MCP/auth/inspector ... 3c62a443d2

- Gateway: remove raw JSON-RPC sockets from HTTP health checks (502 expected)
- Gateway: fix MCP test (POST, not GET)
- Gateway: fix inspector API (use JSON-RPC services.list, skip if scanning)
- Both: mark auth dispatch as known skip (osis aggregates spec but doesn't dispatch)
- Container: fix inspector test to use HTTP over UDS (not raw socat)
- Container: remove hero_embedder_server.sock (doesn't create one)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 02:24:28 +00:00

chore: bump CACHE_BUST=21 for hero_rpc 2-part dispatch fix ... c7b0710f77

Rebuilds hero_osis_server with updated hero_rpc dependency that supports
2-part method names (Type.method) in Unix socket dispatch, fixing all
RPC calls from the WASM UI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 02:25:11 +00:00

fix: auth smoke tests now expect pass (not skip) ... c07c1cc643

With the hero_rpc 2-part dispatch fix, auth methods should work.
Changed auth test fallback from skip to fail so broken auth is
caught as a regression.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 02:46:37 +00:00

chore: bump CACHE_BUST=22 for two-phase dispatch fix ... e810a9068f

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 03:02:41 +00:00

chore: bump CACHE_BUST=23 for error distinction fix ... 6e2a58faff

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 03:03:16 +00:00

fix: update auth smoke tests to use device_sid parameter ... fb23726314

The AuthService.login method now requires device_sid instead of metadata.
Updated both smoke tests to pass device_sid: "smoke-test".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 03:38:07 +00:00

Fix hero_fossil socket name for hero_proxy routing ... 44eab665cb

Rename hero_fossil_server.sock to hero_fossil.sock so hero_proxy
can discover it via path prefix matching. hero_proxy searches for
exact, _http, and _ui suffixes but never _server sockets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 03:38:30 +00:00

Bump CACHE_BUST=24 for Settings dock fix and Files 404 fix ... f056ed154a

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf added 1 commit 2026-03-05 03:54:10 +00:00

Fix smoke tests: hero_fossil socket renamed from hero_fossil_server ... 2d934e0175

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mik-tf closed this pull request 2026-03-05 13:34:43 +00:00

mik-tf referenced this pull request 2026-03-05 13:34:55 +00:00

Combined deploy — unified TOMLs, Docker prod build, TFGrid deployment #46

All checks were successful

Hide all checks

Build and Test / build (pull_request) Successful in 5m16s

Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

lhumina_code/hero_services!43

No description provided.