make development compatible with main and other way around #1
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
so this will allow us to bring main back into development
and be compatible so we can easier make progress and move project per project over
checks
A few notes from the meeting to add on top of this, not replacing anything above.
On the old vs new question: this repo holds the previous, stable macros, the versions main already works with, not the latest fast moving ones. The name previous means the older known good set. The newer macros stay in hero_lib and hero_blueprint and keep evolving. Main pins this repo so it stops breaking when those change, and each repo migrates onto the new library one by one later. When the issue says use these new macros, new means the macros now living in this new repo, not a newer version.
Signed-by: mik-tf mik-tf@noreply.invalid
Summary of the task
The problem
Today the derive macros and their supporting code, including the OpenRPC clients, live in hero_lib and hero_blueprint. Those repos move fast and get big updates, and hero_blueprint in particular is too new for main. If we bring the current hero_lib and hero_blueprint into main, main stops building. So main and development have drifted apart and work gets stuck on one side.
The plan
The payoff
main builds and stays live again, which is what the sandbox needs to deploy from main. The two-way flow between main and development is restored, so newer work can reach the sandbox once each piece has moved over.
Checks
Signed-by: mik-tf mik-tf@noreply.invalid
Status report —
hero_macros_previousfreeze rolloutWhat this repo now provides
A frozen, self-contained snapshot of the macro/codegen toolchain so consumer repos stop tracking the still-moving
hero_lib/hero_blueprint. Two branches, mirroring hero_lib:main— oldhero_rpcstack re-cut from the pre-bridgehero_blueprint@d31c7b6(client macros emit::hero_rpc2::, herolib-free) + a frozen new-gen derivehero_derive(fromhero_lib@9657556a).development— same old stack +hero_derivecut fromhero_lib@d1383f58(for consumers that pinnedherolib_derive@development).Both branches build green and carry zero
git=hero_lib/hero_blueprintdependencies.Migration recipes (mechanical, no source changes)
herolib_derive = { package = "hero_derive", git = .../hero_macros_previous.git, branch = <same branch the repo pinned> }. All otherherolib_*left live — the frozen macro emits against the consumer's live runtime, so hero_lib keeps moving. Thepackage =rename avoids colliding with theherolib_derivethat liveherolib_corepulls transitively.hero_rpc_derive/hero_rpc_openrpc→hero_macros_previous, and addhero_rpc2(the pre-bridge client macro emits::hero_rpc2::, so it must be declared). Fully self-contained afterwards.Rollout result (branch
development_freezepushed per repo; only that branch pushed)✅ DONE — frozen + builds green,
development_freezepushed (8):hero_shrimp,hero_redis(=hero_db),hero_planner,hero_code,hero_codescalers,hero_gpu,hero_indexer,hero_slides⚠️ PARTIAL — integration freeze builds, but the repo's
developmentbranch is broken independent of this change (2):hero_browser,hero_proc(e.g. proc dev referencesherolib_core::logger::LogDeleteScope, missing at its pinned hero_lib). Frozen on integration locally; dev held until their dev baseline is fixed.⚪ NOOP — no
herolib_derive/hero_rpcgit dep, nothing to do (~19):whiteboard, skills, archipelagos, biz, code_indexer, forge, foundry, kimi, kimi_rust, ledger, mail, researcher, router, webbuilder, website_framework, + non-Rust repos (builder, compute_config, compute_registry, deployment_docker).
⏸ DEFERRED — Blocker B (11): these don't build at baseline on integration, because they transitively pull a broken
hero_rpc_derivefromhero_blueprint/hero_rpc(mid-teardown) via other hero SDKs, or their own source uses old transport types:agent,books,office,orchestrator,voice,proxy(transitive dual-source),editor(own source useshero_rpc_openrpc::OpenRpcError),lib_rhai(complex mixed),wallet,voice_provider,matrixchat(baseline-broken via transitive blueprint dep).⏸ TIER-3 — schema/server-macro adopters, deferred for team judgement (7):
compute,drive,logic,mycelium,os,osis,livekit(committed generated#[derive(OsisObject)]/ inline server code — can't go self-contained; need the frozenhero_derivesuperset + live runtime).Checklist status
hero_derivealias removes the direct moving-derive dependency on those. Remaining repos are Blocker-B (need their broken transitivehero_rpcSDK deps fixed first) or Tier-3.macros_previousrepo — in progress (8 pushed ondevelopment_freeze).Key finding / next step
The dominant blocker is not the macro swap itself — it's that many integration repos don't build at baseline because they transitively depend on broken
hero_blueprint/hero_rpccrates left mid-teardown. The freeze succeeds cleanly wherever the repo builds. Recommended next: fix that root cause (extend the freeze/[patch]to the leaf SDKs likehero_*_sdkthat still pullhero_rpc_derivefrom blueprint), which should unblock most of the deferred set, then take Tier-3 with the team.Nothing else pushed; all consumer changes are local on
development_freezebranches pending review.Status report — OSIS/macros freeze rollout (session 2026-06-08/09)
Building on the earlier freeze work. This session extended the freeze from the client/derive layer into the OSIS server stack, rolled out
development_freezebranches across most repos, and nailed down why the hard repos are hard. Nothing pushed except three new cluster branches on this repo; all consumerdevelopment_freezebranches are local commits pending review.New artifacts on
hero_macros_previous(pushed)Three vendored, blueprint-free OSIS clusters, one per serving era (each: vendored osis crates from blueprint, herolib runtime pinned by
revfor churn-isolation, lifecycle/indexer pinned):osis_mainefb3a609herolib_derive→ frozenhero_derivealiasosis_oldservingf6509f9chero_rpc_derivevendored directlyosis_oserverd31c7b6/df2f8b1bfrozen_osis_sid/otoml/base)Key findings
Recoverability is real, not universal. For the 7 OSIS repos, the snapshot each pins in its committed
mainCargo.lockbuilds green--lockedfor 3 of 7 (logic, compute, livekit) — so freezing preserves a known-good state there. The other 4 were committed in a non-building state (e.g.hero_osiscallshero_lifecycle::serve_unix, a function that never existed in any rev).The OSIS-facing herolib surface is churning hard —
core19,derive22,oschema16,openrpc7 commits in 3 weeks, with breaking renames (herolib_derive→herolib_macros,hero_socket_dir→path_sockets_dir, service API moved tohero_lifecycle::base). So "float herolib" re-exposes the churn; the freeze pins herolib by rev instead.The dual-source clash = mixed old/new stack in one dependency graph. e.g.
expected herolib_openrpc::RpcError, found hero_rpc2::error::RpcError. Fix = consolidate each repo's entire graph onto one stack (the side its own source already leans). This is the root cause of the remaining "hard" repos.Repos span multiple serving eras (new-serving vs old-serving, and sub-eras), so a single cluster can't serve all — hence three clusters. Repos converge onto the cluster matching their era.
development_freezerollout status✅ Green + blueprint/derive-independent, committed locally (~19):
hero_logic(→osis_main),hero_livekit(→osis_oldserving),hero_office(moved to frozen new stack)herolib_derive→hero_derive): aibroker, browser, code, codescalers, gpu, indexer, planner, proc, redis, slides, voice_provider, wallethero_rpc2): skills, shrimp, collab, orchestrator⬜ No-op branches created (22): archipelagos, biz, builder, code_indexer, compute_config, compute_registry, coordinator, deployment_docker, forge, foundry, foundry_ui, kimi, kimi_rust, ledger, lib_rhai, mail, researcher, router, services, tfspores, webbuilder, website_framework (no blueprint/derive deps — independent by construction).
Remaining work (with concrete blockers)
hero_compute— coherent era too far from any cluster (osis@7587189e + herolib@b814ec07 + old proc_sdk); cascades through proc_sdk + a dual-source OsisObject. Needs its own 7587189e cluster or a forward-port.hero_osis— committed-broken (serve_unixnever existed). Needs a code fix.hero_osdepends onhero_osis_sdk→ blocked on osis.mycelium,drive— depend onhero_rpc_oserver, a crate deleted 2026-03-11. Port-or-retire decision.hero_proc(hero_proc_sdkusesherolib_openrpc). Since hero_proc is genuinely new-stack, these must either move to new too, or hero_proc needs an old-compatible SDK. One decision gates four repos.hero_service::HeroUiServer(old UI-server, replaced by a builder inhero_admin_libper #142) → small UI-server port.hero_rpc_derive = "^0.5.0"version pins.Checklist progress
Recommended next step: decide the
hero_procSDK question (old-compat shim vs move 4 consumers to new) — highest leverage — then the proxy/matrixchat new-stack ports, then the per-repo OSIS fixes (compute cluster, osis code-fix).hero_macros_previous freeze — session summary (2026-06-08/09)
Extends the freeze from the client/derive layer into the OSIS server stack and rolls out
development_freezebranches across the org. Tracking issue:lhumina_code/hero_macros_previous#1.1. Pushed artifacts
Frozen clusters on
hero_macros_previous(one per serving era)Each is blueprint-free, vendors the osis crates, and pins the herolib runtime by
rev(churn-isolation):
osis_mainefb3a609herolib_derive→ frozenhero_derivealiasosis_oldservingf6509f9chero_rpc_derivevendored directlyosis_oserverdf2f8b1bfrozen_osis_sid/otoml/base)Consumer
development_freezebranches pushed (37)Green + fully independent (blueprint=0, no transitive hero_rpc.git) — the freeze work (18):
hero_logic,hero_livekit,hero_office,hero_proxy(OSIS / new-stack moves)hero_aibroker,hero_browser,hero_code,hero_codescalers,hero_gpu,hero_indexer,hero_planner,hero_proc,hero_redis,hero_slides,hero_voice_provider,hero_wallet(new-stack
herolib_derive→hero_derivealias)hero_skills,hero_shrimp,hero_collab,hero_orchestrator(old-stack repoint → macros_previous)No-op (= main, independent by construction; no blueprint/derive deps) (19):
hero_archipelagos,hero_builder,hero_code_indexer,hero_compute_config,hero_compute_registry,hero_deployment_docker,hero_forge,hero_foundry,hero_foundry_ui,hero_kimi,hero_kimi_rust,hero_lib_rhai,hero_mail,hero_researcher,hero_router,hero_tfspores,hero_webbuilder,hero_website_framework,hero_whiteboard2. The core model — "freeze codegen, float proc" (validated)
The dual-source clash (
expected herolib_openrpc::RpcError, found hero_rpc2::error::RpcError)is OLD vs NEW transport meeting in one dependency graph, not "macros vs proc."
hero_derive(onmacros_previous@main) is a frozen macro thatemits against the live
herolib_openrpctransport. So codegen is pinned while transport,hero_lifecycle, andhero_prockeep floating.deps (provider SDKs) must be on the same side, or they clash.
Recipes (saved at
/tmp/alias.py,/tmp/oldstack2.py,/tmp/macroswap.py):herolib_derive = { package = "hero_derive", git = macros_previous, branch = main }hero_rpc.git/hero_blueprint.git→macros_previous@main(+hero_rpc2w/openrpc-transportfeature where the macro emits::hero_rpc2::)hero_rpc_derive::openrpc_client!→herolib_derive::openrpc_client!,hero_rpc_openrpc::/hero_rpc2::→herolib_openrpc::, drop old deps, pin herolibrev=9657556a,drop hero_rpc patch sections, repoint lifecycle
hero_rpc.git→hero_lib.3. Held back — green but residual transitive blueprint (NOT pushed)
These build green but their lock still pulls
hero_rpc.git(= blueprint via 301 redirect),via an old pinned provider SDK. Need the provider frozen too before they're clean:
hero_computehero_proc_sdk@41d42a14→ hero_rpc_openrpc/derive (era 7587189e). (owner iterating)hero_orchestratorhero_voice_providerhero_bizhero_coordinatorhero_ledger4. Remaining hard work (with concrete blockers)
Follow-proc builder-API port (voice, embedder, matrixchat, agent)
Current
hero_proc_sdkmoved from a struct-literal API (KillOther {…},HealthCheck {…},ActionSpec) to a builder API (ActionBuilder.kill_other(…).health_check(…)inbuilders/factory). These types are gone, not moved — so it's a real per-repo refactor,not a path remap or sed. This is the genuine "repos follow floating proc" cost. (The stale
*.generated.rsdisk-dump still shows the old structs — misleading.)agentis additionally blocked on the brokenhero_osis.High-leverage single fix
hero_osis— committed-broken:hero_osis_admincallshero_lifecycle::serve_unix, afunction that never existed in any lifecycle rev. Fixing it unblocks
hero_os(depends onhero_osis_sdk) and cleans the service layer that agent/archipelagos call.NB: "use OSIS" mostly means the framework (
hero_rpc_osiscodegen, ~8 builder repos), notthe hero_osis service (few direct callers, concentrated through
hero_archipelagos).Team decisions (not coding)
hero_books— adopting the new derive forces renaming its entire dotted-method wire API(
books.export→books_export, 20+ methods). Breaking change → maintainer call.hero_mycelium,hero_drive— depend onhero_rpc_oserver, deleted 2026-03-11 →port-or-retire.
hero_compute— coherent era (osis@7587189e + herolib@b814ec07 + old proc_sdk) too far fromany cluster; needs own 7587189e cluster or forward-port. (owner iterating)
5. Tally
Checklist progress (issue #1):
Status report — 2026-06-10
TL;DR: the freeze is done for every active repo. All 60+ active repos now have a
development_freezebranch on the forge that builds green, has zerohero_rpc.git/hero_blueprintreferences in its Cargo.lock, and contains the latest work frommainandintegration. These branches are the candidate frozendevelopmentstate — flippingdevelopmentto them is now a conflict-free, same-tree operation.The issue's checks
development_freezebranch (the new development baseline).main/integrationwere not rewritten; they converge as these branches are promoted.hero_lib@9657556afor the runtime). Repos with no such deps were verified independent as-is.What was done
main,osis_main,osis_oldserving,osis_oserver) and pristine mirrors (frozen_<rev>, byte-copies consumed via[patch]— these must never receive commits). The${PATH_SOCKET}socket-placeholder fix was cherry-picked fromdevelopmenttomain;developmenthere is unused (zero consumers) and should not be based on.cargo update-style lock refreshes; all merged repos rebuilt green.hero_rpc_osis; note its committed lock was never vanilla-buildable upstream — CI relied on local-path patches), plus cross-org stragglers my_compute_holo and znzfreezone_backend.development_freezeon the forge now carries the development history with the frozen tree (the development merge is already baked in). Pre-merge freeze tips are kept locally asdevelopment_freeze_backup.Exceptions (not blocking)
near-apidrift is ported (~10 call sites; needs NEAR context)zinit_archive2); green freeze kept locallyDecisions / next steps for the team
developmentto thedevelopment_freezestate per repo (fast-forward/same-tree; zero conflict risk). After that, normal work continues on integration→main as agreed, anddevelopmentis frozen against the hero_lib/hero_blueprint churn.lab builddefaults to policy-apply, which rewrites third-party dep versions (thiserror 1→2, axum 0.7→0.8, …) and runs targetedcargo update— fine for live repos, hazardous for frozen ones. Use--policy-mode warnon freeze branches, or add[package.metadata.hero_builder.dep_overrides]to frozen repos, before frozen development becomes the default build target.Cargo.lock; no wholesalecargo update/generate-lockfileon frozen repos; repos consumed as git deps must pin churning deps byrev=, notbranch=(cargo ignores a dependency's lockfile).