herocoordinator/specs/architecture.md

per user

runs in container or VM, one per user

• zinit
• herocoordinator
  • think about like DAG worklflow manager
  • manage jobs who are send around to different nodes
• mycelium address range (part of mycelium on host)
• herodb
  • state manager
  • redis protocol / primitives
  • fs backend (mem and allways append in future)
  • encryption & decryption primitives
  • key mgmt for encryption (creation, deletion)
  • openrpc admin features: user management, role-based access control
• postgresql + postgrest
• AI Agent TBD

%%{init: {"theme":"dark"}}%%
graph TD
    subgraph Per Node System
        N[Node] --> OS(Run on top of ZOS4 or Ubuntu or in a VM)

        subgraph On Node
            OS --> SV(Supervisors)
            OS --> ZN(Zinit)
            OS --> R(Runners)
            OS --> PGN(Some Nodes: PostgreSQL + Postgrest)
            OS --> HDN(Each Node: Herodb)

            subgraph Supervisors Responsibilities
                SV --> SV_MR(Manage runners & scheduling for the node)
                SV --> SV_MJ(Monitor & schedule jobs)
                SV --> SV_RU(Check resource usage)
                SV --> SV_TO(Checks on timeout)
            end

            subgraph Runners Characteristics
                R --> R_LV(V/Python & Rust)
                R --> R_FORK(Uses fork per runner for scalability)
                R --> R_COUNT(Some runners can only run 1, others more)
                R --> R_CONTEXT(Some runners are per context)
            end
        end

        SV -- "Manage" --> R
        SV -- "Schedule jobs via" --> ZN
        ZN -- "Starts" --> R
        R -- "Interacts with" --> PGN
        R -- "Interacts with" --> HDN
    end

per node

• run on top of ZOS4 or Ubuntu or in a VM
• supervisors
  • manage runners and scheduling for the node of these runners
  • monitor & schedule jobs, check resource usage, checks on timout
• zinit
• runners (are scheduled in zinit by supervisor)
  • V/Python & Rust
  • uses fork per runner (process) for scalability
  • some runners can only run 1, others more
  • some runners are per context
• some nodes will have postgresql + postgrest
• each node has herodb

REMARK

• each rhaj or heroscript running on a node can use herodb if needed (careful, because can and will be lost), but cannot communicate with anyone else outside of the node