diff --git a/specs/architecture.md b/specs/architecture.md new file mode 100644 index 0000000..5c1f6e0 --- /dev/null +++ b/specs/architecture.md @@ -0,0 +1,77 @@ + + +## per user + +runs in container or VM, one per user + +- zinit +- herocoordinator + - think about like DAG worklflow manager + - manage jobs who are send around to different nodes +- mycelium address range (part of mycelium on host) +- herodb + - state manager + - redis protocol / primitives + - fs backend (mem and allways append in future) + - encryption & decryption primitives + - key mgmt for encryption (creation, deletion) + - openrpc admin features: user management, role-based access control +- postgresql + postgrest +- AI Agent TBD + +```mermaid +%%{init: {"theme":"dark"}}%% +graph TD + subgraph Per Node System + N[Node] --> OS(Run on top of ZOS4 or Ubuntu or in a VM) + + subgraph On Node + OS --> SV(Supervisors) + OS --> ZN(Zinit) + OS --> R(Runners) + OS --> PGN(Some Nodes: PostgreSQL + Postgrest) + OS --> HDN(Each Node: Herodb) + + subgraph Supervisors Responsibilities + SV --> SV_MR(Manage runners & scheduling for the node) + SV --> SV_MJ(Monitor & schedule jobs) + SV --> SV_RU(Check resource usage) + SV --> SV_TO(Checks on timeout) + end + + subgraph Runners Characteristics + R --> R_LV(V/Python & Rust) + R --> R_FORK(Uses fork per runner for scalability) + R --> R_COUNT(Some runners can only run 1, others more) + R --> R_CONTEXT(Some runners are per context) + end + end + + SV -- "Manage" --> R + SV -- "Schedule jobs via" --> ZN + ZN -- "Starts" --> R + R -- "Interacts with" --> PGN + R -- "Interacts with" --> HDN + end +``` + +## per node + +- run on top of ZOS4 or Ubuntu or in a VM +- supervisors + - manage runners and scheduling for the node of these runners + - monitor & schedule jobs, check resource usage, checks on timout +- zinit +- runners (are scheduled in zinit by supervisor) + - V/Python & Rust + - uses fork per runner (process) for scalability + - some runners can only run 1, others more + - some runners are per context +- some nodes will have postgresql + postgrest +- each node has herodb + +REMARK + +- each rhaj or heroscript running on a node can use herodb if needed (careful, because can and will be lost), but cannot communicate with anyone else outside of the node + + diff --git a/specs/hercoordinator.md b/specs/hercoordinator.md new file mode 100644 index 0000000..0b4549b --- /dev/null +++ b/specs/hercoordinator.md @@ -0,0 +1,16 @@ + + +will have openrpc interface + +- start, stop, delete, list a DAG +- query the DAG and its status + + +## remarks for supervisor + +- no retry +- no dependencies + +## inspiration + +- DAGU \ No newline at end of file