From 0db55fdc6e36ea9bf8b632de25f1af4e14589923 Mon Sep 17 00:00:00 2001 From: Jan De Landtsheer Date: Tue, 9 Sep 2025 16:17:10 +0200 Subject: [PATCH] docs: add comprehensive repository map and operational notes (build flow, branding passwordless policy, path normalization, container tools) --- docs/NOTES.md | 225 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 177 insertions(+), 48 deletions(-) diff --git a/docs/NOTES.md b/docs/NOTES.md index 3b440aa..8e9cff3 100644 --- a/docs/NOTES.md +++ b/docs/NOTES.md @@ -1,57 +1,186 @@ -Zero-OS Branding Diagnostics and Notes +Zero-OS Builder – Working Notes and Repository Map -Context -- Goal: Branding flags should enable passwordless root in initramfs and update /etc/{issue,motd}. -- Source of truth for flags: [config/build.conf](config/build.conf) -- Implementation hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575) called from [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:663) just before CPIO creation. +Purpose +- This document captures operational knowledge of this repository: build flow, key files, flags, and recent behavior decisions (e.g., passwordless root). +- Links below point to exact functions and files for fast triage, using code navigation-friendly anchors. -Observed issue in latest build -- Branding flags were set: logs showed "Branding debug: ZEROOS_BRANDING=true ... _branding=true". -- Both /etc/passwd and /etc/shadow exist in initramfs; Alpine uses shadow for authentication. -- The script only edited /etc/passwd, leaving /etc/shadow unchanged; login still required a password. -- Evidence (from build logs): - - Preview /etc/passwd (pre): root:(x):0:0:root:/root:/bin/sh - - Preview /etc/shadow (pre): root:(***):... - - Preview /etc/passwd (post): root:(x):0:0:root:/root:/bin/sh - - Preview /etc/shadow (post): root:(***):... +Repository Overview +- Build entrypoint: [scripts/build.sh](scripts/build.sh) + - Orchestrates incremental stages using stage markers. + - Runs inside a container defined by [Dockerfile](Dockerfile) for reproducibility. +- Common utilities and config loading: [bash.common.sh](scripts/lib/common.sh:1) + - Loads [config/build.conf](config/build.conf), normalizes directory paths, provides logging and safe execution wrappers. +- Initramfs assembly and finalization: [bash.initramfs_* functions](scripts/lib/initramfs.sh:1) + - Copies components, sets up zinit configs, finalizes branding, creates CPIO archive, validates contents. +- Kernel integration (optional embedded initramfs): [bash.kernel_* functions](scripts/lib/kernel.sh:1) + - Downloads/configures/builds kernel and modules, embeds initramfs, runs depmod. +- zinit configuration: [config/zinit/](config/zinit/) + - YAML service definitions and init scripts used by zinit inside the initramfs rootfs. +- RFS tooling (modules/firmware flists): [scripts/rfs/](scripts/rfs/) + - Packs module/firmware flists and embeds them into initramfs at /etc/rfs. -Root cause -- Editing /etc/passwd is ineffective when /etc/shadow is present; the pw field is ignored in passwd and 'x' indicates to consult shadow. +Container Tooling (dev-container) +- Base image: Alpine 3.22 in [Dockerfile](Dockerfile:1) +- Tools: + - shadow (passwd/chpasswd): required for root password management in initramfs. + - openssl, openssl-dev: kept for other build steps and potential hashing utilities. + - build-base, rustup, kmod, upx, etc.: required by various build stages. +- Removed: perl, not required for password handling after switching to passwd/chpasswd workflow. -Fix implemented -- Change in [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575): - - Prefer editing /etc/shadow for root’s password field; fallback to /etc/passwd if shadow is absent. - - Command used: - - sed -i 's/^root:[^:]*:/root::/' "${initramfs_dir}/etc/shadow" -- Diagnostics retained: - - Logs branding vars, presence/perms of /etc/{shadow,passwd}, and sanitized previews pre/post. +Configuration – build.conf +- File: [config/build.conf](config/build.conf) +- Key variables: + - Versions: ALPINE_VERSION, KERNEL_VERSION + - Directories (relative in config, normalized to absolute during runtime): + - INSTALL_DIR="initramfs" + - COMPONENTS_DIR="components" + - KERNEL_DIR="kernel" + - DIST_DIR="dist" + - Flags: + - ZEROOS_BRANDING="true" + - ZEROOS_REBRANDING="true" + - Branding behavior: + - ZEROOS_PASSWORDLESS_ROOT="true" (default for branded builds in current policy) + - ZEROOS_ROOT_PASSWORD_HASH / ROOT_PASSWORD_HASH (not used in current policy) + - ZEROOS_ROOT_PASSWORD / ROOT_PASSWORD (not used in current policy) + - FIRMWARE_TAG optional for reproducible firmware flist naming. -Verification plan -- Minimal rebuild to re-run finalize: - - rm -f .build-stages/initramfs_create.done .build-stages/initramfs_test.done - - DEBUG=1 ./scripts/build.sh --skip-tests -- Confirm in logs: - - "✓ Root password removed in /etc/shadow (passwordless root)" - - Preview /etc/shadow (post): root:(***): with empty field notation "root::" internally. -- Optional deeper check by inspecting the archive: - - cd dist && mkdir tmp && cd tmp - - xz -dc ../initramfs.cpio.xz | cpio -idv - - grep '^root:' ./etc/shadow | sed 's/^\(root:\)[^:]*:/\1(***):/' - - Expected: the second field is empty (root::...). +Absolute Path Normalization +- Location: [bash.common.sh](scripts/lib/common.sh:225) +- After sourcing build.conf, the following variables are normalized to absolute paths anchored at PROJECT_ROOT: + - INSTALL_DIR, COMPONENTS_DIR, KERNEL_DIR, DIST_DIR +- Rationale: Prevents path resolution errors when CWD changes (e.g., when kernel build operates in /workspace/kernel/current, validation now resolves to /workspace/initramfs instead of /workspace/kernel/current/initramfs). -Behavior and safety notes -- Permissions: /etc/shadow typically 640 root:shadow; the fix does not alter permissions. -- Passwordless root in initramfs is intended only when [config/build.conf](config/build.conf) sets ZEROOS_BRANDING="true" (or ZEROOS_REBRANDING="true"). -- The change affects only the initramfs image; not the host system. +Build Pipeline – High Level +- Orchestrator: [bash.main_build_process()](scripts/build.sh:213) + - Stage list: + - alpine_extract + - alpine_configure + - alpine_packages + - alpine_firmware + - components_build + - components_verify + - kernel_modules + - init_script + - components_copy + - zinit_setup + - modules_setup + - modules_copy + - cleanup + - rfs_flists + - validation + - initramfs_create + - initramfs_test + - kernel_build + - boot_tests + - Each stage wrapped with [bash.stage_run()](scripts/lib/stages.sh:99) and tracked under .build-stages/ +- Container use: + - Always run in container for stable toolchain (podman/docker auto-detected). + - Inside container, CWD normalized to PROJECT_ROOT. -Code references -- Branding guard and customization: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575) -- Archive creation entry point: [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:663) -- Build orchestrator: [bash.main_build_process()](scripts/build.sh:213) +Initramfs Assembly – Key Functions +- zinit setup: [bash.initramfs_setup_zinit()](scripts/lib/initramfs.sh:12) + - Copies [config/zinit](config/zinit/) into /etc/zinit, fixes perms, removes OpenRC remnants. +- Install init script: [bash.initramfs_install_init_script()](scripts/lib/initramfs.sh:74) + - Installs [config/init](config/init) as /init in initramfs root. +- Components copy: [bash.initramfs_copy_components()](scripts/lib/initramfs.sh:101) + - Installs built components (zinit/rfs/mycelium/corex) into proper locations, strips/UPX where applicable. +- Modules setup: [bash.initramfs_setup_modules()](scripts/lib/initramfs.sh:229) + - Reads [config/modules.conf](config/modules.conf), resolves deps via [bash.initramfs_resolve_module_dependencies()](scripts/lib/initramfs.sh:318), generates stage1 list with firmware correlation. +- Create module scripts: [bash.initramfs_create_module_scripts()](scripts/lib/initramfs.sh:427) + - Writes /etc/zinit/init/stage1-modules.sh and stage2-modules.sh for zinit to load modules. +- Binary size optimization: [bash.initramfs_strip_and_upx()](scripts/lib/initramfs.sh:491) +- Final customization: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575) + - See “Branding behavior” below. +- Create archive: [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:662) + - Calls finalize, runs sanity checks, and creates initramfs.cpio.xz. +- Validate: [bash.initramfs_validate()](scripts/lib/initramfs.sh:790) + - Ensures essential items exist, logs debug info: + - Prints “Validation debug:” lines showing input, PWD, PROJECT_ROOT, INSTALL_DIR, and resolved path. -Notes usage -- This file (docs/NOTES.md) is the session-to-session log of debugging and decisions. -- For finalized policies, consider adding docs/DECISIONS.md. +Kernel Integration +- Get full kernel version: [bash.kernel_get_full_version()](scripts/lib/kernel.sh:13) +- Apply config (embed initramfs): [bash.kernel_apply_config()](scripts/lib/kernel.sh:81) + - Updates CONFIG_INITRAMFS_SOURCE to the archive’s absolute path via [bash.kernel_modify_config_for_initramfs()](scripts/lib/kernel.sh:129). +- Build kernel: [bash.kernel_build_with_initramfs()](scripts/lib/kernel.sh:173) +- Build and install modules in container: [bash.kernel_build_modules()](scripts/lib/kernel.sh:228) + - Installs modules to /lib/modules/$FULL_VERSION in container, runs depmod -a. -Change log -- 2025-09-09: Added diagnostics and implemented shadow-first passwordless root; documented verification steps. \ No newline at end of file +RFS Flists (modules/firmware) +- Packing scripts: + - Modules: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1) + - Firmware: [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1) +- Integrated in stage_rfs_flists: + - Embeds /etc/rfs/modules-.fl + - Embeds /etc/rfs/firmware-latest.fl (or tagged by FIRMWARE_TAG) + - See [bash.main_build_process() — stage_rfs_flists](scripts/build.sh:298) + +Branding Behavior (Passwordless Root, motd/issue) +- Finalization hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575) +- Behavior (current policy): + - Passwordless root enforced using passwd for shadow-aware deletion: + - [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:616): + - passwd -d -R "${initramfs_dir}" root + - Ensures /etc/shadow has root:: (empty password) inside initramfs root, not host. + - Branding toggles: ZEROOS_BRANDING and ZEROOS_REBRANDING (branding guard printed in logs). + - Branding also updates /etc/motd and /etc/issue to Zero-OS. + +Console and getty +- Serial and console getty configs (zinit service YAML): + - [config/zinit/getty.yaml](config/zinit/getty.yaml) + - [config/zinit/gettyconsole.yaml](config/zinit/gettyconsole.yaml) + - [config/zinit/console.yaml](config/zinit/console.yaml) +- Optional ash login loop (not enabled unless referenced): + - [bash.ashloging.sh](config/zinit/init/ashloging.sh:1) + +Validation Diagnostics and Triage +- Common error previously observed: + - “Initramfs directory not found: initramfs (resolved: /workspace/kernel/current/initramfs)” +- Root cause: + - INSTALL_DIR re-sourced in a different CWD and interpreted as relative. +- Fix: + - Absolute path normalization of INSTALL_DIR/COMPONENTS_DIR/KERNEL_DIR/DIST_DIR after sourcing build.conf in [bash.common.sh](scripts/lib/common.sh:225). + - Additional “Validation debug” prints added in [bash.initramfs_validate()](scripts/lib/initramfs.sh:790). +- Expected logs now: + - “Validation debug: input='initramfs' PWD=/workspace PROJECT_ROOT=/workspace INSTALL_DIR=/workspace/initramfs” + - Resolves correctly even if called from a different stage CWD. + +How to Verify Passwordless Root +- After build, check archive: + - mkdir -p dist/_inspect && cd dist/_inspect + - xz -dc ../initramfs.cpio.xz | cpio -idmv + - grep '^root:' ./etc/shadow + - Expect root:: (empty field) indicating passwordless root. +- At runtime on console: + - When prompted for root’s password, press Enter. + +Stage System and Incremental Rebuilds +- Stage markers stored in .build-stages/ (one file per stage). +- To minimally rebuild: + - Remove relevant .done files, e.g.: + - initramfs_create.done initramfs_test.done validation.done + - Rerun: DEBUG=1 ./scripts/build.sh --skip-tests +- Show status: + - ./scripts/build.sh --show-stages + +Key Decisions (current) +- Branding enforces passwordless root via passwd -d -R inside initramfs finalization, avoiding direct edits of passwd/shadow files. +- Directory paths normalized to absolute after loading config to avoid CWD-sensitive behavior. +- Container image contains shadow suite to ensure passwd/chpasswd availability; perl removed. + +File Pointers (quick jump) +- Orchestrator: [scripts/build.sh](scripts/build.sh) +- Common and config loading: [bash.common.sh](scripts/lib/common.sh:1) +- Finalization hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575) +- Passwordless deletion: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:616) +- Validation entry: [bash.initramfs_validate()](scripts/lib/initramfs.sh:790) +- CPIO creation: [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:662) +- Kernel embed config: [bash.kernel_modify_config_for_initramfs()](scripts/lib/kernel.sh:129) +- RFS packers: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1), [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1) + +Change Log +- 2025-09-09: + - Enforce passwordless root using passwd -d -R in finalization. + - Normalize INSTALL_DIR/COMPONENTS_DIR/KERNEL_DIR/DIST_DIR to absolute paths post-config load. + - Add validation diagnostics prints (input/PWD/PROJECT_ROOT/INSTALL_DIR/resolved). + - Ensure shadow package in container for passwd/chpasswd; keep openssl and openssl-dev; remove perl earlier. \ No newline at end of file