forked from tfgrid/zosbuilder
docs: add comprehensive repository map and operational notes (build flow, branding passwordless policy, path normalization, container tools)
This commit is contained in:
225
docs/NOTES.md
225
docs/NOTES.md
@@ -1,57 +1,186 @@
|
||||
Zero-OS Branding Diagnostics and Notes
|
||||
Zero-OS Builder – Working Notes and Repository Map
|
||||
|
||||
Context
|
||||
- Goal: Branding flags should enable passwordless root in initramfs and update /etc/{issue,motd}.
|
||||
- Source of truth for flags: [config/build.conf](config/build.conf)
|
||||
- Implementation hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575) called from [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:663) just before CPIO creation.
|
||||
Purpose
|
||||
- This document captures operational knowledge of this repository: build flow, key files, flags, and recent behavior decisions (e.g., passwordless root).
|
||||
- Links below point to exact functions and files for fast triage, using code navigation-friendly anchors.
|
||||
|
||||
Observed issue in latest build
|
||||
- Branding flags were set: logs showed "Branding debug: ZEROOS_BRANDING=true ... _branding=true".
|
||||
- Both /etc/passwd and /etc/shadow exist in initramfs; Alpine uses shadow for authentication.
|
||||
- The script only edited /etc/passwd, leaving /etc/shadow unchanged; login still required a password.
|
||||
- Evidence (from build logs):
|
||||
- Preview /etc/passwd (pre): root:(x):0:0:root:/root:/bin/sh
|
||||
- Preview /etc/shadow (pre): root:(***):...
|
||||
- Preview /etc/passwd (post): root:(x):0:0:root:/root:/bin/sh
|
||||
- Preview /etc/shadow (post): root:(***):...
|
||||
Repository Overview
|
||||
- Build entrypoint: [scripts/build.sh](scripts/build.sh)
|
||||
- Orchestrates incremental stages using stage markers.
|
||||
- Runs inside a container defined by [Dockerfile](Dockerfile) for reproducibility.
|
||||
- Common utilities and config loading: [bash.common.sh](scripts/lib/common.sh:1)
|
||||
- Loads [config/build.conf](config/build.conf), normalizes directory paths, provides logging and safe execution wrappers.
|
||||
- Initramfs assembly and finalization: [bash.initramfs_* functions](scripts/lib/initramfs.sh:1)
|
||||
- Copies components, sets up zinit configs, finalizes branding, creates CPIO archive, validates contents.
|
||||
- Kernel integration (optional embedded initramfs): [bash.kernel_* functions](scripts/lib/kernel.sh:1)
|
||||
- Downloads/configures/builds kernel and modules, embeds initramfs, runs depmod.
|
||||
- zinit configuration: [config/zinit/](config/zinit/)
|
||||
- YAML service definitions and init scripts used by zinit inside the initramfs rootfs.
|
||||
- RFS tooling (modules/firmware flists): [scripts/rfs/](scripts/rfs/)
|
||||
- Packs module/firmware flists and embeds them into initramfs at /etc/rfs.
|
||||
|
||||
Root cause
|
||||
- Editing /etc/passwd is ineffective when /etc/shadow is present; the pw field is ignored in passwd and 'x' indicates to consult shadow.
|
||||
Container Tooling (dev-container)
|
||||
- Base image: Alpine 3.22 in [Dockerfile](Dockerfile:1)
|
||||
- Tools:
|
||||
- shadow (passwd/chpasswd): required for root password management in initramfs.
|
||||
- openssl, openssl-dev: kept for other build steps and potential hashing utilities.
|
||||
- build-base, rustup, kmod, upx, etc.: required by various build stages.
|
||||
- Removed: perl, not required for password handling after switching to passwd/chpasswd workflow.
|
||||
|
||||
Fix implemented
|
||||
- Change in [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575):
|
||||
- Prefer editing /etc/shadow for root’s password field; fallback to /etc/passwd if shadow is absent.
|
||||
- Command used:
|
||||
- sed -i 's/^root:[^:]*:/root::/' "${initramfs_dir}/etc/shadow"
|
||||
- Diagnostics retained:
|
||||
- Logs branding vars, presence/perms of /etc/{shadow,passwd}, and sanitized previews pre/post.
|
||||
Configuration – build.conf
|
||||
- File: [config/build.conf](config/build.conf)
|
||||
- Key variables:
|
||||
- Versions: ALPINE_VERSION, KERNEL_VERSION
|
||||
- Directories (relative in config, normalized to absolute during runtime):
|
||||
- INSTALL_DIR="initramfs"
|
||||
- COMPONENTS_DIR="components"
|
||||
- KERNEL_DIR="kernel"
|
||||
- DIST_DIR="dist"
|
||||
- Flags:
|
||||
- ZEROOS_BRANDING="true"
|
||||
- ZEROOS_REBRANDING="true"
|
||||
- Branding behavior:
|
||||
- ZEROOS_PASSWORDLESS_ROOT="true" (default for branded builds in current policy)
|
||||
- ZEROOS_ROOT_PASSWORD_HASH / ROOT_PASSWORD_HASH (not used in current policy)
|
||||
- ZEROOS_ROOT_PASSWORD / ROOT_PASSWORD (not used in current policy)
|
||||
- FIRMWARE_TAG optional for reproducible firmware flist naming.
|
||||
|
||||
Verification plan
|
||||
- Minimal rebuild to re-run finalize:
|
||||
- rm -f .build-stages/initramfs_create.done .build-stages/initramfs_test.done
|
||||
- DEBUG=1 ./scripts/build.sh --skip-tests
|
||||
- Confirm in logs:
|
||||
- "✓ Root password removed in /etc/shadow (passwordless root)"
|
||||
- Preview /etc/shadow (post): root:(***): with empty field notation "root::" internally.
|
||||
- Optional deeper check by inspecting the archive:
|
||||
- cd dist && mkdir tmp && cd tmp
|
||||
- xz -dc ../initramfs.cpio.xz | cpio -idv
|
||||
- grep '^root:' ./etc/shadow | sed 's/^\(root:\)[^:]*:/\1(***):/'
|
||||
- Expected: the second field is empty (root::...).
|
||||
Absolute Path Normalization
|
||||
- Location: [bash.common.sh](scripts/lib/common.sh:225)
|
||||
- After sourcing build.conf, the following variables are normalized to absolute paths anchored at PROJECT_ROOT:
|
||||
- INSTALL_DIR, COMPONENTS_DIR, KERNEL_DIR, DIST_DIR
|
||||
- Rationale: Prevents path resolution errors when CWD changes (e.g., when kernel build operates in /workspace/kernel/current, validation now resolves to /workspace/initramfs instead of /workspace/kernel/current/initramfs).
|
||||
|
||||
Behavior and safety notes
|
||||
- Permissions: /etc/shadow typically 640 root:shadow; the fix does not alter permissions.
|
||||
- Passwordless root in initramfs is intended only when [config/build.conf](config/build.conf) sets ZEROOS_BRANDING="true" (or ZEROOS_REBRANDING="true").
|
||||
- The change affects only the initramfs image; not the host system.
|
||||
Build Pipeline – High Level
|
||||
- Orchestrator: [bash.main_build_process()](scripts/build.sh:213)
|
||||
- Stage list:
|
||||
- alpine_extract
|
||||
- alpine_configure
|
||||
- alpine_packages
|
||||
- alpine_firmware
|
||||
- components_build
|
||||
- components_verify
|
||||
- kernel_modules
|
||||
- init_script
|
||||
- components_copy
|
||||
- zinit_setup
|
||||
- modules_setup
|
||||
- modules_copy
|
||||
- cleanup
|
||||
- rfs_flists
|
||||
- validation
|
||||
- initramfs_create
|
||||
- initramfs_test
|
||||
- kernel_build
|
||||
- boot_tests
|
||||
- Each stage wrapped with [bash.stage_run()](scripts/lib/stages.sh:99) and tracked under .build-stages/
|
||||
- Container use:
|
||||
- Always run in container for stable toolchain (podman/docker auto-detected).
|
||||
- Inside container, CWD normalized to PROJECT_ROOT.
|
||||
|
||||
Code references
|
||||
- Branding guard and customization: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575)
|
||||
- Archive creation entry point: [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:663)
|
||||
- Build orchestrator: [bash.main_build_process()](scripts/build.sh:213)
|
||||
Initramfs Assembly – Key Functions
|
||||
- zinit setup: [bash.initramfs_setup_zinit()](scripts/lib/initramfs.sh:12)
|
||||
- Copies [config/zinit](config/zinit/) into /etc/zinit, fixes perms, removes OpenRC remnants.
|
||||
- Install init script: [bash.initramfs_install_init_script()](scripts/lib/initramfs.sh:74)
|
||||
- Installs [config/init](config/init) as /init in initramfs root.
|
||||
- Components copy: [bash.initramfs_copy_components()](scripts/lib/initramfs.sh:101)
|
||||
- Installs built components (zinit/rfs/mycelium/corex) into proper locations, strips/UPX where applicable.
|
||||
- Modules setup: [bash.initramfs_setup_modules()](scripts/lib/initramfs.sh:229)
|
||||
- Reads [config/modules.conf](config/modules.conf), resolves deps via [bash.initramfs_resolve_module_dependencies()](scripts/lib/initramfs.sh:318), generates stage1 list with firmware correlation.
|
||||
- Create module scripts: [bash.initramfs_create_module_scripts()](scripts/lib/initramfs.sh:427)
|
||||
- Writes /etc/zinit/init/stage1-modules.sh and stage2-modules.sh for zinit to load modules.
|
||||
- Binary size optimization: [bash.initramfs_strip_and_upx()](scripts/lib/initramfs.sh:491)
|
||||
- Final customization: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575)
|
||||
- See “Branding behavior” below.
|
||||
- Create archive: [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:662)
|
||||
- Calls finalize, runs sanity checks, and creates initramfs.cpio.xz.
|
||||
- Validate: [bash.initramfs_validate()](scripts/lib/initramfs.sh:790)
|
||||
- Ensures essential items exist, logs debug info:
|
||||
- Prints “Validation debug:” lines showing input, PWD, PROJECT_ROOT, INSTALL_DIR, and resolved path.
|
||||
|
||||
Notes usage
|
||||
- This file (docs/NOTES.md) is the session-to-session log of debugging and decisions.
|
||||
- For finalized policies, consider adding docs/DECISIONS.md.
|
||||
Kernel Integration
|
||||
- Get full kernel version: [bash.kernel_get_full_version()](scripts/lib/kernel.sh:13)
|
||||
- Apply config (embed initramfs): [bash.kernel_apply_config()](scripts/lib/kernel.sh:81)
|
||||
- Updates CONFIG_INITRAMFS_SOURCE to the archive’s absolute path via [bash.kernel_modify_config_for_initramfs()](scripts/lib/kernel.sh:129).
|
||||
- Build kernel: [bash.kernel_build_with_initramfs()](scripts/lib/kernel.sh:173)
|
||||
- Build and install modules in container: [bash.kernel_build_modules()](scripts/lib/kernel.sh:228)
|
||||
- Installs modules to /lib/modules/$FULL_VERSION in container, runs depmod -a.
|
||||
|
||||
Change log
|
||||
- 2025-09-09: Added diagnostics and implemented shadow-first passwordless root; documented verification steps.
|
||||
RFS Flists (modules/firmware)
|
||||
- Packing scripts:
|
||||
- Modules: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1)
|
||||
- Firmware: [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1)
|
||||
- Integrated in stage_rfs_flists:
|
||||
- Embeds /etc/rfs/modules-<FULL_KERNEL_VERSION>.fl
|
||||
- Embeds /etc/rfs/firmware-latest.fl (or tagged by FIRMWARE_TAG)
|
||||
- See [bash.main_build_process() — stage_rfs_flists](scripts/build.sh:298)
|
||||
|
||||
Branding Behavior (Passwordless Root, motd/issue)
|
||||
- Finalization hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575)
|
||||
- Behavior (current policy):
|
||||
- Passwordless root enforced using passwd for shadow-aware deletion:
|
||||
- [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:616):
|
||||
- passwd -d -R "${initramfs_dir}" root
|
||||
- Ensures /etc/shadow has root:: (empty password) inside initramfs root, not host.
|
||||
- Branding toggles: ZEROOS_BRANDING and ZEROOS_REBRANDING (branding guard printed in logs).
|
||||
- Branding also updates /etc/motd and /etc/issue to Zero-OS.
|
||||
|
||||
Console and getty
|
||||
- Serial and console getty configs (zinit service YAML):
|
||||
- [config/zinit/getty.yaml](config/zinit/getty.yaml)
|
||||
- [config/zinit/gettyconsole.yaml](config/zinit/gettyconsole.yaml)
|
||||
- [config/zinit/console.yaml](config/zinit/console.yaml)
|
||||
- Optional ash login loop (not enabled unless referenced):
|
||||
- [bash.ashloging.sh](config/zinit/init/ashloging.sh:1)
|
||||
|
||||
Validation Diagnostics and Triage
|
||||
- Common error previously observed:
|
||||
- “Initramfs directory not found: initramfs (resolved: /workspace/kernel/current/initramfs)”
|
||||
- Root cause:
|
||||
- INSTALL_DIR re-sourced in a different CWD and interpreted as relative.
|
||||
- Fix:
|
||||
- Absolute path normalization of INSTALL_DIR/COMPONENTS_DIR/KERNEL_DIR/DIST_DIR after sourcing build.conf in [bash.common.sh](scripts/lib/common.sh:225).
|
||||
- Additional “Validation debug” prints added in [bash.initramfs_validate()](scripts/lib/initramfs.sh:790).
|
||||
- Expected logs now:
|
||||
- “Validation debug: input='initramfs' PWD=/workspace PROJECT_ROOT=/workspace INSTALL_DIR=/workspace/initramfs”
|
||||
- Resolves correctly even if called from a different stage CWD.
|
||||
|
||||
How to Verify Passwordless Root
|
||||
- After build, check archive:
|
||||
- mkdir -p dist/_inspect && cd dist/_inspect
|
||||
- xz -dc ../initramfs.cpio.xz | cpio -idmv
|
||||
- grep '^root:' ./etc/shadow
|
||||
- Expect root:: (empty field) indicating passwordless root.
|
||||
- At runtime on console:
|
||||
- When prompted for root’s password, press Enter.
|
||||
|
||||
Stage System and Incremental Rebuilds
|
||||
- Stage markers stored in .build-stages/ (one file per stage).
|
||||
- To minimally rebuild:
|
||||
- Remove relevant .done files, e.g.:
|
||||
- initramfs_create.done initramfs_test.done validation.done
|
||||
- Rerun: DEBUG=1 ./scripts/build.sh --skip-tests
|
||||
- Show status:
|
||||
- ./scripts/build.sh --show-stages
|
||||
|
||||
Key Decisions (current)
|
||||
- Branding enforces passwordless root via passwd -d -R inside initramfs finalization, avoiding direct edits of passwd/shadow files.
|
||||
- Directory paths normalized to absolute after loading config to avoid CWD-sensitive behavior.
|
||||
- Container image contains shadow suite to ensure passwd/chpasswd availability; perl removed.
|
||||
|
||||
File Pointers (quick jump)
|
||||
- Orchestrator: [scripts/build.sh](scripts/build.sh)
|
||||
- Common and config loading: [bash.common.sh](scripts/lib/common.sh:1)
|
||||
- Finalization hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575)
|
||||
- Passwordless deletion: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:616)
|
||||
- Validation entry: [bash.initramfs_validate()](scripts/lib/initramfs.sh:790)
|
||||
- CPIO creation: [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:662)
|
||||
- Kernel embed config: [bash.kernel_modify_config_for_initramfs()](scripts/lib/kernel.sh:129)
|
||||
- RFS packers: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1), [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1)
|
||||
|
||||
Change Log
|
||||
- 2025-09-09:
|
||||
- Enforce passwordless root using passwd -d -R in finalization.
|
||||
- Normalize INSTALL_DIR/COMPONENTS_DIR/KERNEL_DIR/DIST_DIR to absolute paths post-config load.
|
||||
- Add validation diagnostics prints (input/PWD/PROJECT_ROOT/INSTALL_DIR/resolved).
|
||||
- Ensure shadow package in container for passwd/chpasswd; keep openssl and openssl-dev; remove perl earlier.
|
||||
Reference in New Issue
Block a user