Jan De Landtsheer e13034810d Clean up build system and update documentation
- Updated README.md to reflect current single-stage build architecture
- Removed obsolete scripts/build-boot.sh (unused two-stage boot approach)
- Updated cleanup.sh to remove reference to non-existent setup-submodules.sh
- Documented smart caching and component compilation workflow
- Added proper build variants (minimal vs full) documentation
2025-08-18 14:41:46 +02:00

Alpine-based Zero-OS Initramfs

Modern Alpine Linux based approach to building Zero-OS initramfs with source compilation for Zero-OS components and Alpine packages for system tools.

Architecture Overview

This system uses:

  • Alpine Linux 3.22 as base system with latest packages
  • Individual Rust workspaces with git subtrees for Zero-OS components (zinit, mycelium, rfs)
  • Component-based compilation - each component built from source in its own workspace
  • Alpine packages for system tools (busybox, openssh, btrfs-progs, etc.)
  • Smart caching with Docker layers and volumes for fast rebuilds
  • Single-stage initramfs with minimal (~50MB) or full (~700MB) variants

Directory Structure

alpine-initramfs/
├── build/                     # Build orchestration
│   ├── Dockerfile.alpine      # Alpine build environment  
│   ├── Dockerfile.cached      # Multi-stage cached build
│   └── docker-compose.yml     # Build orchestration with caching
├── components/                # Git subtrees (individual workspaces)
│   ├── zinit/                 # Init system (Rust workspace)
│   ├── mycelium/              # Networking layer (Rust workspace)
│   └── rfs/                   # Filesystem (Rust workspace)
├── configs/                   # Configuration files
│   ├── packages-alpine.txt    # Full Alpine packages (~700MB)
│   ├── packages-minimal.txt   # Minimal packages (~50MB)
│   ├── kernel-version         # Kernel version to build
│   ├── kernel-config-generic  # Kernel configuration
│   ├── init                   # Init script (Alpine → zinit)
│   └── zinit/                 # Zinit service configurations
├── scripts/                   # Build scripts
│   ├── build-smart.sh         # Smart build with caching (main orchestrator)
│   ├── compile-components.sh  # Compile Rust components from source
│   ├── install-packages.sh    # Install Alpine packages
│   ├── setup-initramfs.sh     # Create initramfs structure and archive
│   ├── build-kernel.sh        # Build kernel with embedded initramfs
│   └── cleanup.sh             # Clean build artifacts
├── output/                    # Final build outputs
│   ├── vmlinuz.efi            # Final bootable kernel
│   ├── initramfs.cpio.xz      # Generated initramfs archive
│   └── compiled binaries      # Individual component binaries
└── cache/                     # Build cache for smart rebuilds
    ├── components-marker      # Component build cache marker
    └── kernel-marker          # Kernel build cache marker

Build Flow

The build system uses intelligent caching to minimize rebuild time:

  1. Smart Analysis: Check what components need rebuilding based on file changes
  2. Component Compilation: Build only changed Rust components (zinit, mycelium, rfs)
  3. Package Installation: Install Alpine packages to initramfs root
  4. Initramfs Assembly: Create filesystem structure with compiled binaries
  5. Kernel Build: Compile kernel only if config/version changed
  6. Final Assembly: Embed initramfs into kernel → bootable vmlinuz.efi

Quick Start

Build Everything

# Build with default settings (debug mode, full packages)
./build.sh

# Build minimal embedded initramfs (~50MB)
./build.sh --minimal

# Build release mode
./build.sh --release

# Clean build without cache
./build.sh --clean

Build Variants

# Minimal embedded initramfs (~50MB)
./build.sh --minimal

# Full featured initramfs (~700MB)  
./build.sh

# Development with clean rebuild
./build.sh --clean --dev

Development Workflow

# Start interactive development shell
./build.sh --dev

# Inside container:
/build/scripts/compile-components.sh  # Build all components
/build/scripts/build-smart.sh         # Smart build with caching

# Build individual components manually:
cd /build/components/zinit && cargo build --release --target x86_64-unknown-linux-musl
cd /build/components/mycelium/myceliumd && cargo build --release --target x86_64-unknown-linux-musl
cd /build/components/rfs && cargo build --release --target x86_64-unknown-linux-musl

Key Features

  • Smart caching - Only rebuild components that changed
  • Component workspaces - Each component maintains its own workspace
  • Git subtrees - Source code included, no submodule complexity
  • Source compilation - All Zero-OS components built from source
  • Dual size modes - Minimal (~50MB) or full (~700MB) initramfs
  • Docker layer caching - Fast rebuilds with persistent cache volumes
  • Alpine packages - Latest system tools from Alpine repositories
  • Development mode - Full IDE support and integrated tooling

Build Options

Package Modes

  • Minimal Mode (--minimal): ~50MB embedded initramfs with essential packages only
  • Full Mode (default): ~700MB initramfs with complete package set

Build Modes

  • Debug (default): Faster compilation with debug symbols
  • Release (--release): Optimized binaries for production

Cache Control

  • Smart Build (default): Use cache when possible
  • Clean Build (--clean): Force complete rebuild

Development

Building Individual Components

# Start development shell
./build.sh --dev

# Build all components
/build/scripts/compile-components.sh

# Or build individual components:
cd /build/components/zinit
cargo build --release --target x86_64-unknown-linux-musl

cd /build/components/mycelium/myceliumd  
cargo build --release --target x86_64-unknown-linux-musl

cd /build/components/rfs
cargo build --release --target x86_64-unknown-linux-musl

Cache Management

Smart build uses Docker volumes for caching:

  • build-cache: Build state markers for smart rebuilds
  • source-cache: Downloaded sources and dependencies
  • kernel-cache: Compiled kernel builds
  • target-cache: Rust workspace build artifacts

Updating Components

Update git subtree components:

# Update zinit to latest
git subtree pull --prefix=components/zinit https://github.com/threefoldtech/zinit.git master --squash

# Update mycelium to latest  
git subtree pull --prefix=components/mycelium https://github.com/threefoldtech/mycelium.git master --squash

# Update rfs to latest
git subtree pull --prefix=components/rfs https://github.com/threefoldtech/rfs.git master --squash

Cleanup

# Clean all build artifacts and cache
./build.sh --clean

# Or clean specific components
./scripts/cleanup.sh

Testing

Test the built kernel:

# Boot with QEMU
qemu-system-x86_64 \
  -kernel output/vmlinuz.efi \
  -m 2048 \
  -enable-kvm \
  -cpu host \
  -net nic,model=e1000 \
  -net bridge,br=vm0 \
  -nographic \
  -serial null \
  -serial mon:stdio \
  -append "console=ttyS1,115200n8"

Architecture Notes

Single-Stage Boot

This system uses a single-stage initramfs approach where the entire Zero-OS environment is embedded in the kernel. This provides:

  • Simpler deployment (single vmlinuz.efi file)
  • Faster boot times (no network dependencies)
  • Better reliability (no network mount failures)

Component Compilation

All Zero-OS components (zinit, mycelium, rfs) are compiled from source rather than downloaded as binaries. This ensures:

  • Consistent build environment
  • Version control over all components
  • Easy development and debugging
  • Security through reproducible builds

Smart Caching

The build system tracks file changes and only rebuilds what's necessary:

  • Component sources unchanged → use cached binaries
  • Kernel config unchanged → use cached kernel
  • Only initramfs assembly runs every time (fast)

This typically reduces rebuild time from 10+ minutes to under 2 minutes.

Description
dockerized/podman-ized build system for zos, based on Alpine pakaging system, zinit and zos-specific binaries
Readme 147 MiB
Languages
Shell 99.5%
Dockerfile 0.5%