zosbuilder/README.md

# Alpine-based Zero-OS Initramfs

Modern Alpine Linux based approach to building Zero-OS initramfs with source compilation for Zero-OS components and Alpine packages for system tools.

## Architecture Overview

This system uses:
- **Alpine Linux 3.22** as base system with latest packages
- **Individual Rust workspaces** with git subtrees for Zero-OS components (zinit, mycelium, rfs)
- **Component-based compilation** - each component built from source in its own workspace
- **Alpine packages** for system tools (busybox, openssh, btrfs-progs, etc.)
- **Smart caching** with Docker layers and volumes for fast rebuilds
- **Single-stage initramfs** with minimal (~50MB) or full (~700MB) variants

## Directory Structure

```
alpine-initramfs/
├── build/                     # Build orchestration
│   ├── Dockerfile.alpine      # Alpine build environment  
│   ├── Dockerfile.cached      # Multi-stage cached build
│   └── docker-compose.yml     # Build orchestration with caching
├── components/                # Git subtrees (individual workspaces)
│   ├── zinit/                 # Init system (Rust workspace)
│   ├── mycelium/              # Networking layer (Rust workspace)
│   └── rfs/                   # Filesystem (Rust workspace)
├── configs/                   # Configuration files
│   ├── packages-alpine.txt    # Full Alpine packages (~700MB)
│   ├── packages-minimal.txt   # Minimal packages (~50MB)
│   ├── kernel-version         # Kernel version to build
│   ├── kernel-config-generic  # Kernel configuration
│   ├── init                   # Init script (Alpine → zinit)
│   └── zinit/                 # Zinit service configurations
├── scripts/                   # Build scripts
│   ├── build-smart.sh         # Smart build with caching (main orchestrator)
│   ├── compile-components.sh  # Compile Rust components from source
│   ├── install-packages.sh    # Install Alpine packages
│   ├── setup-initramfs.sh     # Create initramfs structure and archive
│   ├── build-kernel.sh        # Build kernel with embedded initramfs
│   └── cleanup.sh             # Clean build artifacts
├── output/                    # Final build outputs
│   ├── vmlinuz.efi            # Final bootable kernel
│   ├── initramfs.cpio.xz      # Generated initramfs archive
│   └── compiled binaries      # Individual component binaries
└── cache/                     # Build cache for smart rebuilds
    ├── components-marker      # Component build cache marker
    └── kernel-marker          # Kernel build cache marker
```

## Build Flow

The build system uses intelligent caching to minimize rebuild time:

1. **Smart Analysis**: Check what components need rebuilding based on file changes
2. **Component Compilation**: Build only changed Rust components (zinit, mycelium, rfs)
3. **Package Installation**: Install Alpine packages to initramfs root
4. **Initramfs Assembly**: Create filesystem structure with compiled binaries
5. **Kernel Build**: Compile kernel only if config/version changed
6. **Final Assembly**: Embed initramfs into kernel → bootable vmlinuz.efi

## Quick Start

### Build Everything
```bash
# Build with default settings (debug mode, full packages)
./build.sh

# Build minimal embedded initramfs (~50MB)
./build.sh --minimal

# Build release mode
./build.sh --release

# Clean build without cache
./build.sh --clean
```

### Build Variants
```bash
# Minimal embedded initramfs (~50MB)
./build.sh --minimal

# Full featured initramfs (~700MB)  
./build.sh

# Development with clean rebuild
./build.sh --clean --dev
```

### Development Workflow
```bash
# Start interactive development shell
./build.sh --dev

# Inside container:
/build/scripts/compile-components.sh  # Build all components
/build/scripts/build-smart.sh         # Smart build with caching

# Build individual components manually:
cd /build/components/zinit && cargo build --release --target x86_64-unknown-linux-musl
cd /build/components/mycelium/myceliumd && cargo build --release --target x86_64-unknown-linux-musl
cd /build/components/rfs && cargo build --release --target x86_64-unknown-linux-musl
```

## Key Features

- ✅ **Smart caching** - Only rebuild components that changed
- ✅ **Component workspaces** - Each component maintains its own workspace
- ✅ **Git subtrees** - Source code included, no submodule complexity
- ✅ **Source compilation** - All Zero-OS components built from source
- ✅ **Dual size modes** - Minimal (~50MB) or full (~700MB) initramfs
- ✅ **Docker layer caching** - Fast rebuilds with persistent cache volumes
- ✅ **Alpine packages** - Latest system tools from Alpine repositories
- ✅ **Development mode** - Full IDE support and integrated tooling

## Build Options

### Package Modes
- **Minimal Mode** (`--minimal`): ~50MB embedded initramfs with essential packages only
- **Full Mode** (default): ~700MB initramfs with complete package set

### Build Modes  
- **Debug** (default): Faster compilation with debug symbols
- **Release** (`--release`): Optimized binaries for production

### Cache Control
- **Smart Build** (default): Use cache when possible
- **Clean Build** (`--clean`): Force complete rebuild

## Development

### Building Individual Components
```bash
# Start development shell
./build.sh --dev

# Build all components
/build/scripts/compile-components.sh

# Or build individual components:
cd /build/components/zinit
cargo build --release --target x86_64-unknown-linux-musl

cd /build/components/mycelium/myceliumd  
cargo build --release --target x86_64-unknown-linux-musl

cd /build/components/rfs
cargo build --release --target x86_64-unknown-linux-musl
```

### Cache Management
Smart build uses Docker volumes for caching:
- `build-cache`: Build state markers for smart rebuilds
- `source-cache`: Downloaded sources and dependencies
- `kernel-cache`: Compiled kernel builds
- `target-cache`: Rust workspace build artifacts

### Updating Components
Update git subtree components:
```bash
# Update zinit to latest
git subtree pull --prefix=components/zinit https://github.com/threefoldtech/zinit.git master --squash

# Update mycelium to latest  
git subtree pull --prefix=components/mycelium https://github.com/threefoldtech/mycelium.git master --squash

# Update rfs to latest
git subtree pull --prefix=components/rfs https://github.com/threefoldtech/rfs.git master --squash
```

### Cleanup
```bash
# Clean all build artifacts and cache
./build.sh --clean

# Or clean specific components
./scripts/cleanup.sh
```

## Testing

Test the built kernel:
```bash
# Boot with QEMU
qemu-system-x86_64 \
  -kernel output/vmlinuz.efi \
  -m 2048 \
  -enable-kvm \
  -cpu host \
  -net nic,model=e1000 \
  -net bridge,br=vm0 \
  -nographic \
  -serial null \
  -serial mon:stdio \
  -append "console=ttyS1,115200n8"
```

## Architecture Notes

### Single-Stage Boot
This system uses a single-stage initramfs approach where the entire Zero-OS environment is embedded in the kernel. This provides:
- Simpler deployment (single vmlinuz.efi file)
- Faster boot times (no network dependencies)
- Better reliability (no network mount failures)

### Component Compilation
All Zero-OS components (zinit, mycelium, rfs) are compiled from source rather than downloaded as binaries. This ensures:
- Consistent build environment
- Version control over all components
- Easy development and debugging
- Security through reproducible builds

### Smart Caching
The build system tracks file changes and only rebuilds what's necessary:
- Component sources unchanged → use cached binaries
- Kernel config unchanged → use cached kernel
- Only initramfs assembly runs every time (fast)

This typically reduces rebuild time from 10+ minutes to under 2 minutes.