zosbuilder/README.md

# Zero OS Alpine Initramfs Builder

A comprehensive build system for creating custom Alpine Linux 3.22 x86_64 initramfs with zinit process management, designed for Zero OS deployment.

## Features

- **Alpine Linux 3.22** miniroot as base system
- **zinit** process manager (complete OpenRC replacement)
- **Rootless containers** (Docker/Podman compatible)
- **Rust components** with musl targeting (zinit, rfs, mycelium, zosstorage)
- **Aggressive optimization** (strip + UPX compression)
- **2-stage module loading** for hardware support
- **GitHub Actions** compatible build pipeline
- **Final output**: `vmlinuz.efi` with embedded `initramfs.cpio.xz`

## Quick Start

### Prerequisites

#### Ubuntu/Debian
```bash
sudo apt-get update
sudo apt-get install -y \
  build-essential \
  rustc \
  cargo \
  upx-ucl \
  binutils \
  git \
  wget \
  qemu-system-x86 \
  podman

# Add Rust musl target
rustup target add x86_64-unknown-linux-musl
sudo apt-get install -y musl-tools
```

#### Alpine Linux
```bash
apk add --no-cache \
  build-base \
  rust \
  cargo \
  upx \
  git \
  wget \
  qemu-system-x86 \
  podman

# Add Rust musl target
rustup target add x86_64-unknown-linux-musl
```

### Rootless Container Setup

For rootless Docker/Podman support:

```bash
# Configure subuid/subgid (if not already configured)
echo "$(whoami):100000:65536" | sudo tee -a /etc/subuid
echo "$(whoami):100000:65536" | sudo tee -a /etc/subgid

# Verify setup
podman system info
```

### Build

```bash
# Clone the repository
git clone <repository-url>
cd zosbuilder

# Make scripts executable
chmod +x scripts/build.sh scripts/clean.sh

# Build initramfs
./scripts/build.sh

# Output will be in dist/
ls -la dist/
# vmlinuz.efi          - Kernel with embedded initramfs
# initramfs.cpio.xz    - Standalone initramfs archive
```

## Project Structure

```
zosbuilder/
├── config/
│   ├── zinit/              # zinit service definitions
│   │   ├── services/       # individual service files
│   │   └── zinit.conf      # main zinit configuration
│   ├── packages.list       # Alpine packages to install
│   ├── sources.conf        # components to build (ThreeFold)
│   ├── kernel.config       # Linux kernel configuration
│   └── modules.conf        # 2-stage module loading
├── configs/                # existing configurations (migrated)
├── scripts/
│   ├── lib/
│   │   ├── docker.sh       # container management
│   │   ├── alpine.sh       # Alpine operations
│   │   ├── components.sh   # source building
│   │   ├── initramfs.sh    # assembly & optimization
│   │   └── kernel.sh       # kernel building
│   ├── build.sh            # main orchestrator
│   └── clean.sh            # cleanup script
├── initramfs/              # build output (generated)
├── components/             # component sources (generated)
├── kernel/                 # kernel source (generated)
└── dist/                   # final artifacts (generated)
```

## Configuration

### Component Sources (config/sources.conf)

Define components to download and build:

```bash
# Format: TYPE:NAME:URL:VERSION:BUILD_FUNCTION[:EXTRA_OPTIONS]

# Git repositories (Rust components with musl)
git:zinit:https://github.com/threefoldtech/zinit:master:build_zinit
git:mycelium:https://github.com/threefoldtech/mycelium:0.6.1:build_mycelium
git:rfs:https://github.com/threefoldtech/rfs:development:build_rfs

# Pre-built releases
release:corex:https://github.com/threefoldtech/corex/releases/download/2.1.4/corex-2.1.4-amd64-linux-static:2.1.4:install_corex:rename=corex
```

### Package List (config/packages.list)

Alpine packages to install (NO OpenRC):

```bash
# Core system
alpine-baselayout
busybox
musl

# Hardware detection & modules
eudev
eudev-hwids
kmod

# Networking
iproute2
ethtool
dhcpcd

# Filesystems
btrfs-progs
dosfstools

# Security & SSH
haveged
openssh-server

# Tools
zellij
tcpdump
bmon
```

### Module Loading (config/modules.conf)

2-stage hardware module loading:

```bash
# Stage 1: Critical boot modules
stage1:virtio_net
stage1:virtio_scsi
stage1:virtio_blk
stage1:e1000
stage1:e1000e

# Stage 2: Extended hardware support
stage2:igb
stage2:ixgbe
stage2:i40e
stage2:r8169
stage2:bnx2
stage2:bnx2x
```

### zinit Configuration (config/zinit/)

#### Main config (config/zinit/zinit.conf)
```yaml
log_level: debug
init:
  - stage1-modules
  - stage2-modules
  - networking
  - services
```

#### Service definitions (config/zinit/services/)

Services are migrated from existing `configs/zinit/` directory with proper initialization order.

## Build Process

### Phase 1: Environment Setup
1. Create build directories
2. Install build dependencies
3. Setup Rust musl target

### Phase 2: Alpine Base
1. Download Alpine 3.22 miniroot
2. Extract to initramfs directory
3. Install packages from `config/packages.list`
4. **NO OpenRC installation**

### Phase 3: Component Building
1. Parse `config/sources.conf`
2. Download/clone sources to `components/`
3. Build Rust components with musl:
   - **zinit**: Standard cargo build
   - **rfs**: Standard cargo build  
   - **mycelium**: Build in `myceliumd/` subdirectory
   - **zosstorage**: Build from the storage orchestration component for Zero-OS
4. Install binaries to initramfs

### Phase 4: System Configuration
1. Replace `/sbin/init` with zinit
2. Copy zinit configuration
3. Setup 2-stage module loading
4. Configure system services

### Phase 5: Optimization
1. **Aggressive cleanup**:
   - Remove docs, man pages, locales
   - Remove headers, development files
   - Remove APK cache
2. **Binary optimization**:
   - Strip all executables and libraries
   - UPX compress all binaries
3. **Size verification**

### Phase 6: Packaging
1. Create `initramfs.cpio.xz` with XZ compression
2. Build kernel with embedded initramfs
3. Generate `vmlinuz.efi`

## Testing

### QEMU Testing
```bash
# Boot test with QEMU (default)
./runit.sh

# With custom parameters
./runit.sh --hypervisor qemu --memory 2048 --disks 3
```

### cloud-hypervisor Testing
```bash
# Boot test with cloud-hypervisor
./runit.sh --hypervisor ch

# With disk reset
./runit.sh --hypervisor ch --reset --disks 5
```

### Advanced Options
```bash
# See all options
./runit.sh --help

# Custom disk size and bridge
./runit.sh --disk-size 20G --bridge zosbr --disks 4

# Environment variables
HYPERVISOR=ch NUM_DISKS=5 ./runit.sh
```

## Size Optimization

The build system achieves minimal size through:

### Package Selection
- Minimal Alpine packages (~50MB target)
- No OpenRC or systemd
- Essential tools only

### Binary Optimization
- **strip**: Remove debug symbols
- **UPX**: Maximum compression
- **musl static linking**: No runtime dependencies

### Filesystem Cleanup
- Remove documentation
- Remove locales (except C)
- Remove development headers
- Remove package manager cache

### Expected Sizes
- **Base Alpine**: ~5MB
- **With packages**: ~25MB  
- **With components**: ~40MB
- **After optimization**: ~15-20MB
- **Final initramfs.cpio.xz**: ~8-12MB

## GitHub Actions Integration

See [GITHUB_ACTIONS.md](GITHUB_ACTIONS.md) for complete CI/CD setup.

### Basic Workflow
```yaml
name: Build Zero OS
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - name: Setup rootless containers
      run: |
        echo "runner:100000:65536" | sudo tee -a /etc/subuid
        echo "runner:100000:65536" | sudo tee -a /etc/subgid
    - name: Build
      run: ./scripts/build.sh
    - name: Test
      run: ./runit.sh --hypervisor qemu
```

## Advanced Usage

### Custom Components

Add custom components to `config/sources.conf`:

```bash
# Custom Git component
git:myapp:https://github.com/user/myapp:v1.0:build_myapp

# Custom release
release:mytool:https://releases.example.com/mytool-x86_64:v2.0:install_mytool
```

Implement build function in `scripts/lib/components.sh`:

```bash
function build_myapp() {
    local name="$1"
    local component_dir="$2"
    
    # Custom build logic
    export RUSTFLAGS="-C target-feature=+crt-static"
    cargo build --release --target x86_64-unknown-linux-musl
    
    # Install binary
    cp target/x86_64-unknown-linux-musl/release/myapp "${INSTALL_DIR}/usr/bin/"
}
```

### Container Builds

Build in isolated container:

```bash
# Build container image
podman build -t zero-os-builder .

# Run build in container
podman run --rm \
  -v $(pwd):/workspace \
  -w /workspace \
  zero-os-builder \
  ./scripts/build.sh
```

### Cross-Platform Support (totally untestd)

The build system supports multiple architectures:

```bash
# Build for different targets
export RUST_TARGET="aarch64-unknown-linux-musl"
export ALPINE_ARCH="aarch64"
./scripts/build.sh
```

## Troubleshooting

### Common Issues

#### Build Failures
```bash
# Clean and retry
./scripts/clean.sh
./scripts/build.sh

# Check dependencies
./scripts/build.sh --check-deps
```

#### Container Issues
```bash
# Verify rootless setup
podman system info

# Reset user namespace
podman system reset
```

#### Rust Build Issues
```bash
# Verify musl target
rustup target list --installed | grep musl

# Add if missing
rustup target add x86_64-unknown-linux-musl
```

### Debug Mode
```bash
# Enable verbose output
export DEBUG=1
./scripts/build.sh
```

## Contributing

1. **Fork** the repository
2. **Create** feature branch
3. **Test** thoroughly with both QEMU and cloud-hypervisor
4. **Ensure** size optimization targets are met
5. **Submit** pull request with detailed description

### Development Workflow

```bash
# Setup development environment
./scripts/dev-container.sh start

# Run incremental build
./scripts/build.sh

# Test with QEMU
./runit.sh --hypervisor qemu

# Test with cloud-hypervisor
./runit.sh --hypervisor ch
```

## License

[License information]

## Support

- **Issues**: GitHub Issues
- **Documentation**: See `docs/` directory
- **Examples**: See `examples/` directory

## Related Projects

- [ThreeFold Zero OS](https://github.com/threefoldtech/zos)
- [zinit](https://github.com/threefoldtech/zinit)
- [Mycelium](https://github.com/threefoldtech/mycelium)
- [RFS](https://github.com/threefoldtech/rfs)