zosstorage/docs/SPECS.md

zosstorage Detailed Specifications

This document finalizes core specifications required before code skeleton implementation. It complements docs/ARCHITECTURE.md and docs/SCHEMA.md, and references the API declarations listed in docs/API.md.

Linked modules and functions

• Logging module: src/logging/mod.rs
  • fn init_logging(opts: &LogOptions) -> Result<()>
• Report module: src/report/state.rs
  • const REPORT_VERSION: &str
  • fn build_report(...) -> StateReport
  • fn write_report(report: &StateReport) -> Result<()>
• Device module: src/device/discovery.rs
  • fn discover(filter: &DeviceFilter) -> Result<Vec>
• Partitioning module: src/partition/plan.rs
  • fn plan_partitions(disks: &[Disk], cfg: &Config) -> Result
  • fn apply_partitions(plan: &PartitionPlan) -> Result<Vec>
• Filesystems module: src/fs/plan.rs
  • fn plan_filesystems(disks: &[Disk], parts: &[PartitionResult], cfg: &Config) -> Result
  • fn make_filesystems(plan: &FsPlan) -> Result<Vec>
• Mount module: src/mount/ops.rs
  • fn plan_mounts(fs_results: &[FsResult], cfg: &Config) -> Result
  • fn apply_mounts(plan: &MountPlan) -> Result<Vec>
  • fn maybe_write_fstab(mounts: &[MountResult], cfg: &Config) -> Result<()>
• Idempotency module: src/idempotency/mod.rs
  • fn detect_existing_state() -> Result<Option>
  • fn is_empty_disk(disk: &Disk) -> Result
• CLI module: src/cli/args.rs
  • fn from_args() -> Cli
• Orchestrator: src/orchestrator/run.rs
  • fn run(ctx: &Context) -> Result<()>

---

1. Logging and tracing

Goals

• Structured, low-noise logging compatible with initramfs.
• Defaults to stderr. Optional file at /run/zosstorage/zosstorage.log controlled by config or CLI.

Configuration

• Levels: error, warn, info, debug (default info).
• Propagation: single global initialization via fn init_logging. Subsequent calls must be no-ops.

Implementation notes

• Use tracing and tracing-subscriber.
• Format: compact, with fields level, target, message, and optional module path. Avoid timestamps if writing to stderr in initramfs; include timestamps when logging to file.

Example behavior

• CLI --log-level debug sets level to debug.
• CLI --log-to-file or config.logging.to_file true enables file layer at /run/zosstorage/zosstorage.log.

---

2. JSON state report schema v1

Location

• Default output: /run/zosstorage/state.json

Versioning

• Include a top-level string field version equal to REPORT_VERSION. Start with v1.

Schema example

{
  "version": "v1",
  "timestamp": "2025-09-25T12:00:00Z",
  "status": "success",
  "disks": [
    {
      "path": "/dev/nvme0n1",
      "size_bytes": 40007973632,
      "rotational": false,
      "model": "QEMU NVMe Ctrl",
      "serial": "nvme-1234",
      "selected": true,
      "roles": ["esp", "data"]
    }
  ],
  "partitions": [
    {
      "disk": "/dev/nvme0n1",
      "number": 1,
      "role": "bios_boot",
      "gpt_name": "zosboot",
      "uuid": "11111111-1111-1111-1111-111111111111",
      "start_mib": 1,
      "size_mib": 1
    },
    {
      "disk": "/dev/nvme0n1",
      "number": 2,
      "role": "esp",
      "gpt_name": "zosboot",
      "uuid": "22222222-2222-2222-2222-222222222222",
      "start_mib": 2,
      "size_mib": 512,
      "fs_label": "ZOSBOOT"
    },
    {
      "disk": "/dev/nvme0n1",
      "number": 3,
      "role": "data",
      "gpt_name": "zosdata",
      "uuid": "33333333-3333-3333-3333-333333333333",
      "start_mib": 514,
      "size_mib": 39000
    }
  ],
  "filesystems": [
    {
      "kind": "vfat",
      "device": "/dev/nvme0n1p2",
      "uuid": "AAAA-BBBB",
      "label": "ZOSBOOT",
      "mountpoint": null
    },
    {
      "kind": "btrfs",
      "device": "/dev/nvme0n1p3",
      "uuid": "aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff",
      "label": "ZOSDATA",
      "mountpoint": "/var/cache/aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff"
    }
  ],
  "mounts": [
    {
      "source": "/dev/nvme0n1p3",
      "target": "/var/cache/aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff",
      "fstype": "btrfs",
      "options": "defaults,ssd,compress=zstd:3"
    }
  ]
}

Notes

• UUID formats follow tool output: VFAT UUID short form allowed.
• Status values: success, already_provisioned, error. On error, add error field with reason.

---

3. Device discovery and filtering rules

Default include patterns

• ^/dev/sd\w+$
• ^/dev/nvme\w+n\d+$
• ^/dev/vd\w+$

Default exclude patterns

• ^/dev/ram\d+$
• ^/dev/zram\d+$
• ^/dev/loop\d+$
• ^/dev/fd\d+$

Selection policy

• Compile include and exclude regex into DeviceFilter.
• Enumerate device candidates and apply:
  • Must match at least one include.
  • Must not match any exclude.
  • Must be larger than min_size_gib (default 10).
• Probing
  • Gather size, rotational flag, model, serial when available.
  • Expose via struct Disk.

No eligible disks

• Return a specific error variant in enum Error.

---

4. Partitioning plan

Constraints

• GPT only; enforce 1 MiB alignment.
• Abort immediately if any target disk is non-empty when require_empty_disks is true.

Layout defaults

• BIOS boot: 1 MiB first, role BiosBoot, GPT name zosboot, no filesystem.
• ESP: 512 MiB FAT32, GPT name zosboot; filesystem label ZOSBOOT.
• Data: remainder, GPT name zosdata.
• Cache partitions (only in ssd_hdd_bcachefs): GPT name zoscache on SSD.

Per-topology specifics

• single: All roles on the single disk.
• dual_independent: Each disk gets BIOS boot + ESP + data.
• ssd_hdd_bcachefs: SSD gets BIOS boot + ESP + zoscache, HDD gets BIOS boot + ESP + zosdata.

Safety checks

• Ensure unique partition UUIDs.
• Verify no pre-existing partitions or signatures. Use blkid or similar via run_cmd_capture.
• After partition creation, run udev settle via udev_settle.

Application

• Utilize sgdisk helpers in apply_partitions.

---

5. Filesystem provisioning strategies

Kinds

• Vfat for ESP, label ZOSBOOT.
• Btrfs for data on single and dual_independent.
• Bcachefs for ssd_hdd_bcachefs (SSD cache, HDD backing).
• All data filesystems use label ZOSDATA.

Defaults

• btrfs: compression zstd:3, raid_profile none unless explicitly set to raid1 in btrfs_raid1 mode.
• bcachefs: cache_mode promote, compression zstd, checksum crc32c.
• vfat: ESP label ZOSBOOT.

Planning and execution

• Decide mapping of PartitionResult to FsSpec in plan_filesystems.
• Create filesystems in make_filesystems through wrapped mkfs tools.
• Capture resulting identifiers (fs uuid, label) in FsResult.

---

6. Mount scheme and fstab policy

Scheme

• per_uuid under /var/cache: directories named as filesystem UUIDs.

Mount options

• btrfs: ssd when non-rotational underlying device, compress from config, defaults otherwise.
• vfat: defaults, utf8.

fstab

• Disabled by default.
• When enabled, maybe_write_fstab writes deterministic entries sorted by target path.

---

7. Idempotency detection

Signals for already-provisioned system

• Expected GPT names found: zosboot, zosdata, and zoscache when applicable.
• Filesystems with labels ZOSBOOT for ESP and ZOSDATA for all data filesystems.
• When consistent with selected topology, detect_existing_state returns a StateReport and orchestrator exits success without changes.

Disk emptiness

• is_empty_disk checks for absence of partitions and FS signatures before any modification.

---

8. CLI flags and help text outline

Flags mirrored by struct Cli parsed via from_args

• --config PATH
• --log-level LEVEL error | warn | info | debug
• --log-to-file
• --fstab enable fstab generation
• --force present but returns unimplemented error

Kernel cmdline

• zosstorage.config= accepts a path or file: URI or data: URL as described in docs/SCHEMA.md.

Help text sections

• NAME, SYNOPSIS, DESCRIPTION
• CONFIG PRECEDENCE
• TOPOLOGIES: single, dual_independent, ssd_hdd_bcachefs, btrfs_raid1
• SAFETY AND IDEMPOTENCY
• REPORTS
• EXIT CODES: 0 success or already_provisioned, non-zero on error

---

9. Integration testing plan (QEMU KVM)

Scenarios to scaffold in tests/

• Single disk 40 GiB virtio: validates single topology end-to-end smoke.
• Dual NVMe 40 GiB each: validates dual_independent topology.
• SSD NVMe + HDD virtio: validates ssd_hdd_bcachefs topology.
• Negative: no eligible disks, or non-empty disk should abort.

Test strategy

• Tests will be staged as integration test scaffolds that compile and document manual steps or automated harness placeholders.
• Mocks
  • Provide a test DeviceProvider to simulate discovery when running without QEMU.
  • Wrap external tools via utility trait to enable command capture in dry-runs.

Artifacts to validate

• Presence of expected partition GPT names.
• Filesystems created with correct labels.
• Mountpoints under /var/cache/ when running in a VM.
• JSON report validates against v1 schema.

---

10. Documentation deliverables

• README.md
  • Overview, quickstart, config precedence, example YAML, topology walkthroughs, usage, report format, safety, limitations, roadmap.
• docs/ARCHITECTURE.md
• docs/SCHEMA.md
• docs/API.md
• Release notes template and CHANGELOG policy.

---

11. Build and packaging for static musl and Alpine initramfs

Rust build

• Target: x86_64-unknown-linux-musl
• Avoid glibc-only dependencies.

Binary constraints

• No reliance on services; suitable for busybox initramfs.

Embedding in initramfs

• Place the statically linked binary in initramfs.
• Ensure required external tools (sgdisk, blkid, mkfs.vfat, mkfs.btrfs, mkfs.bcachefs, udevadm) are present in the same initramfs environment.

Runtime notes

• Minimal stdout use; all status via tracing.
• Exit codes:
  • 0 on success and on already provisioned.
  • Non-zero on any validation or execution error.

---

12. Open items carried forward

• Confirm exact BIOS boot partition requirements across target platforms; currently set to 1 MiB first.
• Finalize btrfs and bcachefs tuning defaults after stakeholder review.
• Decide if/when to enable fstab generation by default in future.
• Allow removable media policies and additional device classes in configuration.

---

13. Next steps

• Proceed to code-mode to scaffold modules and types as declared in docs/API.md.
• Add dependencies via cargo add as listed in docs/ARCHITECTURE.md.
• Implement bodies with todo!() placeholders and exhaustive doc comments before enabling functional behavior.

End of specifications.