347 lines
13 KiB
Markdown
347 lines
13 KiB
Markdown
# zosstorage Detailed Specifications
|
|
|
|
This document finalizes core specifications required before code skeleton implementation. It complements [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) and [docs/SCHEMA.md](docs/SCHEMA.md), and references the API declarations listed in [docs/API.md](docs/API.md).
|
|
|
|
Linked modules and functions
|
|
- Logging module: [src/logging/mod.rs](../src/logging/mod.rs)
|
|
- [fn init_logging(opts: &LogOptions) -> Result<()>](../src/logging/mod.rs:1)
|
|
- Report module: [src/report/state.rs](../src/report/state.rs)
|
|
- [const REPORT_VERSION: &str](../src/report/state.rs:1)
|
|
- [fn build_report(...) -> StateReport](../src/report/state.rs:1)
|
|
- [fn write_report(report: &StateReport) -> Result<()>](../src/report/state.rs:1)
|
|
- Device module: [src/device/discovery.rs](../src/device/discovery.rs)
|
|
- [fn discover(filter: &DeviceFilter) -> Result<Vec<Disk>>](../src/device/discovery.rs:1)
|
|
- Partitioning module: [src/partition/plan.rs](../src/partition/plan.rs)
|
|
- [fn plan_partitions(disks: &[Disk], cfg: &Config) -> Result<PartitionPlan>](../src/partition/plan.rs:1)
|
|
- [fn apply_partitions(plan: &PartitionPlan) -> Result<Vec<PartitionResult>>](../src/partition/plan.rs:1)
|
|
- Filesystems module: [src/fs/plan.rs](../src/fs/plan.rs)
|
|
- [fn plan_filesystems(disks: &[Disk], parts: &[PartitionResult], cfg: &Config) -> Result<FsPlan>](../src/fs/plan.rs:1)
|
|
- [fn make_filesystems(plan: &FsPlan) -> Result<Vec<FsResult>>](../src/fs/plan.rs:1)
|
|
- Mount module: [src/mount/ops.rs](../src/mount/ops.rs)
|
|
- [fn plan_mounts(fs_results: &[FsResult], cfg: &Config) -> Result<MountPlan>](../src/mount/ops.rs:1)
|
|
- [fn apply_mounts(plan: &MountPlan) -> Result<Vec<MountResult>>](../src/mount/ops.rs:1)
|
|
- [fn maybe_write_fstab(mounts: &[MountResult], cfg: &Config) -> Result<()>](../src/mount/ops.rs:1)
|
|
- Idempotency module: [src/idempotency/mod.rs](../src/idempotency/mod.rs)
|
|
- [fn detect_existing_state() -> Result<Option<StateReport>>](../src/idempotency/mod.rs:1)
|
|
- [fn is_empty_disk(disk: &Disk) -> Result<bool>](../src/idempotency/mod.rs:1)
|
|
- CLI module: [src/cli/args.rs](../src/cli/args.rs)
|
|
- [fn from_args() -> Cli](../src/cli/args.rs:1)
|
|
- Orchestrator: [src/orchestrator/run.rs](../src/orchestrator/run.rs)
|
|
- [fn run(ctx: &Context) -> Result<()>](../src/orchestrator/run.rs:1)
|
|
|
|
---
|
|
|
|
## 1. Logging and tracing
|
|
|
|
Goals
|
|
- Structured, low-noise logging compatible with initramfs.
|
|
- Defaults to stderr. Optional file at /run/zosstorage/zosstorage.log controlled by config or CLI.
|
|
|
|
Configuration
|
|
- Levels: error, warn, info, debug (default info).
|
|
- Propagation: single global initialization via [fn init_logging](../src/logging/mod.rs:1). Subsequent calls must be no-ops.
|
|
|
|
Implementation notes
|
|
- Use tracing and tracing-subscriber.
|
|
- Format: compact, with fields level, target, message, and optional module path. Avoid timestamps if writing to stderr in initramfs; include timestamps when logging to file.
|
|
|
|
Example behavior
|
|
- CLI --log-level debug sets level to debug.
|
|
- CLI --log-to-file or config.logging.to_file true enables file layer at /run/zosstorage/zosstorage.log.
|
|
|
|
---
|
|
|
|
## 2. JSON state report schema v1
|
|
|
|
Location
|
|
- Default output: /run/zosstorage/state.json
|
|
|
|
Versioning
|
|
- Include a top-level string field version equal to [REPORT_VERSION](../src/report/state.rs:1). Start with v1.
|
|
|
|
Schema example
|
|
|
|
```json
|
|
{
|
|
"version": "v1",
|
|
"timestamp": "2025-09-25T12:00:00Z",
|
|
"status": "success",
|
|
"disks": [
|
|
{
|
|
"path": "/dev/nvme0n1",
|
|
"size_bytes": 40007973632,
|
|
"rotational": false,
|
|
"model": "QEMU NVMe Ctrl",
|
|
"serial": "nvme-1234",
|
|
"selected": true,
|
|
"roles": ["esp", "data"]
|
|
}
|
|
],
|
|
"partitions": [
|
|
{
|
|
"disk": "/dev/nvme0n1",
|
|
"number": 1,
|
|
"role": "bios_boot",
|
|
"gpt_name": "zosboot",
|
|
"uuid": "11111111-1111-1111-1111-111111111111",
|
|
"start_mib": 1,
|
|
"size_mib": 1
|
|
},
|
|
{
|
|
"disk": "/dev/nvme0n1",
|
|
"number": 2,
|
|
"role": "esp",
|
|
"gpt_name": "zosboot",
|
|
"uuid": "22222222-2222-2222-2222-222222222222",
|
|
"start_mib": 2,
|
|
"size_mib": 512,
|
|
"fs_label": "ZOSBOOT"
|
|
},
|
|
{
|
|
"disk": "/dev/nvme0n1",
|
|
"number": 3,
|
|
"role": "data",
|
|
"gpt_name": "zosdata",
|
|
"uuid": "33333333-3333-3333-3333-333333333333",
|
|
"start_mib": 514,
|
|
"size_mib": 39000
|
|
}
|
|
],
|
|
"filesystems": [
|
|
{
|
|
"kind": "vfat",
|
|
"device": "/dev/nvme0n1p2",
|
|
"uuid": "AAAA-BBBB",
|
|
"label": "ZOSBOOT",
|
|
"mountpoint": null
|
|
},
|
|
{
|
|
"kind": "btrfs",
|
|
"device": "/dev/nvme0n1p3",
|
|
"uuid": "aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff",
|
|
"label": "ZOSDATA",
|
|
"mountpoint": "/var/cache/aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff"
|
|
}
|
|
],
|
|
"mounts": [
|
|
{
|
|
"source": "/dev/nvme0n1p3",
|
|
"target": "/var/cache/aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff",
|
|
"fstype": "btrfs",
|
|
"options": "defaults,ssd,compress=zstd:3"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
Notes
|
|
- UUID formats follow tool output: VFAT UUID short form allowed.
|
|
- Status values: success, already_provisioned, error. On error, add error field with reason.
|
|
|
|
---
|
|
|
|
## 3. Device discovery and filtering rules
|
|
|
|
Default include patterns
|
|
- ^/dev/sd\\w+$
|
|
- ^/dev/nvme\\w+n\\d+$
|
|
- ^/dev/vd\\w+$
|
|
|
|
Default exclude patterns
|
|
- ^/dev/ram\\d+$
|
|
- ^/dev/zram\\d+$
|
|
- ^/dev/loop\\d+$
|
|
- ^/dev/fd\\d+$
|
|
|
|
Selection policy
|
|
- Compile include and exclude regex into [DeviceFilter](../src/device/discovery.rs).
|
|
- Enumerate device candidates and apply:
|
|
- Must match at least one include.
|
|
- Must not match any exclude.
|
|
- Must be larger than min_size_gib (default 10).
|
|
- Probing
|
|
- Gather size, rotational flag, model, serial when available.
|
|
- Expose via [struct Disk](../src/device/discovery.rs:1).
|
|
|
|
No eligible disks
|
|
- Return a specific error variant in [enum Error](../src/errors.rs:1).
|
|
|
|
---
|
|
|
|
## 4. Partitioning plan
|
|
|
|
Constraints
|
|
- GPT only; enforce 1 MiB alignment.
|
|
- Abort immediately if any target disk is non-empty when require_empty_disks is true.
|
|
|
|
Layout defaults
|
|
- BIOS boot: 1 MiB first, role BiosBoot, GPT name zosboot, no filesystem.
|
|
- ESP: 512 MiB FAT32, GPT name zosboot; filesystem label ZOSBOOT.
|
|
- Data: remainder, GPT name zosdata.
|
|
- Cache partitions (only in ssd_hdd_bcachefs): GPT name zoscache on SSD.
|
|
|
|
Per-topology specifics
|
|
- btrfs_single: All roles on the single disk; data formatted as btrfs.
|
|
- bcachefs_single: All roles on the single disk; data formatted as bcachefs.
|
|
- dual_independent: On each eligible disk (one or more), create BIOS boot (if applicable), ESP, and data.
|
|
- bcachefs_2copy: Create data partitions on two or more disks; later formatted as one multi-device bcachefs spanning all data partitions.
|
|
- ssd_hdd_bcachefs: SSD gets BIOS boot + ESP + zoscache; HDD gets BIOS boot + ESP + zosdata; combined later into one bcachefs.
|
|
- btrfs_raid1: Two disks minimum; data partitions mirrored via btrfs RAID1.
|
|
|
|
Safety checks
|
|
- Ensure unique partition UUIDs.
|
|
- Verify no pre-existing partitions or signatures. Use blkid or similar via [run_cmd_capture](../src/util/mod.rs:1).
|
|
- After partition creation, run udev settle via [udev_settle](../src/util/mod.rs:1).
|
|
|
|
Application
|
|
- Utilize sgdisk helpers in [apply_partitions](../src/partition/plan.rs:1).
|
|
|
|
---
|
|
|
|
## 5. Filesystem provisioning strategies
|
|
|
|
Kinds
|
|
- Vfat for ESP, label ZOSBOOT.
|
|
- Btrfs for data in btrfs_single, dual_independent, and btrfs_raid1 (with RAID1 profile).
|
|
- Bcachefs for data in bcachefs_single, ssd_hdd_bcachefs (SSD cache + HDD backing), and bcachefs_2copy (multi-device).
|
|
- All data filesystems use label ZOSDATA.
|
|
|
|
Defaults
|
|
- btrfs: compression zstd:3, raid_profile none unless explicitly set; for btrfs_raid1 use -m raid1 -d raid1.
|
|
- bcachefs: cache_mode promote, compression zstd, checksum crc32c; for bcachefs_2copy use `--replicas=2` (data and metadata).
|
|
- vfat: ESP label ZOSBOOT.
|
|
|
|
Planning and execution
|
|
- Decide mapping of [PartitionResult](../src/partition/plan.rs:1) to [FsSpec](../src/fs/plan.rs:1) in [plan_filesystems](../src/fs/plan.rs:1).
|
|
- Create filesystems in [make_filesystems](../src/fs/plan.rs:1) through wrapped mkfs tools.
|
|
- Capture resulting identifiers (fs uuid, label) in [FsResult](../src/fs/plan.rs:1).
|
|
|
|
---
|
|
|
|
## 6. Mount scheme and fstab policy
|
|
|
|
Runtime root mounts (all data filesystems)
|
|
- Each data filesystem is root-mounted at `/var/mounts/{UUID}` (runtime only).
|
|
- btrfs root mount options: `rw,noatime,subvolid=5`
|
|
- bcachefs root mount options: `rw,noatime`
|
|
|
|
Final subvolume/subdir mounts (from the primary data filesystem)
|
|
- Create or ensure subvolumes named: `system`, `etc`, `modules`, `vm-meta`
|
|
- Mount targets: `/var/cache/system`, `/var/cache/etc`, `/var/cache/modules`, `/var/cache/vm-meta`
|
|
- btrfs options: `-o rw,noatime,subvol={name}`
|
|
- bcachefs options: `-o rw,noatime,X-mount.subdir={name}`
|
|
|
|
fstab policy
|
|
- Disabled by default.
|
|
- When enabled, [maybe_write_fstab](../src/mount/ops.rs:1) writes only the four final subvolume/subdir entries using `UUID=` sources, in deterministic target order. Root mounts under `/var/mounts` are excluded.
|
|
|
|
---
|
|
|
|
## 7. Idempotency detection
|
|
|
|
Signals for already-provisioned system
|
|
- Expected GPT names found: zosboot, zosdata, and zoscache when applicable.
|
|
- Filesystems with labels ZOSBOOT for ESP and ZOSDATA for all data filesystems.
|
|
- When consistent with selected topology, [detect_existing_state](../src/idempotency/mod.rs:1) returns a StateReport and orchestrator exits success without changes.
|
|
|
|
Disk emptiness
|
|
- [is_empty_disk](../src/idempotency/mod.rs:1) checks for absence of partitions and FS signatures before any modification.
|
|
|
|
---
|
|
|
|
## 8. CLI flags and help text outline
|
|
|
|
Flags mirrored by [struct Cli](../src/cli/args.rs:1) parsed via [from_args](../src/cli/args.rs:1)
|
|
- --config PATH
|
|
- --log-level LEVEL error | warn | info | debug
|
|
- --log-to-file
|
|
- --fstab enable fstab generation
|
|
- --show print preview JSON to stdout (non-destructive)
|
|
- --report PATH write preview JSON to file (non-destructive)
|
|
- --apply perform partitioning, filesystem creation, and mounts (DESTRUCTIVE)
|
|
- --force present but returns unimplemented error
|
|
|
|
Kernel cmdline
|
|
- zosstorage.config= accepts a path or file: URI or data: URL as described in [docs/SCHEMA.md](docs/SCHEMA.md).
|
|
|
|
Help text sections
|
|
- NAME, SYNOPSIS, DESCRIPTION
|
|
- CONFIG PRECEDENCE
|
|
- TOPOLOGIES: btrfs_single, bcachefs_single, dual_independent, bcachefs_2copy, ssd_hdd_bcachefs, btrfs_raid1
|
|
- SAFETY AND IDEMPOTENCY
|
|
- REPORTS
|
|
- EXIT CODES: 0 success or already_provisioned, non-zero on error
|
|
|
|
---
|
|
|
|
## 9. Integration testing plan (QEMU KVM)
|
|
|
|
Scenarios to scaffold in [tests/](tests/)
|
|
- Single disk 40 GiB virtio: validates btrfs_single topology end-to-end smoke.
|
|
- Dual NVMe 40 GiB each: validates dual_independent topology (independent btrfs per disk).
|
|
- SSD NVMe + HDD virtio: validates ssd_hdd_bcachefs topology (bcachefs with SSD cache/promote, HDD backing).
|
|
- Three disks: validates bcachefs_2copy across data partitions using `--replicas=2`.
|
|
- Negative: no eligible disks, or non-empty disk should abort.
|
|
|
|
Test strategy
|
|
- Tests will be staged as integration test scaffolds that compile and document manual steps or automated harness placeholders.
|
|
- Mocks
|
|
- Provide a test DeviceProvider to simulate discovery when running without QEMU.
|
|
- Wrap external tools via utility trait to enable command capture in dry-runs.
|
|
|
|
Artifacts to validate
|
|
- Presence of expected partition GPT names.
|
|
- Filesystems created with correct labels.
|
|
- Runtime root mounts under `/var/mounts/{UUID}` and final subvolume targets at `/var/cache/{system,etc,modules,vm-meta}`.
|
|
- JSON report validates against v1 schema.
|
|
|
|
---
|
|
|
|
## 10. Documentation deliverables
|
|
|
|
- [README.md](README.md)
|
|
- Overview, quickstart, config precedence, example YAML, topology walkthroughs, usage, report format, safety, limitations, roadmap.
|
|
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
|
|
- [docs/SCHEMA.md](docs/SCHEMA.md)
|
|
- [docs/API.md](docs/API.md)
|
|
- Release notes template and CHANGELOG policy.
|
|
|
|
---
|
|
|
|
## 11. Build and packaging for static musl and Alpine initramfs
|
|
|
|
Rust build
|
|
- Target: x86_64-unknown-linux-musl
|
|
- Avoid glibc-only dependencies.
|
|
|
|
Binary constraints
|
|
- No reliance on services; suitable for busybox initramfs.
|
|
|
|
Embedding in initramfs
|
|
- Place the statically linked binary in initramfs.
|
|
- Ensure required external tools (sgdisk, blkid, mkfs.vfat, mkfs.btrfs, mkfs.bcachefs, udevadm) are present in the same initramfs environment.
|
|
|
|
Runtime notes
|
|
- Minimal stdout use; all status via tracing.
|
|
- Exit codes:
|
|
- 0 on success and on already provisioned.
|
|
- Non-zero on any validation or execution error.
|
|
|
|
---
|
|
|
|
## 12. Open items carried forward
|
|
|
|
- Confirm exact BIOS boot partition requirements across target platforms; currently set to 1 MiB first.
|
|
- Finalize btrfs and bcachefs tuning defaults after stakeholder review.
|
|
- Decide if/when to enable fstab generation by default in future.
|
|
- Allow removable media policies and additional device classes in configuration.
|
|
|
|
---
|
|
|
|
## 13. Next steps
|
|
|
|
- Proceed to code-mode to scaffold modules and types as declared in [docs/API.md](docs/API.md).
|
|
- Add dependencies via cargo add as listed in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
|
|
- Implement bodies with todo!() placeholders and exhaustive doc comments before enabling functional behavior.
|
|
|
|
End of specifications. |