[infra] add --debug install path for fast dev iteration (5-10× faster than --release) #189

New issue

Open

opened 2026-05-01 19:40:15 +00:00 by mik-tf · 0 comments

mik-tf commented

2026-05-01 19:40:15 +00:00

Owner

Summary

Add a --debug (or --dev) mode to service_X install (and service_install_all) that builds with cargo's dev profile instead of --release. Production-bound deploys keep using --release (current default); developer iteration uses --debug. 5-10× faster dev cycles because dev profile drops:

lto = true (link-time optimization, single-threaded link)
codegen-units = 1 (per-crate codegen on one thread)
opt-level = 3 (aggressive optimization passes)

These flags are correct for production binaries (smaller, faster runtime), but they're the dominant cost in build time. For dev iteration ("change one line, rebuild, restart, test"), the runtime perf cost of dev profile is irrelevant — what matters is rebuild speed.

Today's behaviour

Every service_X install builds release-profile, even when the operator is iterating on a single line of code in their own service. Per-service rebuild cycles take 30s-5min depending on the service. Over a day of iteration that's 30-60 cumulative minutes spent waiting for codegen.

Proposed

Each service_X.nu install command grows a --debug flag (mutually exclusive with --release):

export def install [
    --root(-r)
    --update(-u)
    --reset
    --release  # Build optimized release binary (production deploy)
    --debug    # Build dev-profile binary (faster iteration; bigger binary, slower runtime)
] {
    # Validate exactly one of --release / --debug (or neither = current default of release)
    let profile = if $debug { "debug" } else if $release { "release" } else { "release" }
    svc_install $SVX_SERVICE_NAME $SVX_FORGE_LOC $SVX_BINARIES $root $update $reset $profile
}

svc_install in lib.nu accepts the profile string and invokes:

let cargo_profile_arg = if $profile == "debug" { [] } else { ["--release"] }
^cargo build ...$cargo_profile_arg ...

Same structure for service_install_all and service_complete (forward --debug through).

Use cases

Dev iteration on a single service: service_X install --update --debug — binary swap in seconds, restart and test.
First-time setup on a fresh dev VM: service_install_all --update --debug — get the whole stack runnable in 5-10 min instead of 30-60 min.
CI smoke tests: dev profile is sufficient for "does it compile + does it boot."
Production deploys: keep --release (existing default); dev profile binaries are larger and runtime-slower.

Tradeoffs

Binary size: dev profile binaries are 5-10× larger. Disk usage on dev VMs grows. Worth a --clear-debug-bins cleanup helper.
Runtime perf: dev profile lacks inlining, vectorization, dead-code elimination. Fine for dev/test; not OK for prod (HTTP throughput, embedder vector math etc. would be visibly slower).
Mixed environments: if some services are debug and others release, dependencies are recompiled per-profile (cargo target dirs are profile-scoped). Disk grows. Workable.

Out of scope

Build profile customisation beyond release / debug (e.g. release-with-debuginfo, release-without-LTO). Add later if there's demand.
Pre-built artifacts (covered by hero_demo#54).

Cross-refs

hero_demo#54 — CI artifacts (the bigger structural win; this --debug path complements it for local iteration).
Build perf quick wins issue (sister issue I'm filing alongside this one): bumps -j default + adds sccache + makes nice/ionice opt-in. Independent of this --debug flag.

ROI estimate

For "developer changes one line in service_X and wants to test":

Today:                    cargo build --release: 30s - 5 min (depending on service)
After --debug option:     cargo build (dev):     3s - 30s
                          → 5-10× faster iteration

Multiplied across multiple iterations per day, this is the difference between "wait 5-10 min for each retry" and "iteration is instant."

## Summary Add a `--debug` (or `--dev`) mode to `service_X install` (and `service_install_all`) that builds with cargo's dev profile instead of `--release`. Production-bound deploys keep using `--release` (current default); developer iteration uses `--debug`. **5-10× faster dev cycles** because dev profile drops: - `lto = true` (link-time optimization, single-threaded link) - `codegen-units = 1` (per-crate codegen on one thread) - `opt-level = 3` (aggressive optimization passes) These flags are correct for production binaries (smaller, faster runtime), but they're the dominant cost in build time. For dev iteration ("change one line, rebuild, restart, test"), the runtime perf cost of dev profile is irrelevant — what matters is rebuild speed. ## Today's behaviour Every `service_X install` builds release-profile, even when the operator is iterating on a single line of code in their own service. Per-service rebuild cycles take 30s-5min depending on the service. Over a day of iteration that's 30-60 cumulative minutes spent waiting for codegen. ## Proposed Each `service_X.nu` install command grows a `--debug` flag (mutually exclusive with `--release`): ```nu export def install [ --root(-r) --update(-u) --reset --release # Build optimized release binary (production deploy) --debug # Build dev-profile binary (faster iteration; bigger binary, slower runtime) ] { # Validate exactly one of --release / --debug (or neither = current default of release) let profile = if $debug { "debug" } else if $release { "release" } else { "release" } svc_install $SVX_SERVICE_NAME $SVX_FORGE_LOC $SVX_BINARIES $root $update $reset $profile } ``` `svc_install` in lib.nu accepts the profile string and invokes: ```nu let cargo_profile_arg = if $profile == "debug" { [] } else { ["--release"] } ^cargo build ...$cargo_profile_arg ... ``` Same structure for `service_install_all` and `service_complete` (forward `--debug` through). ## Use cases - **Dev iteration on a single service**: `service_X install --update --debug` — binary swap in seconds, restart and test. - **First-time setup on a fresh dev VM**: `service_install_all --update --debug` — get the whole stack runnable in 5-10 min instead of 30-60 min. - **CI smoke tests**: dev profile is sufficient for "does it compile + does it boot." - **Production deploys**: keep `--release` (existing default); dev profile binaries are larger and runtime-slower. ## Tradeoffs - **Binary size**: dev profile binaries are 5-10× larger. Disk usage on dev VMs grows. Worth a `--clear-debug-bins` cleanup helper. - **Runtime perf**: dev profile lacks inlining, vectorization, dead-code elimination. Fine for dev/test; not OK for prod (HTTP throughput, embedder vector math etc. would be visibly slower). - **Mixed environments**: if some services are debug and others release, dependencies are recompiled per-profile (cargo target dirs are profile-scoped). Disk grows. Workable. ## Out of scope - Build profile customisation beyond `release` / `debug` (e.g. release-with-debuginfo, release-without-LTO). Add later if there's demand. - Pre-built artifacts (covered by [hero_demo#54](https://forge.ourworld.tf/lhumina_code/hero_demo/issues/54)). ## Cross-refs - [hero_demo#54](https://forge.ourworld.tf/lhumina_code/hero_demo/issues/54) — CI artifacts (the bigger structural win; this `--debug` path complements it for local iteration). - Build perf quick wins issue (sister issue I'm filing alongside this one): bumps `-j` default + adds sccache + makes nice/ionice opt-in. Independent of this `--debug` flag. ## ROI estimate For "developer changes one line in service_X and wants to test": ``` Today: cargo build --release: 30s - 5 min (depending on service) After --debug option: cargo build (dev): 3s - 30s → 5-10× faster iteration ``` Multiplied across multiple iterations per day, this is the difference between "wait 5-10 min for each retry" and "iteration is instant."