diff --git a/.gitea/workflows/README.md b/.gitea/workflows/README.md new file mode 100644 index 0000000..66fed1b --- /dev/null +++ b/.gitea/workflows/README.md @@ -0,0 +1,391 @@ +# Gitea Actions Workflows Documentation + +This directory contains the CI/CD workflows for the Horus project using Gitea Actions. + +## Overview + +The Horus project uses two main workflows: + +1. **[ci.yml](./ci.yml)** - Continuous Integration workflow +2. **[release.yml](./release.yml)** - Release automation workflow + +## Workflow Files + +### ci.yml - Continuous Integration + +**Purpose**: Automatically build, test, and validate code quality on every push and pull request. + +**Triggers**: +- Push to any branch +- Pull request events (opened, synchronized, reopened) + +**What it does**: +1. Sets up Rust toolchain +2. Caches dependencies for faster builds +3. Runs code quality checks (check, test, clippy, fmt) +4. Builds all 7 binaries in release mode +5. Uploads binaries as artifacts + +**Duration**: ~5-15 minutes (first run), ~2-5 minutes (cached runs) + +**Artifacts**: Binaries are stored for 7 days and can be downloaded from the Actions tab + +--- + +### release.yml - Release Automation + +**Purpose**: Automatically create GitHub-style releases with downloadable binaries when version tags are pushed. + +**Triggers**: +- Tags matching `v*.*.*` pattern (e.g., `v1.0.0`, `v2.1.3`) + +**What it does**: +1. Builds optimized release binaries +2. Strips debug symbols to reduce size +3. Packages each binary as a tarball +4. Generates SHA256 checksums +5. Creates a Gitea release with all artifacts attached + +**Duration**: ~5-10 minutes + +**Artifacts**: Permanently attached to the release + +--- + +## Binaries Built + +Both workflows build the following 7 binaries: + +| Binary Name | Description | +|-------------|-------------| +| `supervisor` | Hero Supervisor service | +| `coordinator` | Hero Coordinator service | +| `horus` | Horus main binary | +| `osiris` | Osiris server | +| `herorunner` | Hero runner | +| `runner_osiris` | Osiris runner | +| `runner_sal` | SAL runner | + +--- + +## Usage Guide + +### Testing Code Changes + +Every time you push code or create a pull request, the CI workflow automatically runs: + +```bash +# Make your changes +git add . +git commit -m "Your changes" +git push origin your-branch + +# Or create a pull request +# The CI workflow will run automatically +``` + +**Check Results**: +1. Go to your Gitea repository +2. Click on the **Actions** tab +3. Find your workflow run +4. Click to see detailed logs + +--- + +### Creating a Release + +To create a new release with binaries: + +```bash +# 1. Ensure your code is ready for release +# 2. Create a version tag (use semantic versioning) +git tag v1.0.0 + +# 3. Push the tag +git push origin v1.0.0 + +# 4. The release workflow will automatically: +# - Build all binaries +# - Create a release +# - Attach binaries and checksums +``` + +**View Release**: +1. Go to your Gitea repository +2. Click on the **Releases** tab +3. Your new release will be listed with downloadable artifacts + +--- + +### Downloading Release Binaries + +Users can download binaries from releases: + +```bash +# Download a specific binary +wget https://git.ourworld.tf/peternashaat/horus/releases/download/v1.0.0/supervisor-v1.0.0-linux-x86_64.tar.gz + +# Extract +tar -xzf supervisor-v1.0.0-linux-x86_64.tar.gz + +# Make executable +chmod +x supervisor + +# Optionally move to system path +sudo mv supervisor /usr/local/bin/ + +# Verify it works +supervisor --help +``` + +**Verify Integrity**: +```bash +# Download checksums +wget https://git.ourworld.tf/peternashaat/horus/releases/download/v1.0.0/checksums.txt + +# Verify a binary +sha256sum -c checksums.txt +``` + +--- + +## Workflow Requirements + +### Runner Configuration + +Your Gitea Actions runner must be configured with these labels: +- `ubuntu-latest` (recommended) +- `ubuntu-22.04` (alternative) +- `ubuntu-20.04` (alternative) + +### Permissions + +The workflows require: +- Read access to repository code +- Write access to create releases (for release.yml) +- Access to `GITHUB_TOKEN` secret (automatically provided by Gitea) + +### Dependencies + +The workflows automatically install: +- Rust stable toolchain +- rustfmt (code formatter) +- clippy (linter) + +No manual setup required! + +--- + +## Caching Strategy + +The CI workflow uses three levels of caching to speed up builds: + +1. **Cargo Registry Cache** - Downloaded crate metadata +2. **Cargo Index Cache** - Git index of crates.io +3. **Build Cache** - Compiled dependencies + +**Benefits**: +- First build: ~10-15 minutes +- Cached builds: ~2-5 minutes +- Saves bandwidth and runner resources + +--- + +## Troubleshooting + +### CI Workflow Fails + +**Check these common issues**: + +1. **Compilation Errors** + - Review the "Check code" step logs + - Fix Rust compilation errors locally first + +2. **Test Failures** + - Review the "Run tests" step logs + - Run `cargo test --workspace` locally to reproduce + +3. **Clippy Warnings** + - Review the "Run clippy" step logs + - Fix with: `cargo clippy --workspace --fix` + +4. **Formatting Issues** + - Review the "Check formatting" step logs + - Fix with: `cargo fmt --all` + +5. **Runner Offline** + - Check if your Gitea Actions runner is running + - Verify runner labels match workflow requirements + +### Release Workflow Fails + +**Check these common issues**: + +1. **Tag Format** + - Ensure tag matches `v*.*.*` pattern + - Examples: `v1.0.0`, `v2.1.3`, `v0.1.0-beta` + +2. **Binary Not Found** + - Check if all binaries built successfully + - Review the "Build release binaries" step logs + +3. **Permission Denied** + - Ensure runner has write access to create releases + - Check repository settings + +4. **Release Already Exists** + - Delete the existing release first + - Or use a different version tag + +--- + +## Best Practices + +### Version Tagging + +Use [Semantic Versioning](https://semver.org/): +- `v1.0.0` - Major release (breaking changes) +- `v1.1.0` - Minor release (new features) +- `v1.0.1` - Patch release (bug fixes) +- `v1.0.0-beta.1` - Pre-release + +### Commit Messages + +Write clear commit messages for better release notes: +```bash +git commit -m "feat: Add new authentication system" +git commit -m "fix: Resolve memory leak in supervisor" +git commit -m "docs: Update installation guide" +``` + +### Testing Before Release + +Always test before creating a release: +```bash +# Run all checks locally +cargo check --workspace +cargo test --workspace +cargo clippy --workspace -- -D warnings +cargo fmt --all -- --check + +# Build release binaries locally +cargo build --workspace --release + +# Test the binaries +./target/release/supervisor --help +``` + +--- + +## Workflow Customization + +### Changing Rust Version + +Edit the toolchain in both workflows: +```yaml +- name: Setup Rust toolchain + uses: actions-rust-lang/setup-rust-toolchain@v1 + with: + toolchain: 1.75.0 # Specify exact version +``` + +### Adding More Binaries + +If you add new binaries to the workspace: + +1. Update `ci.yml` - Add to the upload artifacts step +2. Update `release.yml` - Add to strip and package steps +3. Update this README + +### Changing Artifact Retention + +In `ci.yml`, modify the retention period: +```yaml +retention-days: 30 # Keep for 30 days instead of 7 +``` + +### Adding Build Matrix + +To build for multiple platforms, add a matrix strategy: +```yaml +jobs: + build: + strategy: + matrix: + os: [ubuntu-latest, macos-latest, windows-latest] + runs-on: ${{ matrix.os }} +``` + +--- + +## Monitoring + +### View Workflow Status + +**In Gitea UI**: +1. Repository → Actions tab +2. See all workflow runs +3. Click any run for detailed logs + +**Via Git**: +```bash +# List recent tags +git tag -l + +# Show tag details +git show v1.0.0 +``` + +### Workflow Badges + +Add status badges to your README: +```markdown +![CI Status](https://git.ourworld.tf/peternashaat/horus/actions/workflows/ci.yml/badge.svg) +``` + +--- + +## Security Considerations + +### Secrets + +The workflows use `GITHUB_TOKEN` which is automatically provided by Gitea. This token: +- Has repository-scoped permissions +- Expires after the workflow run +- Cannot be accessed by pull requests from forks (for security) + +### Binary Verification + +Always verify downloaded binaries: +```bash +# Check SHA256 hash +sha256sum binary-name +# Compare with checksums.txt +``` + +### Supply Chain Security + +The workflows: +- Use pinned action versions (`@v4`, `@v1`) +- Build from source (no pre-built binaries) +- Generate checksums for verification + +--- + +## Additional Resources + +- [Gitea Actions Documentation](https://docs.gitea.com/usage/actions/overview) +- [GitHub Actions Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) (Gitea is compatible) +- [Rust CI Best Practices](https://doc.rust-lang.org/cargo/guide/continuous-integration.html) +- [Semantic Versioning](https://semver.org/) + +--- + +## Support + +For issues with: +- **Workflows**: Check the troubleshooting section above +- **Horus Project**: See the main [README.md](../../README.md) +- **Gitea Actions**: Consult [Gitea documentation](https://docs.gitea.com) + +For detailed line-by-line explanation of the workflows, see [WORKFLOW_EXPLAINED.md](./WORKFLOW_EXPLAINED.md). diff --git a/.gitea/workflows/WORKFLOW_EXPLAINED.md b/.gitea/workflows/WORKFLOW_EXPLAINED.md new file mode 100644 index 0000000..bc16faf --- /dev/null +++ b/.gitea/workflows/WORKFLOW_EXPLAINED.md @@ -0,0 +1,881 @@ +# Gitea Actions Workflows - Line-by-Line Explanation + +This document provides a detailed explanation of every line in the CI/CD workflows, explaining what each line does, why it's needed, and how it works. + +--- + +## Table of Contents + +1. [CI Workflow (ci.yml)](#ci-workflow-ciyml) +2. [Release Workflow (release.yml)](#release-workflow-releaseyml) +3. [Testing Guide](#testing-guide) +4. [Common Patterns Explained](#common-patterns-explained) + +--- + +## CI Workflow (ci.yml) + +### Header Section + +```yaml +name: CI +``` +**What**: Defines the workflow name displayed in the Gitea Actions UI +**Why**: Helps identify this workflow among multiple workflows +**How**: Gitea reads this and displays "CI" in the Actions tab + +--- + +### Trigger Configuration + +```yaml +on: +``` +**What**: Starts the trigger configuration section +**Why**: Tells Gitea when to run this workflow +**How**: Gitea monitors repository events and matches them against these triggers + +```yaml + push: + branches: + - '**' +``` +**What**: Triggers workflow on push to any branch +**Why**: We want to test every commit on every branch +**How**: `'**'` is a glob pattern matching all branch names (main, develop, feature/*, etc.) +**Alternative**: Use `- main` to only run on main branch + +```yaml + pull_request: + types: [opened, synchronize, reopened] +``` +**What**: Triggers workflow on pull request events +**Why**: Test code before merging into main branch +**How**: +- `opened` - When PR is first created +- `synchronize` - When new commits are pushed to the PR +- `reopened` - When a closed PR is reopened +**Note**: Does not run on `closed` to save resources + +--- + +### Job Definition + +```yaml +jobs: +``` +**What**: Starts the jobs section +**Why**: Workflows contain one or more jobs that run tasks +**How**: Each job runs in a fresh virtual environment + +```yaml + build-and-test: +``` +**What**: Job identifier (internal name) +**Why**: Unique ID for this job, used in logs and dependencies +**How**: Must be unique within the workflow, use kebab-case + +```yaml + name: Build & Test +``` +**What**: Human-readable job name +**Why**: Displayed in the Gitea UI for better readability +**How**: Shows in the Actions tab instead of "build-and-test" + +```yaml + runs-on: ubuntu-latest +``` +**What**: Specifies which runner to use +**Why**: Determines the OS and environment for the job +**How**: Gitea matches this label with available runners +**Note**: Your runner must have the `ubuntu-latest` label configured + +--- + +### Steps Section + +```yaml + steps: +``` +**What**: Starts the list of steps to execute +**Why**: Steps are the individual tasks that make up a job +**How**: Steps run sequentially in order + +--- + +#### Step 1: Checkout Code + +```yaml + - name: Checkout code +``` +**What**: Human-readable step name +**Why**: Helps identify this step in logs +**How**: Displayed in the workflow run UI + +```yaml + uses: actions/checkout@v4 +``` +**What**: Uses a pre-built action to checkout code +**Why**: Clones your repository into the runner's workspace +**How**: +- `actions/checkout` - GitHub's official checkout action (Gitea compatible) +- `@v4` - Pins to version 4 for stability +**What it does**: +1. Clones the repository +2. Checks out the commit that triggered the workflow +3. Sets up git configuration + +**Why needed**: Without this, the runner has no access to your code + +--- + +#### Step 2: Setup Rust + +```yaml + - name: Setup Rust toolchain + uses: actions-rust-lang/setup-rust-toolchain@v1 +``` +**What**: Installs Rust toolchain on the runner +**Why**: Needed to compile Rust code +**How**: Downloads and installs rustc, cargo, and related tools +**What it does**: +1. Installs Rust stable version +2. Adds cargo to PATH +3. Configures cargo home directory + +```yaml + with: + toolchain: stable +``` +**What**: Specifies which Rust version to install +**Why**: `stable` ensures we use the latest stable Rust release +**How**: Downloads from rust-lang.org +**Alternatives**: `nightly`, `beta`, or specific version like `1.75.0` + +```yaml + components: rustfmt, clippy +``` +**What**: Installs additional Rust components +**Why**: +- `rustfmt` - Code formatter (needed for formatting check) +- `clippy` - Linter (needed for lint check) +**How**: Installed via rustup alongside the toolchain + +--- + +#### Step 3-5: Caching + +**Why caching is needed**: +- Rust compilation is slow +- Dependencies rarely change +- Caching speeds up builds from ~15 minutes to ~2 minutes + +```yaml + - name: Cache cargo registry + uses: actions/cache@v4 +``` +**What**: Caches the Cargo registry +**Why**: Stores downloaded crate metadata to avoid re-downloading +**How**: Uses GitHub's cache action (Gitea compatible) + +```yaml + with: + path: ~/.cargo/registry +``` +**What**: Directory to cache +**Why**: This is where Cargo stores downloaded crate files +**How**: Entire directory is compressed and stored + +```yaml + key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }} +``` +**What**: Unique cache key +**Why**: Identifies this specific cache +**How**: +- `${{ runner.os }}` - OS name (e.g., "Linux") +- `cargo-registry` - Cache identifier +- `${{ hashFiles('**/Cargo.lock') }}` - Hash of Cargo.lock +**Result**: Key like `Linux-cargo-registry-a1b2c3d4` +**When it changes**: When Cargo.lock changes (dependencies updated) + +```yaml + restore-keys: | + ${{ runner.os }}-cargo-registry- +``` +**What**: Fallback cache keys +**Why**: If exact match not found, use partial match +**How**: Tries to find any cache starting with `Linux-cargo-registry-` +**Benefit**: Even with dependency changes, most cache is still valid + +**Same pattern repeats for**: +- Cargo index cache (`~/.cargo/git`) +- Build cache (`target/`) + +--- + +#### Step 6: Check Code + +```yaml + - name: Check code + run: cargo check --workspace --verbose +``` +**What**: Runs `cargo check` command +**Why**: Fast compilation check without producing binaries +**How**: +- `cargo check` - Compiles code but doesn't generate executables +- `--workspace` - Check all packages in the workspace +- `--verbose` - Show detailed output +**Benefit**: Catches compilation errors quickly (~2x faster than full build) +**Exit code**: Non-zero if compilation fails, which fails the workflow + +--- + +#### Step 7: Run Tests + +```yaml + - name: Run tests + run: cargo test --workspace --verbose +``` +**What**: Runs all tests in the workspace +**Why**: Ensures code changes don't break functionality +**How**: +- Compiles test code +- Runs all `#[test]` functions +- Runs integration tests in `tests/` directory +**Exit code**: Non-zero if any test fails + +--- + +#### Step 8: Run Clippy + +```yaml + - name: Run clippy + run: cargo clippy --workspace -- -D warnings +``` +**What**: Runs Rust linter +**Why**: Catches common mistakes and enforces best practices +**How**: +- `cargo clippy` - Runs the clippy linter +- `--workspace` - Lint all packages +- `--` - Separator between cargo args and clippy args +- `-D warnings` - Treat warnings as errors +**Result**: Fails if any clippy warnings are found +**Examples of what clippy catches**: +- Unused variables +- Inefficient code patterns +- Potential bugs + +--- + +#### Step 9: Check Formatting + +```yaml + - name: Check formatting + run: cargo fmt --all -- --check +``` +**What**: Checks if code is properly formatted +**Why**: Enforces consistent code style +**How**: +- `cargo fmt` - Rust formatter +- `--all` - Check all packages +- `--` - Separator +- `--check` - Don't modify files, just check +**Exit code**: Non-zero if any file is not formatted +**To fix locally**: Run `cargo fmt --all` without `--check` + +--- + +#### Step 10: Build Release Binaries + +```yaml + - name: Build release binaries + run: cargo build --workspace --release --verbose +``` +**What**: Builds all binaries with optimizations +**Why**: Ensures release builds work and produces artifacts +**How**: +- `cargo build` - Compile code +- `--workspace` - Build all packages +- `--release` - Enable optimizations (from Cargo.toml profile.release) +- `--verbose` - Show detailed output +**Result**: Binaries in `target/release/` directory +**Time**: ~5-10 minutes (first run), ~2-5 minutes (cached) + +--- + +#### Step 11: List Binaries + +```yaml + - name: List built binaries + run: | + echo "Built binaries:" + ls -lh target/release/ | grep -E '^-.*x.*' +``` +**What**: Lists built executable files +**Why**: Helps verify all binaries were built successfully +**How**: +- `ls -lh` - List files with human-readable sizes +- `grep -E '^-.*x.*'` - Filter for executable files +**Output**: Shows binary names and sizes in logs + +--- + +#### Step 12: Upload Artifacts + +```yaml + - name: Upload build artifacts + uses: actions/upload-artifact@v4 +``` +**What**: Uploads files to Gitea for download +**Why**: Makes binaries available for testing without creating a release +**How**: Uses GitHub's upload action (Gitea compatible) + +```yaml + with: + name: binaries-${{ github.sha }} +``` +**What**: Artifact name +**Why**: Unique name for this set of binaries +**How**: `${{ github.sha }}` - Git commit SHA (e.g., `binaries-a1b2c3d4`) + +```yaml + path: | + target/release/supervisor + target/release/coordinator + target/release/horus + target/release/osiris + target/release/herorunner + target/release/runner_osiris + target/release/runner_sal +``` +**What**: Files to upload +**Why**: These are the 7 binaries we want to preserve +**How**: Each line is a file path, `|` allows multi-line list + +```yaml + retention-days: 7 +``` +**What**: How long to keep artifacts +**Why**: Saves storage space by auto-deleting old artifacts +**How**: Gitea automatically deletes after 7 days + +```yaml + if-no-files-found: warn +``` +**What**: What to do if files don't exist +**Why**: `warn` logs a warning but doesn't fail the workflow +**How**: Useful if some binaries fail to build +**Alternatives**: `error` (fail workflow), `ignore` (silent) + +--- + +## Release Workflow (release.yml) + +### Header and Triggers + +```yaml +name: Release +``` +**What**: Workflow name +**Why**: Identifies this as the release workflow + +```yaml +on: + push: + tags: + - 'v*.*.*' +``` +**What**: Triggers on version tags +**Why**: Only create releases for version tags +**How**: +- `tags:` - Watches for tag pushes +- `'v*.*.*'` - Glob pattern matching semantic versions +**Matches**: `v1.0.0`, `v2.1.3`, `v0.1.0` +**Doesn't match**: `v1.0`, `1.0.0`, `release-1.0.0` + +--- + +### Job Setup + +```yaml +jobs: + build-release: + name: Build Release Binaries + runs-on: ubuntu-latest +``` +**Same as CI workflow** - See above for explanation + +--- + +### Steps 1-2: Checkout and Setup + +**Same as CI workflow** - See above for explanation + +--- + +#### Step 3: Extract Version + +```yaml + - name: Extract version from tag + id: version +``` +**What**: Names this step and gives it an ID +**Why**: `id` allows other steps to reference this step's outputs +**How**: Use `${{ steps.version.outputs.VERSION }}` in later steps + +```yaml + run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT +``` +**What**: Extracts version from tag name +**Why**: Needed for naming release files +**How**: +- `GITHUB_REF` - Full ref like `refs/tags/v1.0.0` +- `${GITHUB_REF#refs/tags/}` - Bash parameter expansion, removes `refs/tags/` prefix +- Result: `v1.0.0` +- `>> $GITHUB_OUTPUT` - Sets output variable +**Usage**: Later steps access via `${{ steps.version.outputs.VERSION }}` + +--- + +#### Step 4: Build Release Binaries + +```yaml + - name: Build release binaries + run: cargo build --workspace --release --verbose +``` +**Same as CI workflow** - Builds optimized binaries +**Why here**: Need fresh release builds for distribution + +--- + +#### Step 5: Strip Binaries + +```yaml + - name: Strip binaries + run: | + strip target/release/supervisor || true + strip target/release/coordinator || true + # ... etc for all 7 binaries +``` +**What**: Removes debug symbols from binaries +**Why**: Reduces binary size by 50-90% +**How**: +- `strip` - Linux command that removes debugging symbols +- `|| true` - Don't fail if strip fails (some binaries might not exist) +**Result**: Smaller binaries, faster downloads +**Example**: 50MB binary → 5MB binary + +--- + +#### Step 6: Create Release Directory + +```yaml + - name: Create release directory + run: mkdir -p release-artifacts +``` +**What**: Creates directory for release files +**Why**: Organize artifacts before uploading +**How**: `mkdir -p` creates directory (doesn't fail if exists) + +--- + +#### Step 7: Package Binaries + +```yaml + - name: Package binaries + run: | + # Package each binary as a tarball + for binary in supervisor coordinator horus osiris herorunner runner_osiris runner_sal; do +``` +**What**: Loops through all binary names +**Why**: Package each binary separately +**How**: Bash for loop + +```yaml + if [ -f "target/release/$binary" ]; then +``` +**What**: Checks if binary file exists +**Why**: Skip if binary wasn't built +**How**: `-f` tests if file exists + +```yaml + tar -czf "release-artifacts/${binary}-${{ steps.version.outputs.VERSION }}-linux-x86_64.tar.gz" \ + -C target/release "$binary" +``` +**What**: Creates compressed tarball +**Why**: Standard distribution format for Linux binaries +**How**: +- `tar` - Archive tool +- `-c` - Create archive +- `-z` - Compress with gzip +- `-f` - Output filename +- `-C target/release` - Change to this directory first +- `"$binary"` - File to archive +**Result**: File like `supervisor-v1.0.0-linux-x86_64.tar.gz` +**Naming convention**: `{name}-{version}-{platform}-{arch}.tar.gz` + +```yaml + echo "Packaged $binary" + else + echo "Warning: $binary not found, skipping" + fi + done +``` +**What**: Logs success or warning +**Why**: Helps debug if binaries are missing +**How**: Simple echo statements + +--- + +#### Step 8: Generate Checksums + +```yaml + - name: Generate checksums + run: | + cd release-artifacts + sha256sum *.tar.gz > checksums.txt + cat checksums.txt +``` +**What**: Creates SHA256 checksums for all tarballs +**Why**: Allows users to verify download integrity +**How**: +- `cd release-artifacts` - Change to artifact directory +- `sha256sum *.tar.gz` - Calculate SHA256 hash for each tarball +- `> checksums.txt` - Save to file +- `cat checksums.txt` - Display in logs +**Result**: File with lines like: +``` +a1b2c3d4... supervisor-v1.0.0-linux-x86_64.tar.gz +e5f6g7h8... coordinator-v1.0.0-linux-x86_64.tar.gz +``` + +--- + +#### Step 9: Create Release + +```yaml + - name: Create Release + uses: actions/gitea-release@v1 +``` +**What**: Uses Gitea's release action +**Why**: Creates a release with attached files +**How**: Calls Gitea API to create release + +```yaml + with: + files: release-artifacts/* +``` +**What**: Files to attach to release +**Why**: Makes binaries downloadable +**How**: Glob pattern uploads all files in directory + +```yaml + token: ${{ secrets.GITHUB_TOKEN }} +``` +**What**: Authentication token +**Why**: Needed to create releases via API +**How**: Gitea automatically provides this secret +**Security**: Token is scoped to this repository only + +```yaml + tag_name: ${{ steps.version.outputs.VERSION }} +``` +**What**: Tag to create release for +**Why**: Associates release with the tag +**How**: Uses version extracted in step 3 + +```yaml + name: Release ${{ steps.version.outputs.VERSION }} +``` +**What**: Release title +**Why**: Displayed in releases page +**How**: Combines "Release" + version (e.g., "Release v1.0.0") + +```yaml + body: | + ## Horus Release ${{ steps.version.outputs.VERSION }} + + ### Binaries + This release includes the following binaries for Linux x86_64: + - `supervisor` - Hero Supervisor service + # ... etc +``` +**What**: Release description (markdown) +**Why**: Provides context and instructions +**How**: Multi-line string with markdown formatting +**Result**: Rendered as formatted text in release page + +```yaml + draft: false +``` +**What**: Publish immediately +**Why**: Make release public right away +**How**: `false` = published, `true` = draft +**Alternative**: Set to `true` to review before publishing + +```yaml + prerelease: false +``` +**What**: Mark as stable release +**Why**: Indicates this is production-ready +**How**: `false` = stable, `true` = pre-release (beta, alpha) +**When to use `true`**: For tags like `v1.0.0-beta.1` + +--- + +## Testing Guide + +### Testing CI Workflow Locally + +Before pushing, test locally: + +```bash +# 1. Check compilation +cargo check --workspace --verbose + +# 2. Run tests +cargo test --workspace --verbose + +# 3. Run clippy +cargo clippy --workspace -- -D warnings + +# 4. Check formatting +cargo fmt --all -- --check + +# 5. Build release +cargo build --workspace --release --verbose + +# 6. Verify binaries exist +ls -lh target/release/ | grep -E '^-.*x.*' +``` + +**Expected result**: All commands should succeed with exit code 0 + +--- + +### Testing CI Workflow in Gitea + +```bash +# 1. Create a test branch +git checkout -b test-ci + +# 2. Make a small change (e.g., add a comment) +echo "// Test CI" >> bin/supervisor/src/main.rs + +# 3. Commit and push +git add . +git commit -m "test: Trigger CI workflow" +git push origin test-ci + +# 4. Check Gitea Actions +# Navigate to: https://git.ourworld.tf/peternashaat/horus/actions +``` + +**Expected result**: +- Workflow appears in Actions tab +- All steps complete successfully (green checkmarks) +- Artifacts are uploaded + +**To download artifacts**: +1. Click on the workflow run +2. Scroll to "Artifacts" section +3. Click to download + +--- + +### Testing Release Workflow Locally + +Simulate release build: + +```bash +# 1. Build release binaries +cargo build --workspace --release --verbose + +# 2. Strip binaries +strip target/release/supervisor || true +strip target/release/coordinator || true +# ... etc + +# 3. Create test directory +mkdir -p test-release + +# 4. Package binaries +for binary in supervisor coordinator horus osiris herorunner runner_osiris runner_sal; do + if [ -f "target/release/$binary" ]; then + tar -czf "test-release/${binary}-v0.0.1-test-linux-x86_64.tar.gz" \ + -C target/release "$binary" + echo "Packaged $binary" + fi +done + +# 5. Generate checksums +cd test-release +sha256sum *.tar.gz > checksums.txt +cat checksums.txt +cd .. + +# 6. Test extraction +cd test-release +tar -xzf supervisor-v0.0.1-test-linux-x86_64.tar.gz +./supervisor --help +cd .. +``` + +**Expected result**: +- All binaries package successfully +- Checksums are generated +- Binary extracts and runs + +--- + +### Testing Release Workflow in Gitea + +```bash +# 1. Ensure code is ready +git checkout main +git pull + +# 2. Create a test tag +git tag v0.1.0-test + +# 3. Push the tag +git push origin v0.1.0-test + +# 4. Check Gitea +# Navigate to: https://git.ourworld.tf/peternashaat/horus/releases +``` + +**Expected result**: +- Release appears in Releases tab +- All 7 binaries are attached as `.tar.gz` files +- `checksums.txt` is attached +- Release notes are properly formatted + +**To test download**: +```bash +# Download a binary +wget https://git.ourworld.tf/peternashaat/horus/releases/download/v0.1.0-test/supervisor-v0.1.0-test-linux-x86_64.tar.gz + +# Extract +tar -xzf supervisor-v0.1.0-test-linux-x86_64.tar.gz + +# Test +chmod +x supervisor +./supervisor --help +``` + +**Cleanup test release**: +1. Go to Releases page +2. Click on the test release +3. Click "Delete" button + +--- + +## Common Patterns Explained + +### Why `|| true`? + +```yaml +strip target/release/supervisor || true +``` + +**What**: Bash OR operator +**Why**: Prevents step from failing if command fails +**How**: +- If `strip` succeeds (exit 0), continue +- If `strip` fails (exit non-zero), `|| true` runs and returns 0 +**Use case**: Some binaries might not exist, don't fail the whole workflow + +--- + +### Why `${{ }}` Syntax? + +```yaml +${{ github.sha }} +${{ steps.version.outputs.VERSION }} +``` + +**What**: GitHub Actions expression syntax +**Why**: Access variables and context +**How**: Gitea evaluates these at runtime +**Types**: +- `github.*` - Workflow context (sha, ref, actor, etc.) +- `secrets.*` - Secret variables +- `steps.*` - Output from previous steps +- `runner.*` - Runner information (os, arch, etc.) + +--- + +### Why `--workspace`? + +```yaml +cargo build --workspace +``` + +**What**: Cargo flag to include all workspace members +**Why**: Your project is a workspace with multiple packages +**How**: Cargo reads `Cargo.toml` [workspace] section +**Without it**: Only builds the root package +**With it**: Builds all 7 binaries + +--- + +### Why Separate CI and Release Workflows? + +**CI Workflow**: +- Runs frequently (every push) +- Fast feedback +- Doesn't create releases + +**Release Workflow**: +- Runs rarely (only on tags) +- Slower (includes packaging) +- Creates permanent artifacts + +**Benefit**: Fast CI doesn't slow down development, releases are deliberate + +--- + +### Why Cache Three Directories? + +1. **`~/.cargo/registry`** - Downloaded crate files + - Changes when: Dependencies are added/updated + - Size: ~500MB - 2GB + +2. **`~/.cargo/git`** - Git dependencies + - Changes when: Git dependencies are updated + - Size: ~100MB - 500MB + +3. **`target/`** - Compiled artifacts + - Changes when: Code or dependencies change + - Size: ~2GB - 10GB + +**Together**: Reduce build time from 15 minutes to 2 minutes + +--- + +### Why `hashFiles('**/Cargo.lock')`? + +**What**: Generates hash of Cargo.lock +**Why**: Cache key changes when dependencies change +**How**: +- `**/Cargo.lock` - Find all Cargo.lock files +- `hashFiles()` - Generate SHA256 hash +**Result**: Different hash = different cache = rebuild dependencies +**Benefit**: Cache is invalidated when dependencies change + +--- + +## Summary + +Both workflows follow best practices: + +✅ **Fast feedback** - CI runs on every push +✅ **Comprehensive testing** - Check, test, lint, format +✅ **Optimized builds** - Caching reduces build time +✅ **Automated releases** - Tag-based release creation +✅ **Secure** - Uses scoped tokens, no manual secrets +✅ **Reproducible** - Pinned action versions +✅ **User-friendly** - Clear release notes and instructions + +For more information, see [README.md](./README.md). diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml new file mode 100644 index 0000000..0d94d3c --- /dev/null +++ b/.gitea/workflows/ci.yml @@ -0,0 +1,82 @@ +name: CI + +on: + push: + branches: + - '**' + pull_request: + types: [opened, synchronize, reopened] + +jobs: + build-and-test: + name: Build & Test + runs-on: ubuntu-latest + + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Rust toolchain + uses: actions-rust-lang/setup-rust-toolchain@v1 + with: + toolchain: stable + components: rustfmt, clippy + + - name: Cache cargo registry + uses: actions/cache@v4 + with: + path: ~/.cargo/registry + key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }} + restore-keys: | + ${{ runner.os }}-cargo-registry- + + - name: Cache cargo index + uses: actions/cache@v4 + with: + path: ~/.cargo/git + key: ${{ runner.os }}-cargo-index-${{ hashFiles('**/Cargo.lock') }} + restore-keys: | + ${{ runner.os }}-cargo-index- + + - name: Cache cargo build + uses: actions/cache@v4 + with: + path: target + key: ${{ runner.os }}-cargo-build-target-${{ hashFiles('**/Cargo.lock') }} + restore-keys: | + ${{ runner.os }}-cargo-build-target- + + - name: Check code + run: cargo check --workspace --verbose + + - name: Run tests + run: cargo test --workspace --verbose + + - name: Run clippy + run: cargo clippy --workspace -- -D warnings + + - name: Check formatting + run: cargo fmt --all -- --check + + - name: Build release binaries + run: cargo build --workspace --release --verbose + + - name: List built binaries + run: | + echo "Built binaries:" + ls -lh target/release/ | grep -E '^-.*x.*' + + - name: Upload build artifacts + uses: actions/upload-artifact@v4 + with: + name: binaries-${{ github.sha }} + path: | + target/release/supervisor + target/release/coordinator + target/release/horus + target/release/osiris + target/release/herorunner + target/release/runner_osiris + target/release/runner_sal + retention-days: 7 + if-no-files-found: warn diff --git a/.gitea/workflows/release.yml b/.gitea/workflows/release.yml new file mode 100644 index 0000000..8290dfd --- /dev/null +++ b/.gitea/workflows/release.yml @@ -0,0 +1,100 @@ +name: Release + +on: + push: + tags: + - 'v*.*.*' + +jobs: + build-release: + name: Build Release Binaries + runs-on: ubuntu-latest + + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Rust toolchain + uses: actions-rust-lang/setup-rust-toolchain@v1 + with: + toolchain: stable + + - name: Extract version from tag + id: version + run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT + + - name: Build release binaries + run: cargo build --workspace --release --verbose + + - name: Strip binaries + run: | + strip target/release/supervisor || true + strip target/release/coordinator || true + strip target/release/horus || true + strip target/release/osiris || true + strip target/release/herorunner || true + strip target/release/runner_osiris || true + strip target/release/runner_sal || true + + - name: Create release directory + run: mkdir -p release-artifacts + + - name: Package binaries + run: | + # Package each binary as a tarball + for binary in supervisor coordinator horus osiris herorunner runner_osiris runner_sal; do + if [ -f "target/release/$binary" ]; then + tar -czf "release-artifacts/${binary}-${{ steps.version.outputs.VERSION }}-linux-x86_64.tar.gz" \ + -C target/release "$binary" + echo "Packaged $binary" + else + echo "Warning: $binary not found, skipping" + fi + done + + - name: Generate checksums + run: | + cd release-artifacts + sha256sum *.tar.gz > checksums.txt + cat checksums.txt + + - name: Create Release + uses: actions/gitea-release@v1 + with: + files: release-artifacts/* + token: ${{ secrets.GITHUB_TOKEN }} + tag_name: ${{ steps.version.outputs.VERSION }} + name: Release ${{ steps.version.outputs.VERSION }} + body: | + ## Horus Release ${{ steps.version.outputs.VERSION }} + + ### Binaries + This release includes the following binaries for Linux x86_64: + - `supervisor` - Hero Supervisor service + - `coordinator` - Hero Coordinator service + - `horus` - Horus main binary + - `osiris` - Osiris server + - `herorunner` - Hero runner + - `runner_osiris` - Osiris runner + - `runner_sal` - SAL runner + + ### Installation + Download the appropriate binary for your system: + ```bash + # Example: Download and install supervisor + wget https://git.ourworld.tf/peternashaat/horus/releases/download/${{ steps.version.outputs.VERSION }}/supervisor-${{ steps.version.outputs.VERSION }}-linux-x86_64.tar.gz + tar -xzf supervisor-${{ steps.version.outputs.VERSION }}-linux-x86_64.tar.gz + chmod +x supervisor + sudo mv supervisor /usr/local/bin/ + ``` + + ### Verification + Verify the integrity of downloaded files using the checksums: + ```bash + sha256sum -c checksums.txt + ``` + + ### Changes + See commit history for detailed changes in this release. + draft: false + prerelease: false