forked from herocode/horus
Compare commits
3 Commits
09c4444db5
...
main_cicdp
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aa30c54742 | ||
| 8fdfdd1746 | |||
|
|
dffb17ffbc |
392
.gitea/workflows/README.md
Normal file
392
.gitea/workflows/README.md
Normal file
@@ -0,0 +1,392 @@
|
||||
# Gitea Actions Workflows Documentation
|
||||
|
||||
This directory contains the CI/CD workflows for the Horus project using Gitea Actions.
|
||||
|
||||
## Overview
|
||||
|
||||
The Horus project uses two main workflows:
|
||||
|
||||
1. **[ci.yml](./ci.yml)** - Continuous Integration workflow
|
||||
2. **[release.yml](./release.yml)** - Release automation workflow
|
||||
|
||||
## Workflow Files
|
||||
|
||||
### ci.yml - Continuous Integration
|
||||
|
||||
**Purpose**: Automatically build, test, and validate code quality on every push and pull request.
|
||||
|
||||
**Triggers**:
|
||||
- Push to any branch
|
||||
- Pull request events (opened, synchronized, reopened)
|
||||
|
||||
**What it does**:
|
||||
1. Sets up Rust toolchain
|
||||
2. Caches dependencies for faster builds
|
||||
3. Runs code quality checks (check, test, clippy, fmt)
|
||||
4. Builds all 7 binaries in release mode
|
||||
5. Uploads binaries as artifacts
|
||||
|
||||
**Duration**: ~5-15 minutes (first run), ~2-5 minutes (cached runs)
|
||||
|
||||
**Artifacts**: Binaries are stored for 7 days and can be downloaded from the Actions tab
|
||||
|
||||
---
|
||||
|
||||
### release.yml - Release Automation
|
||||
|
||||
**Purpose**: Automatically create GitHub-style releases with downloadable binaries when version tags are pushed.
|
||||
|
||||
**Triggers**:
|
||||
- Tags matching `v*.*.*` pattern (e.g., `v1.0.0`, `v2.1.3`)
|
||||
|
||||
**What it does**:
|
||||
1. Builds optimized release binaries
|
||||
2. Strips debug symbols to reduce size
|
||||
3. Packages each binary as a tarball
|
||||
4. Generates SHA256 checksums
|
||||
5. Creates a Gitea release with all artifacts attached
|
||||
|
||||
**Duration**: ~5-10 minutes
|
||||
|
||||
**Artifacts**: Permanently attached to the release
|
||||
|
||||
---
|
||||
|
||||
## Binaries Built
|
||||
|
||||
Both workflows build the following 7 binaries:
|
||||
|
||||
| Binary Name | Description |
|
||||
|-------------|-------------|
|
||||
| `supervisor` | Hero Supervisor service |
|
||||
| `coordinator` | Hero Coordinator service |
|
||||
| `horus` | Horus main binary |
|
||||
| `osiris` | Osiris server |
|
||||
| `herorunner` | Hero runner |
|
||||
| `runner_osiris` | Osiris runner |
|
||||
| `runner_sal` | SAL runner |
|
||||
|
||||
---
|
||||
|
||||
## Usage Guide
|
||||
|
||||
### Testing Code Changes
|
||||
|
||||
Every time you push code or create a pull request, the CI workflow automatically runs:
|
||||
|
||||
```bash
|
||||
# Make your changes
|
||||
git add .
|
||||
git commit -m "Your changes"
|
||||
git push origin your-branch
|
||||
|
||||
# Or create a pull request
|
||||
# The CI workflow will run automatically
|
||||
```
|
||||
|
||||
**Check Results**:
|
||||
1. Go to your Gitea repository
|
||||
2. Click on the **Actions** tab
|
||||
3. Find your workflow run
|
||||
4. Click to see detailed logs
|
||||
|
||||
---
|
||||
|
||||
### Creating a Release
|
||||
|
||||
To create a new release with binaries:
|
||||
|
||||
```bash
|
||||
# 1. Ensure your code is ready for release
|
||||
# 2. Create a version tag (use semantic versioning)
|
||||
git tag v1.0.0
|
||||
|
||||
# 3. Push the tag
|
||||
git push origin v1.0.0
|
||||
|
||||
# 4. The release workflow will automatically:
|
||||
# - Build all binaries
|
||||
# - Create a release
|
||||
# - Attach binaries and checksums
|
||||
```
|
||||
|
||||
**View Release**:
|
||||
1. Go to your Gitea repository
|
||||
2. Click on the **Releases** tab
|
||||
3. Your new release will be listed with downloadable artifacts
|
||||
|
||||
---
|
||||
|
||||
### Downloading Release Binaries
|
||||
|
||||
Users can download binaries from releases:
|
||||
|
||||
```bash
|
||||
# Download a specific binary
|
||||
wget https://git.ourworld.tf/peternashaat/horus/releases/download/v1.0.0/supervisor-v1.0.0-linux-x86_64.tar.gz
|
||||
|
||||
# Extract
|
||||
tar -xzf supervisor-v1.0.0-linux-x86_64.tar.gz
|
||||
|
||||
# Make executable
|
||||
chmod +x supervisor
|
||||
|
||||
# Optionally move to system path
|
||||
sudo mv supervisor /usr/local/bin/
|
||||
|
||||
# Verify it works
|
||||
supervisor --help
|
||||
```
|
||||
|
||||
**Verify Integrity**:
|
||||
```bash
|
||||
# Download checksums
|
||||
wget https://git.ourworld.tf/peternashaat/horus/releases/download/v1.0.0/checksums.txt
|
||||
|
||||
# Verify a binary
|
||||
sha256sum -c checksums.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Requirements
|
||||
|
||||
### Runner Configuration
|
||||
|
||||
Your Gitea Actions runner must be configured with these labels:
|
||||
- `ubuntu-latest` (recommended)
|
||||
- `ubuntu-22.04` (alternative)
|
||||
- `ubuntu-20.04` (alternative)
|
||||
|
||||
### Permissions
|
||||
|
||||
The workflows require:
|
||||
- Read access to repository code
|
||||
- Write access to create releases (for release.yml)
|
||||
- Access to `GITHUB_TOKEN` secret (automatically provided by Gitea)
|
||||
|
||||
### Dependencies
|
||||
|
||||
The workflows automatically install:
|
||||
- Rust stable toolchain
|
||||
- rustfmt (code formatter)
|
||||
- clippy (linter)
|
||||
|
||||
No manual setup required!
|
||||
|
||||
---
|
||||
|
||||
## Caching Strategy
|
||||
|
||||
The CI workflow uses three levels of caching to speed up builds:
|
||||
|
||||
1. **Cargo Registry Cache** - Downloaded crate metadata
|
||||
2. **Cargo Index Cache** - Git index of crates.io
|
||||
3. **Build Cache** - Compiled dependencies
|
||||
|
||||
**Benefits**:
|
||||
- First build: ~10-15 minutes
|
||||
- Cached builds: ~2-5 minutes
|
||||
- Saves bandwidth and runner resources
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### CI Workflow Fails
|
||||
|
||||
**Check these common issues**:
|
||||
|
||||
1. **Compilation Errors**
|
||||
- Review the "Check code" step logs
|
||||
- Fix Rust compilation errors locally first
|
||||
|
||||
2. **Test Failures**
|
||||
- Review the "Run tests" step logs
|
||||
- Run `cargo test --workspace` locally to reproduce
|
||||
|
||||
3. **Clippy Warnings**
|
||||
- Review the "Run clippy" step logs
|
||||
- Fix with: `cargo clippy --workspace --fix`
|
||||
|
||||
4. **Formatting Issues**
|
||||
- Review the "Check formatting" step logs
|
||||
- Fix with: `cargo fmt --all`
|
||||
|
||||
5. **Runner Offline**
|
||||
- Check if your Gitea Actions runner is running
|
||||
- Verify runner labels match workflow requirements
|
||||
|
||||
### Release Workflow Fails
|
||||
|
||||
**Check these common issues**:
|
||||
|
||||
1. **Tag Format**
|
||||
- Ensure tag matches `v*.*.*` pattern
|
||||
- Examples: `v1.0.0`, `v2.1.3`, `v0.1.0-beta`
|
||||
|
||||
2. **Binary Not Found**
|
||||
- Check if all binaries built successfully
|
||||
- Review the "Build release binaries" step logs
|
||||
|
||||
3. **Permission Denied**
|
||||
- Ensure runner has write access to create releases
|
||||
- Check repository settings
|
||||
|
||||
4. **Release Already Exists**
|
||||
- Delete the existing release first
|
||||
- Or use a different version tag
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Version Tagging
|
||||
|
||||
Use [Semantic Versioning](https://semver.org/):
|
||||
- `v1.0.0` - Major release (breaking changes)
|
||||
- `v1.1.0` - Minor release (new features)
|
||||
- `v1.0.1` - Patch release (bug fixes)
|
||||
- `v1.0.0-beta.1` - Pre-release
|
||||
|
||||
### Commit Messages
|
||||
|
||||
Write clear commit messages for better release notes:
|
||||
```bash
|
||||
git commit -m "feat: Add new authentication system"
|
||||
git commit -m "fix: Resolve memory leak in supervisor"
|
||||
git commit -m "docs: Update installation guide"
|
||||
```
|
||||
|
||||
### Testing Before Release
|
||||
|
||||
Always test before creating a release:
|
||||
```bash
|
||||
# Run all checks locally
|
||||
cargo check --workspace
|
||||
cargo test --workspace
|
||||
cargo clippy --workspace -- -D warnings
|
||||
cargo fmt --all -- --check
|
||||
|
||||
# Build release binaries locally
|
||||
cargo build --workspace --release
|
||||
|
||||
# Test the binaries
|
||||
./target/release/supervisor --help
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Customization
|
||||
|
||||
### Changing Rust Version
|
||||
|
||||
Edit the toolchain in both workflows:
|
||||
```yaml
|
||||
- name: Setup Rust toolchain
|
||||
uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
toolchain: 1.75.0 # Specify exact version
|
||||
```
|
||||
|
||||
### Adding More Binaries
|
||||
|
||||
If you add new binaries to the workspace:
|
||||
|
||||
1. Update `ci.yml` - Add to the upload artifacts step
|
||||
2. Update `release.yml` - Add to strip and package steps
|
||||
3. Update this README
|
||||
|
||||
### Changing Artifact Retention
|
||||
|
||||
In `ci.yml`, modify the retention period:
|
||||
```yaml
|
||||
retention-days: 30 # Keep for 30 days instead of 7
|
||||
```
|
||||
|
||||
### Adding Build Matrix
|
||||
|
||||
To build for multiple platforms, add a matrix strategy:
|
||||
```yaml
|
||||
jobs:
|
||||
build:
|
||||
strategy:
|
||||
matrix:
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
runs-on: ${{ matrix.os }}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring
|
||||
|
||||
### View Workflow Status
|
||||
|
||||
**In Gitea UI**:
|
||||
1. Repository → Actions tab
|
||||
2. See all workflow runs
|
||||
3. Click any run for detailed logs
|
||||
|
||||
**Via Git**:
|
||||
```bash
|
||||
# List recent tags
|
||||
git tag -l
|
||||
|
||||
# Show tag details
|
||||
git show v1.0.0
|
||||
```
|
||||
|
||||
### Workflow Badges
|
||||
|
||||
Add status badges to your README:
|
||||
```markdown
|
||||

|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Secrets
|
||||
|
||||
The workflows use `GITHUB_TOKEN` which is automatically provided by Gitea. This token:
|
||||
- Has repository-scoped permissions
|
||||
- Expires after the workflow run
|
||||
- Cannot be accessed by pull requests from forks (for security)
|
||||
|
||||
### Binary Verification
|
||||
|
||||
Always verify downloaded binaries:
|
||||
```bash
|
||||
# Check SHA256 hash
|
||||
sha256sum binary-name
|
||||
# Compare with checksums.txt
|
||||
```
|
||||
|
||||
### Supply Chain Security
|
||||
|
||||
The workflows:
|
||||
- Use pinned action versions (`@v4`, `@v1`)
|
||||
- Build from source (no pre-built binaries)
|
||||
- Generate checksums for verification
|
||||
|
||||
---
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [Gitea Actions Documentation](https://docs.gitea.com/usage/actions/overview)
|
||||
- [GitHub Actions Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) (Gitea is compatible)
|
||||
- [Rust CI Best Practices](https://doc.rust-lang.org/cargo/guide/continuous-integration.html)
|
||||
- [Semantic Versioning](https://semver.org/)
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
For issues with:
|
||||
- **Workflows**: Check the troubleshooting section above
|
||||
- **Horus Project**: See the main [README.md](../../README.md)
|
||||
- **Gitea Actions**: Consult [Gitea documentation](https://docs.gitea.com)
|
||||
|
||||
For detailed line-by-line explanation of the workflows, see [WORKFLOW_EXPLAINED.md](./WORKFLOW_EXPLAINED.md).
|
||||
|
||||
111
.gitea/workflows/ci.yml
Normal file
111
.gitea/workflows/ci.yml
Normal file
@@ -0,0 +1,111 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- '**'
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened]
|
||||
|
||||
jobs:
|
||||
build-and-test:
|
||||
name: Build & Test
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Rust toolchain
|
||||
uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
toolchain: stable
|
||||
components: rustfmt, clippy
|
||||
|
||||
- name: Cache cargo registry
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cargo/registry
|
||||
key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-cargo-registry-
|
||||
|
||||
- name: Cache cargo index
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cargo/git
|
||||
key: ${{ runner.os }}-cargo-index-${{ hashFiles('**/Cargo.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-cargo-index-
|
||||
|
||||
- name: Cache cargo build
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: target
|
||||
key: ${{ runner.os }}-cargo-build-target-${{ hashFiles('**/Cargo.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-cargo-build-target-
|
||||
|
||||
# 👇 Don't fail CI on hero-runner's warnings
|
||||
- name: Check code (all crates under bin/)
|
||||
env:
|
||||
RUSTFLAGS: "--cap-lints=warn"
|
||||
run: |
|
||||
set -euo pipefail
|
||||
find bin -name Cargo.toml -print0 | while IFS= read -r -d '' manifest; do
|
||||
echo "=== cargo check --manifest-path $manifest ==="
|
||||
cargo check --manifest-path "$manifest" --verbose
|
||||
done
|
||||
|
||||
# 👇 Same trick for tests, otherwise they’d fail for the same reason
|
||||
- name: Run tests (all crates under bin/)
|
||||
env:
|
||||
RUSTFLAGS: "--cap-lints=warn"
|
||||
run: |
|
||||
set -euo pipefail
|
||||
find bin -name Cargo.toml -print0 | while IFS= read -r -d '' manifest; do
|
||||
echo "=== cargo test --manifest-path $manifest ==="
|
||||
cargo test --manifest-path "$manifest" --verbose
|
||||
done
|
||||
|
||||
# Clippy stays strict (still uses -D warnings for clippy lints).
|
||||
# If this later fails because of hero-runner, we can also add RUSTFLAGS here.
|
||||
- name: Run clippy (all crates under bin/)
|
||||
run: |
|
||||
set -euo pipefail
|
||||
find bin -name Cargo.toml -print0 | while IFS= read -r -d '' manifest; do
|
||||
echo "=== cargo clippy --manifest-path $manifest ==="
|
||||
cargo clippy --manifest-path "$manifest" -- -D warnings
|
||||
done
|
||||
|
||||
- name: Check formatting
|
||||
run: cargo fmt --all -- --check
|
||||
|
||||
# Build was already succeeding; leaving it without cap-lints is fine.
|
||||
- name: Build release binaries (all crates under bin/)
|
||||
run: |
|
||||
set -euo pipefail
|
||||
find bin -name Cargo.toml -print0 | while IFS= read -r -d '' manifest; do
|
||||
echo "=== cargo build --manifest-path $manifest --release ==="
|
||||
cargo build --manifest-path "$manifest" --release --verbose
|
||||
done
|
||||
|
||||
- name: List built binaries
|
||||
run: |
|
||||
echo "Built binaries:"
|
||||
ls -lh target/release/ | grep -E '^-.*x.*'
|
||||
|
||||
- name: Upload build artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: binaries-${{ github.sha }}
|
||||
path: |
|
||||
target/release/supervisor
|
||||
target/release/coordinator
|
||||
target/release/horus
|
||||
target/release/osiris
|
||||
target/release/herorunner
|
||||
target/release/runner_osiris
|
||||
target/release/runner_sal
|
||||
retention-days: 7
|
||||
if-no-files-found: warn
|
||||
115
.gitea/workflows/release.yml
Normal file
115
.gitea/workflows/release.yml
Normal file
@@ -0,0 +1,115 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*.*.*'
|
||||
|
||||
jobs:
|
||||
build-release:
|
||||
name: Build Release Binaries via hero-git installer
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
env:
|
||||
IMAGE_NAME: hero-git:latest
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Extract version from tag
|
||||
id: version
|
||||
run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Ensure hero-git image exists
|
||||
run: |
|
||||
if ! docker image inspect "$IMAGE_NAME" >/dev/null 2>&1; then
|
||||
echo "ERROR: Docker image '$IMAGE_NAME' not found on runner."
|
||||
echo "Build it manually on the runner with:"
|
||||
echo " docker build -t $IMAGE_NAME -f Dockerfile ."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Prepare bin output dir
|
||||
run: mkdir -p hero-bin
|
||||
|
||||
- name: Run horus_full_install installer in container
|
||||
run: |
|
||||
set -euxo pipefail
|
||||
|
||||
docker run --rm \
|
||||
-v "$PWD/hero-bin:/root/hero/bin" \
|
||||
-e HEROLIB_REF=development_nile_installers \
|
||||
"$IMAGE_NAME" \
|
||||
bash -lc '
|
||||
set -euxo pipefail
|
||||
cd /opt/herolib
|
||||
./examples/installers/horus/horus_full_install.vsh
|
||||
echo "===== AFTER INSTALL, ls -R /root/hero ====="
|
||||
ls -R /root/hero || true
|
||||
'
|
||||
|
||||
- name: List built binaries
|
||||
run: ls -al hero-bin
|
||||
|
||||
- name: Create release directory
|
||||
run: mkdir -p release-artifacts
|
||||
|
||||
- name: Package binaries
|
||||
run: |
|
||||
for binary in supervisor coordinator horus osiris herorunner runner_osiris runner_sal; do
|
||||
if [ -f "hero-bin/$binary" ]; then
|
||||
tar -czf "release-artifacts/${binary}-${{ steps.version.outputs.VERSION }}-linux-x86_64.tar.gz" \
|
||||
-C hero-bin "$binary"
|
||||
echo "Packaged $binary"
|
||||
else
|
||||
echo "Warning: $binary not found, skipping"
|
||||
fi
|
||||
done
|
||||
|
||||
- name: Generate checksums
|
||||
run: |
|
||||
cd release-artifacts
|
||||
if ls *.tar.gz >/dev/null 2>&1; then
|
||||
sha256sum *.tar.gz > checksums.txt
|
||||
cat checksums.txt
|
||||
else
|
||||
echo "ERROR: no .tar.gz artifacts were produced; check previous steps (likely the installer didn’t build any binaries)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Create Release
|
||||
uses: akkuman/gitea-release-action@v1
|
||||
with:
|
||||
files: release-artifacts/*
|
||||
token: ${{ secrets.TOKEN_GITEA }}
|
||||
tag_name: ${{ steps.version.outputs.VERSION }}
|
||||
name: Release ${{ steps.version.outputs.VERSION }}
|
||||
body: |
|
||||
## Horus Release ${{ steps.version.outputs.VERSION }}
|
||||
|
||||
### Binaries
|
||||
Built via the herolib V installer (`horus_full_install.vsh`) inside
|
||||
the `hero-git:latest` image and packaged for Linux x86_64:
|
||||
- `supervisor`
|
||||
- `coordinator`
|
||||
- `horus`
|
||||
- `osiris`
|
||||
- `herorunner`
|
||||
- `runner_osiris`
|
||||
- `runner_sal`
|
||||
|
||||
### Installation (example)
|
||||
```bash
|
||||
wget https://git.ourworld.tf/herocode/horus/releases/download/${{ steps.version.outputs.VERSION }}/supervisor-${{ steps.version.outputs.VERSION }}-linux-x86_64.tar.gz
|
||||
tar -xzf supervisor-${{ steps.version.outputs.VERSION }}-linux-x86_64.tar.gz
|
||||
chmod +x supervisor
|
||||
sudo mv supervisor /usr/local/bin/
|
||||
```
|
||||
|
||||
### Verification
|
||||
```bash
|
||||
sha256sum -c checksums.txt
|
||||
```
|
||||
draft: false
|
||||
prerelease: false
|
||||
43
docker/Dockerfile
Normal file
43
docker/Dockerfile
Normal file
@@ -0,0 +1,43 @@
|
||||
FROM ubuntu:24.04 AS base
|
||||
|
||||
# Includes a bunch of stuff hero will try to install later, so we don't have to
|
||||
# do that on each new container launch
|
||||
RUN apt update && apt install -y ssh wget unzip build-essential git redis-server sudo autoconf libtool iputils-ping net-tools rsync curl mc tmux libsqlite3-dev xz-utils git-lfs ufw libpq-dev
|
||||
|
||||
RUN wget https://github.com/vlang/v/releases/latest/download/v_linux.zip && \
|
||||
unzip v_linux.zip && \
|
||||
cd v && \
|
||||
./v symlink
|
||||
|
||||
# Install bun.sh for docusaurus
|
||||
RUN curl -fsSL https://bun.sh/install | bash
|
||||
|
||||
# Make a new stage so we can invalidate the cache by passing --no-cache-filter
|
||||
FROM base AS clone
|
||||
|
||||
RUN git clone https://github.com/Incubaid/herolib /opt/herolib
|
||||
|
||||
RUN git clone https://github.com/incubaid/docusaurus_template /root/code/github/incubaid/docusaurus_template
|
||||
|
||||
# Warm the bun node modules cache
|
||||
RUN cd /root/code/github/incubaid/docusaurus_template/template && /root/.bun/bin/bun install
|
||||
|
||||
RUN mkdir -p /root/.vmodules/incubaid
|
||||
|
||||
# Make a place for users to mount their ssh key file. We will copy to .ssh and
|
||||
# change permissions in entrypoint script
|
||||
RUN mkdir -p /root/ssh
|
||||
|
||||
COPY entrypoint.sh /bin/entrypoint.sh
|
||||
|
||||
RUN chmod +x /bin/entrypoint.sh
|
||||
|
||||
# Make a new stage so we can invalidate the cache by passing --no-cache-filter
|
||||
FROM clone AS fetch
|
||||
|
||||
# Fetch to freshen the repos with minimal work
|
||||
RUN cd /opt/herolib && git fetch
|
||||
RUN cd /root/code/github/incubaid/docusaurus_template && git fetch
|
||||
|
||||
|
||||
ENTRYPOINT ["/bin/entrypoint.sh"]
|
||||
268
docker/Readme.md
Normal file
268
docker/Readme.md
Normal file
@@ -0,0 +1,268 @@
|
||||
# `hero-git` Docker Image
|
||||
|
||||
This directory contains a reusable Docker image and entrypoint script used to build Hero / Horus binaries via the V-based **herolib** installers.
|
||||
|
||||
It is designed for two main use cases:
|
||||
|
||||
1. **Local development / testing** – run the herolib CLI (`hero`) and Horus installers in a clean, reproducible environment.
|
||||
2. **CI / Release builds** – build Horus binaries inside this image (using `horus_full_install.vsh`) and export them as release artifacts.
|
||||
|
||||
---
|
||||
|
||||
## Files
|
||||
|
||||
### `Dockerfile`
|
||||
|
||||
The `Dockerfile` builds an Ubuntu-based image with all dependencies needed for `herolib` and its installers.
|
||||
|
||||
Key points:
|
||||
|
||||
* Base: `ubuntu:24.04`
|
||||
* Installs system tooling:
|
||||
|
||||
* `ssh`, `wget`, `curl`, `unzip`, `build-essential`, `git`, `git-lfs`
|
||||
* `redis-server`, `libsqlite3-dev`, `libpq-dev`, `autoconf`, `libtool`, `net-tools`, `iputils-ping`, `rsync`, `mc`, `tmux`, `ufw`, `xz-utils`
|
||||
* Installs **V** compiler:
|
||||
|
||||
* Downloads `v_linux.zip` from the official V GitHub releases
|
||||
* Unzips it and runs `./v symlink` so `v` is on the `PATH`
|
||||
* Installs **Bun** (used for the docusaurus docs template):
|
||||
|
||||
* `curl -fsSL https://bun.sh/install | bash`
|
||||
* Clones:
|
||||
|
||||
* `https://github.com/Incubaid/herolib` into `/opt/herolib`
|
||||
* `https://github.com/incubaid/docusaurus_template` into `/root/code/github/incubaid/docusaurus_template`
|
||||
* Warms Bun dependencies for the docusaurus template (for faster re-use):
|
||||
|
||||
* `cd /root/code/github/incubaid/docusaurus_template/template && bun install`
|
||||
* Prepares directories:
|
||||
|
||||
* `/root/.vmodules/incubaid` – used by V module resolution
|
||||
* `/root/ssh` – optional mount point for SSH keys
|
||||
* Copies `entrypoint.sh` into the image and marks it executable.
|
||||
|
||||
The container’s entrypoint is set to the custom script `/bin/entrypoint.sh`.
|
||||
|
||||
---
|
||||
|
||||
### `entrypoint.sh`
|
||||
|
||||
The entrypoint script is responsible for:
|
||||
|
||||
1. Starting **Redis** in the background.
|
||||
2. Optionally configuring **SSH** (if keys are provided).
|
||||
3. Selecting the correct `herolib` source (mounted or cloned).
|
||||
4. Checking out the desired `herolib` branch.
|
||||
5. Building the `hero` CLI with V and making it available as `/bin/hero`.
|
||||
6. Finally, executing any command passed to `docker run`.
|
||||
|
||||
#### Behavior in detail
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
# Start Redis in the background
|
||||
redis-server --daemonize yes
|
||||
|
||||
# Optional SSH setup: only if /root/ssh has keys
|
||||
if [ -d /root/ssh ] && compgen -G "/root/ssh/*" > /dev/null; then
|
||||
mkdir -p /root/.ssh
|
||||
cp -r /root/ssh/* /root/.ssh/
|
||||
chmod 600 /root/.ssh/*
|
||||
eval "$(ssh-agent)"
|
||||
ssh-add /root/.ssh/*
|
||||
fi
|
||||
|
||||
# Support optionally bind-mounting a local herolib into the container.
|
||||
# If /opt/herolib_mount exists, we use that; otherwise we use the cloned /opt/herolib
|
||||
rm -f /root/.vmodules/freeflowuniverse/herolib
|
||||
if [ -d "/opt/herolib_mount" ]; then
|
||||
ln -s /opt/herolib_mount/lib /root/.vmodules/incubaid/herolib
|
||||
cd /opt/herolib_mount
|
||||
else
|
||||
ln -s /opt/herolib/lib /root/.vmodules/incubaid/herolib
|
||||
cd /opt/herolib
|
||||
git fetch
|
||||
git checkout "${HEROLIB_REF:-development}"
|
||||
git pull
|
||||
fi
|
||||
|
||||
cd cli
|
||||
echo "Building hero..."
|
||||
v -enable-globals hero.v > build.log 2>&1 || (cat build.log && exit 1)
|
||||
ln -s "$(realpath hero)" /bin/hero
|
||||
|
||||
cd /root
|
||||
|
||||
# If a command was provided to `docker run`, execute it as-is.
|
||||
# Example: docker run hero-git:latest bash -lc 'cd /opt/herolib && ./examples/installers/horus/horus_full_install.vsh'
|
||||
if [ "$#" -gt 0 ]; then
|
||||
exec "$@"
|
||||
else
|
||||
# No command passed → give an interactive shell
|
||||
exec bash
|
||||
fi
|
||||
```
|
||||
|
||||
#### Important environment variables
|
||||
|
||||
* `HEROLIB_REF`
|
||||
|
||||
* Git ref (branch, tag, or commit) used when checking out `/opt/herolib`.
|
||||
* Default: `development`.
|
||||
* Example for CI: `HEROLIB_REF=development_nile_installers`.
|
||||
|
||||
#### Optional mounts
|
||||
|
||||
* `/opt/herolib_mount`
|
||||
|
||||
* If you bind-mount your local `herolib` repo here, the container will use it instead of the cloned `/opt/herolib`.
|
||||
* Useful for local development when you want to test uncommitted changes.
|
||||
* `/root/ssh`
|
||||
|
||||
* Optional directory containing SSH keys (e.g. `id_ed25519`) if you need to access private repositories.
|
||||
* If present and non-empty, keys are copied to `/root/.ssh` and added to an SSH agent.
|
||||
|
||||
---
|
||||
|
||||
## What this image is used for
|
||||
|
||||
### 1. Local development / experimentation
|
||||
|
||||
You can drop into the container and manually run installers or commands against `herolib`:
|
||||
|
||||
```bash
|
||||
# Interactive shell with default branch
|
||||
docker run -it --rm hero-git:latest
|
||||
|
||||
# Use a specific herolib ref
|
||||
docker run -it --rm -e HEROLIB_REF=development_nile_installers hero-git:latest
|
||||
|
||||
# Mount a local herolib repo and use that instead of the cloned one
|
||||
docker run -it --rm \
|
||||
-v "$PWD/herolib:/opt/herolib_mount" \
|
||||
hero-git:latest
|
||||
```
|
||||
|
||||
Inside the container you can then run:
|
||||
|
||||
```bash
|
||||
cd /opt/herolib
|
||||
./examples/installers/horus/horus_full_install.vsh
|
||||
```
|
||||
|
||||
### 2. CI / Release builds for Horus
|
||||
|
||||
In CI (e.g. the `release.yml` workflow), this image is used to build Horus binaries via the V-based installer script:
|
||||
|
||||
* The workflow:
|
||||
|
||||
1. Ensures `hero-git:latest` is available on the self-hosted runner.
|
||||
2. Creates a local directory (e.g. `hero-bin/`) on the runner.
|
||||
3. Runs the container, mounting `hero-bin/` into `/root/hero/bin`.
|
||||
4. Inside the container, executes `./examples/installers/horus/horus_full_install.vsh` from `/opt/herolib`.
|
||||
5. The installer writes the compiled Horus binaries into `/root/hero/bin`, which appear on the host inside `hero-bin/`.
|
||||
6. The workflow then packages those binaries into `.tar.gz` archives and uploads them as release artifacts.
|
||||
|
||||
Example CI step:
|
||||
|
||||
```yaml
|
||||
- name: Run horus_full_install installer in container
|
||||
run: |
|
||||
set -euxo pipefail
|
||||
|
||||
docker run --rm \
|
||||
-v "$PWD/hero-bin:/root/hero/bin" \
|
||||
-e HEROLIB_REF=development_nile_installers \
|
||||
hero-git:latest \
|
||||
bash -lc '
|
||||
set -euxo pipefail
|
||||
cd /opt/herolib
|
||||
./examples/installers/horus/horus_full_install.vsh
|
||||
echo "===== AFTER INSTALL, ls -R /root/hero ====="
|
||||
ls -R /root/hero || true
|
||||
'
|
||||
```
|
||||
|
||||
After this step, `hero-bin/` on the host should contain binaries like:
|
||||
|
||||
* `supervisor`
|
||||
* `coordinator`
|
||||
* `horus`
|
||||
* `osiris`
|
||||
* `herorunner`
|
||||
* `runner_osiris`
|
||||
* `runner_sal`
|
||||
|
||||
These are then packaged and attached to the Gitea release.
|
||||
|
||||
---
|
||||
|
||||
## How to build the image
|
||||
|
||||
From the directory containing the `Dockerfile` and `entrypoint.sh`:
|
||||
|
||||
```bash
|
||||
# Build the image (force a clean rebuild)
|
||||
docker build --no-cache -t hero-git:latest .
|
||||
|
||||
# Verify entrypoint contents
|
||||
docker run --rm --entrypoint cat hero-git:latest /bin/entrypoint.sh | tail -n 20
|
||||
```
|
||||
|
||||
You should see the `exec "$@"` block at the end of the script, which ensures that commands passed to `docker run` are executed as-is inside the container.
|
||||
|
||||
---
|
||||
|
||||
## How to run the image (examples)
|
||||
|
||||
### Basic interactive usage
|
||||
|
||||
```bash
|
||||
# Default branch (HEROLIB_REF=development)
|
||||
docker run -it --rm hero-git:latest
|
||||
|
||||
# Specific herolib ref
|
||||
export HEROLIB_REF=development_nile_installers
|
||||
|
||||
docker run -it --rm \
|
||||
-e HEROLIB_REF=$HEROLIB_REF \
|
||||
hero-git:latest
|
||||
```
|
||||
|
||||
### Build Horus binaries into a host directory
|
||||
|
||||
```bash
|
||||
mkdir -p hero-bin
|
||||
|
||||
docker run --rm \
|
||||
-v "$PWD/hero-bin:/root/hero/bin" \
|
||||
-e HEROLIB_REF=development_nile_installers \
|
||||
hero-git:latest \
|
||||
bash -lc '
|
||||
set -euxo pipefail
|
||||
cd /opt/herolib
|
||||
./examples/installers/horus/horus_full_install.vsh
|
||||
ls -al /root/hero/bin
|
||||
'
|
||||
|
||||
ls -al hero-bin
|
||||
```
|
||||
|
||||
You should now see the built Horus binaries on the host in `hero-bin/`.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
* The `hero-git:latest` image encapsulates all dependencies required to build Horus via `herolib`.
|
||||
* `entrypoint.sh`:
|
||||
|
||||
* Starts Redis
|
||||
* Optionally configures SSH
|
||||
* Selects and updates the `herolib` checkout
|
||||
* Builds the `hero` CLI once
|
||||
* Runs any command passed to the container (e.g. Horus installers)
|
||||
* CI uses this image to run `horus_full_install.vsh` and collect Horus binaries from `/root/hero/bin` for releases.
|
||||
44
docker/entrypoint.sh
Normal file
44
docker/entrypoint.sh
Normal file
@@ -0,0 +1,44 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
# Start Redis in the background
|
||||
redis-server --daemonize yes
|
||||
|
||||
# Optional SSH setup: only if /root/ssh has keys
|
||||
if [ -d /root/ssh ] && compgen -G "/root/ssh/*" > /dev/null; then
|
||||
mkdir -p /root/.ssh
|
||||
cp -r /root/ssh/* /root/.ssh/
|
||||
chmod 600 /root/.ssh/*
|
||||
eval "$(ssh-agent)"
|
||||
ssh-add /root/.ssh/*
|
||||
fi
|
||||
|
||||
# Support optionally bind-mounting a local herolib into the container.
|
||||
# If /opt/herolib_mount exists, we use that; otherwise we use the cloned /opt/herolib
|
||||
rm -f /root/.vmodules/freeflowuniverse/herolib
|
||||
if [ -d "/opt/herolib_mount" ]; then
|
||||
ln -s /opt/herolib_mount/lib /root/.vmodules/incubaid/herolib
|
||||
cd /opt/herolib_mount
|
||||
else
|
||||
ln -s /opt/herolib/lib /root/.vmodules/incubaid/herolib
|
||||
cd /opt/herolib
|
||||
git fetch
|
||||
git checkout "${HEROLIB_REF:-development}"
|
||||
git pull
|
||||
fi
|
||||
|
||||
# Build hero CLI once so it's available as /bin/hero
|
||||
cd cli
|
||||
echo "Building hero..."
|
||||
v -enable-globals hero.v > build.log 2>&1 || (cat build.log && exit 1)
|
||||
ln -s "$(realpath hero)" /bin/hero
|
||||
|
||||
cd /root
|
||||
|
||||
# If a command was provided to `docker run`, execute it as-is.
|
||||
if [ "$#" -gt 0 ]; then
|
||||
exec "$@"
|
||||
else
|
||||
# No command passed → give an interactive shell
|
||||
exec bash
|
||||
fi
|
||||
Reference in New Issue
Block a user