add complete binary and benchmarking

This commit is contained in:
Timur Gordon
2025-11-18 20:39:25 +01:00
parent f66edba1d3
commit 4142f62e54
17 changed files with 2559 additions and 2 deletions

129
benches/QUICK_START.md Normal file
View File

@@ -0,0 +1,129 @@
# Horus Benchmarks - Quick Start
## 1. Start the Stack
```bash
# Terminal 1: Start Redis
redis-server
# Terminal 2: Start Horus
cd /Users/timurgordon/code/git.ourworld.tf/herocode/horus
RUST_LOG=info ./target/release/horus all --admin-secret SECRET --kill-ports
```
## 2. Run Benchmarks
### Option A: Use the helper script (recommended)
```bash
./benches/run_benchmarks.sh
```
### Option B: Run directly with cargo
```bash
# All benchmarks
cargo bench
# Specific benchmark suite
cargo bench --bench horus_stack
cargo bench --bench stress_test
# Specific test
cargo bench --bench horus_stack -- supervisor_discovery
# Quick run (fewer samples)
cargo bench -- --quick
```
## 3. View Results
```bash
# Open HTML report in browser
open target/criterion/report/index.html
# Or on Linux
xdg-open target/criterion/report/index.html
```
## Available Benchmark Suites
### `horus_stack` - Standard Performance Tests
- API discovery and metadata
- Runner management
- Job operations
- Concurrency tests
- Health checks
- API latency measurements
### `stress_test` - Load & Stress Tests
- High-frequency job submissions (50-200 jobs)
- Sustained load testing
- Large payload handling (1KB-100KB)
- Rapid API calls (100 calls/test)
- Mixed workload scenarios
- Connection pool exhaustion (10-100 clients)
### `memory_usage` - Memory Profiling
- Job object memory footprint (10-200 jobs)
- Client instance memory overhead (1-100 clients)
- Payload size impact on memory (1KB-1MB)
- Memory growth patterns under load
## Common Commands
```bash
# Run only fast benchmarks
cargo bench -- --quick
# Save baseline for comparison
cargo bench -- --save-baseline main
# Compare against baseline
cargo bench -- --baseline main
# Run with verbose output
cargo bench -- --verbose
# Filter by name
cargo bench -- concurrent
cargo bench -- stress
# Run specific benchmark group
cargo bench --bench horus_stack -- api_latency
# Run memory benchmarks
cargo bench --bench memory_usage
# Run memory benchmarks with verbose output (shows memory deltas)
cargo bench --bench memory_usage -- --verbose
```
## Troubleshooting
**"Connection refused"**
- Make sure Horus stack is running
- Check ports: 3030 (supervisor), 8081 (osiris), 9652/9653 (coordinator)
**"Job timeout"**
- Increase timeout in benchmark code
- Check that runners are registered: `curl http://127.0.0.1:3030` (requires POST)
**Slow benchmarks**
- Close other applications
- Use `--quick` flag for faster runs
- Reduce sample size in benchmark code
## Performance Expectations
| Test | Expected Time |
|------|---------------|
| supervisor_discovery | < 10ms |
| supervisor_get_info | < 5ms |
| job_full_lifecycle | < 100ms |
| concurrent_jobs (10) | < 500ms |
| stress_high_frequency (50) | < 2s |
## Next Steps
- See `benches/README.md` for detailed documentation
- Modify `benches/horus_stack.rs` to add custom tests
- Check `target/criterion/` for detailed reports