130 lines
2.9 KiB
Markdown
130 lines
2.9 KiB
Markdown
# Horus Benchmarks - Quick Start
|
|
|
|
## 1. Start the Stack
|
|
|
|
```bash
|
|
# Terminal 1: Start Redis
|
|
redis-server
|
|
|
|
# Terminal 2: Start Horus
|
|
cd /Users/timurgordon/code/git.ourworld.tf/herocode/horus
|
|
RUST_LOG=info ./target/release/horus all --admin-secret SECRET --kill-ports
|
|
```
|
|
|
|
## 2. Run Benchmarks
|
|
|
|
### Option A: Use the helper script (recommended)
|
|
```bash
|
|
./benches/run_benchmarks.sh
|
|
```
|
|
|
|
### Option B: Run directly with cargo
|
|
```bash
|
|
# All benchmarks
|
|
cargo bench
|
|
|
|
# Specific benchmark suite
|
|
cargo bench --bench horus_stack
|
|
cargo bench --bench stress_test
|
|
|
|
# Specific test
|
|
cargo bench --bench horus_stack -- supervisor_discovery
|
|
|
|
# Quick run (fewer samples)
|
|
cargo bench -- --quick
|
|
```
|
|
|
|
## 3. View Results
|
|
|
|
```bash
|
|
# Open HTML report in browser
|
|
open target/criterion/report/index.html
|
|
|
|
# Or on Linux
|
|
xdg-open target/criterion/report/index.html
|
|
```
|
|
|
|
## Available Benchmark Suites
|
|
|
|
### `horus_stack` - Standard Performance Tests
|
|
- API discovery and metadata
|
|
- Runner management
|
|
- Job operations
|
|
- Concurrency tests
|
|
- Health checks
|
|
- API latency measurements
|
|
|
|
### `stress_test` - Load & Stress Tests
|
|
- High-frequency job submissions (50-200 jobs)
|
|
- Sustained load testing
|
|
- Large payload handling (1KB-100KB)
|
|
- Rapid API calls (100 calls/test)
|
|
- Mixed workload scenarios
|
|
- Connection pool exhaustion (10-100 clients)
|
|
|
|
### `memory_usage` - Memory Profiling
|
|
- Job object memory footprint (10-200 jobs)
|
|
- Client instance memory overhead (1-100 clients)
|
|
- Payload size impact on memory (1KB-1MB)
|
|
- Memory growth patterns under load
|
|
|
|
## Common Commands
|
|
|
|
```bash
|
|
# Run only fast benchmarks
|
|
cargo bench -- --quick
|
|
|
|
# Save baseline for comparison
|
|
cargo bench -- --save-baseline main
|
|
|
|
# Compare against baseline
|
|
cargo bench -- --baseline main
|
|
|
|
# Run with verbose output
|
|
cargo bench -- --verbose
|
|
|
|
# Filter by name
|
|
cargo bench -- concurrent
|
|
cargo bench -- stress
|
|
|
|
# Run specific benchmark group
|
|
cargo bench --bench horus_stack -- api_latency
|
|
|
|
# Run memory benchmarks
|
|
cargo bench --bench memory_usage
|
|
|
|
# Run memory benchmarks with verbose output (shows memory deltas)
|
|
cargo bench --bench memory_usage -- --verbose
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
**"Connection refused"**
|
|
- Make sure Horus stack is running
|
|
- Check ports: 3030 (supervisor), 8081 (osiris), 9652/9653 (coordinator)
|
|
|
|
**"Job timeout"**
|
|
- Increase timeout in benchmark code
|
|
- Check that runners are registered: `curl http://127.0.0.1:3030` (requires POST)
|
|
|
|
**Slow benchmarks**
|
|
- Close other applications
|
|
- Use `--quick` flag for faster runs
|
|
- Reduce sample size in benchmark code
|
|
|
|
## Performance Expectations
|
|
|
|
| Test | Expected Time |
|
|
|------|---------------|
|
|
| supervisor_discovery | < 10ms |
|
|
| supervisor_get_info | < 5ms |
|
|
| job_full_lifecycle | < 100ms |
|
|
| concurrent_jobs (10) | < 500ms |
|
|
| stress_high_frequency (50) | < 2s |
|
|
|
|
## Next Steps
|
|
|
|
- See `benches/README.md` for detailed documentation
|
|
- Modify `benches/horus_stack.rs` to add custom tests
|
|
- Check `target/criterion/` for detailed reports
|