add complete binary and benchmarking
This commit is contained in:
129
benches/QUICK_START.md
Normal file
129
benches/QUICK_START.md
Normal file
@@ -0,0 +1,129 @@
|
||||
# Horus Benchmarks - Quick Start
|
||||
|
||||
## 1. Start the Stack
|
||||
|
||||
```bash
|
||||
# Terminal 1: Start Redis
|
||||
redis-server
|
||||
|
||||
# Terminal 2: Start Horus
|
||||
cd /Users/timurgordon/code/git.ourworld.tf/herocode/horus
|
||||
RUST_LOG=info ./target/release/horus all --admin-secret SECRET --kill-ports
|
||||
```
|
||||
|
||||
## 2. Run Benchmarks
|
||||
|
||||
### Option A: Use the helper script (recommended)
|
||||
```bash
|
||||
./benches/run_benchmarks.sh
|
||||
```
|
||||
|
||||
### Option B: Run directly with cargo
|
||||
```bash
|
||||
# All benchmarks
|
||||
cargo bench
|
||||
|
||||
# Specific benchmark suite
|
||||
cargo bench --bench horus_stack
|
||||
cargo bench --bench stress_test
|
||||
|
||||
# Specific test
|
||||
cargo bench --bench horus_stack -- supervisor_discovery
|
||||
|
||||
# Quick run (fewer samples)
|
||||
cargo bench -- --quick
|
||||
```
|
||||
|
||||
## 3. View Results
|
||||
|
||||
```bash
|
||||
# Open HTML report in browser
|
||||
open target/criterion/report/index.html
|
||||
|
||||
# Or on Linux
|
||||
xdg-open target/criterion/report/index.html
|
||||
```
|
||||
|
||||
## Available Benchmark Suites
|
||||
|
||||
### `horus_stack` - Standard Performance Tests
|
||||
- API discovery and metadata
|
||||
- Runner management
|
||||
- Job operations
|
||||
- Concurrency tests
|
||||
- Health checks
|
||||
- API latency measurements
|
||||
|
||||
### `stress_test` - Load & Stress Tests
|
||||
- High-frequency job submissions (50-200 jobs)
|
||||
- Sustained load testing
|
||||
- Large payload handling (1KB-100KB)
|
||||
- Rapid API calls (100 calls/test)
|
||||
- Mixed workload scenarios
|
||||
- Connection pool exhaustion (10-100 clients)
|
||||
|
||||
### `memory_usage` - Memory Profiling
|
||||
- Job object memory footprint (10-200 jobs)
|
||||
- Client instance memory overhead (1-100 clients)
|
||||
- Payload size impact on memory (1KB-1MB)
|
||||
- Memory growth patterns under load
|
||||
|
||||
## Common Commands
|
||||
|
||||
```bash
|
||||
# Run only fast benchmarks
|
||||
cargo bench -- --quick
|
||||
|
||||
# Save baseline for comparison
|
||||
cargo bench -- --save-baseline main
|
||||
|
||||
# Compare against baseline
|
||||
cargo bench -- --baseline main
|
||||
|
||||
# Run with verbose output
|
||||
cargo bench -- --verbose
|
||||
|
||||
# Filter by name
|
||||
cargo bench -- concurrent
|
||||
cargo bench -- stress
|
||||
|
||||
# Run specific benchmark group
|
||||
cargo bench --bench horus_stack -- api_latency
|
||||
|
||||
# Run memory benchmarks
|
||||
cargo bench --bench memory_usage
|
||||
|
||||
# Run memory benchmarks with verbose output (shows memory deltas)
|
||||
cargo bench --bench memory_usage -- --verbose
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"Connection refused"**
|
||||
- Make sure Horus stack is running
|
||||
- Check ports: 3030 (supervisor), 8081 (osiris), 9652/9653 (coordinator)
|
||||
|
||||
**"Job timeout"**
|
||||
- Increase timeout in benchmark code
|
||||
- Check that runners are registered: `curl http://127.0.0.1:3030` (requires POST)
|
||||
|
||||
**Slow benchmarks**
|
||||
- Close other applications
|
||||
- Use `--quick` flag for faster runs
|
||||
- Reduce sample size in benchmark code
|
||||
|
||||
## Performance Expectations
|
||||
|
||||
| Test | Expected Time |
|
||||
|------|---------------|
|
||||
| supervisor_discovery | < 10ms |
|
||||
| supervisor_get_info | < 5ms |
|
||||
| job_full_lifecycle | < 100ms |
|
||||
| concurrent_jobs (10) | < 500ms |
|
||||
| stress_high_frequency (50) | < 2s |
|
||||
|
||||
## Next Steps
|
||||
|
||||
- See `benches/README.md` for detailed documentation
|
||||
- Modify `benches/horus_stack.rs` to add custom tests
|
||||
- Check `target/criterion/` for detailed reports
|
||||
Reference in New Issue
Block a user