218 lines
5.8 KiB
Markdown
218 lines
5.8 KiB
Markdown
# Memory Usage Benchmarks
|
|
|
|
Benchmarks for measuring memory consumption of the Horus stack components.
|
|
|
|
## Overview
|
|
|
|
The memory benchmarks measure heap memory usage for various operations:
|
|
- Job creation and storage
|
|
- Client instantiation
|
|
- Payload size impact
|
|
- Memory growth under load
|
|
|
|
## Benchmarks
|
|
|
|
### 1. `memory_job_creation`
|
|
Measures memory usage when creating multiple Job objects in memory.
|
|
|
|
**Test sizes**: 10, 50, 100, 200 jobs
|
|
|
|
**What it measures**:
|
|
- Memory allocated per job object
|
|
- Heap growth with increasing job count
|
|
- Memory efficiency of Job structure
|
|
|
|
**Expected results**:
|
|
- Linear memory growth with job count
|
|
- ~1-2 KB per job object (depending on payload)
|
|
|
|
### 2. `memory_client_creation`
|
|
Measures memory overhead of creating multiple Supervisor client instances.
|
|
|
|
**Test sizes**: 1, 10, 50, 100 clients
|
|
|
|
**What it measures**:
|
|
- Memory per client instance
|
|
- Connection pool overhead
|
|
- HTTP client memory footprint
|
|
|
|
**Expected results**:
|
|
- ~10-50 KB per client instance
|
|
- Includes HTTP client, connection pools, and buffers
|
|
|
|
### 3. `memory_payload_sizes`
|
|
Measures memory usage with different payload sizes.
|
|
|
|
**Test sizes**: 1KB, 10KB, 100KB, 1MB payloads
|
|
|
|
**What it measures**:
|
|
- Memory overhead of JSON serialization
|
|
- String allocation costs
|
|
- Payload storage efficiency
|
|
|
|
**Expected results**:
|
|
- Memory usage should scale linearly with payload size
|
|
- Small overhead for JSON structure (~5-10%)
|
|
|
|
## Running Memory Benchmarks
|
|
|
|
```bash
|
|
# Run all memory benchmarks
|
|
cargo bench --bench memory_usage
|
|
|
|
# Run specific memory test
|
|
cargo bench --bench memory_usage -- memory_job_creation
|
|
|
|
# Run with verbose output to see memory deltas
|
|
cargo bench --bench memory_usage -- --verbose
|
|
```
|
|
|
|
## Interpreting Results
|
|
|
|
The benchmarks print memory deltas to stderr during execution:
|
|
|
|
```
|
|
Memory delta for 100 jobs: 156 KB
|
|
Memory delta for 50 clients: 2048 KB
|
|
Memory delta for 100KB payload: 105 KB
|
|
```
|
|
|
|
### Memory Delta Interpretation
|
|
|
|
- **Positive delta**: Memory was allocated during the operation
|
|
- **Zero delta**: No significant memory change (may be reusing existing allocations)
|
|
- **Negative delta**: Memory was freed (garbage collection, deallocations)
|
|
|
|
### Platform Differences
|
|
|
|
**macOS**: Uses `ps` command to read RSS (Resident Set Size)
|
|
**Linux**: Reads `/proc/self/status` for VmRSS
|
|
|
|
RSS includes:
|
|
- Heap allocations
|
|
- Stack memory
|
|
- Shared libraries (mapped into process)
|
|
- Memory-mapped files
|
|
|
|
## Limitations
|
|
|
|
1. **Granularity**: OS-level memory reporting may not capture small allocations
|
|
2. **Timing**: Memory measurements happen before/after operations, not continuously
|
|
3. **GC effects**: Rust's allocator may not immediately release memory to OS
|
|
4. **Shared memory**: RSS includes shared library memory
|
|
|
|
## Best Practices
|
|
|
|
### For Accurate Measurements
|
|
|
|
1. **Run multiple iterations**: Criterion handles this automatically
|
|
2. **Warm up**: First iterations may show higher memory due to lazy initialization
|
|
3. **Isolate tests**: Run memory benchmarks separately from performance benchmarks
|
|
4. **Monitor trends**: Compare results over time, not absolute values
|
|
|
|
### Memory Optimization Tips
|
|
|
|
If benchmarks show high memory usage:
|
|
|
|
1. **Check payload sizes**: Large payloads consume proportional memory
|
|
2. **Limit concurrent operations**: Too many simultaneous jobs/clients increase memory
|
|
3. **Review data structures**: Ensure efficient serialization
|
|
4. **Profile with tools**: Use `heaptrack` (Linux) or `instruments` (macOS) for detailed analysis
|
|
|
|
## Advanced Profiling
|
|
|
|
For detailed memory profiling beyond these benchmarks:
|
|
|
|
### macOS
|
|
```bash
|
|
# Use Instruments
|
|
instruments -t Allocations -D memory_trace.trace ./target/release/horus
|
|
|
|
# Use heap profiler
|
|
cargo install cargo-instruments
|
|
cargo instruments --bench memory_usage --template Allocations
|
|
```
|
|
|
|
### Linux
|
|
```bash
|
|
# Use Valgrind massif
|
|
valgrind --tool=massif --massif-out-file=massif.out \
|
|
./target/release/deps/memory_usage-*
|
|
|
|
# Visualize with massif-visualizer
|
|
massif-visualizer massif.out
|
|
|
|
# Use heaptrack
|
|
heaptrack ./target/release/deps/memory_usage-*
|
|
heaptrack_gui heaptrack.memory_usage.*.gz
|
|
```
|
|
|
|
### Cross-platform
|
|
```bash
|
|
# Use dhat (heap profiler)
|
|
cargo install dhat
|
|
# Add dhat to your benchmark and run
|
|
cargo bench --bench memory_usage --features dhat-heap
|
|
```
|
|
|
|
## Continuous Monitoring
|
|
|
|
Integrate memory benchmarks into CI/CD:
|
|
|
|
```bash
|
|
# Run and save baseline
|
|
cargo bench --bench memory_usage -- --save-baseline memory-main
|
|
|
|
# Compare in PR
|
|
cargo bench --bench memory_usage -- --baseline memory-main
|
|
|
|
# Fail if memory usage increases >10%
|
|
# (requires custom scripting to parse Criterion output)
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### "Memory delta is always 0"
|
|
- OS may not update RSS immediately
|
|
- Allocations might be too small to measure
|
|
- Try increasing iteration count or operation size
|
|
|
|
### "Memory keeps growing"
|
|
- Check for memory leaks
|
|
- Verify objects are being dropped
|
|
- Use `cargo clippy` to find potential issues
|
|
|
|
### "Results are inconsistent"
|
|
- Other processes may be affecting measurements
|
|
- Run benchmarks on idle system
|
|
- Increase sample size in benchmark code
|
|
|
|
## Example Output
|
|
|
|
```
|
|
memory_job_creation/10 time: [45.2 µs 46.1 µs 47.3 µs]
|
|
Memory delta for 10 jobs: 24 KB
|
|
|
|
memory_job_creation/50 time: [198.4 µs 201.2 µs 204.8 µs]
|
|
Memory delta for 50 jobs: 98 KB
|
|
|
|
memory_job_creation/100 time: [387.6 µs 392.1 µs 397.4 µs]
|
|
Memory delta for 100 jobs: 187 KB
|
|
|
|
memory_client_creation/1 time: [234.5 µs 238.2 µs 242.6 µs]
|
|
Memory delta for 1 clients: 45 KB
|
|
|
|
memory_payload_sizes/1KB time: [12.3 µs 12.6 µs 13.0 µs]
|
|
Memory delta for 1KB payload: 2 KB
|
|
|
|
memory_payload_sizes/100KB time: [156.7 µs 159.4 µs 162.8 µs]
|
|
Memory delta for 100KB payload: 105 KB
|
|
```
|
|
|
|
## Related Documentation
|
|
|
|
- [Performance Benchmarks](./README.md)
|
|
- [Stress Tests](./README.md#stress-tests)
|
|
- [Rust Performance Book](https://nnethercote.github.io/perf-book/)
|
|
- [Criterion.rs Documentation](https://bheisler.github.io/criterion.rs/book/)
|