refactor wip

core/worker/examples/README.md

@@ -0,0 +1,197 @@
+# Worker Examples
+
+This directory contains example configurations and test scripts for both OSIS and System worker binaries.
+
+## Overview
+
+Both examples demonstrate the ping/pong functionality built into the Hero workers:
+- Workers automatically detect jobs with script content "ping"
+- They respond immediately with "pong" without executing the Rhai engine
+- This provides a fast health check and connectivity test mechanism
+
+## Prerequisites
+
+1. **Redis Server**: Both examples require a running Redis server
+   ```bash
+   # Install Redis (macOS)
+   brew install redis
+   
+   # Start Redis server
+   redis-server
+   ```
+
+2. **Rust Environment**: Make sure you can build the worker binaries
+   ```bash
+   cd /path/to/herocode/hero/core/worker
+   cargo build --bin osis --bin system
+   ```
+
+## OSIS Worker Example
+
+**Location**: `examples/osis/`
+
+The OSIS (Operating System Integration Service) worker processes jobs synchronously, one at a time.
+
+### Files
+- `config.toml` - Configuration for the OSIS worker
+- `example.sh` - Test script that demonstrates ping/pong functionality
+
+### Usage
+```bash
+cd examples/osis
+./example.sh
+```
+
+### What the script does:
+1. Checks Redis connectivity
+2. Cleans up any existing jobs
+3. Starts the OSIS worker in the background
+4. Sends 3 ping jobs sequentially
+5. Verifies each job receives a "pong" response
+6. Reports success/failure statistics
+7. Cleans up worker and Redis data
+
+### Expected Output
+```
+=== OSIS Worker Example ===
+✅ Redis is running
+✅ OSIS worker started (PID: 12345)
+📤 Sending ping job: ping_job_1_1234567890
+✅ Job ping_job_1_1234567890 completed successfully with result: pong
+...
+🎉 All tests passed! OSIS worker is working correctly.
+```
+
+## System Worker Example
+
+**Location**: `examples/system/`
+
+The System worker processes jobs asynchronously, handling multiple jobs concurrently.
+
+### Files
+- `config.toml` - Configuration for the System worker (includes async settings)
+- `example.sh` - Test script that demonstrates concurrent ping/pong functionality
+
+### Usage
+```bash
+cd examples/system
+./example.sh
+```
+
+### What the script does:
+1. Checks Redis connectivity
+2. Cleans up any existing jobs
+3. Starts the System worker with stats reporting
+4. Sends 5 concurrent ping jobs
+5. Sends 10 rapid-fire ping jobs to test async capabilities
+6. Verifies all jobs receive "pong" responses
+7. Reports comprehensive success/failure statistics
+8. Cleans up worker and Redis data
+
+### Expected Output
+```
+=== System Worker Example ===
+✅ Redis is running
+✅ System worker started (PID: 12345)
+📤 Sending ping job: ping_job_1_1234567890123
+✅ Job ping_job_1_1234567890123 completed successfully with result: pong
+...
+🎉 All tests passed! System worker is handling concurrent jobs correctly.
+Overall success rate: 15/15
+```
+
+## Configuration Details
+
+### OSIS Configuration (`examples/osis/config.toml`)
+```toml
+worker_id = "osis_example_worker"
+redis_url = "redis://localhost:6379"
+db_path = "/tmp/osis_example_db"
+preserve_tasks = false
+
+[worker_type]
+type = "sync"
+
+[logging]
+timestamps = true
+level = "info"
+```
+
+### System Configuration (`examples/system/config.toml`)
+```toml
+worker_id = "system_example_worker"
+redis_url = "redis://localhost:6379"
+db_path = "/tmp/system_example_db"
+preserve_tasks = false
+
+[worker_type]
+type = "async"
+default_timeout_seconds = 30
+
+[logging]
+timestamps = true
+level = "info"
+```
+
+## Key Differences
+
+| Feature | OSIS Worker | System Worker |
+|---------|-------------|---------------|
+| **Processing** | Sequential (one job at a time) | Concurrent (multiple jobs simultaneously) |
+| **Use Case** | System-level operations requiring resource management | High-throughput job processing |
+| **Timeout** | No timeout configuration | Configurable job timeouts |
+| **Stats** | Basic logging | Optional statistics reporting (`--show-stats`) |
+| **Job Handling** | Blocking job execution | Non-blocking async job execution |
+
+## Troubleshooting
+
+### Redis Connection Issues
+```bash
+# Check if Redis is running
+redis-cli ping
+
+# Check Redis logs
+redis-server --loglevel verbose
+```
+
+### Worker Compilation Issues
+```bash
+# Clean and rebuild
+cargo clean
+cargo build --bin osis --bin system
+```
+
+### Job Processing Issues
+- Check Redis for stuck jobs: `redis-cli keys "hero:*"`
+- Clear all Hero jobs: `redis-cli eval "return redis.call('del', unpack(redis.call('keys', 'hero:*')))" 0`
+- Check worker logs for detailed error messages
+
+## Extending the Examples
+
+### Adding Custom Jobs
+To test with custom Rhai scripts instead of ping jobs:
+
+1. Modify the job creation in the shell scripts:
+   ```bash
+   # Replace "ping" with your Rhai script
+   redis-cli -u "$REDIS_URL" hset "hero:job:$job_id" \
+       script "your_rhai_script_here"
+   ```
+
+2. Update result verification to expect your script's output instead of "pong"
+
+### Testing Different Configurations
+- Modify `config.toml` files to test different Redis URLs, database paths, or logging levels
+- Test with `preserve_tasks = true` to inspect job details after completion
+- Adjust timeout values in the System worker configuration
+
+## Architecture Notes
+
+Both examples demonstrate the unified Worker trait architecture:
+- **Common Interface**: Both workers implement the same `Worker` trait
+- **Ping/Pong Handling**: Built into the trait's `spawn` method before job delegation
+- **Redis Integration**: Uses the shared Job struct from `hero_job` crate
+- **Configuration**: TOML-based configuration with CLI overrides
+- **Graceful Shutdown**: Both workers handle SIGTERM/SIGINT properly
+
+This architecture allows for easy extension with new worker types while maintaining consistent behavior and configuration patterns.