refactor wip
This commit is contained in:
Timur Gordon
@@ -1,190 +0,0 @@
|
||||
# Architecture of the `rhai_supervisor` Crate
|
||||
|
||||
The `rhai_supervisor` crate provides a Redis-based client library for submitting Rhai scripts to distributed worker services and awaiting their execution results. It implements a request-reply pattern using Redis as the message broker.
|
||||
|
||||
## Core Architecture
|
||||
|
||||
The client follows a builder pattern design with clear separation of concerns:
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[RhaiSupervisorBuilder] --> B[RhaiSupervisor]
|
||||
B --> C[PlayRequestBuilder]
|
||||
C --> D[PlayRequest]
|
||||
D --> E[Redis Task Queue]
|
||||
E --> F[Worker Service]
|
||||
F --> G[Redis Reply Queue]
|
||||
G --> H[Client Response]
|
||||
|
||||
subgraph "Client Components"
|
||||
A
|
||||
B
|
||||
C
|
||||
D
|
||||
end
|
||||
|
||||
subgraph "Redis Infrastructure"
|
||||
E
|
||||
G
|
||||
end
|
||||
|
||||
subgraph "External Services"
|
||||
F
|
||||
end
|
||||
```
|
||||
|
||||
## Key Components
|
||||
|
||||
### 1. RhaiSupervisorBuilder
|
||||
|
||||
A builder pattern implementation for constructing `RhaiSupervisor` instances with proper configuration validation.
|
||||
|
||||
**Responsibilities:**
|
||||
- Configure Redis connection URL
|
||||
- Set caller ID for task attribution
|
||||
- Validate configuration before building client
|
||||
|
||||
**Key Methods:**
|
||||
- `caller_id(id: &str)` - Sets the caller identifier
|
||||
- `redis_url(url: &str)` - Configures Redis connection
|
||||
- `build()` - Creates the final `RhaiSupervisor` instance
|
||||
|
||||
### 2. RhaiSupervisor
|
||||
|
||||
The main client interface that manages Redis connections and provides factory methods for creating play requests.
|
||||
|
||||
**Responsibilities:**
|
||||
- Maintain Redis connection pool
|
||||
- Provide factory methods for request builders
|
||||
- Handle low-level Redis operations
|
||||
- Manage task status queries
|
||||
|
||||
**Key Methods:**
|
||||
- `new_play_request()` - Creates a new `PlayRequestBuilder`
|
||||
- `get_task_status(task_id)` - Queries task status from Redis
|
||||
- Internal methods for Redis operations
|
||||
|
||||
### 3. PlayRequestBuilder
|
||||
|
||||
A fluent builder for constructing and submitting script execution requests.
|
||||
|
||||
**Responsibilities:**
|
||||
- Configure script execution parameters
|
||||
- Handle script loading from files or strings
|
||||
- Manage request timeouts
|
||||
- Provide submission methods (fire-and-forget vs await-response)
|
||||
|
||||
**Key Methods:**
|
||||
- `worker_id(id: &str)` - Target worker queue (determines which worker processes the task)
|
||||
- `context_id(id: &str)` - Target context ID (determines execution context/circle)
|
||||
- `script(content: &str)` - Set script content directly
|
||||
- `script_path(path: &str)` - Load script from file
|
||||
- `timeout(duration: Duration)` - Set execution timeout
|
||||
- `submit()` - Fire-and-forget submission
|
||||
- `await_response()` - Submit and wait for result
|
||||
|
||||
**Architecture Note:** The decoupling of `worker_id` and `context_id` allows a single worker to process tasks for multiple contexts (circles), providing greater deployment flexibility.
|
||||
|
||||
### 4. Data Structures
|
||||
|
||||
#### RhaiTaskDetails
|
||||
Represents the complete state of a task throughout its lifecycle.
|
||||
|
||||
```rust
|
||||
pub struct RhaiTaskDetails {
|
||||
pub task_id: String,
|
||||
pub script: String,
|
||||
pub status: String, // "pending", "processing", "completed", "error"
|
||||
pub output: Option<String>,
|
||||
pub error: Option<String>,
|
||||
pub created_at: DateTime<Utc>,
|
||||
pub updated_at: DateTime<Utc>,
|
||||
pub caller_id: String,
|
||||
}
|
||||
```
|
||||
|
||||
#### RhaiSupervisorError
|
||||
Comprehensive error handling for various failure scenarios:
|
||||
- `RedisError` - Redis connection/operation failures
|
||||
- `SerializationError` - JSON serialization/deserialization issues
|
||||
- `Timeout` - Task execution timeouts
|
||||
- `TaskNotFound` - Missing tasks after submission
|
||||
|
||||
## Communication Protocol
|
||||
|
||||
### Task Submission Flow
|
||||
|
||||
1. **Task Creation**: Client generates unique UUID for task identification
|
||||
2. **Task Storage**: Task details stored in Redis hash: `rhailib:<task_id>`
|
||||
3. **Queue Submission**: Task ID pushed to worker queue: `rhailib:<worker_id>`
|
||||
4. **Reply Queue Setup**: Client listens on: `rhailib:reply:<task_id>`
|
||||
|
||||
### Redis Key Patterns
|
||||
|
||||
- **Task Storage**: `rhailib:<task_id>` (Redis Hash)
|
||||
- **Worker Queues**: `rhailib:<worker_id>` (Redis List)
|
||||
- **Reply Queues**: `rhailib:reply:<task_id>` (Redis List)
|
||||
|
||||
### Message Flow Diagram
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant C as Client
|
||||
participant R as Redis
|
||||
participant W as Worker
|
||||
|
||||
C->>R: HSET rhailib:task_id (task details)
|
||||
C->>R: LPUSH rhailib:worker_id task_id
|
||||
C->>R: BLPOP rhailib:reply:task_id (blocking)
|
||||
|
||||
W->>R: BRPOP rhailib:worker_id (blocking)
|
||||
W->>W: Execute Rhai Script
|
||||
W->>R: LPUSH rhailib:reply:task_id (result)
|
||||
|
||||
R->>C: Return result from BLPOP
|
||||
C->>R: DEL rhailib:reply:task_id (cleanup)
|
||||
```
|
||||
|
||||
## Concurrency and Async Design
|
||||
|
||||
The client is built on `tokio` for asynchronous operations:
|
||||
|
||||
- **Connection Pooling**: Uses Redis multiplexed connections for efficiency
|
||||
- **Non-blocking Operations**: All Redis operations are async
|
||||
- **Timeout Handling**: Configurable timeouts with proper cleanup
|
||||
- **Error Propagation**: Comprehensive error handling with context
|
||||
|
||||
## Configuration and Deployment
|
||||
|
||||
### Prerequisites
|
||||
- Redis server accessible to both client and workers
|
||||
- Proper network connectivity between components
|
||||
- Sufficient Redis memory for task storage
|
||||
|
||||
### Configuration Options
|
||||
- **Redis URL**: Connection string for Redis instance
|
||||
- **Caller ID**: Unique identifier for client instance
|
||||
- **Timeouts**: Per-request timeout configuration
|
||||
- **Worker Targeting**: Direct worker queue addressing
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Task Isolation**: Each task uses unique identifiers
|
||||
- **Queue Separation**: Worker-specific queues prevent cross-contamination
|
||||
- **Cleanup**: Automatic cleanup of reply queues after completion
|
||||
- **Error Handling**: Secure error propagation without sensitive data leakage
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
- **Scalability**: Horizontal scaling through multiple worker instances
|
||||
- **Throughput**: Limited by Redis performance and network latency
|
||||
- **Memory Usage**: Efficient with connection pooling and cleanup
|
||||
- **Latency**: Low latency for local Redis deployments
|
||||
|
||||
## Integration Points
|
||||
|
||||
The client integrates with:
|
||||
- **Worker Services**: Via Redis queue protocol
|
||||
- **Monitoring Systems**: Through structured logging
|
||||
- **Application Code**: Via builder pattern API
|
||||
- **Configuration Systems**: Through environment variables and builders
|
||||
Reference in New Issue
Block a user