move rhailib to herolib

rhailib/src/monitor/Cargo.toml

@@ -0,0 +1,23 @@
+[package]
+name = "monitor"
+version = "0.1.0"
+edition = "2021"
+
+[dependencies]
+anyhow = "1.0"
+clap = { version = "4.4", features = ["derive"] }
+tokio = { version = "1", features = ["macros", "rt-multi-thread", "signal", "time"] } # time feature might be needed later
+tracing = "0.1"
+tracing-subscriber = { version = "0.3", features = ["fmt"] }
+redis = { version = "0.25.0", features = ["tokio-comp"] } # For Redis communication
+prettytable-rs = "0.10.0" # For displaying tasks in a formatted table
+clearscreen = "2.0.1"      # For clearing the terminal screen
+chrono = { version = "0.4", features = ["serde"] } # For timestamps
+futures = "0.3"
+
+# If the monitor library needs to use parts of rhailib (e.g. Redis connections, task definitions):
+# rhailib = { path = ".." } # Assuming monitor is a direct sub-directory of rhailib workspace member
+
+[[bin]]
+name = "monitor"
+path = "src/main.rs"

rhailib/src/monitor/README.md

@@ -0,0 +1,67 @@
+# Rhai Worker Monitor (`monitor`)
+
+`monitor` is a command-line tool designed to observe and display live information about Rhai workers managed by `rhailib`. It provides insights into Redis queue congestion and a table of tasks being processed by specified workers.
+
+## Features (Planned)
+
+*   **Live Redis Queue Visualization**: Displays a textual, horizontal plot showing the number of tasks in the Redis queue for each monitored worker. The plot will be color-coded to indicate congestion levels and will update by polling the queue size.
+*   **Task Table**: Shows a table of tasks associated with each worker, including task hash, creation date, status (e.g., pending, running, completed, failed), and potentially other details.
+
+## Prerequisites
+
+*   Rust and Cargo installed.
+*   Access to the Redis instance used by the Rhai workers.
+
+## Building
+
+Navigate to the `rhailib/monitor` crate's root directory and build the project:
+
+```bash
+cargo build
+```
+
+## Usage
+
+To run the monitor, you need to specify which worker queues you want to observe using the `--workers` (or `-w`) flag. Provide a comma-separated list of worker names.
+
+From the `rhailib/monitor` root directory:
+
+```bash
+cargo run -- --workers <worker_name_1>[,<worker_name_2>,...]
+```
+
+Or from the parent `rhailib` directory (workspace root):
+
+```bash
+cargo run -p monitor -- --workers <worker_name_1>[,<worker_name_2>,...]
+```
+
+**Examples:**
+
+*   Monitor a single worker named `my_default_worker` (from `rhailib/monitor`):
+
+    ```bash
+    cargo run -- --workers my_default_worker
+    ```
+
+*   Monitor multiple workers, `image_processing_worker` and `data_analysis_worker` (from `rhailib` workspace root):
+
+    ```bash
+    cargo run -p monitor -- --workers image_processing_worker,data_analysis_worker
+    ```
+
+### Command-Line Options
+
+*   `-w, --workers <WORKERS>`: (Required) A comma-separated list of worker names to monitor.
+
+(Future options might include Redis connection parameters, polling intervals, etc.)
+
+## Development
+
+The core logic for the monitor is located in `rhailib/monitor/src/`:
+*   `lib.rs`: Main library file, defines modules.
+*   `cli_logic.rs`: Handles argument parsing, Redis interaction, and orchestrates the display.
+*   `plot.rs`: Responsible for generating the textual queue visualization.
+*   `tasks.rs`: Responsible for fetching and displaying the task table.
+
+The binary entry point is `rhailib/monitor/src/main.rs`.

rhailib/src/monitor/cmd/main.rs

@@ -0,0 +1,37 @@
+// File: /Users/timurgordon/code/git.ourworld.tf/herocode/rhailib/src/monitor/cmd/main.rs
+use anyhow::Result;
+use clap::Parser;
+
+// This assumes that `rhailib/src/lib.rs` will have `pub mod monitor;`
+// and `rhailib/src/monitor/mod.rs` will have `pub mod cli_logic;`
+// and `cli_logic.rs` will contain `pub async fn start_monitoring`.
+// The `crate::` prefix refers to the `rhailib` crate root.
+
+#[derive(Parser, Debug)]
+#[clap(author, version, about = "Rhai Worker Live Monitor", long_about = None)]
+struct Args {
+    /// Comma-separated list of worker names to monitor
+    #[clap(short, long, value_delimiter = ',')]
+    workers: Vec<String>,
+}
+
+#[tokio::main]
+async fn main() -> Result<()> {
+    tracing_subscriber::fmt::init();
+
+    let args = Args::parse();
+
+    if args.workers.is_empty() {
+        eprintln!("Error: At least one worker name must be provided via --workers.");
+        // Consider returning an Err or using clap's built-in required attributes.
+        std::process::exit(1); 
+    }
+
+    tracing::info!("Monitor CLI starting for workers: {:?}", args.workers);
+    
+    // Call the monitoring logic from the `cli_logic` submodule within the `monitor` module
+    crate::monitor::cli_logic::start_monitoring(&args.workers).await?;
+
+    tracing::info!("Monitor CLI finished.");
+    Ok(())
+}

rhailib/src/monitor/docs/ARCHITECTURE.md

@@ -0,0 +1,61 @@
+# Architecture of the `monitor` Crate
+
+The `monitor` crate provides a command-line interface for monitoring and managing Rhai task execution across the rhailib ecosystem. It offers real-time visibility into task queues, execution status, and system performance.
+
+## Core Architecture
+
+```mermaid
+graph TD
+    A[Monitor CLI] --> B[Task Monitoring]
+    A --> C[Queue Management]
+    A --> D[Performance Metrics]
+    
+    B --> B1[Redis Task Tracking]
+    B --> B2[Status Visualization]
+    B --> B3[Real-time Updates]
+    
+    C --> C1[Queue Inspection]
+    C --> C2[Task Management]
+    C --> C3[Worker Status]
+    
+    D --> D1[Performance Plotting]
+    D --> D2[Metrics Collection]
+    D --> D3[Historical Analysis]
+```
+
+## Key Components
+
+### 1. CLI Logic (`cli_logic.rs`)
+- **Command Processing**: Handles user commands and interface
+- **Real-time Monitoring**: Continuous task status updates
+- **Interactive Interface**: User-friendly command-line experience
+
+### 2. Task Management (`tasks.rs`)
+- **Task Discovery**: Finds and tracks tasks across Redis queues
+- **Status Reporting**: Provides detailed task execution information
+- **Queue Analysis**: Monitors queue depths and processing rates
+
+### 3. Performance Plotting (`plot.rs`)
+- **Metrics Visualization**: Creates performance charts and graphs
+- **Trend Analysis**: Historical performance tracking
+- **System Health**: Overall system performance indicators
+
+## Features
+
+- **Real-time Task Monitoring**: Live updates of task execution status
+- **Queue Management**: Inspection and management of Redis task queues
+- **Performance Metrics**: System performance visualization and analysis
+- **Interactive CLI**: User-friendly command-line interface
+- **Multi-worker Support**: Monitoring across multiple worker instances
+
+## Dependencies
+
+- **Redis Integration**: Direct Redis connectivity for queue monitoring
+- **CLI Framework**: Clap for command-line argument parsing
+- **Async Runtime**: Tokio for asynchronous operations
+- **Visualization**: Pretty tables and terminal clearing for UI
+- **Logging**: Tracing for structured logging and debugging
+
+## Usage Patterns
+
+The monitor serves as a central observability tool for rhailib deployments, providing operators with comprehensive visibility into system behavior and performance characteristics.

rhailib/src/monitor/src/cli_logic.rs

@@ -0,0 +1,104 @@
+// rhailib/monitor/src/cli_logic.rs
+use anyhow::Result;
+use futures::stream::{self, StreamExt};
+
+// Import functions from sibling modules within the same crate
+use crate::plot;
+use crate::tasks::{self, RhaiTask};
+use redis::{AsyncCommands, Client as RedisClient};
+use std::collections::HashMap;
+use tokio::signal;
+use tokio::time::{sleep, Duration};
+
+const REDIS_URL: &str = "redis://127.0.0.1/";
+const POLLING_INTERVAL_MILLISECONDS: u64 = 10; // Increased polling interval for SCAN
+const SCAN_COUNT: isize = 100; // Number of keys to fetch per SCAN iteration
+
+/// Main monitoring logic.
+pub async fn start_monitoring(worker_names: &[String]) -> Result<()> {
+    tracing::info!("Attempting to connect to Redis at {}", REDIS_URL);
+    let client = RedisClient::open(REDIS_URL)?;
+    let mut con = client.get_multiplexed_async_connection().await?;
+    tracing::info!("Successfully connected to Redis.");
+
+    let ping_result: String = redis::cmd("PING").query_async(&mut con).await?;
+    tracing::info!("Redis PING response: {}", ping_result);
+
+    tracing::info!(
+        "Starting live monitor. Configured workers: {:?}. Press Ctrl+C to exit.",
+        worker_names
+    );
+
+    loop {
+        tokio::select! {
+            _ = signal::ctrl_c() => {
+                print!("\r");
+                println!("Exiting Rhai Worker Monitor...");
+                break;
+            }
+            _ = async {
+                let mut current_con = con.clone(); // Clone for this iteration
+                clearscreen::clear().unwrap_or_else(|e| tracing::warn!("Failed to clear screen: {}", e));
+                println!("Rhai Worker Monitor (Press Ctrl+C to exit)");
+                println!(
+                    "Polling Redis every {}ms. Last update: {}. Configured workers: {:?}",
+                    POLLING_INTERVAL_MILLISECONDS,
+                    chrono::Local::now().format("%Y-%m-%d %H:%M:%S"),
+                    worker_names
+                );
+
+                let mut all_rhai_tasks: Vec<RhaiTask> = Vec::new();
+                let mut cursor: isize = 0;
+                loop {
+                    // SCAN returns a tuple: (new_cursor, keys_array)
+                    let (new_cursor, task_detail_keys): (isize, Vec<String>) = redis::cmd("SCAN")
+                        .arg(cursor)
+                        .arg("MATCH")
+                        .arg("rhai_task_details:*")
+                        .arg("COUNT")
+                        .arg(SCAN_COUNT)
+                        .query_async(&mut current_con)
+                        .await?;
+
+                    // Process keys found in this scan iteration
+                    let tasks_futures = stream::iter(task_detail_keys)
+                        .map(|key_with_prefix| {
+                            let mut task_con = current_con.clone();
+                            async move {
+                                let task_id = key_with_prefix.strip_prefix("rhai_task_details:").unwrap_or(&key_with_prefix).to_string();
+                                match task_con.hgetall::<_, HashMap<String, String>>(&key_with_prefix).await {
+                                    Ok(details_map) => Some(RhaiTask::from_redis_hash(task_id, &details_map)),
+                                    Err(e) => {
+                                        tracing::warn!("Could not fetch details for task key {}: {}", key_with_prefix, e);
+                                        None
+                                    }
+                                }
+                            }
+                        })
+                        .buffer_unordered(10) // Concurrently fetch details for 10 tasks
+                        .collect::<Vec<_>>()
+                        .await;
+
+                    all_rhai_tasks.extend(tasks_futures.into_iter().flatten());
+
+                    cursor = new_cursor;
+                    if cursor == 0 { // SCAN returns 0 when iteration is complete
+                        break;
+                    }
+                }
+
+                // Sort tasks by creation date (optional, assuming created_at is parsable)
+                // For simplicity, we'll skip sorting for now as created_at is a string.
+
+                let pending_tasks_count = all_rhai_tasks.iter().filter(|task| task.status.to_lowercase() == "pending").count();
+
+                plot::display_queue_plot("Total Pending Tasks", pending_tasks_count).await?;
+                tasks::display_task_table(&all_rhai_tasks).await?;
+
+                sleep(Duration::from_millis(POLLING_INTERVAL_MILLISECONDS)).await;
+                Result::<()>::Ok(())
+            } => {}
+        }
+    }
+    Ok(())
+}

rhailib/src/monitor/src/lib.rs

@@ -0,0 +1,10 @@
+// rhailib/monitor/src/lib.rs
+
+// Declare the modules that make up this crate's library
+pub mod cli_logic;
+pub mod plot;
+pub mod tasks;
+
+// Re-export the main function to be used by the binary (src/main.rs)
+// and potentially by other crates if this library is used as a dependency.
+pub use cli_logic::start_monitoring;

rhailib/src/monitor/src/main.rs

@@ -0,0 +1,32 @@
+// rhailib/monitor/src/main.rs
+use anyhow::Result;
+use clap::Parser;
+
+// Use the start_monitoring function from the monitor crate's library
+use monitor::start_monitoring;
+
+#[derive(Parser, Debug)]
+#[clap(author, version, about = "Rhai Worker Monitor CLI", long_about = None)]
+struct Args {
+    /// List of worker names to monitor, comma-separated
+    #[clap(short, long, value_delimiter = ',', required = true, num_args = 1..)]
+    workers: Vec<String>,
+    // TODO: Add other options like Redis connection details if not using a config file or env vars.
+}
+
+#[tokio::main]
+async fn main() -> Result<()> {
+    // Initialize logging (e.g., tracing-subscriber)
+    // Consider making log level configurable via CLI args or env var.
+    tracing_subscriber::fmt::init();
+
+    let args = Args::parse();
+
+    tracing::info!("Starting monitor for workers: {:?}", args.workers);
+
+    // Call the main logic function from the monitor library
+    start_monitoring(&args.workers).await?;
+
+    tracing::info!("Monitor finished.");
+    Ok(())
+}

rhailib/src/monitor/src/plot.rs

@@ -0,0 +1,23 @@
+// rhailib/monitor/src/plot.rs
+use anyhow::Result;
+
+/// Placeholder for queue plotting logic.
+const MAX_BAR_WIDTH: usize = 50; // Max width of the bar in characters
+const BAR_CHAR: char = '█'; // Character to use for the bar
+
+pub async fn display_queue_plot(plot_label: &str, count: usize) -> Result<()> {
+    let bar_width = std::cmp::min(count, MAX_BAR_WIDTH);
+    let bar: String = std::iter::repeat(BAR_CHAR).take(bar_width).collect();
+
+    // ANSI escape code for green color can be added here if desired
+    // Example: let green_bar = format!("\x1b[32m{}\x1b[0m", bar);
+
+    println!(
+        "{:<27} [{:<width$}] ({})", // Adjusted label spacing
+        plot_label,
+        bar,
+        count,
+        width = MAX_BAR_WIDTH
+    );
+    Ok(())
+}

rhailib/src/monitor/src/tasks.rs

@@ -0,0 +1,93 @@
+// rhailib/monitor/src/tasks.rs
+use anyhow::Result;
+use prettytable::{format, Cell, Row, Table};
+use std::collections::HashMap;
+
+#[derive(Debug, Clone)]
+pub struct RhaiTask {
+    pub id: String,
+    pub script: Option<String>,
+    pub status: String,
+    pub created_at: Option<String>, // Keep as string for display, parsing can be complex
+    pub updated_at: Option<String>, // Keep as string, might be RFC3339 or Unix timestamp
+    pub client_rpc_id: Option<String>,
+    pub reply_to_queue: Option<String>,
+    pub output: Option<String>,
+    pub error: Option<String>,
+}
+
+impl RhaiTask {
+    pub fn from_redis_hash(task_id: String, details: &HashMap<String, String>) -> Self {
+        // Helper to get optional string, converting "null" string to None
+        let get_opt_string = |key: &str| -> Option<String> {
+            details.get(key).and_then(|s| {
+                if s.to_lowercase() == "null" || s.is_empty() {
+                    None
+                } else {
+                    Some(s.clone())
+                }
+            })
+        };
+
+        RhaiTask {
+            id: task_id,
+            script: get_opt_string("script"),
+            status: details
+                .get("status")
+                .cloned()
+                .unwrap_or_else(|| "unknown".to_string()),
+            created_at: get_opt_string("createdAt"),
+            updated_at: get_opt_string("updatedAt"),
+            client_rpc_id: get_opt_string("clientRpcId"),
+            reply_to_queue: get_opt_string("replyToQueue"),
+            output: get_opt_string("output"),
+            error: get_opt_string("error"),
+        }
+    }
+}
+
+/// Displays all monitored Rhai tasks in a formatted table.
+pub async fn display_task_table(tasks: &[RhaiTask]) -> Result<()> {
+    println!("\nAll Monitored Rhai Tasks:");
+
+    if tasks.is_empty() {
+        println!("  No tasks to display.");
+        return Ok(());
+    }
+
+    let mut table = Table::new();
+    table.set_format(*format::consts::FORMAT_BOX_CHARS);
+    table.add_row(Row::new(vec![
+        Cell::new("Task ID").style_spec("bFg"),
+        Cell::new("Status").style_spec("bFg"),
+        Cell::new("Created At").style_spec("bFg"),
+        Cell::new("Updated At").style_spec("bFg"),
+        Cell::new("Details (Output/Error)").style_spec("bFg"),
+        // Cell::new("Script (Excerpt)").style_spec("bFg"), // Optional: Add if needed
+    ]));
+
+    for task in tasks {
+        let details_str = match (&task.output, &task.error) {
+            (Some(out), None) => format!("Output: {:.50}", out), // Truncate for display
+            (None, Some(err)) => format!("Error: {:.50}", err),  // Truncate for display
+            (Some(out), Some(err)) => format!("Output: {:.30}... Error: {:.30}...", out, err),
+            (None, None) => "N/A".to_string(),
+        };
+
+        // let script_excerpt = task.script.as_ref().map_or("N/A".to_string(), |s| {
+        //     if s.len() > 30 { format!("{:.27}...", s) } else { s.clone() }
+        // });
+
+        table.add_row(Row::new(vec![
+            Cell::new(&task.id[..std::cmp::min(task.id.len(), 12)]), // Show first 12 chars of ID
+            Cell::new(&task.status),
+            Cell::new(task.created_at.as_deref().unwrap_or("N/A")),
+            Cell::new(task.updated_at.as_deref().unwrap_or("N/A")),
+            Cell::new(&details_str),
+            // Cell::new(&script_excerpt),
+        ]));
+    }
+
+    table.printstd();
+    Ok(())
+}