sal/README.md

# SAL (System Abstraction Layer)

**Version: 0.1.0**

SAL is a comprehensive Rust library designed to provide a unified and simplified interface for a wide array of system-level operations and interactions. It abstracts platform-specific details, enabling developers to write robust, cross-platform code with greater ease. SAL also includes `herodo`, a powerful command-line tool for executing Rhai scripts that leverage SAL's capabilities for automation and system management tasks.

## 🏗️ **Cargo Workspace Structure**

SAL is organized as a **Cargo workspace** with 16 specialized crates:

- **Root Package**: `sal` - Umbrella crate that re-exports all modules
- **13 Library Crates**: Specialized SAL modules (git, text, os, net, etc.)
- **1 Binary Crate**: `herodo` - Rhai script execution engine
- **1 Integration Crate**: `rhai` - Rhai scripting integration layer

This workspace structure provides excellent build performance, dependency management, and maintainability.

### **🚀 Workspace Benefits**
- **Unified Dependency Management**: Shared dependencies across all crates with consistent versions
- **Optimized Build Performance**: Parallel compilation and shared build artifacts
- **Simplified Testing**: Run tests across all modules with a single command
- **Modular Architecture**: Each module is independently maintainable while sharing common infrastructure
- **Production Ready**: 100% test coverage with comprehensive Rhai integration tests

## Core Features

SAL offers a broad spectrum of functionalities, including:

- **System Operations**: File and directory management, environment variable access, system information retrieval, and OS-specific commands.
- **Process Management**: Create, monitor, control, and interact with system processes.
- **Containerization Tools**: 
    - Integration with **Buildah** for building OCI/Docker-compatible container images.
    - Integration with **nerdctl** for managing containers (run, stop, list, build, etc.).
- **Version Control**: Programmatic interaction with Git repositories (clone, commit, push, pull, status, etc.).
- **Database Clients**:
    - **Redis**: Robust client for interacting with Redis servers.
    - **PostgreSQL**: Client for executing queries and managing PostgreSQL databases.
- **Scripting Engine**: In-built support for the **Rhai** scripting language, allowing SAL functionalities to be scripted and automated, primarily through the `herodo` tool.
- **Networking & Services**:
    - **Mycelium**: Tools for Mycelium network peer management and message passing.
    - **Zinit**: Client for interacting with the Zinit process supervision system.
    - **RFS (Remote/Virtual Filesystem)**: Mount, manage, pack, and unpack various types of filesystems (local, SSH, S3, WebDAV).
- **Text Processing**: A suite of utilities for text manipulation, formatting, and regular expressions.
- **Cryptography (`vault`)**: Functions for common cryptographic operations.

## `herodo`: The SAL Scripting Tool

`herodo` is a command-line utility bundled with SAL that executes Rhai scripts. It empowers users to automate tasks and orchestrate complex workflows by leveraging SAL's diverse modules directly from scripts.

### Usage

```bash
# Execute a single Rhai script
herodo script.rhai

# Execute a script with arguments
herodo script.rhai arg1 arg2

# Execute all .rhai scripts in a directory
herodo /path/to/scripts/
```

If a directory is provided, `herodo` will execute all `.rhai` scripts within that directory (and its subdirectories) in alphabetical order.

### Scriptable SAL Modules via `herodo`

The following SAL modules and functionalities are exposed to the Rhai scripting environment through `herodo`:

- **OS (`os`)**: Comprehensive file system operations, file downloading & installation, and system package management. [Documentation](os/README.md)
- **Process (`process`)**: Robust command and script execution, plus process management (listing, finding, killing, checking command existence). [Documentation](process/README.md)
- **Text (`text`)**: String manipulation, prefixing, path/name fixing, text replacement, and templating. [Documentation](text/README.md)
- **Net (`net`)**: Network operations, HTTP requests, and connectivity utilities. [Documentation](net/README.md)
- **Git (`git`)**: High-level repository management and generic Git command execution with Redis-backed authentication (clone, pull, push, commit, etc.). [Documentation](git/README.md)
- **Vault (`vault`)**: Cryptographic operations, keypair management, encryption, decryption, hashing, etc. [Documentation](vault/README.md)
- **Redis Client (`redisclient`)**: Execute Redis commands (`redis_get`, `redis_set`, `redis_execute`, etc.). [Documentation](redisclient/README.md)
- **PostgreSQL Client (`postgresclient`)**: Execute SQL queries against PostgreSQL databases. [Documentation](postgresclient/README.md)
- **Zinit (`zinit_client`)**: Client for Zinit process supervisor (service management, logs). [Documentation](zinit_client/README.md)
- **Mycelium (`mycelium`)**: Client for Mycelium decentralized networking API (node info, peer management, messaging). [Documentation](mycelium/README.md)
- **Virtualization (`virt`)**:
  - **Buildah**: OCI/Docker image building functions. [Documentation](virt/README.md)
  - **nerdctl**: Container lifecycle management (`nerdctl_run`, `nerdctl_stop`, `nerdctl_images`, `nerdctl_image_build`, etc.)
  - **RFS**: Mount various filesystems (local, SSH, S3, etc.), pack/unpack filesystem layers.

### Example `herodo` Rhai Script

```rhai
// file: /opt/scripts/example_task.rhai

// OS operations
println("Checking for /tmp/my_app_data...");
if !exist("/tmp/my_app_data") {
    mkdir("/tmp/my_app_data");
    println("Created directory /tmp/my_app_data");
}

// Redis operations
println("Setting Redis key 'app_status' to 'running'");
redis_set("app_status", "running");
let status = redis_get("app_status");
println("Current app_status from Redis: " + status);

// Process execution
println("Listing files in /tmp:");
let output = run("ls -la /tmp");
println(output.stdout);

println("Script finished.");
```

Run with: `herodo /opt/scripts/example_task.rhai`

For more examples, check the individual module test directories (e.g., `text/tests/rhai/`, `os/tests/rhai/`, etc.) in this repository.

## Using SAL as a Rust Library

Add SAL as a dependency to your `Cargo.toml`:

```toml
[dependencies]
sal = "0.1.0" # Or the latest version
```

### Rust Example: Using Redis Client

```rust
use sal::redisclient::{get_global_client, execute_cmd_with_args};
use redis::RedisResult;

async fn example_redis_interaction() -> RedisResult<()> {
    // Get a connection from the global pool
    let mut conn = get_global_client().await?.get_async_connection().await?;

    // Set a value
    execute_cmd_with_args(&mut conn, "SET", vec!["my_key", "my_value"]).await?;
    println!("Set 'my_key' to 'my_value'");

    // Get a value
    let value: String = execute_cmd_with_args(&mut conn, "GET", vec!["my_key"]).await?;
    println!("Retrieved value for 'my_key': {}", value);

    Ok(())
}

#[tokio::main]
async fn main() {
    if let Err(e) = example_redis_interaction().await {
        eprintln!("Redis Error: {}", e);
    }
}
```
*(Note: The Redis client API might have evolved; please refer to `src/redisclient/mod.rs` and its documentation for the most current usage.)*

## 📦 **Workspace Modules Overview**

SAL is organized as a Cargo workspace with the following crates:

### **Core Library Modules**
- **`sal-os`**: Core OS interactions, file system operations, environment access
- **`sal-process`**: Process creation, management, and control
- **`sal-text`**: Utilities for text processing and manipulation
- **`sal-net`**: Network operations, HTTP requests, and connectivity utilities

### **Integration Modules**
- **`sal-git`**: Git repository management and operations
- **`sal-vault`**: Cryptographic functions and keypair management
- **`sal-rhai`**: Integration layer for the Rhai scripting engine, used by `herodo`

### **Client Modules**
- **`sal-redisclient`**: Client for Redis database interactions
- **`sal-postgresclient`**: Client for PostgreSQL database interactions
- **`sal-zinit-client`**: Client for Zinit process supervisor
- **`sal-mycelium`**: Client for Mycelium network operations

### **Specialized Modules**
- **`sal-virt`**: Virtualization-related utilities (buildah, nerdctl, rfs)

### **Root Package & Binary**
- **`sal`**: Root umbrella crate that re-exports all modules
- **`herodo`**: Command-line binary for executing Rhai scripts

## 🔨 **Building SAL**

Build the entire workspace (all crates) using Cargo:

```bash
# Build all workspace members
cargo build --workspace

# Build for release
cargo build --workspace --release

# Build specific crate
cargo build -p sal-text
cargo build -p herodo
```

The `herodo` executable will be located at `target/debug/herodo` or `target/release/herodo`.

## 🧪 **Running Tests**

### **Rust Unit Tests**
```bash
# Run all workspace tests
cargo test --workspace

# Run tests for specific crate
cargo test -p sal-text
cargo test -p sal-os

# Run only library tests (faster)
cargo test --workspace --lib
```

### **Rhai Integration Tests**
Run comprehensive Rhai script tests that exercise `herodo` and SAL's scripted functionalities:

```bash
# Run all Rhai integration tests (16 modules)
./run_rhai_tests.sh

# Results: 16/16 modules pass with 100% success rate
```

The Rhai tests validate real-world functionality across all SAL modules and provide comprehensive integration testing.

## License

SAL is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.