sal/README.md

SAL (System Abstraction Layer)

Version: 0.1.0

SAL is a comprehensive Rust library designed to provide a unified and simplified interface for a wide array of system-level operations and interactions. It abstracts platform-specific details, enabling developers to write robust, cross-platform code with greater ease. SAL also includes herodo, a powerful command-line tool for executing Rhai scripts that leverage SAL's capabilities for automation and system management tasks.

🏗️ Cargo Workspace Structure

SAL is organized as a Cargo workspace with 16 specialized crates:

• Root Package: sal - Umbrella crate that re-exports all modules
• 13 Library Crates: Specialized SAL modules (git, text, os, net, etc.)
• 1 Binary Crate: herodo - Rhai script execution engine
• 1 Integration Crate: rhai - Rhai scripting integration layer

This workspace structure provides excellent build performance, dependency management, and maintainability.

🚀 Workspace Benefits

• Unified Dependency Management: Shared dependencies across all crates with consistent versions
• Optimized Build Performance: Parallel compilation and shared build artifacts
• Simplified Testing: Run tests across all modules with a single command
• Modular Architecture: Each module is independently maintainable while sharing common infrastructure
• Production Ready: 100% test coverage with comprehensive Rhai integration tests

Core Features

SAL offers a broad spectrum of functionalities, including:

• System Operations: File and directory management, environment variable access, system information retrieval, and OS-specific commands.
• Process Management: Create, monitor, control, and interact with system processes.
• Containerization Tools:
  • Integration with Buildah for building OCI/Docker-compatible container images.
  • Integration with nerdctl for managing containers (run, stop, list, build, etc.).
• Version Control: Programmatic interaction with Git repositories (clone, commit, push, pull, status, etc.).
• Database Clients:
  • Redis: Robust client for interacting with Redis servers.
  • PostgreSQL: Client for executing queries and managing PostgreSQL databases.
• Scripting Engine: In-built support for the Rhai scripting language, allowing SAL functionalities to be scripted and automated, primarily through the herodo tool.
• Networking & Services:
  • Mycelium: Tools for Mycelium network peer management and message passing.
  • Zinit: Client for interacting with the Zinit process supervision system.
  • RFS (Remote/Virtual Filesystem): Mount, manage, pack, and unpack various types of filesystems (local, SSH, S3, WebDAV).
• Text Processing: A suite of utilities for text manipulation, formatting, and regular expressions.
• Cryptography (vault): Functions for common cryptographic operations.

herodo: The SAL Scripting Tool

herodo is a command-line utility bundled with SAL that executes Rhai scripts. It empowers users to automate tasks and orchestrate complex workflows by leveraging SAL's diverse modules directly from scripts.

Usage

# Execute a single Rhai script
herodo script.rhai

# Execute a script with arguments
herodo script.rhai arg1 arg2

# Execute all .rhai scripts in a directory
herodo /path/to/scripts/

If a directory is provided, herodo will execute all .rhai scripts within that directory (and its subdirectories) in alphabetical order.

Scriptable SAL Modules via herodo

The following SAL modules and functionalities are exposed to the Rhai scripting environment through herodo:

• OS (os): Comprehensive file system operations, file downloading & installation, and system package management. Documentation
• Process (process): Robust command and script execution, plus process management (listing, finding, killing, checking command existence). Documentation
• Text (text): String manipulation, prefixing, path/name fixing, text replacement, and templating. Documentation
• Net (net): Network operations, HTTP requests, and connectivity utilities. Documentation
• Git (git): High-level repository management and generic Git command execution with Redis-backed authentication (clone, pull, push, commit, etc.). Documentation
• Vault (vault): Cryptographic operations, keypair management, encryption, decryption, hashing, etc. Documentation
• Redis Client (redisclient): Execute Redis commands (redis_get, redis_set, redis_execute, etc.). Documentation
• PostgreSQL Client (postgresclient): Execute SQL queries against PostgreSQL databases. Documentation
• Zinit (zinit_client): Client for Zinit process supervisor (service management, logs). Documentation
• Mycelium (mycelium): Client for Mycelium decentralized networking API (node info, peer management, messaging). Documentation
• Virtualization (virt):
  • Buildah: OCI/Docker image building functions. Documentation
  • nerdctl: Container lifecycle management (nerdctl_run, nerdctl_stop, nerdctl_images, nerdctl_image_build, etc.)
  • RFS: Mount various filesystems (local, SSH, S3, etc.), pack/unpack filesystem layers.

Example herodo Rhai Script

// file: /opt/scripts/example_task.rhai

// OS operations
println("Checking for /tmp/my_app_data...");
if !exist("/tmp/my_app_data") {
    mkdir("/tmp/my_app_data");
    println("Created directory /tmp/my_app_data");
}

// Redis operations
println("Setting Redis key 'app_status' to 'running'");
redis_set("app_status", "running");
let status = redis_get("app_status");
println("Current app_status from Redis: " + status);

// Process execution
println("Listing files in /tmp:");
let output = run("ls -la /tmp");
println(output.stdout);

println("Script finished.");

Run with: herodo /opt/scripts/example_task.rhai

For more examples, check the individual module test directories (e.g., text/tests/rhai/, os/tests/rhai/, etc.) in this repository.

Using SAL as a Rust Library

Add SAL as a dependency to your Cargo.toml:

[dependencies]
sal = "0.1.0" # Or the latest version

Rust Example: Using Redis Client

use sal::redisclient::{get_global_client, execute_cmd_with_args};
use redis::RedisResult;

async fn example_redis_interaction() -> RedisResult<()> {
    // Get a connection from the global pool
    let mut conn = get_global_client().await?.get_async_connection().await?;

    // Set a value
    execute_cmd_with_args(&mut conn, "SET", vec!["my_key", "my_value"]).await?;
    println!("Set 'my_key' to 'my_value'");

    // Get a value
    let value: String = execute_cmd_with_args(&mut conn, "GET", vec!["my_key"]).await?;
    println!("Retrieved value for 'my_key': {}", value);

    Ok(())
}

#[tokio::main]
async fn main() {
    if let Err(e) = example_redis_interaction().await {
        eprintln!("Redis Error: {}", e);
    }
}

(Note: The Redis client API might have evolved; please refer to src/redisclient/mod.rs and its documentation for the most current usage.)

📦 Workspace Modules Overview

SAL is organized as a Cargo workspace with the following crates:

Core Library Modules

• sal-os: Core OS interactions, file system operations, environment access
• sal-process: Process creation, management, and control
• sal-text: Utilities for text processing and manipulation
• sal-net: Network operations, HTTP requests, and connectivity utilities

Integration Modules

• sal-git: Git repository management and operations
• sal-vault: Cryptographic functions and keypair management
• sal-rhai: Integration layer for the Rhai scripting engine, used by herodo

Client Modules

• sal-redisclient: Client for Redis database interactions
• sal-postgresclient: Client for PostgreSQL database interactions
• sal-zinit-client: Client for Zinit process supervisor
• sal-mycelium: Client for Mycelium network operations

Specialized Modules

• sal-virt: Virtualization-related utilities (buildah, nerdctl, rfs)

Root Package & Binary

• sal: Root umbrella crate that re-exports all modules
• herodo: Command-line binary for executing Rhai scripts

🔨 Building SAL

Build the entire workspace (all crates) using Cargo:

# Build all workspace members
cargo build --workspace

# Build for release
cargo build --workspace --release

# Build specific crate
cargo build -p sal-text
cargo build -p herodo

The herodo executable will be located at target/debug/herodo or target/release/herodo.

🧪 Running Tests

Rust Unit Tests

# Run all workspace tests
cargo test --workspace

# Run tests for specific crate
cargo test -p sal-text
cargo test -p sal-os

# Run only library tests (faster)
cargo test --workspace --lib

Rhai Integration Tests

Run comprehensive Rhai script tests that exercise herodo and SAL's scripted functionalities:

# Run all Rhai integration tests (16 modules)
./run_rhai_tests.sh

# Results: 16/16 modules pass with 100% success rate

The Rhai tests validate real-world functionality across all SAL modules and provide comprehensive integration testing.

License

SAL is licensed under the Apache License 2.0. See the LICENSE file for details.