merge branches and document

Cargo.toml

@@ -46,7 +46,7 @@ tokio = "1.45.0"
 tokio-postgres = "0.7.8" # Async PostgreSQL client
 tokio-test = "0.4.4"
 uuid = { version = "1.16.0", features = ["v4"] }
-zinit-client = { git = "https://github.com/threefoldtech/zinit", branch = "json_rpc",  package = "zinit-client" }
+zinit-client = { path = "/Users/timurgordon/code/github/threefoldtech/zinit/zinit-client" }
 reqwest = { version = "0.12.15", features = ["json"] }
 urlencoding = "2.1.3"
 

README.md

@@ -1,73 +1,184 @@
 # SAL (System Abstraction Layer)
 
-A Rust library that provides a unified interface for interacting with operating system features across different platforms. It abstracts away platform-specific details, allowing developers to write cross-platform code with ease.
+**Version: 0.1.0**
 
-## Features
+SAL is a comprehensive Rust library designed to provide a unified and simplified interface for a wide array of system-level operations and interactions. It abstracts platform-specific details, enabling developers to write robust, cross-platform code with greater ease. SAL also includes `herodo`, a powerful command-line tool for executing Rhai scripts that leverage SAL's capabilities for automation and system management tasks.
 
-- **File System Operations**: Simplified file and directory management
-- **Process Management**: Create, monitor, and control processes
-- **System Information**: Access system details and metrics
-- **Git Integration**: Interface with Git repositories
-- **Redis Client**: Robust Redis connection management and command execution
-- **Text Processing**: Utilities for text manipulation and formatting
+## Core Features
 
-## Modules
+SAL offers a broad spectrum of functionalities, including:
 
-### Redis Client
+- **System Operations**: File and directory management, environment variable access, system information retrieval, and OS-specific commands.
+- **Process Management**: Create, monitor, control, and interact with system processes.
+- **Containerization Tools**: 
+    - Integration with **Buildah** for building OCI/Docker-compatible container images.
+    - Integration with **nerdctl** for managing containers (run, stop, list, build, etc.).
+- **Version Control**: Programmatic interaction with Git repositories (clone, commit, push, pull, status, etc.).
+- **Database Clients**:
+    - **Redis**: Robust client for interacting with Redis servers.
+    - **PostgreSQL**: Client for executing queries and managing PostgreSQL databases.
+- **Scripting Engine**: In-built support for the **Rhai** scripting language, allowing SAL functionalities to be scripted and automated, primarily through the `herodo` tool.
+- **Networking & Services**:
+    - **Mycelium**: Tools for Mycelium network peer management and message passing.
+    - **Zinit**: Client for interacting with the Zinit process supervision system.
+    - **RFS (Remote/Virtual Filesystem)**: Mount, manage, pack, and unpack various types of filesystems (local, SSH, S3, WebDAV).
+- **Text Processing**: A suite of utilities for text manipulation, formatting, and regular expressions.
+- **Cryptography (`vault`)**: Functions for common cryptographic operations.
 
-The Redis client module provides a robust wrapper around the Redis client library for Rust, offering:
+## `herodo`: The SAL Scripting Tool
 
-- Automatic connection management and reconnection
-- Support for both Unix socket and TCP connections
-- Database selection via environment variables
-- Thread-safe global client instance
-- Simple command execution interface
+`herodo` is a command-line utility bundled with SAL that executes Rhai scripts. It empowers users to automate tasks and orchestrate complex workflows by leveraging SAL's diverse modules directly from scripts.
 
-[View Redis Client Documentation](src/redisclient/README.md)
+### Usage
 
-### OS Module
+```bash
+herodo -p <path_to_script.rhai>
+# or
+herodo -p <path_to_directory_with_scripts/>
+```
 
-Provides platform-independent interfaces for operating system functionality.
+If a directory is provided, `herodo` will execute all `.rhai` scripts within that directory (and its subdirectories) in alphabetical order.
 
-### Git Module
+### Scriptable SAL Modules via `herodo`
 
-Tools for interacting with Git repositories programmatically.
+The following SAL modules and functionalities are exposed to the Rhai scripting environment through `herodo`:
 
-### Process Module
+- **OS (`os`)**: Comprehensive file system operations, file downloading & installation, and system package management. [Detailed OS Module Documentation](src/os/README.md)
+- **Process (`process`)**: Robust command and script execution, plus process management (listing, finding, killing, checking command existence). [Detailed Process Module Documentation](src/process/README.md)
+- **Buildah (`buildah`)**: OCI/Docker image building functions. [Detailed Buildah Module Documentation](src/virt/buildah/README.md)
+- **nerdctl (`nerdctl`)**: Container lifecycle management (`nerdctl_run`, `nerdctl_stop`, `nerdctl_images`, `nerdctl_image_build`, etc.). [Detailed Nerdctl Module Documentation](src/virt/nerdctl/README.md)
+- **Git (`git`)**: High-level repository management and generic Git command execution with Redis-backed authentication (clone, pull, push, commit, etc.). [Detailed Git Module Documentation](src/git/README.md)
+- **Zinit (`zinit_client`)**: Client for Zinit process supervisor (service management, logs). [Detailed Zinit Client Module Documentation](src/zinit_client/README.md)
+- **Mycelium (`mycelium`)**: Client for Mycelium decentralized networking API (node info, peer management, messaging). [Detailed Mycelium Module Documentation](src/mycelium/README.md)
+- **Text (`text`)**: String manipulation, prefixing, path/name fixing, text replacement, and templating. [Detailed Text Module Documentation](src/text/README.md)
+- **RFS (`rfs`)**: Mount various filesystems (local, SSH, S3, etc.), pack/unpack filesystem layers. [Detailed RFS Module Documentation](src/virt/rfs/README.md)
+- **Cryptography (`crypto` from `vault`)**: Encryption, decryption, hashing, etc.
+- **Redis Client (`redis`)**: Execute Redis commands (`redis_get`, `redis_set`, `redis_execute`, etc.).
+- **PostgreSQL Client (`postgres`)**: Execute SQL queries against PostgreSQL databases.
 
-Utilities for process creation, monitoring, and management.
+### Example `herodo` Rhai Script
 
-### Text Module
+```rhai
+// file: /opt/scripts/example_task.rhai
 
-Text processing utilities for common operations.
+// OS operations
+println("Checking for /tmp/my_app_data...");
+if !exist("/tmp/my_app_data") {
+    mkdir("/tmp/my_app_data");
+    println("Created directory /tmp/my_app_data");
+}
 
-## Usage
+// Redis operations
+println("Setting Redis key 'app_status' to 'running'");
+redis_set("app_status", "running");
+let status = redis_get("app_status");
+println("Current app_status from Redis: " + status);
 
-Add this to your `Cargo.toml`:
+// Process execution
+println("Listing files in /tmp:");
+let output = run("ls -la /tmp");
+println(output.stdout);
+
+println("Script finished.");
+```
+
+Run with: `herodo -p /opt/scripts/example_task.rhai`
+
+For more examples, check the `examples/` and `rhai_tests/` directories in this repository.
+
+## Using SAL as a Rust Library
+
+Add SAL as a dependency to your `Cargo.toml`:
 
 ```toml
 [dependencies]
-sal = "0.1.0"
+sal = "0.1.0" # Or the latest version
 ```
 
-Basic example:
+### Rust Example: Using Redis Client
 
 ```rust
-use sal::redisclient::execute;
-use redis::cmd;
+use sal::redisclient::{get_global_client, execute_cmd_with_args};
+use redis::RedisResult;
+
+async fn example_redis_interaction() -> RedisResult<()> {
+    // Get a connection from the global pool
+    let mut conn = get_global_client().await?.get_async_connection().await?;
+
+    // Set a value
+    execute_cmd_with_args(&mut conn, "SET", vec!["my_key", "my_value"]).await?;
+    println!("Set 'my_key' to 'my_value'");
+
+    // Get a value
+    let value: String = execute_cmd_with_args(&mut conn, "GET", vec!["my_key"]).await?;
+    println!("Retrieved value for 'my_key': {}", value);
 
-fn main() -> redis::RedisResult<()> {
-    // Execute a Redis command
-    let mut cmd = redis::cmd("SET");
-    cmd.arg("example_key").arg("example_value");
-    execute(&mut cmd)?;
-    
-    // Retrieve the value
-    let mut get_cmd = redis::cmd("GET");
-    get_cmd.arg("example_key");
-    let value: String = execute(&mut get_cmd)?;
-    println!("Value: {}", value);
-    
     Ok(())
 }
+
+#[tokio::main]
+asynchronous fn main() {
+    if let Err(e) = example_redis_interaction().await {
+        eprintln!("Redis Error: {}", e);
+    }
+}
 ```
+*(Note: The Redis client API might have evolved; please refer to `src/redisclient/mod.rs` and its documentation for the most current usage.)*
+
+## Modules Overview (Rust Library)
+
+SAL is organized into several modules, each providing specific functionalities:
+
+- **`sal::os`**: Core OS interactions, file system operations, environment access.
+- **`sal::process`**: Process creation, management, and control.
+- **`sal::git`**: Git repository management.
+- **`sal::redisclient`**: Client for Redis database interactions. (See also `src/redisclient/README.md`)
+- **`sal::postgresclient`**: Client for PostgreSQL database interactions.
+- **`sal::rhai`**: Integration layer for the Rhai scripting engine, used by `herodo`.
+- **`sal::text`**: Utilities for text processing and manipulation.
+- **`sal::vault`**: Cryptographic functions.
+- **`sal::virt`**: Virtualization-related utilities, including `rfs` for remote/virtual filesystems.
+- **`sal::mycelium`**: Client for Mycelium network operations.
+- **`sal::zinit_client`**: Client for Zinit process supervisor.
+- **`sal::cmd`**: Implements the command logic for `herodo`.
+- **(Internal integrations for `buildah`, `nerdctl` primarily exposed via Rhai)**
+
+## Building SAL
+
+Build the library and the `herodo` binary using Cargo:
+
+```bash
+cargo build
+```
+
+For a release build:
+
+```bash
+cargo build --release
+```
+
+The `herodo` executable will be located at `target/debug/herodo` or `target/release/herodo`.
+
+The `build_herodo.sh` script is also available for building `herodo`.
+
+## Running Tests
+
+Run Rust unit and integration tests:
+
+```bash
+cargo test
+```
+
+Run Rhai script tests (which exercise `herodo` and SAL's scripted functionalities):
+
+```bash
+./run_rhai_tests.sh
+```
+
+## License
+
+SAL is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit pull requests or open issues.

rhai_tests/run_all_tests.sh

@@ -4,7 +4,10 @@
 # This script runs all the Rhai tests in the rhai_tests directory
 
 # Set the base directory
-BASE_DIR="src/rhai_tests"
+BASE_DIR="."
+
+# Path to herodo executable (assuming debug build)
+HERODO_CMD="../target/debug/herodo"
 
 # Define colors for output
 GREEN='\033[0;32m'
@@ -27,7 +30,7 @@ run_tests_in_dir() {
     # Check if the directory has a run_all_tests.rhai script
     if [ -f "${dir}/run_all_tests.rhai" ]; then
         echo "Using module's run_all_tests.rhai script"
-        herodo --path "${dir}/run_all_tests.rhai"
+        ${HERODO_CMD} --path "${dir}/run_all_tests.rhai"
         
         if [ $? -eq 0 ]; then
             echo -e "${GREEN}✓ All tests passed for module: ${module_name}${NC}"
@@ -43,7 +46,7 @@ run_tests_in_dir() {
         
         for test_file in $test_files; do
             echo "Running test: $(basename $test_file)"
-            herodo --path "$test_file"
+            ${HERODO_CMD} --path "$test_file"
             
             if [ $? -ne 0 ]; then
                 all_passed=false

src/git/README.md

@@ -0,0 +1,86 @@
+# SAL `git` Module
+
+The `git` module in SAL provides comprehensive functionalities for interacting with Git repositories. It offers both high-level abstractions for common Git workflows and a flexible executor for running arbitrary Git commands with integrated authentication.
+
+This module is central to SAL's capabilities for managing source code, enabling automation of development tasks, and integrating with version control systems.
+
+## Core Components
+
+The module is primarily composed of two main parts:
+
+1.  **Repository and Tree Management (`git.rs`)**: Defines `GitTree` and `GitRepo` structs for a more structured, object-oriented approach to Git operations.
+2.  **Command Execution with Authentication (`git_executor.rs`)**: Provides `GitExecutor` for running any Git command, with a focus on handling authentication via configurations stored in Redis.
+
+### 1. Repository and Tree Management (`GitTree` & `GitRepo`)
+
+These components allow for programmatic management of Git repositories.
+
+*   **`GitTree`**: Represents a directory (base path) that can contain multiple Git repositories.
+    *   `new(base_path)`: Creates a new `GitTree` instance for the given base path.
+    *   `list()`: Lists all Git repositories found under the base path.
+    *   `find(pattern)`: Finds repositories within the tree that match a given name pattern (supports wildcards).
+    *   `get(path_or_url)`: Retrieves `GitRepo` instances. If a local path/pattern is given, it finds existing repositories. If a Git URL is provided, it will clone the repository into a structured path (`base_path/server/account/repo`) if it doesn't already exist.
+
+*   **`GitRepo`**: Represents a single Git repository.
+    *   `new(path)`: Creates a `GitRepo` instance for the repository at the given path.
+    *   `path()`: Returns the local file system path to the repository.
+    *   `has_changes()`: Checks if the repository has uncommitted local changes.
+    *   `pull()`: Pulls the latest changes from the remote. Fails if local changes exist.
+    *   `reset()`: Performs a hard reset (`git reset --hard HEAD`) and cleans untracked files (`git clean -fd`).
+    *   `commit(message)`: Stages all changes (`git add .`) and commits them with the given message.
+    *   `push()`: Pushes committed changes to the remote repository.
+
+*   **`GitError`**: A comprehensive enum for errors related to `GitTree` and `GitRepo` operations (e.g., Git not installed, invalid URL, repository not found, local changes exist).
+
+*   **`parse_git_url(url)`**: A utility function to parse HTTPS and SSH Git URLs into server, account, and repository name components.
+
+### 2. Command Execution with Authentication (`GitExecutor`)
+
+`GitExecutor` is designed for flexible execution of any Git command, with a special emphasis on handling authentication for remote operations.
+
+*   **`GitExecutor::new()` / `GitExecutor::default()`**: Creates a new executor instance.
+*   **`GitExecutor::init()`**: Initializes the executor by attempting to load authentication configurations from Redis (key: `herocontext:git`). If Redis is unavailable or the config is missing, it proceeds without specific auth configurations, relying on system defaults.
+*   **`GitExecutor::execute(args: &[&str])`**: The primary method to run a Git command (e.g., `executor.execute(&["clone", "https://github.com/user/repo.git", "myrepo"])`).
+    *   It intelligently attempts to apply authentication based on the command and the loaded configuration.
+
+#### Authentication Configuration (`herocontext:git` in Redis)
+
+The `GitExecutor` can load its authentication settings from a JSON object stored in Redis under the key `herocontext:git`. The structure is as follows:
+
+```json
+{
+  "status": "ok", // or "error"
+  "auth": {
+    "github.com": {
+      "sshagent": true // Use SSH agent for github.com
+    },
+    "gitlab.example.com": {
+      "key": "/path/to/ssh/key_for_gitlab" // Use specific SSH key
+    },
+    "dev.azure.com": {
+      "username": "your_username",
+      "password": "your_pat_or_password" // Use HTTPS credentials
+    }
+    // ... other server configurations
+  }
+}
+```
+
+*   **Authentication Methods Supported**:
+    *   **SSH Agent**: If `sshagent: true` is set for a server, and an SSH agent is loaded with identities.
+    *   **SSH Key**: If `key: "/path/to/key"` is specified, `GIT_SSH_COMMAND` is used to point to this key.
+    *   **Username/Password (HTTPS)**: If `username` and `password` are provided, HTTPS URLs are rewritten to include these credentials (e.g., `https://user:pass@server/repo.git`).
+
+*   **`GitExecutorError`**: An enum for errors specific to `GitExecutor`, including command failures, Redis errors, JSON parsing issues, and authentication problems (e.g., `SshAgentNotLoaded`, `InvalidAuthConfig`).
+
+## Usage with `herodo`
+
+The `herodo` CLI tool likely leverages `GitExecutor` to provide its scriptable Git functionalities. This allows Rhai scripts executed by `herodo` to perform Git operations using the centrally managed authentication configurations from Redis, promoting secure and consistent access to Git repositories.
+
+## Error Handling
+
+Both `git.rs` and `git_executor.rs` define their own specific error enums (`GitError` and `GitExecutorError` respectively) to provide detailed information about issues encountered during Git operations. These errors cover a wide range of scenarios from command execution failures to authentication problems and invalid configurations.
+
+## Summary
+
+The `git` module offers a powerful and flexible interface to Git, catering to both simple, high-level repository interactions and complex, authenticated command execution scenarios. Its integration with Redis for authentication configuration makes it particularly well-suited for automated systems and tools like `herodo`.

src/mycelium/README.md

@@ -0,0 +1,126 @@
+# SAL Mycelium Module (`sal::mycelium`)
+
+## Overview
+
+The `sal::mycelium` module provides a client interface for interacting with a [Mycelium](https://mycelium.com/) node's HTTP API. Mycelium is a decentralized networking project, and this SAL module allows Rust applications and `herodo` Rhai scripts to manage and communicate over a Mycelium network.
+
+The module enables operations such as:
+- Querying node status and information.
+- Managing peer connections (listing, adding, removing).
+- Inspecting routing tables (selected and fallback routes).
+- Sending messages to other Mycelium nodes.
+- Receiving messages from subscribed topics.
+
+All interactions with the Mycelium API are performed asynchronously.
+
+## Key Design Points
+
+- **Async HTTP Client**: Leverages `reqwest` for asynchronous HTTP requests to the Mycelium node's API, ensuring non-blocking operations suitable for concurrent applications.
+- **JSON Interaction**: Expects and processes JSON-formatted data from the Mycelium API, using `serde_json::Value` for flexible data handling.
+- **Base64 Encoding**: Message payloads and topics are Base64 encoded/decoded when communicating with the Mycelium API, as per its expected format.
+- **Rhai Scriptability**: All core functionalities are exposed to Rhai scripts via `herodo` through the `sal::rhai::mycelium` bridge. This allows for easy automation of Mycelium network tasks.
+- **Error Handling**: Provides clear error messages, converting HTTP and parsing errors into `String` results in Rust, which are then translated to `EvalAltResult` for Rhai.
+- **Tokio Runtime Management**: For Rhai script execution, a Tokio runtime is managed internally by the wrapper functions to bridge Rhai's synchronous world with the asynchronous Rust client.
+
+## Rhai Scripting with `herodo`
+
+The `sal::mycelium` module can be scripted using `herodo`. The following functions are available in Rhai, typically prefixed with `mycelium_`:
+
+All functions take `api_url` (String) as their first argument, which is the base URL of the Mycelium node's HTTP API (e.g., `"http://localhost:7777"`).
+
+- `mycelium_get_node_info(api_url: String) -> Dynamic`
+  - Retrieves general information about the Mycelium node.
+  - Returns a dynamic object (map) representing the JSON response.
+
+- `mycelium_list_peers(api_url: String) -> Dynamic`
+  - Lists all peers currently connected to the node.
+  - Returns a dynamic array of peer information objects.
+
+- `mycelium_add_peer(api_url: String, peer_address: String) -> Dynamic`
+  - Adds a new peer to the node.
+  - `peer_address`: The endpoint address of the peer to add (e.g., `"tcp://192.168.1.10:7778"`).
+  - Returns a success status or an error.
+
+- `mycelium_remove_peer(api_url: String, peer_id: String) -> Dynamic`
+  - Removes a peer from the node.
+  - `peer_id`: The ID of the peer to remove.
+  - Returns a success status or an error.
+
+- `mycelium_list_selected_routes(api_url: String) -> Dynamic`
+  - Lists the currently selected (active) routes in the node's routing table.
+  - Returns a dynamic array of route objects.
+
+- `mycelium_list_fallback_routes(api_url: String) -> Dynamic`
+  - Lists the fallback routes in the node's routing table.
+  - Returns a dynamic array of route objects.
+
+- `mycelium_send_message(api_url: String, destination: String, topic: String, message: String, reply_deadline_secs: Int) -> Dynamic`
+  - Sends a message to a specific destination over the Mycelium network.
+  - `destination`: The Mycelium address of the recipient node.
+  - `topic`: The topic for the message (will be Base64 encoded).
+  - `message`: The content of the message (will be Base64 encoded).
+  - `reply_deadline_secs`: An integer specifying the timeout in seconds to wait for a reply. If negative, no reply is waited for.
+  - Returns a response from the Mycelium API, potentially including a reply if waited for.
+
+- `mycelium_receive_messages(api_url: String, topic: String, wait_deadline_secs: Int) -> Dynamic`
+  - Subscribes to a topic and waits for messages.
+  - `topic`: The topic to subscribe to (will be Base64 encoded).
+  - `wait_deadline_secs`: An integer specifying the maximum time in seconds to wait for a message. If negative, waits indefinitely (or until the API's default timeout).
+  - Returns an array of received messages, or an empty array if the deadline is met before messages arrive.
+
+### Rhai Example
+
+```rhai
+// Assuming a Mycelium node is running and accessible at http://localhost:7777
+let api_url = "http://localhost:7777";
+
+// Get Node Info
+print("Fetching node info...");
+let node_info = mycelium_get_node_info(api_url);
+if node_info.is_ok() {
+    print(`Node Info: ${node_info}`);
+} else {
+    print(`Error fetching node info: ${node_info}`);
+}
+
+// List Peers
+print("\nListing peers...");
+let peers = mycelium_list_peers(api_url);
+if peers.is_ok() {
+    print(`Peers: ${peers}`);
+} else {
+    print(`Error listing peers: ${peers}`);
+}
+
+// Example: Send a message (destination and topic are illustrative)
+let dest_addr = "some_mycelium_destination_address"; // Replace with actual address
+let msg_topic = "sal/test_topic";
+let msg_content = "Hello from SAL Mycelium via Rhai!";
+
+print(`\nSending message to '${dest_addr}' on topic '${msg_topic}'...`);
+// No reply wait (deadline = -1)
+let send_result = mycelium_send_message(api_url, dest_addr, msg_topic, msg_content, -1);
+if send_result.is_ok() {
+    print(`Send Result: ${send_result}`);
+} else {
+    print(`Error sending message: ${send_result}`);
+}
+
+// Example: Receive messages (topic is illustrative)
+// This will block for up to 10 seconds, or until a message arrives.
+print(`\nAttempting to receive messages on topic '${msg_topic}' for 10 seconds...`);
+let received = mycelium_receive_messages(api_url, msg_topic, 10);
+if received.is_ok() {
+    if received.len() > 0 {
+        print(`Received Messages: ${received}`);
+    } else {
+        print("No messages received within the deadline.");
+    }
+} else {
+    print(`Error receiving messages: ${received}`);
+}
+
+print("\nMycelium Rhai script finished.");
+```
+
+This module facilitates integration with Mycelium networks, enabling automation of peer management, message exchange, and network monitoring through `herodo` scripts or direct Rust integration.

src/os/README.md

@@ -0,0 +1,245 @@
+# SAL OS Module (`sal::os`)
+
+The `sal::os` module provides a comprehensive suite of operating system interaction utilities. It aims to offer a cross-platform abstraction layer for common OS-level tasks, simplifying system programming in Rust.
+
+This module is composed of three main sub-modules:
+- [`fs`](#fs): File system operations.
+- [`download`](#download): File downloading and basic installation.
+- [`package`](#package): System package management.
+
+## Key Design Points
+
+The `sal::os` module is engineered with several core principles to provide a robust and developer-friendly interface for OS interactions:
+
+- **Cross-Platform Abstraction**: A primary goal is to offer a unified API for common OS tasks, smoothing over differences between operating systems (primarily Linux and macOS). While it strives for abstraction, it leverages platform-specific tools (e.g., `rsync` on Linux, `robocopy` on Windows for `fs::copy` or `fs::rsync`; `apt` on Debian-based systems, `brew` on macOS for `package` management) for optimal performance and behavior when necessary.
+- **Modular Structure**: Functionality is organized into logical sub-modules:
+    - `fs`: For comprehensive file and directory manipulation.
+    - `download`: For retrieving files from URLs, with support for extraction and basic installation.
+    - `package`: For interacting with system package managers.
+- **Granular Error Handling**: Each sub-module features custom error enums (`FsError`, `DownloadError`, `PackageError`) to provide specific and actionable feedback, aiding in debugging and robust error management.
+- **Sensible Defaults and Defensive Operations**: Many functions are designed to be "defensive," e.g., `mkdir` creates parent directories if they don't exist and doesn't fail if the directory already exists. `delete` doesn't error if the target is already gone.
+- **Facade for Simplicity**: The `package` sub-module uses a `PackHero` facade to provide a simple entry point for common package operations, automatically detecting the underlying OS and package manager.
+- **Rhai Scriptability**: A significant portion of the `sal::os` module's functionality is exposed to Rhai scripts via `herodo`, enabling powerful automation of OS-level tasks.
+
+## `fs` - File System Operations
+
+The `fs` sub-module (`sal::os::fs`) offers a robust set of functions for interacting with the file system.
+
+**Key Features:**
+
+*   **Error Handling**: A custom `FsError` enum for detailed error reporting on file system operations.
+*   **File Operations**:
+    *   `copy(src, dest)`: Copies files and directories, with support for wildcards and recursive copying. Uses platform-specific commands (`cp -R`, `robocopy /MIR`).
+    *   `exist(path)`: Checks if a file or directory exists.
+    *   `find_file(dir, filename_pattern)`: Finds a single file in a directory, supporting wildcards.
+    *   `find_files(dir, filename_pattern)`: Finds multiple files in a directory, supporting wildcards.
+    *   `file_size(path)`: Returns the size of a file in bytes.
+    *   `file_read(path)`: Reads the entire content of a file into a string.
+    *   `file_write(path, content)`: Writes content to a file, overwriting if it exists, and creating parent directories if needed.
+    *   `file_write_append(path, content)`: Appends content to a file, creating it and parent directories if needed.
+*   **Directory Operations**:
+    *   `find_dir(parent_dir, dirname_pattern)`: Finds a single directory within a parent directory, supporting wildcards.
+    *   `find_dirs(parent_dir, dirname_pattern)`: Finds multiple directories recursively within a parent directory, supporting wildcards.
+    *   `delete(path)`: Deletes files or directories.
+    *   `mkdir(path)`: Creates a directory, including parent directories if necessary.
+    *   `rsync(src, dest)`: Synchronizes directories using platform-specific commands (`rsync -a --delete`, `robocopy /MIR`).
+    *   `chdir(path)`: Changes the current working directory.
+*   **Path Operations**:
+    *   `mv(src, dest)`: Moves or renames files and directories. Handles cross-device moves by falling back to copy-then-delete.
+*   **Command Utilities**:
+    *   `which(command_name)`: Checks if a command exists in the system's PATH and returns its path.
+
+**Usage Example (fs):**
+
+```rust
+use sal::os::fs;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    if !fs::exist("my_dir") {
+        fs::mkdir("my_dir")?;
+        println!("Created directory 'my_dir'");
+    }
+
+    fs::file_write("my_dir/example.txt", "Hello from SAL!")?;
+    let content = fs::file_read("my_dir/example.txt")?;
+    println!("File content: {}", content);
+
+    Ok(())
+}
+```
+
+## `download` - File Downloading and Installation
+
+The `download` sub-module (`sal::os::download`) provides utilities for downloading files from URLs and performing basic installation tasks.
+
+**Key Features:**
+
+*   **Error Handling**: A custom `DownloadError` enum for download-specific errors.
+*   **File Downloading**:
+    *   `download(url, dest_dir, min_size_kb)`: Downloads a file to a specified directory.
+        *   Uses `curl` with progress display.
+        *   Supports minimum file size checks.
+        *   Automatically extracts common archive formats (`.tar.gz`, `.tgz`, `.tar`, `.zip`) into `dest_dir`.
+    *   `download_file(url, dest_file_path, min_size_kb)`: Downloads a file to a specific file path without automatic extraction.
+*   **File Permissions**:
+    *   `chmod_exec(path)`: Makes a file executable (equivalent to `chmod +x` on Unix-like systems).
+*   **Download and Install**:
+    *   `download_install(url, min_size_kb)`: Downloads a file (to `/tmp/`) and attempts to install it if it's a supported package format.
+        *   Currently supports `.deb` packages on Debian-based systems.
+        *   For `.deb` files, it uses `sudo dpkg --install` and attempts `sudo apt-get install -f -y` to fix dependencies if needed.
+        *   Handles archives by extracting them to `/tmp/` first.
+
+**Usage Example (download):**
+
+```rust
+use sal::os::download;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let archive_url = "https://example.com/my_archive.tar.gz";
+    let output_dir = "/tmp/my_app";
+    
+    // Download and extract an archive
+    let extracted_path = download::download(archive_url, output_dir, 1024)?; // Min 1MB
+    println!("Archive extracted to: {}", extracted_path);
+
+    // Download a script and make it executable
+    let script_url = "https://example.com/my_script.sh";
+    let script_path = "/tmp/my_script.sh";
+    download::download_file(script_url, script_path, 0)?;
+    download::chmod_exec(script_path)?;
+    println!("Script downloaded and made executable at: {}", script_path);
+
+    Ok(())
+}
+```
+
+## `package` - System Package Management
+
+The `package` sub-module (`sal::os::package`) offers an abstraction layer for interacting with system package managers like APT (for Debian/Ubuntu) and Homebrew (for macOS).
+
+**Key Features:**
+
+*   **Error Handling**: A custom `PackageError` enum.
+*   **Platform Detection**: Identifies the current OS (Ubuntu, macOS, or Unknown) to use the appropriate package manager.
+*   **`PackageManager` Trait**: Defines a common interface for package operations:
+    *   `install(package_name)`
+    *   `remove(package_name)`
+    *   `update()` (updates package lists)
+    *   `upgrade()` (upgrades all installed packages)
+    *   `list_installed()`
+    *   `search(query)`
+    *   `is_installed(package_name)`
+*   **Implementations**:
+    *   `AptPackageManager`: For Debian/Ubuntu systems (uses `apt-get`, `dpkg`).
+    *   `BrewPackageManager`: For macOS systems (uses `brew`).
+*   **`PackHero` Facade**: A simple entry point to access package management functions in a platform-agnostic way.
+    *   `PackHero::new().install("nginx")?`
+
+**Usage Example (package):**
+
+```rust
+use sal::os::package::PackHero;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let pack_hero = PackHero::new();
+
+    // Check if a package is installed
+    if !pack_hero.is_installed("htop")? {
+        println!("htop is not installed. Attempting to install...");
+        pack_hero.install("htop")?;
+        println!("htop installed successfully.");
+    } else {
+        println!("htop is already installed.");
+    }
+
+    // Update package lists
+    println!("Updating package lists...");
+    pack_hero.update()?;
+    println!("Package lists updated.");
+
+    Ok(())
+}
+```
+
+## Rhai Scripting with `herodo`
+
+The `sal::os` module is extensively scriptable via `herodo`, allowing for automation of various operating system tasks directly from Rhai scripts. The `sal::rhai::os` module registers the necessary functions.
+
+### File System (`fs`) Functions
+
+- `copy(src: String, dest: String) -> String`: Copies files/directories (supports wildcards).
+- `exist(path: String) -> bool`: Checks if a file or directory exists.
+- `find_file(dir: String, filename_pattern: String) -> String`: Finds a single file in `dir` matching `filename_pattern`.
+- `find_files(dir: String, filename_pattern: String) -> Array`: Finds multiple files in `dir` (recursive).
+- `find_dir(parent_dir: String, dirname_pattern: String) -> String`: Finds a single directory in `parent_dir`.
+- `find_dirs(parent_dir: String, dirname_pattern: String) -> Array`: Finds multiple directories in `parent_dir` (recursive).
+- `delete(path: String) -> String`: Deletes a file or directory.
+- `mkdir(path: String) -> String`: Creates a directory (and parents if needed).
+- `file_size(path: String) -> Int`: Returns file size in bytes.
+- `rsync(src: String, dest: String) -> String`: Synchronizes directories.
+- `chdir(path: String) -> String`: Changes the current working directory.
+- `file_read(path: String) -> String`: Reads entire file content.
+- `file_write(path: String, content: String) -> String`: Writes content to a file (overwrites).
+- `file_write_append(path: String, content: String) -> String`: Appends content to a file.
+- `mv(src: String, dest: String) -> String`: Moves/renames a file or directory.
+- `which(command_name: String) -> String`: Checks if a command exists in PATH and returns its path.
+- `cmd_ensure_exists(commands: String) -> String`: Ensures one or more commands (comma-separated) exist in PATH; throws an error if any are missing.
+
+### Download Functions
+
+- `download(url: String, dest_dir: String, min_size_kb: Int) -> String`: Downloads from `url` to `dest_dir`, extracts common archives.
+- `download_file(url: String, dest_file_path: String, min_size_kb: Int) -> String`: Downloads from `url` to `dest_file_path` (no extraction).
+- `download_install(url: String, min_size_kb: Int) -> String`: Downloads and attempts to install (e.g., `.deb` packages).
+- `chmod_exec(path: String) -> String`: Makes a file executable (`chmod +x`).
+
+### Package Management Functions
+
+- `package_install(package_name: String) -> String`: Installs a package.
+- `package_remove(package_name: String) -> String`: Removes a package.
+- `package_update() -> String`: Updates package lists.
+- `package_upgrade() -> String`: Upgrades all installed packages.
+- `package_list() -> Array`: Lists all installed packages.
+- `package_search(query: String) -> Array`: Searches for packages.
+- `package_is_installed(package_name: String) -> bool`: Checks if a package is installed.
+- `package_set_debug(debug: bool) -> bool`: Enables/disables debug logging for package operations.
+- `package_platform() -> String`: Returns the detected package platform (e.g., "Ubuntu", "MacOS").
+
+### Rhai Example
+
+```rhai
+// File system operations
+let test_dir = "/tmp/sal_os_rhai_test";
+if exist(test_dir) {
+    delete(test_dir);
+}
+mkdir(test_dir);
+print(`Created directory: ${test_dir}`);
+
+file_write(`${test_dir}/message.txt`, "Hello from Rhai OS module!");
+let content = file_read(`${test_dir}/message.txt`);
+print(`File content: ${content}`);
+
+// Download operation (example URL, may not be active)
+// let script_url = "https://raw.githubusercontent.com/someuser/somescript/main/script.sh";
+// let script_path = `${test_dir}/downloaded_script.sh`;
+// try {
+//   download_file(script_url, script_path, 0);
+//   chmod_exec(script_path);
+//   print(`Downloaded and made executable: ${script_path}`);
+// } catch (e) {
+//   print(`Download example failed (this is okay for a test): ${e}`);
+// }
+
+// Package management (illustrative, requires sudo for install/remove/update)
+print(`Package platform: ${package_platform()}`);
+if !package_is_installed("htop") {
+    print("htop is not installed.");
+    // package_install("htop"); // Would require sudo
+} else {
+    print("htop is already installed.");
+}
+
+print("OS module Rhai script finished.");
+```
+
+This module provides a powerful and convenient way to handle common OS-level tasks within your Rust applications.

src/process/README.md

@@ -1,75 +1,150 @@
-# Process Module
-====================
-## Overview
-The process module is responsible for managing and running system processes.
+# SAL Process Module (`sal::process`)
 
-## Usage
-To use the process module, import it in your Rust file and call the desired functions.
+The `process` module in the SAL (System Abstraction Layer) library provides a robust and cross-platform interface for creating, managing, and interacting with system processes. It is divided into two main sub-modules: `run` for command and script execution, and `mgmt` for process management tasks like listing, finding, and terminating processes.
 
-## Functions
-### mgmt.rs
-The mgmt.rs file contains functions for managing system processes.
+## Core Functionalities
 
-### run.rs
-The run.rs file contains functions for running system processes.
+### 1. Command and Script Execution (`run.rs`)
 
-#### Input Flexibility
-The `run` function in run.rs is designed to be flexible with its input:
+The `run.rs` sub-module offers flexible ways to execute external commands and multi-line scripts.
 
-1. **One-liner Commands**: If the input is a single line, it's treated as a command with arguments.
-   ```rust
-   run("ls -la");  // Runs the 'ls -la' command
-   ```
+#### `RunBuilder`
 
-2. **Multi-line Scripts**: If the input contains newlines, it's treated as a script. The script is automatically dedented using the `dedent` function from `src/text/dedent.rs` before execution.
-   ```rust
-   run("    echo 'Hello'\n    ls -la");  // Common indentation is removed before execution
-   ```
+The primary interface for execution is the `RunBuilder`, obtained via `sal::process::run("your_command_or_script")`. It allows for fluent configuration:
+
+-   `.die(bool)`: If `true` (default), an error is returned if the command fails. If `false`, a `CommandResult` with `success: false` is returned instead.
+-   `.silent(bool)`: If `true` (default is `false`), suppresses `stdout` and `stderr` from being printed to the console during execution. Output is still captured in `CommandResult`.
+-   `.async_exec(bool)`: If `true` (default is `false`), executes the command or script in a separate thread, returning an immediate placeholder `CommandResult`.
+-   `.log(bool)`: If `true` (default is `false`), prints a log message before executing the command.
+-   `.execute() -> Result<CommandResult, RunError>`: Executes the configured command or script.
+
+**Input Handling**:
+-   **Single-line commands**: Treated as a command and its arguments (e.g., `"ls -la"`).
+-   **Multi-line scripts**: If the input string contains newline characters (`\n`), it's treated as a script.
+    -   The script content is automatically dedented.
+    -   On Unix-like systems, `#!/bin/bash -e` is prepended (if no shebang exists) to ensure the script exits on error.
+    -   A temporary script file is created, made executable, and then run.
+
+#### `CommandResult`
+
+All execution functions return a `Result<CommandResult, RunError>`. The `CommandResult` struct contains:
+-   `stdout: String`: Captured standard output.
+-   `stderr: String`: Captured standard error.
+-   `success: bool`: `true` if the command exited with a zero status code.
+-   `code: i32`: The exit code of the command.
+
+#### Convenience Functions:
+-   `sal::process::run_command("cmd_or_script")`: Equivalent to `run("cmd_or_script").execute()`.
+-   `sal::process::run_silent("cmd_or_script")`: Equivalent to `run("cmd_or_script").silent(true).execute()`.
+
+#### Error Handling:
+-   `RunError`: Enum for errors specific to command/script execution (e.g., `EmptyCommand`, `CommandExecutionFailed`, `ScriptPreparationFailed`).
+
+### 2. Process Management (`mgmt.rs`)
+
+The `mgmt.rs` sub-module provides tools for querying and managing system processes.
+
+#### `ProcessInfo`
+A struct holding basic process information:
+-   `pid: i64`
+-   `name: String`
+-   `memory: f64` (currently a placeholder)
+-   `cpu: f64` (currently a placeholder)
+
+#### Functions:
+-   `sal::process::which(command_name: &str) -> Option<String>`:
+    Checks if a command exists in the system's `PATH`. Returns the full path if found.
+    ```rust
+    if let Some(path) = sal::process::which("git") {
+        println!("Git found at: {}", path);
+    }
+    ```
+
+-   `sal::process::kill(pattern: &str) -> Result<String, ProcessError>`:
+    Kills processes matching the given `pattern` (name or part of the command line).
+    Uses `taskkill` on Windows and `pkill -f` on Unix-like systems.
+    ```rust
+    match sal::process::kill("my-server-proc") {
+        Ok(msg) => println!("{}", msg), // "Successfully killed processes" or "No matching processes found"
+        Err(e) => eprintln!("Error killing process: {}", e),
+    }
+    ```
+
+-   `sal::process::process_list(pattern: &str) -> Result<Vec<ProcessInfo>, ProcessError>`:
+    Lists running processes, optionally filtering by a `pattern` (substring match on name). If `pattern` is empty, lists all accessible processes.
+    Uses `wmic` on Windows and `ps` on Unix-like systems.
+    ```rust
+    match sal::process::process_list("nginx") {
+        Ok(procs) => {
+            for p in procs {
+                println!("PID: {}, Name: {}", p.pid, p.name);
+            }
+        },
+        Err(e) => eprintln!("Error listing processes: {}", e),
+    }
+    ```
+
+-   `sal::process::process_get(pattern: &str) -> Result<ProcessInfo, ProcessError>`:
+    Retrieves a single `ProcessInfo` for a process matching `pattern`.
+    Returns an error if zero or multiple processes match.
+    ```rust
+    match sal::process::process_get("unique_process_name") {
+        Ok(p) => println!("Found: PID {}, Name {}", p.pid, p.name),
+        Err(sal::process::ProcessError::NoProcessFound(patt)) => eprintln!("No process like '{}'", patt),
+        Err(sal::process::ProcessError::MultipleProcessesFound(patt, count)) => {
+            eprintln!("Found {} processes like '{}'", count, patt);
+        }
+        Err(e) => eprintln!("Error: {}", e),
+    }
+    ```
+
+#### Error Handling:
+-   `ProcessError`: Enum for errors specific to process management (e.g., `CommandExecutionFailed`, `NoProcessFound`, `MultipleProcessesFound`).
 
 ## Examples
-### Example 1: Running a Process
-To run a process, use the `run` function from the `run.rs` file:
-```rust
-use process::run;
 
-fn main() {
-    run("ls -l");
+### Running a simple command
+```rust
+use sal::process;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let result = process::run("echo 'Hello from SAL!'").execute()?;
+    println!("Output: {}", result.stdout);
+    Ok(())
 }
 ```
 
-### Example 2: Running a Multi-line Script
+### Running a multi-line script silently
 ```rust
-use process::run;
+use sal::process;
 
-fn main() {
-    let result = run(r#"
-        echo "Hello, world!"
-        ls -la
-        echo "Script complete"
-    "#);
-}
-```
-### Example 2: Managing a Process
-To manage a process, use the `mgmt` function from the `mgmt.rs` file:
-```rust
-use process::mgmt;
-
-fn main() {
-    mgmt("start");
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let script = r#"
+        echo "Starting script..."
+        date
+        echo "Script finished."
+    "#;
+    let result = process::run(script).silent(true).execute()?;
+    if result.success {
+        println!("Script executed successfully. Output:\n{}", result.stdout);
+    } else {
+        eprintln!("Script failed. Error:\n{}", result.stderr);
+    }
+    Ok(())
 }
 ```
 
-## Automatic Dedentation
-When a multi-line script is provided to the `run` function, it automatically uses the `dedent` function from `src/text/dedent.rs` to remove common leading whitespace from all lines. This allows you to write scripts with proper indentation in your code without affecting the execution.
-
-For example, this indented script:
+### Checking if a command exists and then running it
 ```rust
-run(r#"
-    echo "This line has 4 spaces of indentation in the source"
-    echo "This line also has 4 spaces"
-        echo "This line has 8 spaces (4 more than the common indentation)"
-"#);
+use sal::process;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    if process::which("figlet").is_some() {
+        process::run("figlet 'SAL Process'").execute()?;
+    } else {
+        println!("Figlet not found, using echo instead:");
+        process::run("echo 'SAL Process'").execute()?;
+    }
+    Ok(())
+}
 ```
-Will be executed with the common indentation (4 spaces) removed, preserving only the relative indentation between lines.
-
-

src/text/README.md

@@ -1,189 +1,307 @@
-# Text Processing Utilities
+# SAL Text Module (`sal::text`)
 
-A collection of Rust utilities for common text processing operations.
+This module provides a collection of utilities for common text processing and manipulation tasks in Rust, with bindings for Rhai scripting.
 
 ## Overview
 
-This module provides functions for text manipulation tasks such as:
-- Removing indentation from multiline strings
-- Adding prefixes to multiline strings
-- Normalizing filenames and paths
-- Text replacement (regex and literal) with file operations
+The `sal::text` module offers functionalities for:
+- **Indentation**: Removing common leading whitespace (`dedent`) and adding prefixes to lines (`prefix`).
+- **Normalization**: Sanitizing strings for use as filenames (`name_fix`) or fixing filename components within paths (`path_fix`).
+- **Replacement**: A powerful `TextReplacer` for performing single or multiple regex or literal text replacements in strings or files.
+- **Templating**: A `TemplateBuilder` using the Tera engine to render text templates with dynamic data.
 
-## Functions
+## Rust API
 
-### Text Indentation
+### 1. Text Indentation
 
-#### `dedent(text: &str) -> String`
+Located in `src/text/dedent.rs` (for `dedent`) and `src/text/fix.rs` (likely contains `prefix`, though not explicitly confirmed by file view, its Rhai registration implies existence).
 
-Removes common leading whitespace from multiline strings.
+-   **`dedent(text: &str) -> String`**: Removes common leading whitespace from a multiline string. Tabs are treated as 4 spaces. Ideal for cleaning up heredocs or indented code snippets.
+    ```rust
+    use sal::text::dedent;
+    let indented_text = "    Hello\n      World";
+    assert_eq!(dedent(indented_text), "Hello\n  World");
+    ```
+
+-   **`prefix(text: &str, prefix_str: &str) -> String`**: Adds `prefix_str` to the beginning of each line in `text`.
+    ```rust
+    use sal::text::prefix;
+    let text = "line1\nline2";
+    assert_eq!(prefix(text, "> "), "> line1\n> line2");
+    ```
+
+### 2. Filename and Path Normalization
+
+Located in `src/text/fix.rs`.
+
+-   **`name_fix(text: &str) -> String`**: Sanitizes a string to be suitable as a name or filename component. It converts to lowercase, replaces whitespace and various special characters with underscores, and removes non-ASCII characters.
+    ```rust
+    use sal::text::name_fix;
+    assert_eq!(name_fix("My File (New).txt"), "my_file_new_.txt");
+    assert_eq!(name_fix("Café crème.jpg"), "caf_crm.jpg");
+    ```
+
+-   **`path_fix(text: &str) -> String`**: Applies `name_fix` to the filename component of a given path string, leaving the directory structure intact.
+    ```rust
+    use sal::text::path_fix;
+    assert_eq!(path_fix("/some/path/My Document.docx"), "/some/path/my_document.docx");
+    ```
+
+### 3. Text Replacement (`TextReplacer`)
+
+Located in `src/text/replace.rs`. Provides `TextReplacer` and `TextReplacerBuilder`.
+
+The `TextReplacer` allows for complex, chained replacement operations on strings or file contents.
+
+**Builder Pattern:**
 
 ```rust
-let indented = "    line 1\n    line 2\n        line 3";
-let dedented = dedent(indented);
-assert_eq!(dedented, "line 1\nline 2\n    line 3");
-```
+use sal::text::TextReplacer;
 
-**Features:**
-- Analyzes all non-empty lines to determine minimum indentation
-- Preserves empty lines but removes all leading whitespace from them
-- Treats tabs as 4 spaces for indentation purposes
-
-#### `prefix(text: &str, prefix: &str) -> String`
-
-Adds a specified prefix to each line of a multiline string.
-
-```rust
-let text = "line 1\nline 2\nline 3";
-let prefixed = prefix(text, "    ");
-assert_eq!(prefixed, "    line 1\n    line 2\n    line 3");
-```
-
-### Filename and Path Normalization
-
-#### `name_fix(text: &str) -> String`
-
-Normalizes filenames by:
-- Converting to lowercase
-- Replacing whitespace and special characters with underscores
-- Removing non-ASCII characters
-- Collapsing consecutive special characters into a single underscore
-
-```rust
-assert_eq!(name_fix("Hello World"), "hello_world");
-assert_eq!(name_fix("File-Name.txt"), "file_name.txt");
-assert_eq!(name_fix("Résumé"), "rsum");
-```
-
-#### `path_fix(text: &str) -> String`
-
-Applies `name_fix()` to the filename portion of a path while preserving the directory structure.
-
-```rust
-assert_eq!(path_fix("/path/to/File Name.txt"), "/path/to/file_name.txt");
-assert_eq!(path_fix("./relative/path/to/DOCUMENT-123.pdf"), "./relative/path/to/document_123.pdf");
-```
-
-**Features:**
-- Preserves paths ending with `/` (directories)
-- Only normalizes the filename portion, leaving the path structure intact
-- Handles both absolute and relative paths
-
-### Text Replacement
-
-#### `TextReplacer`
-
-A flexible text replacement utility that supports both regex and literal replacements with a builder pattern.
-
-```rust
-// Regex replacement
+// Example: Multiple replacements, regex and literal
 let replacer = TextReplacer::builder()
-    .pattern(r"\bfoo\b")
-    .replacement("bar")
+    .pattern(r"\d+") // Regex: match one or more digits
+    .replacement("NUMBER")
     .regex(true)
-    .add_replacement()
-    .unwrap()
+    .and() // Chain another replacement
+    .pattern("World") // Literal string
+    .replacement("Universe")
+    .regex(false) // Explicitly literal, though default
     .build()
-    .unwrap();
+    .expect("Failed to build replacer");
 
-let result = replacer.replace("foo bar foo baz"); // "bar bar bar baz"
-```
+let original_text = "Hello World, item 123 and item 456.";
+let modified_text = replacer.replace(original_text);
+assert_eq!(modified_text, "Hello Universe, item NUMBER and item NUMBER.");
 
-**Features:**
-- Supports both regex and literal string replacements
-- Builder pattern for fluent configuration
-- Multiple replacements in a single pass
-- Case-insensitive matching (for regex replacements)
-- File reading and writing operations
-
-#### Multiple Replacements
-
-```rust
-let replacer = TextReplacer::builder()
-    .pattern("foo")
-    .replacement("qux")
-    .add_replacement()
-    .unwrap()
-    .pattern("bar")
-    .replacement("baz")
-    .add_replacement()
-    .unwrap()
-    .build()
-    .unwrap();
-
-let result = replacer.replace("foo bar foo"); // "qux baz qux"
-```
-
-#### File Operations
-
-```rust
-// Replace in a file and get the result as a string
-let result = replacer.replace_file("input.txt")?;
-
-// Replace in a file and write back to the same file
-replacer.replace_file_in_place("input.txt")?;
-
-// Replace in a file and write to a new file
-replacer.replace_file_to("input.txt", "output.txt")?;
-```
-
-#### Case-Insensitive Matching
-
-```rust
-let replacer = TextReplacer::builder()
-    .pattern("foo")
-    .replacement("bar")
+// Case-insensitive regex example
+let case_replacer = TextReplacer::builder()
+    .pattern("apple")
+    .replacement("FRUIT")
     .regex(true)
     .case_insensitive(true)
-    .add_replacement()
-    .unwrap()
     .build()
     .unwrap();
-
-let result = replacer.replace("FOO foo Foo"); // "bar bar bar"
+assert_eq!(case_replacer.replace("Apple and apple"), "FRUIT and FRUIT");
 ```
 
-## Usage
+**Key `TextReplacerBuilder` methods:**
+-   `pattern(pat: &str)`: Sets the search pattern (string or regex).
+-   `replacement(rep: &str)`: Sets the replacement string.
+-   `regex(yes: bool)`: If `true`, treats `pattern` as a regex. Default is `false` (literal).
+-   `case_insensitive(yes: bool)`: If `true` (and `regex` is `true`), performs case-insensitive matching.
+-   `and()`: Finalizes the current replacement operation and prepares for a new one.
+-   `build()`: Consumes the builder and returns a `Result<TextReplacer, String>`.
 
-Import the functions from the module:
+**`TextReplacer` methods:**
+-   `replace(input: &str) -> String`: Applies all configured replacements to the input string.
+-   `replace_file(path: P) -> io::Result<String>`: Reads a file, applies replacements, returns the result.
+-   `replace_file_in_place(path: P) -> io::Result<()>`: Replaces content in the specified file directly.
+-   `replace_file_to(input_path: P1, output_path: P2) -> io::Result<()>`: Reads from `input_path`, applies replacements, writes to `output_path`.
+
+### 4. Text Templating (`TemplateBuilder`)
+
+Located in `src/text/template.rs`. Uses the Tera templating engine.
+
+**Builder Pattern:**
 
 ```rust
-use your_crate::text::{dedent, prefix, name_fix, path_fix, TextReplacer};
+use sal::text::TemplateBuilder;
+use std::collections::HashMap;
+
+// Assume "./my_template.txt" contains: "Hello, {{ name }}! You are {{ age }}."
+
+// Create a temporary template file for the example
+std::fs::write("./my_template.txt", "Hello, {{ name }}! You are {{ age }}.").unwrap();
+
+let mut builder = TemplateBuilder::open("./my_template.txt").expect("Template not found");
+
+// Add variables individually
+builder = builder.add_var("name", "Alice").add_var("age", 30);
+
+let rendered_string = builder.render().expect("Rendering failed");
+assert_eq!(rendered_string, "Hello, Alice! You are 30.");
+
+// Or add multiple variables from a HashMap
+let mut vars = HashMap::new();
+vars.insert("name", "Bob");
+vars.insert("age", "25"); // Values in HashMap are typically strings or serializable types
+
+let mut builder2 = TemplateBuilder::open("./my_template.txt").unwrap();
+builder2 = builder2.add_vars(vars);
+let rendered_string2 = builder2.render().unwrap();
+assert_eq!(rendered_string2, "Hello, Bob! You are 25.");
+
+// Render directly to a file
+// builder.render_to_file("output.txt").expect("Failed to write to file");
+
+// Clean up temporary file
+std::fs::remove_file("./my_template.txt").unwrap();
 ```
 
-## Examples
+**Key `TemplateBuilder` methods:**
+-   `open(template_path: P) -> io::Result<Self>`: Loads the template file.
+-   `add_var(name: S, value: V) -> Self`: Adds a single variable to the context.
+-   `add_vars(vars: HashMap<S, V>) -> Self`: Adds multiple variables from a HashMap.
+-   `render() -> Result<String, tera::Error>`: Renders the template to a string.
+-   `render_to_file(output_path: P) -> io::Result<()>`: Renders the template and writes it to the specified file.
 
-### Cleaning up indented text from a template
+## Rhai Scripting with `herodo`
 
-```rust
-let template = "
-    <div>
-        <h1>Title</h1>
-        <p>
-            Some paragraph text
-            with multiple lines
-        </p>
-    </div>
-";
+The `sal::text` module's functionalities are exposed to Rhai scripts when using `herodo`.
+
+### Direct Functions
+
+-   **`dedent(text_string)`**: Removes common leading whitespace.
+    -   Example: `let clean_script = dedent("    if true {\n        print(\"indented\");\n    }");`
+-   **`prefix(text_string, prefix_string)`**: Adds `prefix_string` to each line of `text_string`.
+    -   Example: `let prefixed_text = prefix("hello\nworld", "# ");`
+-   **`name_fix(text_string)`**: Normalizes a string for use as a filename.
+    -   Example: `let filename = name_fix("My Document (V2).docx"); // "my_document_v2_.docx"`
+-   **`path_fix(path_string)`**: Normalizes the filename part of a path.
+    -   Example: `let fixed_path = path_fix("/uploads/User Files/Report [Final].pdf");`
+
+### TextReplacer
+
+Provides text replacement capabilities through a builder pattern.
+
+1.  **Create a builder**: `let builder = text_replacer_new();`
+2.  **Configure replacements** (methods return the builder for chaining):
+    -   `builder = builder.pattern(search_pattern_string);`
+    -   `builder = builder.replacement(replacement_string);`
+    -   `builder = builder.regex(is_regex_bool);` (default `false`)
+    -   `builder = builder.case_insensitive(is_case_insensitive_bool);` (default `false`, only applies if `regex` is `true`)
+    -   `builder = builder.and();` (to add the current replacement and start a new one)
+3.  **Build the replacer**: `let replacer = builder.build();`
+4.  **Use the replacer**:
+    -   `let modified_text = replacer.replace(original_text_string);`
+    -   `let modified_text_from_file = replacer.replace_file(input_filepath_string);`
+    -   `replacer.replace_file_in_place(filepath_string);`
+    -   `replacer.replace_file_to(input_filepath_string, output_filepath_string);`
+
+### TemplateBuilder
+
+Provides text templating capabilities.
+
+1.  **Open a template file**: `let tpl_builder = template_builder_open(template_filepath_string);`
+2.  **Add variables** (methods return the builder for chaining):
+    -   `tpl_builder = tpl_builder.add_var(name_string, value);` (value can be string, int, float, bool, or array)
+    -   `tpl_builder = tpl_builder.add_vars(map_object);` (map keys are variable names, values are their corresponding values)
+3.  **Render the template**:
+    -   `let rendered_string = tpl_builder.render();`
+    -   `tpl_builder.render_to_file(output_filepath_string);`
+
+## Rhai Example
+
+```rhai
+// Create a temporary file for template demonstration
+let template_content = "Report for {{user}}:\nItems processed: {{count}}.\nStatus: {{status}}.";
+let template_path = "./temp_report_template.txt";
+
+// Using file.write (assuming sal::file module is available and registered)
+// For this example, we'll assume a way to write this file or that it exists.
+// For a real script, ensure the file module is used or the file is pre-existing.
+print(`Intending to write template to: ${template_path}`);
+// In a real scenario: file.write(template_path, template_content);
+
+// For demonstration, let's simulate it exists for the template_builder_open call.
+// If file module is not used, this script part needs adjustment or pre-existing file.
+
+// --- Text Normalization ---
+let raw_filename = "User's Report [Draft 1].md";
+let safe_filename = name_fix(raw_filename);
+print(`Safe filename: ${safe_filename}`); // E.g., "users_report_draft_1_.md"
+
+let raw_path = "/data/project files/Final Report (2023).pdf";
+let safe_path = path_fix(raw_path);
+print(`Safe path: ${safe_path}`); // E.g., "/data/project files/final_report_2023_.pdf"
+
+// --- Dedent and Prefix ---
+let script_block = "\n    for item in items {\n        print(item);\n    }\n";
+let dedented_script = dedent(script_block);
+print("Dedented script:\n" + dedented_script);
+
+let prefixed_log = prefix("Operation successful.\nDetails logged.", "LOG: ");
+print(prefixed_log);
+
+// --- TextReplacer Example ---
+let text_to_modify = "The quick brown fox jumps over the lazy dog. The dog was very lazy.";
+
+let replacer_builder = text_replacer_new()
+    .pattern("dog")
+    .replacement("cat")
+    .case_insensitive(true) // Replace 'dog', 'Dog', 'DOG', etc.
+    .and()
+    .pattern("lazy")
+    .replacement("energetic")
+    .regex(false); // This is the default, explicit for clarity
+
+let replacer = replacer_builder.build();
+let replaced_text = replacer.replace(text_to_modify);
+print(`Replaced text: ${replaced_text}`);
+// Expected: The quick brown fox jumps over the energetic cat. The cat was very energetic.
+
+// --- TemplateBuilder Example ---
+// This part assumes 'temp_report_template.txt' was successfully created with content:
+// "Report for {{user}}:\nItems processed: {{count}}.\nStatus: {{status}}."
+// If not, template_builder_open will fail. For a robust script, check file existence or create it.
+
+// Create a dummy template file if it doesn't exist for the example to run
+// This would typically be done using the file module, e.g. file.write()
+// For simplicity here, we'll just print a message if it's missing.
+// In a real script: if !file.exists(template_path) { file.write(template_path, template_content); }
+
+// Let's try to proceed assuming the template might exist or skip if not.
+// A more robust script would handle the file creation explicitly.
+
+// For the sake of this example, let's create it directly if possible (conceptual)
+// This is a placeholder for actual file writing logic.
+// if (true) { // Simulate file creation for example purpose
+//    std.os.remove_file(template_path); // Clean up if exists
+//    let f = std.io.open(template_path, "w"); f.write(template_content); f.close();
+// }
+
+// Due to the sandbox, direct file system manipulation like above isn't typically done in Rhai examples
+// without relying on registered SAL functions. We'll assume the file exists.
+
+print("Attempting to use template: " + template_path);
+// It's better to ensure the file exists before calling template_builder_open
+// For this example, we'll proceed, but in a real script, handle file creation.
+
+// Create a dummy file for the template example to work in isolation
+// This is not ideal but helps for a self-contained example if file module isn't used prior.
+// In a real SAL script, you'd use `file.write`.
+let _dummy_template_file_path = "./example_template.rhai.tmp";
+// file.write(_dummy_template_file_path, "Name: {{name}}, Age: {{age}}");
+
+// Using a known, simple template string for robustness if file ops are tricky in example context
+let tpl_builder = template_builder_open(_dummy_template_file_path); // Use the dummy/known file
+
+if tpl_builder.is_ok() {
+    let mut template_engine = tpl_builder.unwrap();
+    template_engine = template_engine.add_var("user", "Jane Doe");
+    template_engine = template_engine.add_var("count", 150);
+    template_engine = template_engine.add_var("status", "Completed");
+
+    let report_output = template_engine.render();
+    if report_output.is_ok() {
+        print("Generated Report:\n" + report_output.unwrap());
+    } else {
+        print("Error rendering template: " + report_output.unwrap_err());
+    }
+
+    // Example: Render to file
+    // template_engine.render_to_file("./generated_report.txt");
+    // print("Report also written to ./generated_report.txt");
+} else {
+    print("Skipping TemplateBuilder example as template file '" + _dummy_template_file_path + "' likely missing or unreadable.");
+    print("Error: " + tpl_builder.unwrap_err());
+    print("To run this part, ensure '" + _dummy_template_file_path + "' exists with content like: 'Name: {{name}}, Age: {{age}}'");
+}
+
+// Clean up dummy file
+// file.remove(_dummy_template_file_path);
 
-let clean = dedent(template);
-// Result:
-// <div>
-//     <h1>Title</h1>
-//     <p>
-//         Some paragraph text
-//         with multiple lines
-//     </p>
-// </div>
 ```
 
-### Normalizing user-provided filenames
-
-```rust
-let user_filename = "My Document (2023).pdf";
-let safe_filename = name_fix(user_filename);
-// Result: "my_document_2023_.pdf"
-
-let user_path = "/uploads/User Files/Report #123.xlsx";
-let safe_path = path_fix(user_path);
-// Result: "/uploads/User Files/report_123.xlsx"
+**Note on Rhai Example File Operations:** The Rhai example above includes comments about file creation for the `TemplateBuilder` part. In a real `herodo` script, you would use `sal::file` module functions (e.g., `file.write`, `file.exists`, `file.remove`) to manage the template file. For simplicity and to avoid making the example dependent on another module's full setup path, it highlights where such operations would occur. The example tries to use a dummy path and gracefully skips if the template isn't found, which is a common issue when running examples in restricted environments or without proper setup. The core logic of using `TemplateBuilder` once the template is loaded remains the same.

src/virt/buildah/README.md

@@ -0,0 +1,232 @@
+# SAL Buildah Module (`sal::virt::buildah`)
+
+## Overview
+
+The Buildah module in SAL provides a comprehensive Rust interface for interacting with the `buildah` command-line tool. It allows users to build OCI (Open Container Initiative) and Docker-compatible container images programmatically. The module offers both a high-level `Builder` API for step-by-step image construction and static functions for managing images in local storage.
+
+A Rhai script interface for this module is also available via `sal::rhai::buildah`, making these functionalities accessible from `herodo` scripts.
+
+## Core Components
+
+### 1. `Builder` Struct (`sal::virt::buildah::Builder`)
+
+The `Builder` struct is the primary entry point for constructing container images. It encapsulates a Buildah working container, created from a base image, and provides methods to modify this container and eventually commit it as a new image.
+
+- **Creation**: `Builder::new(name: &str, image: &str) -> Result<Builder, BuildahError>`
+  - Creates a new working container (or re-attaches to an existing one with the same name) from the specified base `image`.
+- **Debug Mode**: `builder.set_debug(true)` / `builder.debug()`
+  - Enables/disables verbose logging for Buildah commands executed by this builder instance.
+
+#### Working Container Operations:
+
+- `builder.run(command: &str) -> Result<CommandResult, BuildahError>`: Executes a shell command inside the working container (e.g., `buildah run <container> -- <command>`).
+- `builder.run_with_isolation(command: &str, isolation: &str) -> Result<CommandResult, BuildahError>`: Runs a command with specified isolation (e.g., "chroot").
+- `builder.copy(source_on_host: &str, dest_in_container: &str) -> Result<CommandResult, BuildahError>`: Copies files/directories from the host to the container (`buildah copy`).
+- `builder.add(source_on_host: &str, dest_in_container: &str) -> Result<CommandResult, BuildahError>`: Adds files/directories to the container (`buildah add`), potentially handling URLs and archive extraction.
+- `builder.config(options: HashMap<String, String>) -> Result<CommandResult, BuildahError>`: Modifies image metadata (e.g., environment variables, labels, entrypoint, cmd). Example options: `{"env": "MYVAR=value", "label": "mylabel=myvalue"}`.
+- `builder.set_entrypoint(entrypoint: &str) -> Result<CommandResult, BuildahError>`: Sets the image entrypoint.
+- `builder.set_cmd(cmd: &str) -> Result<CommandResult, BuildahError>`: Sets the default command for the image.
+- `builder.commit(image_name: &str) -> Result<CommandResult, BuildahError>`: Commits the current state of the working container to a new image named `image_name`.
+- `builder.remove() -> Result<CommandResult, BuildahError>`: Removes the working container (`buildah rm`).
+- `builder.reset() -> Result<(), BuildahError>`: Removes the working container and resets the builder state.
+
+### 2. Static Image Management Functions (on `Builder`)
+
+These functions operate on images in the local Buildah storage and are not tied to a specific `Builder` instance.
+
+- `Builder::images() -> Result<Vec<Image>, BuildahError>`: Lists all images available locally (`buildah images --json`). Returns a vector of `Image` structs.
+- `Builder::image_remove(image_ref: &str) -> Result<CommandResult, BuildahError>`: Removes an image (`buildah rmi <image_ref>`).
+- `Builder::image_pull(image_name: &str, tls_verify: bool) -> Result<CommandResult, BuildahError>`: Pulls an image from a registry (`buildah pull`).
+- `Builder::image_push(image_ref: &str, destination: &str, tls_verify: bool) -> Result<CommandResult, BuildahError>`: Pushes an image to a registry (`buildah push`).
+- `Builder::image_tag(image_ref: &str, new_name: &str) -> Result<CommandResult, BuildahError>`: Tags an image (`buildah tag`).
+- `Builder::image_commit(container_ref: &str, image_name: &str, format: Option<&str>, squash: bool, rm: bool) -> Result<CommandResult, BuildahError>`: A static version to commit any existing container to an image, with options for format (e.g., "oci", "docker"), squashing layers, and removing the container post-commit.
+- `Builder::build(tag: Option<&str>, context_dir: &str, file: &str, isolation: Option<&str>) -> Result<CommandResult, BuildahError>`: Builds an image from a Dockerfile/Containerfile (`buildah bud`).
+
+*Note: Many static image functions also have a `_with_debug(..., debug: bool)` variant for explicit debug control.*
+
+### 3. `Image` Struct (`sal::virt::buildah::Image`)
+
+Represents a container image as listed by `buildah images`.
+
+```rust
+pub struct Image {
+    pub id: String,      // Image ID
+    pub names: Vec<String>, // Image names/tags
+    pub size: String,    // Image size
+    pub created: String, // Creation timestamp (as string)
+}
+```
+
+### 4. `ContentOperations` (`sal::virt::buildah::ContentOperations`)
+
+Provides static methods for reading and writing file content directly within a container, useful for dynamic configuration or inspection.
+
+- `ContentOperations::write_content(container_id: &str, content: &str, dest_path_in_container: &str) -> Result<CommandResult, BuildahError>`: Writes string content to a file inside the specified container.
+- `ContentOperations::read_content(container_id: &str, source_path_in_container: &str) -> Result<String, BuildahError>`: Reads the content of a file from within the specified container into a string.
+
+### 5. `BuildahError` Enum (`sal::virt::buildah::BuildahError`)
+
+Defines the error types that can occur during Buildah operations:
+- `CommandExecutionFailed(io::Error)`: The `buildah` command itself failed to start.
+- `CommandFailed(String)`: The `buildah` command ran but returned a non-zero exit code or error.
+- `JsonParseError(String)`: Failed to parse JSON output from Buildah.
+- `ConversionError(String)`: Error during data conversion.
+- `Other(String)`: Generic error.
+
+## Key Design Points
+
+The SAL Buildah module is designed with the following principles:
+
+- **Builder Pattern**: The `Builder` struct (`sal::virt::buildah::Builder`) employs a builder pattern, enabling a fluent, step-by-step, and stateful approach to constructing container images. Each `Builder` instance manages a specific working container.
+- **Separation of Concerns**:
+    - **Instance Methods**: Operations specific to a working container (e.g., `run`, `copy`, `config`, `commit`) are methods on the `Builder` instance.
+    - **Static Methods**: General image management tasks (e.g., listing images with `Builder::images()`, removing images with `Builder::image_remove()`, pulling, pushing, tagging, and building from a Dockerfile with `Builder::build()`) are provided as static functions on the `Builder` struct.
+- **Direct Content Manipulation**: The `ContentOperations` struct provides static methods (`write_content`, `read_content`) to directly interact with files within a Buildah container. This is typically achieved by temporarily mounting the container or using `buildah add` with temporary files, abstracting the complexity from the user.
+- **Debuggability**: Fine-grained control over `buildah` command logging is provided. The `builder.set_debug(true)` method enables verbose output for a specific `Builder` instance. Many static functions also offer `_with_debug(..., debug: bool)` variants. This is managed internally via a thread-local flag passed to the core `execute_buildah_command` function.
+- **Comprehensive Rhai Integration**: Most functionalities of the Buildah module are exposed to Rhai scripts executed via `herodo`, allowing for powerful automation of image building workflows. This is facilitated by the `sal::rhai::buildah` module.
+
+## Low-Level Command Execution
+
+- `execute_buildah_command(args: &[&str]) -> Result<CommandResult, BuildahError>` (in `sal::virt::buildah::cmd`):
+  The core function that executes `buildah` commands. It handles debug logging based on a thread-local flag, which is managed by the higher-level `Builder` methods and `_with_debug` static function variants.
+
+## Usage Example (Rust)
+
+```rust
+use sal::virt::buildah::{Builder, BuildahError, ContentOperations};
+use std::collections::HashMap;
+
+fn build_custom_image() -> Result<String, BuildahError> {
+    // Create a new builder from a base image (e.g., alpine)
+    let mut builder = Builder::new("my-custom-container", "docker.io/library/alpine:latest")?;
+    builder.set_debug(true);
+
+    // Run some commands
+    builder.run("apk add --no-cache curl")?;
+    builder.run("mkdir /app")?;
+
+    // Add a file
+    ContentOperations::write_content(builder.container_id().unwrap(), "Hello from SAL!", "/app/hello.txt")?;
+
+    // Set image configuration
+    let mut config_opts = HashMap::new();
+    config_opts.insert("workingdir".to_string(), "/app".to_string());
+    config_opts.insert("label".to_string(), "maintainer=sal-user".to_string());
+    builder.config(config_opts)?;
+    builder.set_entrypoint("["/usr/bin/curl"]")?;
+    builder.set_cmd("["http://ifconfig.me"]")?;
+
+    // Commit the image
+    let image_tag = "localhost/my-custom-image:latest";
+    builder.commit(image_tag)?;
+
+    println!("Successfully built image: {}", image_tag);
+
+    // Clean up the working container
+    builder.remove()?;
+
+    Ok(image_tag.to_string())
+}
+
+fn main() {
+    match build_custom_image() {
+        Ok(tag) => println!("Image {} created.", tag),
+        Err(e) => eprintln!("Error building image: {}", e),
+    }
+}
+```
+
+## Rhai Scripting with `herodo`
+
+The Buildah module's capabilities are extensively exposed to Rhai scripts, enabling automation of image building and management tasks via the `herodo` CLI tool. The `sal::rhai::buildah` module registers the necessary functions and types.
+
+Below is a summary of commonly used Rhai functions for Buildah. (Note: `builder` refers to an instance of `BuildahBuilder` obtained typically via `bah_new`).
+
+### Builder Object Management
+- `bah_new(name: String, image: String) -> BuildahBuilder`: Creates a new Buildah builder instance (working container) from a base `image` with a given `name`.
+- `builder.remove()`: Removes the working container associated with the `builder`.
+- `builder.reset()`: Removes the working container and resets the `builder` state.
+
+### Builder Configuration & Operations
+- `builder.set_debug(is_debug: bool)`: Enables or disables verbose debug logging for commands executed by this `builder`.
+- `builder.debug_mode` (property): Get or set the debug mode (e.g., `let mode = builder.debug_mode; builder.debug_mode = true;`).
+- `builder.container_id` (property): Returns the ID of the working container (e.g., `let id = builder.container_id;`).
+- `builder.name` (property): Returns the name of the builder/working container.
+- `builder.image` (property): Returns the base image name used by the builder.
+- `builder.run(command: String)`: Executes a shell command inside the `builder`'s working container.
+- `builder.run_with_isolation(command: String, isolation: String)`: Runs a command with specified isolation (e.g., "chroot").
+- `builder.copy(source_on_host: String, dest_in_container: String)`: Copies files/directories from the host to the `builder`'s container.
+- `builder.add(source_on_host: String, dest_in_container: String)`: Adds files/directories to the `builder`'s container (can handle URLs and auto-extract archives).
+- `builder.config(options: Map)`: Modifies image metadata. `options` is a Rhai map, e.g., `#{ "env": "MYVAR=value", "label": "foo=bar" }`.
+- `builder.set_entrypoint(entrypoint: String)`: Sets the image entrypoint (e.g., `builder.set_entrypoint("[/app/run.sh]")`).
+- `builder.set_cmd(cmd: String)`: Sets the default command for the image (e.g., `builder.set_cmd("[--help]")`).
+- `builder.commit(image_tag: String)`: Commits the current state of the `builder`'s working container to a new image with `image_tag`.
+
+### Content Operations (with a Builder instance)
+- `bah_write_content(builder: BuildahBuilder, content: String, dest_path_in_container: String)`: Writes string `content` to a file at `dest_path_in_container` inside the `builder`'s container.
+- `bah_read_content(builder: BuildahBuilder, source_path_in_container: String) -> String`: Reads the content of a file from `source_path_in_container` within the `builder`'s container.
+
+### Global Image Operations
+These functions generally correspond to static methods in Rust and operate on the local Buildah image storage.
+- `bah_images() -> Array`: Lists all images available locally. Returns an array of `BuildahImage` objects.
+- `bah_image_remove(image_ref: String)`: Removes an image (e.g., by ID or tag) from local storage.
+- `bah_image_pull(image_name: String, tls_verify: bool)`: Pulls an image from a registry.
+- `bah_image_push(image_ref: String, destination: String, tls_verify: bool)`: Pushes a local image to a registry.
+- `bah_image_tag(image_ref: String, new_name: String)`: Adds a new tag (`new_name`) to an existing image (`image_ref`).
+- `bah_build(tag: String, context_dir: String, file: String, isolation: String)`: Builds an image from a Dockerfile/Containerfile (equivalent to `buildah bud`). `file` is the path to the Dockerfile relative to `context_dir`. `isolation` can be e.g., "chroot".
+
+### Example `herodo` Rhai Script (Revisited)
+
+```rhai
+// Create a new builder
+let builder = bah_new("my-rhai-app", "docker.io/library/alpine:latest");
+builder.debug_mode = true; // Enable debug logging for this builder
+
+// Run commands in the container
+builder.run("apk add --no-cache figlet curl");
+builder.run("mkdir /data");
+
+// Write content to a file in the container
+bah_write_content(builder, "Hello from SAL Buildah via Rhai!", "/data/message.txt");
+
+// Configure image metadata
+builder.config(#{ 
+    "env": "APP_VERSION=1.0",
+    "label": "author=HerodoUser"
+});
+builder.set_entrypoint('["figlet"]');
+builder.set_cmd('["Rhai Build"]');
+
+// Commit the image
+let image_name = "localhost/my-rhai-app:v1";
+builder.commit(image_name);
+print(`Image committed: ${image_name}`);
+
+// Clean up the working container
+builder.remove();
+print("Builder container removed.");
+
+// List local images
+print("Current local images:");
+let images = bah_images();
+for img in images {
+    print(`  ID: ${img.id}, Name(s): ${img.names}, Size: ${img.size}`);
+}
+
+// Example: Build from a Dockerfile (assuming Dockerfile exists at /tmp/build_context/Dockerfile)
+// Ensure /tmp/build_context/Dockerfile exists with simple content like:
+// FROM alpine
+// RUN echo "Built with bah_build" > /built.txt
+// CMD cat /built.txt
+//
+// if exist("/tmp/build_context/Dockerfile") {
+//   print("Building from Dockerfile...");
+//   bah_build("localhost/from-dockerfile:latest", "/tmp/build_context", "Dockerfile", "chroot");
+//   print("Dockerfile build complete.");
+//   bah_image_remove("localhost/from-dockerfile:latest"); // Clean up
+// } else {
+//   print("Skipping Dockerfile build example: /tmp/build_context/Dockerfile not found.");
+// }
+```
+
+This README provides a guide to using the SAL Buildah module. For more detailed information on specific functions and their parameters, consult the Rust doc comments within the source code.

src/virt/nerdctl/README.md

@@ -1,374 +1,223 @@
-# Container API for nerdctl
-
-This module provides a Rust API for managing containers using nerdctl, a Docker-compatible CLI for containerd.
+# SAL `nerdctl` Module (`sal::virt::nerdctl`)
 
 ## Overview
 
-The Container API is designed with a builder pattern to make it easy to create and manage containers. It provides a fluent interface for configuring container options and performing operations on containers.
+The `sal::virt::nerdctl` module provides a comprehensive Rust interface for interacting with `nerdctl`, a command-line tool for `containerd`.
+It allows for managing container lifecycles, images, and other `nerdctl` functionalities programmatically from Rust and through Rhai scripts via `herodo`.
 
-## Key Components
+This module offers two primary ways to interact with `nerdctl`:
+1.  A fluent **`Container` builder pattern** for defining, creating, and managing containers with detailed configurations.
+2.  **Direct static functions** that wrap common `nerdctl` commands for quick operations on containers and images.
 
-- `Container`: The main struct representing a container
-- `HealthCheck`: Configuration for container health checks
-- `ContainerStatus`: Information about a container's status
-- `ResourceUsage`: Information about a container's resource usage
+## Core Components
 
-## Getting Started
+### 1. `NerdctlError` (in `mod.rs`)
 
-### Prerequisites
+An enum defining specific error types for `nerdctl` operations:
+- `CommandExecutionFailed(io::Error)`: `nerdctl` command failed to start (e.g., not found).
+- `CommandFailed(String)`: `nerdctl` command executed but returned an error.
+- `JsonParseError(String)`: Failure to parse JSON output from `nerdctl`.
+- `ConversionError(String)`: Error during data type conversions.
+- `Other(String)`: Generic errors.
 
-- nerdctl must be installed on your system
-- containerd must be running
+### 2. `execute_nerdctl_command` (in `cmd.rs`)
 
-### Basic Usage
-
-Add the following to your `Cargo.toml`:
-
-```toml
-[dependencies]
-sal = { path = "/path/to/sal" }
-```
-
-Then import the Container API in your Rust code:
+The core function for executing `nerdctl` commands. It takes an array of string arguments, runs the command, and returns a `CommandResult` or `NerdctlError`.
 
 ```rust
-use sal::virt::nerdctl::Container;
+// Example (internal usage)
+// use sal::virt::nerdctl::execute_nerdctl_command;
+// let result = execute_nerdctl_command(&["ps", "-a"]);
 ```
 
+### 3. `Container` Struct (defined in `container_types.rs`, builder in `container_builder.rs`, operations in `container_operations.rs`)
+
+Represents a `nerdctl` container and is the centerpiece of the builder pattern.
+
+**Fields (Configuration):**
+- `name: String`: Name of the container.
+- `container_id: Option<String>`: ID of the container (populated after creation).
+- `image: Option<String>`: Base image for the container.
+- `ports: Vec<String>`: Port mappings (e.g., `"8080:80"`).
+- `volumes: Vec<String>`: Volume mounts (e.g., `"/host/path:/container/path"`).
+- `env_vars: HashMap<String, String>`: Environment variables.
+- `network: Option<String>`: Network to connect to.
+- `network_aliases: Vec<String>`: Network aliases.
+- `cpu_limit: Option<String>`, `memory_limit: Option<String>`, `memory_swap_limit: Option<String>`, `cpu_shares: Option<String>`: Resource limits.
+- `restart_policy: Option<String>`: Restart policy (e.g., `"always"`).
+- `health_check: Option<HealthCheck>`: Health check configuration.
+- `detach: bool`: Whether to run in detached mode (default: `false`, but Rhai `container_build` implies `true` often).
+- `snapshotter: Option<String>`: Snapshotter to use.
+
+**Builder Methods (Fluent Interface - `impl Container` in `container_builder.rs`):**
+These methods configure the `Container` object and return `Self` for chaining.
+- `Container::new(name: &str, image: &str)`: Constructor (Note: Rhai uses `nerdctl_container_new(name)` and `nerdctl_container_from_image(name, image)` which call underlying Rust constructors).
+- `reset()`: Resets configuration, stops/removes existing container with the same name.
+- `with_port(port: &str)`, `with_ports(ports: &[&str])`
+- `with_volume(volume: &str)`, `with_volumes(volumes: &[&str])`
+- `with_env(key: &str, value: &str)`, `with_envs(env_map: &HashMap<&str, &str>)`
+- `with_network(network: &str)`
+- `with_network_alias(alias: &str)`, `with_network_aliases(aliases: &[&str])`
+- `with_cpu_limit(cpus: &str)`
+- `with_memory_limit(memory: &str)`
+- `with_memory_swap_limit(memory_swap: &str)`
+- `with_cpu_shares(shares: &str)`
+- `with_restart_policy(policy: &str)`
+- `with_health_check(cmd: &str)`
+- `with_health_check_options(cmd, interval, timeout, retries, start_period)`
+- `with_snapshotter(snapshotter: &str)`
+- `with_detach(detach: bool)`
+
+**Action Methods (on `Container` instances):
+- `build()` (in `container_builder.rs`): Assembles and executes `nerdctl run` with all configured options. Populates `container_id` on success.
+- `start()` (in `container_operations.rs`): Starts the container. If not yet built, it attempts to pull the image and build the container first. Verifies the container is running and provides detailed logs/status on failure.
+- `stop()` (in `container_operations.rs`): Stops the container.
+- `remove()` (in `container_operations.rs`): Removes the container.
+- `exec(command: &str)` (in `container_operations.rs`): Executes a command in the container.
+- `copy(source: &str, dest: &str)` (in `container_operations.rs`): Copies files/folders. `source`/`dest` must be formatted like `container_name_or_id:/path` or `/local/path`.
+- `status()` (in `container_operations.rs`): Returns `ContainerStatus` by parsing `nerdctl inspect`.
+- `health_status()` (in `container_operations.rs`): Returns the health status string from `nerdctl inspect`.
+- `logs()` (in `container_operations.rs`): Fetches container logs.
+- `resources()` (in `container_operations.rs`): Returns `ResourceUsage` by parsing `nerdctl stats`.
+- `commit(image_name: &str)` (in `container_operations.rs`): Commits the container to a new image.
+- `export(path: &str)` (in `container_operations.rs`): Exports the container's filesystem as a tarball.
+
+### 4. `HealthCheck` Struct (in `container_types.rs`)
+
+Defines health check parameters:
+- `cmd: String`: Command to execute.
+- `interval: Option<String>`
+- `timeout: Option<String>`
+- `retries: Option<u32>`
+- `start_period: Option<String>`
+
+### 5. `prepare_health_check_command` (in `health_check_script.rs`)
+
+A helper function that takes a health check command string. If it's multi-line, it attempts to save it as an executable script in `/root/hero/var/containers/healthcheck_<container_name>.sh` and returns the script path. Otherwise, returns the command as is. The path `/root/hero/var/containers` implies this script needs to be accessible from within the target container at that specific location if a multi-line script is used.
+
+### 6. `Image` Struct (in `images.rs`)
+
+Represents a `nerdctl` image, typically from `nerdctl images` output.
+- `id: String`
+- `repository: String`
+- `tag: String`
+- `size: String`
+- `created: String`
+
+### 7. Static Image Functions (in `images.rs`)
+
+These functions operate on images:
+- `images() -> Result<CommandResult, NerdctlError>`: Lists images (`nerdctl images`).
+- `image_remove(image: &str)`: Removes an image (`nerdctl rmi`).
+- `image_push(image: &str, destination: &str)`: Pushes an image (`nerdctl push`).
+- `image_tag(image: &str, new_name: &str)`: Tags an image (`nerdctl tag`).
+- `image_pull(image: &str)`: Pulls an image (`nerdctl pull`).
+- `image_commit(container: &str, image_name: &str)`: Commits a container to an image (`nerdctl commit`).
+- `image_build(tag: &str, context_path: &str)`: Builds an image from a Dockerfile (`nerdctl build -t <tag> <context_path>`).
+
+### 8. Static Container Functions (in `container_functions.rs`)
+
+Direct wrappers for `nerdctl` commands, an alternative to the builder pattern:
+- `run(image: &str, name: Option<&str>, detach: bool, ports: Option<&[&str]>, snapshotter: Option<&str>)`: Runs a container.
+- `exec(container: &str, command: &str)`: Executes a command in a running container.
+- `copy(source: &str, dest: &str)`: Copies files.
+- `stop(container: &str)`: Stops a container.
+- `remove(container: &str)`: Removes a container.
+- `list(all: bool)`: Lists containers (`nerdctl ps`).
+- `logs(container: &str)`: Fetches logs for a container.
+
+### 9. `ContainerStatus` and `ResourceUsage` Structs (in `container_types.rs`)
+- `ContainerStatus`: Holds parsed data from `nerdctl inspect` (state, status, created, started, health info).
+- `ResourceUsage`: Holds parsed data from `nerdctl stats` (CPU, memory, network, block I/O, PIDs).
+
 ## Usage Examples
 
-### Getting a Container by Name
-
-You can get a reference to an existing container by name:
+### Rust Example (Builder Pattern)
 
 ```rust
-use sal::virt::nerdctl::Container;
+use sal::virt::nerdctl::{Container, NerdctlError};
+use std::collections::HashMap;
 
-// Get a container by name (if it exists)
-match Container::new("existing-container") {
-    Ok(container) => {
-        if container.container_id.is_some() {
-            println!("Found container with ID: {}", container.container_id.unwrap());
-            
-            // Perform operations on the existing container
-            let status = container.status()?;
-            println!("Container status: {}", status.status);
-        } else {
-            println!("Container exists but has no ID");
-        }
-    },
-    Err(e) => {
-        println!("Error getting container: {}", e);
-    }
-}
-```
+fn main() -> Result<(), NerdctlError> {
+    let mut envs = HashMap::new();
+    envs.insert("MY_VAR", "my_value");
 
-### Creating a Container
-
-You can create a new container from an image using the builder pattern:
-
-```rust
-use sal::virt::nerdctl::Container;
-
-// Create a container from an image
-let container = Container::from_image("my-nginx", "nginx:latest")?
-    .with_port("8080:80")
-    .with_env("NGINX_HOST", "example.com")
-    .with_volume("/tmp/nginx:/usr/share/nginx/html")
-    .with_health_check("curl -f http://localhost/ || exit 1")
-    .with_detach(true)
-    .build()?;
-```
-
-### Container Operations
-
-Once you have a container, you can perform various operations on it:
-
-```rust
-// Execute a command in the container
-let result = container.exec("echo 'Hello from container'")?;
-println!("Command output: {}", result.stdout);
-
-// Get container status
-let status = container.status()?;
-println!("Container status: {}", status.status);
-
-// Get resource usage
-let resources = container.resources()?;
-println!("CPU usage: {}", resources.cpu_usage);
-println!("Memory usage: {}", resources.memory_usage);
-
-// Stop and remove the container
-container.stop()?;
-container.remove()?;
-```
-
-## Container Configuration Options
-
-The Container API supports a wide range of configuration options through its builder pattern:
-
-### Ports
-
-Map container ports to host ports:
-
-```rust
-// Map a single port
-.with_port("8080:80")
-
-// Map multiple ports
-.with_ports(&["8080:80", "8443:443"])
-```
-
-### Volumes
-
-Mount host directories or volumes in the container:
-
-```rust
-// Mount a single volume
-.with_volume("/host/path:/container/path")
-
-// Mount multiple volumes
-.with_volumes(&["/host/path1:/container/path1", "/host/path2:/container/path2"])
-```
-
-### Environment Variables
-
-Set environment variables in the container:
-
-```rust
-// Set a single environment variable
-.with_env("KEY", "value")
-
-// Set multiple environment variables
-let mut env_map = HashMap::new();
-env_map.insert("KEY1", "value1");
-env_map.insert("KEY2", "value2");
-.with_envs(&env_map)
-```
-
-### Network Configuration
-
-Configure container networking:
-
-```rust
-// Set the network
-.with_network("bridge")
-
-// Add a network alias
-.with_network_alias("my-container")
-
-// Add multiple network aliases
-.with_network_aliases(&["alias1", "alias2"])
-```
-
-### Resource Limits
-
-Set CPU and memory limits:
-
-```rust
-// Set CPU limit (e.g., 0.5 for half a CPU, 2 for 2 CPUs)
-.with_cpu_limit("0.5")
-
-// Set memory limit (e.g., 512m for 512MB, 1g for 1GB)
-.with_memory_limit("512m")
-
-// Set memory swap limit
-.with_memory_swap_limit("1g")
-
-// Set CPU shares (relative weight)
-.with_cpu_shares("1024")
-```
-
-### Health Checks
-
-Configure container health checks:
-
-```rust
-// Simple health check
-.with_health_check("curl -f http://localhost/ || exit 1")
-
-// Health check with custom options
-.with_health_check_options(
-    "curl -f http://localhost/ || exit 1",  // Command
-    Some("30s"),                           // Interval
-    Some("10s"),                           // Timeout
-    Some(3),                               // Retries
-    Some("5s")                             // Start period
-)
-```
-
-### Other Options
-
-Other container configuration options:
-
-```rust
-// Set restart policy
-.with_restart_policy("always")  // Options: no, always, on-failure, unless-stopped
-
-// Set snapshotter
-.with_snapshotter("native")     // Options: native, fuse-overlayfs, etc.
-
-// Set detach mode
-.with_detach(true)              // Run in detached mode
-```
-
-## Container Operations
-
-Once a container is created, you can perform various operations on it:
-
-### Basic Operations
-
-```rust
-// Start the container
-container.start()?;
-
-// Stop the container
-container.stop()?;
-
-// Remove the container
-container.remove()?;
-```
-
-### Command Execution
-
-```rust
-// Execute a command in the container
-let result = container.exec("echo 'Hello from container'")?;
-println!("Command output: {}", result.stdout);
-```
-
-### File Operations
-
-```rust
-// Copy files between the container and host
-container.copy("container_name:/path/in/container", "/path/on/host")?;
-container.copy("/path/on/host", "container_name:/path/in/container")?;
-
-// Export the container to a tarball
-container.export("/path/to/export.tar")?;
-```
-
-### Image Operations
-
-```rust
-// Commit the container to an image
-container.commit("my-custom-image:latest")?;
-```
-
-### Status and Monitoring
-
-```rust
-// Get container status
-let status = container.status()?;
-println!("Container state: {}", status.state);
-println!("Container status: {}", status.status);
-println!("Created: {}", status.created);
-println!("Started: {}", status.started);
-
-// Get health status
-let health_status = container.health_status()?;
-println!("Health status: {}", health_status);
-
-// Get resource usage
-let resources = container.resources()?;
-println!("CPU usage: {}", resources.cpu_usage);
-println!("Memory usage: {}", resources.memory_usage);
-println!("Memory limit: {}", resources.memory_limit);
-println!("Memory percentage: {}", resources.memory_percentage);
-println!("Network I/O: {} / {}", resources.network_input, resources.network_output);
-println!("Block I/O: {} / {}", resources.block_input, resources.block_output);
-println!("PIDs: {}", resources.pids);
-```
-
-## Error Handling
-
-The Container API uses a custom error type `NerdctlError` that can be one of the following:
-
-- `CommandExecutionFailed`: The nerdctl command failed to execute
-- `CommandFailed`: The nerdctl command executed but returned an error
-- `JsonParseError`: Failed to parse JSON output
-- `ConversionError`: Failed to convert data
-- `Other`: Generic error
-
-Example error handling:
-
-```rust
-match Container::new("non-existent-container") {
-    Ok(container) => {
-        // Container exists
-        println!("Container found");
-    },
-    Err(e) => {
-        match e {
-            NerdctlError::CommandExecutionFailed(io_error) => {
-                println!("Failed to execute nerdctl command: {}", io_error);
-            },
-            NerdctlError::CommandFailed(error_msg) => {
-                println!("nerdctl command failed: {}", error_msg);
-            },
-            _ => {
-                println!("Other error: {}", e);
-            }
-        }
-    }
-}
-```
-
-## Implementation Details
-
-The Container API is implemented in several modules:
-
-- `container_types.rs`: Contains the struct definitions
-- `container.rs`: Contains the main Container implementation
-- `container_builder.rs`: Contains the builder pattern methods
-- `container_operations.rs`: Contains the container operations
-- `health_check.rs`: Contains the HealthCheck implementation
-
-This modular approach makes the code more maintainable and easier to understand.
-
-## Complete Example
-
-Here's a complete example that demonstrates the Container API:
-
-```rust
-use std::error::Error;
-use sal::virt::nerdctl::Container;
-
-fn main() -> Result<(), Box<dyn Error>> {
-    // Create a container from an image
-    println!("Creating container from image...");
-    let container = Container::from_image("my-nginx", "nginx:latest")?
+    let container_config = Container::new("my_nginx_container", "nginx:latest") // Assuming a constructor like this exists or is adapted
         .with_port("8080:80")
-        .with_env("NGINX_HOST", "example.com")
-        .with_volume("/tmp/nginx:/usr/share/nginx/html")
-        .with_health_check("curl -f http://localhost/ || exit 1")
+        .with_envs(&envs)
         .with_detach(true)
-        .build()?;
-    
-    println!("Container created successfully");
-    
-    // Execute a command in the container
-    println!("Executing command in container...");
-    let result = container.exec("echo 'Hello from container'")?;
-    println!("Command output: {}", result.stdout);
-    
-    // Get container status
-    println!("Getting container status...");
-    let status = container.status()?;
-    println!("Container status: {}", status.status);
-    
-    // Get resource usage
-    println!("Getting resource usage...");
-    let resources = container.resources()?;
-    println!("CPU usage: {}", resources.cpu_usage);
-    println!("Memory usage: {}", resources.memory_usage);
-    
-    // Stop and remove the container
-    println!("Stopping and removing container...");
-    container.stop()?;
-    container.remove()?;
-    
-    println!("Container stopped and removed");
-    
+        .with_restart_policy("always");
+
+    // Build (create and run) the container
+    let built_container = container_config.build()?;
+    println!("Container {} created with ID: {:?}", built_container.name, built_container.container_id);
+
+    // Perform operations
+    let status = built_container.status()?;
+    println!("Status: {}, State: {}", status.status, status.state);
+
+    // Stop and remove
+    built_container.stop()?;
+    built_container.remove()?;
+    println!("Container stopped and removed.");
+
     Ok(())
-}
+}
+```
+
+*Note: The direct `Container::new(name, image)` constructor isn't explicitly shown in the provided Rust code snippets for `Container` itself, but the Rhai bindings `nerdctl_container_new` and `nerdctl_container_from_image` imply underlying Rust constructors that initialize a `Container` struct. The `build()` method is the primary way to run it after configuration.*
+
+### Rhai Script Example (using `herodo`)
+
+```rhai
+// Create and configure a container using the builder pattern
+let c = nerdctl_container_from_image("my_redis", "redis:alpine")
+    .with_port("6379:6379")
+    .with_restart_policy("unless-stopped");
+
+// Build and run the container
+let running_container = c.build();
+
+if running_container.is_ok() {
+    print(`Container ${running_container.name} ID: ${running_container.container_id}`);
+    
+    // Get status
+    let status = running_container.status();
+    if status.is_ok() {
+        print(`Status: ${status.state}, Health: ${status.health_status}`);
+    }
+    
+    // Stop the container (example, might need a mutable borrow or re-fetch)
+    // running_container.stop(); // Assuming stop is available and works on the result
+    // running_container.remove();
+} else {
+    print(`Error building container: ${running_container.error()}`);
+}
+
+// Direct command example
+let images = nerdctl_images();
+print(images.stdout);
+
+nerdctl_image_pull("alpine:latest");
+```
+
+## Key Design Points
+
+- **Fluent Builder**: The `Container` struct uses a builder pattern, allowing for clear and chainable configuration of container parameters before execution.
+- **Comprehensive Operations**: Covers most common `nerdctl` functionalities for containers and images.
+- **Error Handling**: `NerdctlError` provides typed errors. The Rhai layer adds more descriptive error messages for common scenarios.
+- **Dual API**: Offers both a detailed builder pattern and simpler static functions for flexibility.
+- **Health Check Scripting**: Supports multi-line shell scripts for health checks by saving them to a file, though care must be taken regarding the script's accessibility from within the target container.
+- **Resource Parsing**: Includes parsing for `nerdctl inspect` (JSON) and `nerdctl stats` (tabular text) to provide structured information.
+
+## File Structure
+
+- `src/virt/nerdctl/mod.rs`: Main module file, error definitions, sub-module declarations.
+- `src/virt/nerdctl/cmd.rs`: Core `execute_nerdctl_command` function.
+- `src/virt/nerdctl/container_types.rs`: Definitions for `Container`, `HealthCheck`, `ContainerStatus`, `ResourceUsage`.
+- `src/virt/nerdctl/container_builder.rs`: Implements the builder pattern methods for the `Container` struct.
+- `src/virt/nerdctl/container_operations.rs`: Implements instance methods on `Container` (start, stop, status, etc.).
+- `src/virt/nerdctl/images.rs`: `Image` struct and static functions for image management.
+- `src/virt/nerdctl/container_functions.rs`: Static functions for direct container commands.
+- `src/virt/nerdctl/health_check_script.rs`: Logic for `prepare_health_check_command`.
+- `src/rhai/nerdctl.rs`: Rhai script bindings for `herodo`.

src/virt/rfs/README.md

@@ -0,0 +1,165 @@
+# SAL RFS (Remote File System) Module (`sal::virt::rfs`)
+
+## Overview
+
+The `sal::virt::rfs` module provides a Rust interface for interacting with an underlying `rfs` command-line tool. This tool facilitates mounting various types of remote and local filesystems and managing packed filesystem layers.
+
+The module allows Rust applications and `herodo` Rhai scripts to:
+- Mount and unmount filesystems from different sources (e.g., local paths, SSH, S3, WebDAV).
+- List currently mounted filesystems and retrieve information about specific mounts.
+- Pack directories into filesystem layers, potentially using specified storage backends.
+- Unpack, list contents of, and verify these filesystem layers.
+
+All operations are performed by invoking the `rfs` CLI tool and parsing its output.
+
+## Key Design Points
+
+- **CLI Wrapper**: This module acts as a wrapper around an external `rfs` command-line utility. The actual filesystem operations and layer management are delegated to this tool.
+- **Asynchronous Operations (Implicit)**: While the Rust functions themselves might be synchronous, the underlying `execute_rfs_command` (presumably from `super::cmd`) likely handles command execution, which could be asynchronous or blocking depending on its implementation.
+- **Filesystem Abstraction**: Supports mounting diverse filesystem types such as `local`, `ssh`, `s3`, and `webdav` through the `rfs` tool's capabilities.
+- **Layer Management**: Provides functionalities to `pack` directories into portable layers, `unpack` them, `list_contents`, and `verify` their integrity. This is useful for creating and managing reproducible filesystem snapshots or components.
+- **Store Specifications (`StoreSpec`)**: The packing functionality allows specifying `StoreSpec` types, suggesting that packed layers can be stored or referenced using different backend mechanisms (e.g., local files, S3 buckets). This enables flexible storage and retrieval of filesystem layers.
+- **Builder Pattern**: Uses `RfsBuilder` for constructing mount commands with various options and `PackBuilder` for packing operations, providing a fluent interface for complex configurations.
+- **Rhai Scriptability**: Most functionalities are exposed to Rhai scripts via `herodo` through the `sal::rhai::rfs` bridge, enabling automation of filesystem and layer management tasks.
+- **Structured Error Handling**: Defines `RfsError` for specific error conditions encountered during `rfs` command execution or output parsing.
+
+## Rhai Scripting with `herodo`
+
+The `sal::virt::rfs` module is scriptable via `herodo`. The following functions are available in Rhai, prefixed with `rfs_`:
+
+### Mount Operations
+
+- `rfs_mount(source: String, target: String, mount_type: String, options: Map) -> Map`
+  - Mounts a filesystem.
+  - `source`: The source path or URL (e.g., `/path/to/local_dir`, `ssh://user@host:/remote/path`, `s3://bucket/key`).
+  - `target`: The local path where the filesystem will be mounted.
+  - `mount_type`: A string specifying the type of filesystem (e.g., "local", "ssh", "s3", "webdav").
+  - `options`: A Rhai map of additional mount options (e.g., `#{ "read_only": true, "uid": 1000 }`).
+  - Returns a map containing details of the mount (id, source, target, fs_type, options) on success.
+
+- `rfs_unmount(target: String) -> ()`
+  - Unmounts the filesystem at the specified target path.
+
+- `rfs_list_mounts() -> Array`
+  - Lists all currently mounted filesystems managed by `rfs`.
+  - Returns an array of maps, each representing a mount with its details.
+
+- `rfs_unmount_all() -> ()`
+  - Unmounts all filesystems currently managed by `rfs`.
+
+- `rfs_get_mount_info(target: String) -> Map`
+  - Retrieves information about a specific mounted filesystem.
+  - Returns a map with mount details if found.
+
+### Pack/Layer Operations
+
+- `rfs_pack(directory: String, output: String, store_specs: String) -> ()`
+  - Packs the contents of a `directory` into an `output` file (layer).
+  - `store_specs`: A comma-separated string defining storage specifications for the layer (e.g., `"file:path=/path/to/local_store,s3:bucket=my-archive,region=us-west-1"`). Each spec is `type:key=value,key2=value2`.
+
+- `rfs_unpack(input: String, directory: String) -> ()`
+  - Unpacks an `input` layer file into the specified `directory`.
+
+- `rfs_list_contents(input: String) -> String`
+  - Lists the contents of an `input` layer file.
+  - Returns a string containing the file listing (raw output from the `rfs` tool).
+
+- `rfs_verify(input: String) -> bool`
+  - Verifies the integrity of an `input` layer file.
+  - Returns `true` if the layer is valid, `false` otherwise.
+
+### Rhai Example
+
+```rhai
+// Example: Mounting a local directory (ensure /mnt/my_local_mount exists)
+let source_dir = "/tmp/my_data_source"; // Create this directory first
+let target_mount = "/mnt/my_local_mount";
+
+// Create source_dir if it doesn't exist for the example to run
+// In a real script, you might use sal::os::dir_create or ensure it exists.
+// For this example, assume it's manually created or use: os_run_command(`mkdir -p ${source_dir}`);
+
+print(`Mounting ${source_dir} to ${target_mount}...`);
+let mount_result = rfs_mount(source_dir, target_mount, "local", #{});
+if mount_result.is_ok() {
+    print(`Mount successful: ${mount_result}`);
+} else {
+    print(`Mount failed: ${mount_result}`);
+}
+
+// List mounts
+print("\nCurrent mounts:");
+let mounts = rfs_list_mounts();
+if mounts.is_ok() {
+    for m in mounts {
+        print(`  Target: ${m.target}, Source: ${m.source}, Type: ${m.fs_type}`);
+    }
+} else {
+    print(`Error listing mounts: ${mounts}`);
+}
+
+// Example: Packing a directory
+let dir_to_pack = "/tmp/pack_this_dir"; // Create and populate this directory
+let packed_file = "/tmp/my_layer.pack";
+// os_run_command(`mkdir -p ${dir_to_pack}`); 
+// os_run_command(`echo 'hello' > ${dir_to_pack}/file1.txt`);
+
+print(`\nPacking ${dir_to_pack} to ${packed_file}...`);
+// Using a file-based store spec for simplicity
+let pack_store_specs = "file:path=/tmp/rfs_store"; 
+// os_run_command(`mkdir -p /tmp/rfs_store`);
+
+let pack_result = rfs_pack(dir_to_pack, packed_file, pack_store_specs);
+if pack_result.is_ok() {
+    print("Packing successful.");
+
+    // List contents of the packed file
+    print(`\nContents of ${packed_file}:`);
+    let contents = rfs_list_contents(packed_file);
+    if contents.is_ok() {
+        print(contents);
+    } else {
+        print(`Error listing contents: ${contents}`);
+    }
+
+    // Verify the packed file
+    print(`\nVerifying ${packed_file}...`);
+    let verify_result = rfs_verify(packed_file);
+    if verify_result.is_ok() && verify_result {
+        print("Verification successful: Layer is valid.");
+    } else {
+        print(`Verification failed or error: ${verify_result}`);
+    }
+
+    // Example: Unpacking
+    let unpack_dir = "/tmp/unpacked_layer_here";
+    // os_run_command(`mkdir -p ${unpack_dir}`);
+    print(`\nUnpacking ${packed_file} to ${unpack_dir}...`);
+    let unpack_result = rfs_unpack(packed_file, unpack_dir);
+    if unpack_result.is_ok() {
+        print("Unpacking successful.");
+        // You would typically check contents of unpack_dir here
+        // os_run_command(`ls -la ${unpack_dir}`);
+    } else {
+        print(`Error unpacking: ${unpack_result}`);
+    }
+
+} else {
+    print(`Error packing: ${pack_result}`);
+}
+
+// Cleanup: Unmount the local mount
+if mount_result.is_ok() {
+    print(`\nUnmounting ${target_mount}...`);
+    rfs_unmount(target_mount);
+}
+
+// To run this example, ensure the 'rfs' command-line tool is installed and configured,
+// and that the necessary directories (/tmp/my_data_source, /mnt/my_local_mount, etc.)
+// exist and have correct permissions.
+// You might need to run herodo with sudo for mount/unmount operations.
+
+print("\nRFS Rhai script finished.");
+```
+
+This module provides a flexible way to manage diverse filesystems and filesystem layers, making it a powerful tool for system automation and deployment tasks within the SAL ecosystem.

src/zinit_client/README.md

@@ -0,0 +1,163 @@
+# SAL Zinit Client Module (`sal::zinit_client`)
+
+## Overview
+
+The `sal::zinit_client` module provides a Rust interface for interacting with a [Zinit](https://github.com/systeminit/zinit) process supervisor daemon. Zinit is a process and service manager for Linux systems, designed for simplicity and robustness.
+
+This SAL module allows Rust applications and `herodo` Rhai scripts to:
+- List and manage Zinit services (get status, start, stop, restart, monitor, forget, kill).
+- Define and manage service configurations (create, delete, get).
+- Retrieve logs from Zinit.
+
+The client communicates with the Zinit daemon over a Unix domain socket. All operations are performed asynchronously.
+
+## Key Design Points
+
+- **Async Operations**: Leverages `tokio` for asynchronous communication with the Zinit daemon, ensuring non-blocking calls suitable for concurrent applications.
+- **Unix Socket Communication**: Connects to the Zinit daemon via a specified Unix domain socket path (e.g., `/var/run/zinit.sock`).
+- **Global Client Instance**: Manages a global, lazily-initialized `Arc<ZinitClientWrapper>` to reuse the Zinit client connection across multiple calls within the same process, improving efficiency.
+- **Comprehensive Service Management**: Exposes a wide range of Zinit's service management capabilities, from basic lifecycle control to service definition and log retrieval.
+- **Rhai Scriptability**: A significant portion of the Zinit client's functionality is exposed to Rhai scripts via `herodo` through the `sal::rhai::zinit` bridge, enabling automation of service management tasks.
+- **Error Handling**: Converts errors from the underlying `zinit_client` crate into `zinit_client::ClientError`, which are then translated to `EvalAltResult` for Rhai, providing clear feedback.
+- **Simplified Rhai Interface**: For some operations like service creation, the Rhai interface offers a simplified parameter set compared to the direct Rust API for ease of use in scripts.
+
+## Rhai Scripting with `herodo`
+
+The `sal::zinit_client` module is scriptable via `herodo`. The following functions are available in Rhai, prefixed with `zinit_`. All functions require `socket_path` (String) as their first argument, specifying the path to the Zinit Unix domain socket.
+
+- `zinit_list(socket_path: String) -> Map`
+  - Lists all services managed by Zinit and their states.
+  - Returns a map where keys are service names and values are their current states (e.g., "Running", "Stopped").
+
+- `zinit_status(socket_path: String, name: String) -> Map`
+  - Retrieves the detailed status of a specific service.
+  - `name`: The name of the service.
+  - Returns a map containing status details like PID, state, target state, and dependencies.
+
+- `zinit_start(socket_path: String, name: String) -> bool`
+  - Starts the specified service.
+  - Returns `true` on success.
+
+- `zinit_stop(socket_path: String, name: String) -> bool`
+  - Stops the specified service.
+  - Returns `true` on success.
+
+- `zinit_restart(socket_path: String, name: String) -> bool`
+  - Restarts the specified service.
+  - Returns `true` on success.
+
+- `zinit_monitor(socket_path: String, name: String) -> bool`
+  - Enables monitoring for the specified service (Zinit will attempt to keep it running).
+  - Returns `true` on success.
+
+- `zinit_forget(socket_path: String, name: String) -> bool`
+  - Disables monitoring for the specified service (Zinit will no longer attempt to restart it if it stops).
+  - Returns `true` on success.
+
+- `zinit_kill(socket_path: String, name: String, signal: String) -> bool`
+  - Sends a specific signal (e.g., "TERM", "KILL", "HUP") to the specified service.
+  - Returns `true` on success.
+
+- `zinit_create_service(socket_path: String, name: String, exec: String, oneshot: bool) -> String`
+  - Creates a new service configuration in Zinit.
+  - `name`: The name for the new service.
+  - `exec`: The command to execute for the service.
+  - `oneshot`: A boolean indicating if the service is a one-shot task (true) or a long-running process (false).
+  - Returns a confirmation message or an error.
+
+- `zinit_delete_service(socket_path: String, name: String) -> String`
+  - Deletes the specified service configuration from Zinit.
+  - Returns a confirmation message or an error.
+
+- `zinit_get_service(socket_path: String, name: String) -> Dynamic`
+  - Retrieves the configuration of the specified service as a dynamic map.
+
+- `zinit_logs(socket_path: String, filter: String) -> Array`
+  - Retrieves logs for a specific service or component matching the filter.
+  - `filter`: The name of the service/component to get logs for.
+  - Returns an array of log lines.
+
+- `zinit_logs_all(socket_path: String) -> Array`
+  - Retrieves all available logs from Zinit.
+  - Returns an array of log lines.
+
+### Rhai Example
+
+```rhai
+// Default Zinit socket path
+let zinit_socket = "/var/run/zinit.sock";
+
+// Ensure Zinit is running and socket exists before running this script.
+
+// List all services
+print("Listing Zinit services...");
+let services = zinit_list(zinit_socket);
+if services.is_ok() {
+    print(`Services: ${services}`);
+} else {
+    print(`Error listing services: ${services}`);
+    // exit(); // Or handle error appropriately
+}
+
+// Define a test service
+let service_name = "my_test_app";
+let service_exec = "/usr/bin/sleep 300"; // Example command
+
+// Try to get service info first, to see if it exists
+let existing_service = zinit_get_service(zinit_socket, service_name);
+if !existing_service.is_ok() { // Assuming error means it doesn't exist or can't be fetched
+    print(`\nService '${service_name}' not found or error. Attempting to create...`);
+    let create_result = zinit_create_service(zinit_socket, service_name, service_exec, false);
+    if create_result.is_ok() {
+        print(`Service '${service_name}' created successfully.`);
+    } else {
+        print(`Error creating service '${service_name}': ${create_result}`);
+        // exit();
+    }
+} else {
+    print(`\nService '${service_name}' already exists: ${existing_service}`);
+}
+
+// Get status of the service
+print(`\nFetching status for '${service_name}'...`);
+let status = zinit_status(zinit_socket, service_name);
+if status.is_ok() {
+    print(`Status for '${service_name}': ${status}`);
+    // Example: Start if not running (simplified check)
+    if status.state != "Running" && status.state != "Starting" {
+        print(`Attempting to start '${service_name}'...`);
+        zinit_start(zinit_socket, service_name);
+    }
+} else {
+    print(`Error fetching status for '${service_name}': ${status}`);
+}
+
+// Get some logs for the service (if it produced any)
+// Note: Logs might be empty if service just started or hasn't output anything.
+print(`\nFetching logs for '${service_name}'...`);
+let logs = zinit_logs(zinit_socket, service_name);
+if logs.is_ok() {
+    if logs.len() > 0 {
+        print(`Logs for '${service_name}':`);
+        for log_line in logs {
+            print(`  ${log_line}`);
+        }
+    } else {
+        print(`No logs found for '${service_name}'.`);
+    }
+} else {
+    print(`Error fetching logs for '${service_name}': ${logs}`);
+}
+
+// Example: Stop and delete the service (cleanup)
+// print(`\nStopping service '${service_name}'...`);
+// zinit_stop(zinit_socket, service_name);
+// print(`Forgetting service '${service_name}'...`);
+// zinit_forget(zinit_socket, service_name); // Stop monitoring before delete
+// print(`Deleting service '${service_name}'...`);
+// zinit_delete_service(zinit_socket, service_name);
+
+print("\nZinit Rhai script finished.");
+```
+
+This module provides a powerful way to automate service management and interaction with Zinit-supervised systems directly from Rust or `herodo` scripts.