sal-modular/docs/Architecture.md

Architecture and Implementation Plan for the Rust Modular System

The system is organized into three core Rust crates (kvstore, vault, evm_client) plus two front‐end targets (a CLI and a WASM web app). The kvstore crate defines an async KVStore trait and provides two implementations: on native platforms it uses sled, while in WASM/browser it uses IndexedDB via the idb crate (selected by Cargo feature flags or #[cfg(target_arch = "wasm32")]). For example, Wire’s core-crypto keystore uses IndexedDB with AES-GCM for WASM and SQLCipher on native platforms. The vault crate manages an encrypted keyspace of multiple keypairs (password-protected), performing cryptographic operations (sign/verify, sym/asym encryption) and persisting data through kvstore. The evm_client crate handles EVM RPC calls (using alloy), depending on vault to sign transactions with stored keys. A Rust CLI binary ties these together with a Rhai scripting engine: Rhai scripts invoke async APIs via a message-passing pattern. The browser target compiles to Wasm (with wasm-bindgen); it exposes the same APIs to JavaScript or to Rhai compiled for Wasm.

Crate and Module Structure

• Cargo workspace: top-level Cargo.toml lists members kvstore/, vault/, evm_client/, cli_app/, web_app/. Common dev-dependencies and CI config are shared at the workspace root.

• Features & cfg: In kvstore, define Cargo features or use #[cfg] to toggle backends. E.g. cfg(not(target_arch = "wasm32")) for sled, and cfg(target_arch = "wasm32") for IndexedDB. Use async_trait for the KVStore trait so implementations can be async. Similar conditional compilation applies to any platform-specific code (e.g. using WebCrypto APIs only under WASM).

• Dependencies:

  • kvstore depends on sled (native) and idb (WASM), and defines async fn methods. Blocking DB calls (sled) must be offloaded via a spawn_blocking provided by the caller.
  • vault depends on kvstore and various crypto crates (e.g. aes-gcm or chacha20poly1305 for symmetric encryption; k256/rust-crypto for signatures). For WASM compatibility, ensure chosen crypto crates support wasm32-unknown-unknown. Keys are encrypted at rest with a password-derived key (AES-256-GCM or similar).
  • evm_client depends on vault (for signing) and an Ethereum library (e.g. alloy with an async HTTP provider). On WASM, use wasm-bindgen-futures to call JavaScript fetch or use a crate like reqwest with the wasm feature.
  • The CLI (binary) depends on Rhai (rhai crate), tokio or similar for async execution, and the above libraries. It sets up an async runtime (e.g. Tokio) to run tasks.
  • The web_app (WASM target) depends on wasm-bindgen/wasm-bindgen-futures and vault/evm_client. It uses wasm-bindgen to expose Rust functions to JS. Rhai can also be compiled to WASM for scripting in-browser, but must be integrated via the same message-passing pattern (see below).

kvstore Crate Design

The kvstore crate defines:

#[async_trait]
pub trait KVStore {
    async fn get(&self, key: &str) -> Option<Vec<u8>>;
    async fn put(&self, key: &str, value: &[u8]) -> ();
    async fn delete(&self, key: &str) -> ();
    // ...
}

It then provides two modules implementing this trait:

• Native backend (sled): A wrapper around sled::Db. Since sled I/O is blocking, each call should be executed in a blocking context (e.g. using tokio::task::spawn_blocking) so as not to block the async runtime.
• WASM/browser backend (IndexedDB): Uses the idb crate (or web-sys/gloo) to store data in the browser’s IndexedDB. This implementation is inherently async (Promise-based) and works in wasm32-unknown-unknown. On compilation, one can use Cargo features like default-features = false and features = ["native", "wasm"], or simply #[cfg] to select the correct backend.

Citing best practice: the pattern of having an encrypted keystore use IndexedDB on WASM is standard (e.g. Wire’s core-crypto keystore). We will mirror that by encrypting data before put-ting it. The kvstore implementation will automatically be runtime-agnostic (using only std::future::Future in its APIs).

vault Crate Design

The vault crate implements a WebAssembly-compatible cryptographic keystore. It manages:

• Encrypted keyspace: A password protects all key material. On open, derive an encryption key (e.g. via scrypt or PBKDF2) and decrypt the stored vault (a blob in kvstore). Inside, multiple keypairs (e.g. Ethereum secp256k1 keys, Ed25519 keys, etc.) are stored.
• Crypto APIs: Expose async functions to create new keys, list keys, and to perform crypto operations: e.g. async fn sign_transaction(&self, key_id: &str, tx: &Transaction) -> Signature, async fn verify(&self, ...) -> bool, async fn encrypt(&self, ...)->Ciphertext, etc.
• Storage: Internally uses the kvstore::KVStore trait to persist the encrypted vault. For example, on each change, it re-encrypts the whole keyspace and puts it under a fixed ID key.
• WASM Compatibility: All operations must compile to Wasm. Use Rust crypto crates compatible with no_std/WASM (e.g. aes-gcm, k256, rand_core with getrandom support). Alternatively, one could use the browser’s WebCrypto via wasm-bindgen for symmetric operations, but for simplicity we can rely on Rust crates (AES-GCM implementations that compile to WASM).

Internally, vault ensures all operations return Futures. It will not assume any particular async runtime – for example, file I/O or crypto is fast in memory, but if any blocking work is needed (like PBKDF2 hashing), it should be done via a provided spawn_blocking (as recommended by forum answers). On WASM, such heavy work would yield to the JS event loop via spawn_local (see below).

When open, vault authenticates the user’s password, loads (via kvstore) the encrypted blob of keys, and allows operations. Fig. above illustrates a cryptographic network: keys stored securely (vault) are used for signing on behalf of the user. Internally, best practice is to use an authenticated cipher (e.g. AES-256-GCM) with a strong KDF, as noted in existing systems.

evm_client Crate Design

The evm_client crate provides async interfaces to interact with an EVM blockchain:

• Dependencies: It uses the alloy crate for building transactions, ABI encoding, and an async HTTP provider for RPC calls.
• Signing: It calls into vault when a transaction must be signed. For example, evm_client.sign_and_send(tx) will invoke vault.sign(key_id, tx_bytes) to get a signature.
• Async RPC: All RPC calls (eth_sendRawTransaction, eth_call, etc.) are async fns returning Futures. These futures must be runtime-agnostic: they use standard async/await and do not tie to Tokio specifically. For HTTP, on native targets use reqwest with Tokio, while on WASM use reqwest with its wasm feature or gloo-net with wasm-bindgen-futures.
• Configuration: Provide a flexible config (e.g. chain ID, gas price options) via plain structs. Errors should use a common error enum or thiserror crate.
• Features: Could have a feature flag to choose between alloy and ethers. Both are fully async.

The evm_client crate itself should be purely async and not block. It will typically run on Tokio in the CLI, and on the browser’s single-threaded event loop with spawn_local in the web app.

CLI Binary (Rhai Scripting)

The CLI binary (cli_app) binds everything with a user interface. Its design:

• Command loop: On startup it spawns a Rhai Engine in a separate OS thread. The main thread runs a Tokio async runtime (or other) to handle network and I/O.
• Message-passing: Use two MPSC channels: one for messages to the engine, and one for replies from the engine. According to Rhai’s multi-threaded pattern, we register API functions in the engine that send commands via channel to the main thread. The main thread processes commands (e.g. “sign this tx”, “send transaction”, etc.) using vault/evm_client, then sends back results.
• Blocking calls: In Rhai, all calls are blocking from the script’s perspective. Under the hood, the registered API calls serialize the request (e.g. to JSON) and send it on the command channel. The Rhai engine will block until a reply arrives on the reply channel. This pattern ensures the script can call async Rust code seamlessly (step 6–8 in Rhai docs).
• Example flow: A Rhai script calls let res = send_tx(data). The send_tx function (registered in the engine) captures the channel handles, packages data into a message, and sends it. The engine thread blocks. The main thread’s async runtime reads the message, calls evm_client.send_transaction(data).await, then sends the result back. The Rhai engine thread receives it and returns it to the script.

This design follows Rhai’s recommended “blocking/async” pattern. It keeps the library usage runtime-agnostic, while allowing user-defined scripts to trigger asynchronous operations.

Browser Application (WASM)

The browser target (web_app) is compiled with wasm-bindgen to Wasm. It provides the same core functionality via a JS API (or Rhai in WASM). Key points:

• Exports: Use #[wasm_bindgen] to expose async functions to JavaScript. For example, expose async fn create_key(name: String) -> JsValue that returns a JavaScript Promise. The wasm-bindgen-futures crate will convert Rust Futures into JS Promises automatically.
• Async runtime: WebAssembly runs on the browser’s single thread. To perform async Rust code, we use wasm_bindgen_futures::spawn_local to drive futures on the JS event loop. For example, in an exported function we might do spawn_local(async move { /* call vault, evm_client */ }). According to docs, spawn_local “runs a Rust Future on the current thread” and schedules it as a microtask. This lets our async functions execute to completion without blocking the event loop.
• Promises and interop: Return types must be JsValue or types convertible by wasm_bindgen. Complex data (e.g. byte arrays) can be passed as Uint8Array or encoded (e.g. hex).
• Rhai in WASM: Optionally, we can compile Rhai to WebAssembly as well. In that case, we would run the Rhai engine in a WebWorker (since WASM threads are limited) and use MessageChannel for communication. The same message-passing pattern applies: a script call in the worker posts a message to the main thread with request data, and awaits a message back. The main thread (browser UI) handles the request using the exposed Rust APIs. This is analogous to the CLI pattern but using Web APIs. (Implementation note: enabling threading in WASM requires wasm-bindgen with the --target bundler or using web-sys Worker APIs.)
• Integration tips: Use the wasm-bindgen guide to share data types (strings, structs) between JS and Rust. For async tests, wasm-bindgen-futures has examples.

In summary, the web app compiles the same crates to Wasm and exposes them. The figure above (a network on a globe) conceptually represents the global connectivity: the browser connects to EVM nodes via WebAssembly modules, invoking Rust code. All async boundaries are handled with spawn_local and JS Promises (as wasm-bindgen-futures outlines).

Async and Runtime-Agnostic Best Practices

Throughout all crates we adhere to runtime-agnostic async principles:

• Use std::future::Future in public APIs, not a specific runtime’s types. Internally, any async work (I/O, network) should be done with async/await.
• Feature-gate runtime-specific code: If we need to call tokio::spawn or async-std, isolate that behind #[cfg(feature = "tokio")] or similar. Initially, one can pick one runtime (e.g. Tokio) and make the library depend on it, then add cfg-features later.
• Blocking calls: Any blocking work (file I/O, heavy crypto) is executed via a passed-in executor (e.g. require a spawn_blocking: Fn(Box<dyn FnOnce() + Send>) callback), as recommended by Rust forum advice. This way the library never forces a specific thread pool. For example, in kvstore’s sled backend, all operations are done in spawn_blocking.
• Testing: Include tests for both native and WASM targets (using wasm-pack test or headless browser tests) to catch platform differences.
• Error handling: Use Result types, with a shared error enum. Avoid panic paths – return errors across FFI boundaries.

By decoupling logic from the runtime (using channels for Rhai, spawn_local for WASM, cfg-features for backends), the libraries remain flexible. As one Rust discussion notes, “using cfg(feature = "...") to isolate the pieces that have to be runtime specific” is key. We ensure all public async APIs are async fn so they can be awaited in any context.

Workspace Layout and Features

The recommended workspace layout is:

/Cargo.toml           # workspace manifest
/kvstore/Cargo.toml   # kvstore crate
/vault/Cargo.toml     # vault crate
/evm_client/Cargo.toml
/cli_app/Cargo.toml   # binary (depends on kvstore, vault, evm_client, rhai)
/web_app/Cargo.toml   # cdylib (wasm) crate (depends on kvstore, vault, evm_client, wasm-bindgen)

Each crate’s Cargo.toml lists its dependencies. For kvstore, an example feature setup:

[features]
default = ["native"]
native = ["sled"]
web = ["idb"]

In code:

#[cfg(feature = "native")]
mod sled_backend;
#[cfg(feature = "web")]
mod indexeddb_backend;

One could also omit features and just use #[cfg(target_arch = "wasm32")] for the web backend. The wasm-bindgen crate is included under the web_app for browser integration.

Conclusion

This plan lays out a clear, modular architecture. Diagrams (above) conceptually show how the crates interact: kvstore underlies vault, which together support evm_client; the CLI and WASM targets invoke them asynchronously. We use message-passing (channels) to bridge Rhai scripts with async Rust code, and spawn_local in the browser to schedule futures. By following Rust async best practices (runtime-agnostic Futures, careful use of cfg and spawn-blocking) and wasm-bindgen conventions, the system will work seamlessly both on the desktop/CLI and in the browser.

Sources: Concepts and patterns are drawn from Rust async and WASM guidelines. For example, using IndexedDB with AES-GCM in WASM keystores is inspired by existing systems. These sources guided the design of a flexible, secure architecture.

🔐 kvstore Crate: Pluggable Key-Value Storage Layer

Purpose: Provide an abstraction for key-value storage with async-compatible traits, supporting both native and WASM environments.

Public API

#[async_trait]
pub trait KVStore {
    async fn get(&self, key: &str) -> Result<Option<Vec<u8>>, KVError>;
    async fn set(&self, key: &str, value: &[u8]) -> Result<(), KVError>;
    async fn delete(&self, key: &str) -> Result<(), KVError>;
    async fn exists(&self, key: &str) -> Result<bool, KVError>;
}

Backends:

• Native: sled
• WASM: idb (IndexedDB)

Features:

• Compile-time target detection via #[cfg(target_arch = "wasm32")]
• Enables usage in both CLI and browser environments

---

🛡️ vault Crate: Core Cryptography Module

Purpose: Manage secure key storage, cryptographic operations, and password-protected keyspaces.

Public API

pub struct HeroVault;

impl HeroVault {
    pub async fn create_keyspace(name: &str, password: &str) -> Result<(), VaultError>;
    pub async fn load_keyspace(name: &str, password: &str) -> Result<(), VaultError>;
    pub async fn logout() -> Result<(), VaultError>;

    pub async fn create_keypair(label: &str) -> Result<(), VaultError>;
    pub async fn select_keypair(label: &str) -> Result<(), VaultError>;
    pub async fn list_keypairs() -> Result<Vec<String>, VaultError>;
    pub async fn get_public_key(label: &str) -> Result<Vec<u8>, VaultError>;

    pub async fn sign_message(message: &[u8]) -> Result<Vec<u8>, VaultError>;
    pub async fn verify_signature(message: &[u8], signature: &[u8], public_key: &[u8]) -> Result<bool, VaultError>;

    pub async fn encrypt(data: &[u8], password: &str) -> Result<Vec<u8>, VaultError>;
    pub async fn decrypt(data: &[u8], password: &str) -> Result<Vec<u8>, VaultError>;
}

Security:

• All sensitive data encrypted at rest using AES-GCM or ChaCha20-Poly1305
• Passwords stretched via Argon2id or PBKDF2

---

⚙️ evm_client Crate: EVM Integration Layer

Purpose: Interact with Ethereum-compatible chains using key material from vault.

Public API

pub struct EvmClient;

impl EvmClient {
    pub async fn connect(rpc_url: &str) -> Result<Self, EvmError>;
    pub async fn get_balance(&self, address: &str) -> Result<U256, EvmError>;
    pub async fn send_transaction(&self, to: &str, value: U256, data: &[u8]) -> Result<TxHash, EvmError>;
    pub async fn call_contract(&self, to: &str, data: &[u8]) -> Result<Vec<u8>, EvmError>;
}

Options:

• ethers-rs (default, mature)
• alloy (alternative, lightweight and WASM-friendly)(Stack Overflow)

Usage:

• Transaction signing using vault keys
• Account management and EIP-1559 support
• Modular pluggability to support multiple networks(Medium)

---

🧰 CLI Interface

Purpose: Provide a command-line interface for interacting with the vault and evm_client crates, with scripting capabilities via Rhai.

Features

• Built with rhai scripting engine for dynamic workflows
• Thin wrapper over vault and evm_client
• Exposes custom functions to Rhai:

fn sign_tx(...) -> Result<String, Box<EvalAltResult>>;
fn create_keyspace(...) -> ...;

• Asynchronous operations managed via tokio or async-std

---

🌐 WebAssembly (Browser) Target

Purpose: Provide a browser-compatible interface for the vault and evm_client crates, compiled to WebAssembly.

Features

• Exposed using wasm-bindgen

• No Rhai scripting in browser due to native-only dependencies

• Interaction model:

  • Expose WebAssembly bindings (async Promise-compatible)
  • Front-end (e.g., React) calls functions via JS bridge
  • Keyspace and signing operations run within WASM memory

---

🧠 Rhai Integration Strategy

• Only used in CLI
• Bind only synchronous APIs
• Asynchronous work handled by sending commands to a background task(Deno)

rhai.register_fn("sign", move |input: String| -> String {
    let (tx, rx) = oneshot::channel();
    command_sender.send(VaultCommand::SignMessage { input, resp: tx });
    rx.blocking_recv().unwrap()
});

---

🔧 Runtime Strategy

• Library (vault, kvstore, evm_client):

  • Must be async-runtime agnostic
  • No global runtime should be spawned
  • Use async-trait, Send + 'static futures

• CLI & Web Targets:

  • CLI: Use tokio or async-std
  • WASM: Use wasm-bindgen-futures and spawn_local

---

📐 Architecture Diagram

[ CLI (Rhai) ]          [ Browser (WASM) ]
       |                        |
    [ Scripts ]            [ JS / TS ]
       |                        |
     [ Runtime ]         [ wasm-bindgen ]
       |                        |
   [ vault (async) ]  [ vault (wasm32) ]
       |                        |
   [ kvstore (sled) ]       [ kvstore (idb) ]

---

📦 Dependency Overview

Crate	Key Deps	WASM Support
kvstore	sled, idb	✅
hero_vault	aes-gcm, argon2, rand	✅
evm_client	alloy	✅
CLI	rhai, tokio	❌
Web Target	wasm-bindgen, idb	✅

---

📝 Implementation Plan

1. Scaffold Crates:

   • kvstore
   • vault
   • evm_client

2. Implement KVStore Trait:

   • Implement sled backend for native
   • Implement idb backend for WASM

3. Develop vault:

   • Implement password-based encrypted keyspaces
   • Integrate with kvstore for persistence
   • Implement cryptographic operations (signing, encryption, etc.)(GitHub)

4. Develop evm_client:

   • Integrate with alloy
   • Implement transaction signing using vault keys
   • Implement account management and contract interaction

5. Develop CLI Interface:

   • Integrate rhai scripting engine
   • Expose vault and evm_client functionalities
   • Implement message-passing for async operations

6. Develop WebAssembly Target:

   • Compile vault and evm_client to WASM using wasm-bindgen
   • Expose functionalities to JavaScript
   • Implement frontend interface (e.g., React)

7. Testing and Documentation:

   • Write unit and integration tests for all functionalities
   • Document public APIs and usage examples

---

This comprehensive plan ensures a modular, secure, and cross-platform cryptographic system, drawing inspiration from the herocode/webassembly project. The design facilitates both command-line and browser-based applications, providing