circles/ARCHITECTURE.md

Circles: Peer-Centric Application Layer Architecture

This document provides a comprehensive overview of the Circles project architecture - a distributed, peer-contextual application layer runtime where peers can send scripts to each other for execution within isolated contexts.

1. System Overview

1.1. Core Concept

Circles implements a peer-centric backend architecture where each user is treated as an autonomous execution context rather than a passive database entry. The system enables cross-peer script execution where:

• Peer A sends a script to Peer B
• The script executes within Peer B's context (filesystem namespace, data isolation)
• The script runs with awareness that Peer A initiated the execution
• Peer B controls what scripts it accepts and how they execute

1.2. Key Characteristics

• Peer-Centric Execution: Each peer is an isolated context, not a row in a global database
• Context-Based Routing: Execution flows are scoped by peer identity (public keys)
• Decentralizable: Circle nodes can be deployed independently and serve separate peer sets
• Script-Triggered Workflows: Cross-peer interactions happen via script execution
• Namespace Isolation: Workers enforce isolation through filesystem namespacing

2. Core Components

2.1. Peer Context

A peer context represents an isolated execution environment identified by a secp256k1 public key:

• Identity: Unique secp256k1 public key
• Filesystem Namespace: Isolated root directory (DB_PATH/{peer_public_key}/)
• Execution Environment: Rhai script engine with peer-specific variables
• Policy Boundaries: Controls which peers can execute scripts and what operations are allowed

2.2. Circle (WebSocket Gateway)

The server crate provides the WebSocket server that acts as a gateway for peer interactions:

• Peer Authentication: secp256k1 signature-based authentication via JSON-RPC
• Script Routing: Routes incoming script execution requests to Redis queues
• Session Management: Maintains authenticated WebSocket connections per peer
• Multi-Peer Support: Single server instance can handle multiple peer contexts

Key Features:

• Built on Actix Web with WebSocket actors (CircleWs)
• JSON-RPC 2.0 API for authentication and script execution
• Redis integration for decoupled worker communication

2.3. Worker Engine

The worker crate implements the script execution engine:

• Context-Aware Execution: Loads and executes scripts within specific peer contexts
• Namespace Isolation: Each peer gets isolated filesystem access
• Caller Awareness: Scripts execute with knowledge of both caller and target peer
• Multiplexing Support: Single worker can serve multiple peer contexts

Execution Environment Variables:

CALLER_PUBLIC_KEY    // The peer who sent the script
CIRCLE_PUBLIC_KEY    // The peer in whose context the script runs
DB_PATH             // Namespaced database path for this peer

2.4. Client Library

The client_ws crate provides cross-platform WebSocket client functionality:

• Cross-Platform: Native (tokio-tungstenite) and WASM (gloo-net) support
• Authentication: Automated secp256k1 signature-based authentication
• Script Execution: High-level API for sending scripts to other peers
• Builder Pattern: Flexible client construction with optional authentication

3. Communication Topology

3.1. Peer-to-Peer Script Execution Flow

sequenceDiagram
    participant PeerA as Peer A (Client)
    participant Circle as Circle Node (WebSocket)
    participant Redis as Redis Queue
    participant Worker as Worker Engine
    participant FS as Peer B Filesystem

    Note over PeerA,FS: Peer A sends script to execute in Peer B's context

    PeerA->>Circle: WebSocket: JSON-RPC "play" request
    Note right of PeerA: Script + Target Peer B's public key
    
    Circle->>Circle: Validate authentication
    Circle->>Redis: LPUSH to peer B's queue
    Note right of Circle: Queue: rhailib:{peer_b_pubkey}
    
    Worker->>Redis: BLPOP from peer B's queue
    Worker->>Worker: Load Peer B's context
    Note right of Worker: Set CALLER_PUBLIC_KEY=peer_a<br/>Set CIRCLE_PUBLIC_KEY=peer_b<br/>Set DB_PATH=peer_b_namespace/
    
    Worker->>FS: Execute script in Peer B's namespace
    FS-->>Worker: Script execution result
    
    Worker->>Redis: LPUSH result to reply queue
    Redis-->>Circle: Result notification
    Circle-->>PeerA: WebSocket: JSON-RPC response

3.2. System Component Diagram

graph TB
    subgraph "Peer A Environment"
        ClientA[Client A<br/>client_ws]
        KeyA[Private Key A]
        ClientA -.-> KeyA
    end
    
    subgraph "Peer B Environment"
        ClientB[Client B<br/>client_ws]
        KeyB[Private Key B]
        ClientB -.-> KeyB
    end
    
    subgraph "Circle Infrastructure"
        Circle[Circle Node<br/>server]
        Redis[(Redis<br/>Task Queues)]
        Worker1[Worker Engine<br/>Peer A Context]
        Worker2[Worker Engine<br/>Peer B Context]
        
        Circle <--> Redis
        Redis <--> Worker1
        Redis <--> Worker2
    end
    
    subgraph "Isolated Storage"
        FSA[Peer A Namespace<br/>DB_PATH/peer_a/]
        FSB[Peer B Namespace<br/>DB_PATH/peer_b/]
        
        Worker1 <--> FSA
        Worker2 <--> FSB
    end
    
    ClientA <-->|WebSocket<br/>Authenticated| Circle
    ClientB <-->|WebSocket<br/>Authenticated| Circle
    
    style ClientA fill:#e1f5fe
    style ClientB fill:#e8f5e8
    style Circle fill:#fff3e0
    style Worker1 fill:#e1f5fe
    style Worker2 fill:#e8f5e8

3.3. Multi-Circle Deployment Architecture

graph TB
    subgraph "Launcher Configuration"
        Launcher[Launcher<br/>circles.json]
        Config[["OurWorld: port 8090<br/>Threefold: port 8091<br/>Sikana: port 8092<br/>..."]]
        Launcher --> Config
    end
    
    subgraph "Circle Nodes"
        Circle1[Circle 1<br/>:8090]
        Circle2[Circle 2<br/>:8091]
        Circle3[Circle 3<br/>:8092]
    end
    
    subgraph "Worker Pool"
        Worker1[Worker<br/>OurWorld Context]
        Worker2[Worker<br/>Threefold Context]
        Worker3[Worker<br/>Sikana Context]
    end
    
    subgraph "Shared Infrastructure"
        Redis[(Redis<br/>Coordination)]
        Storage[(Namespaced Storage<br/>peer_contexts/)]
    end
    
    Launcher --> Circle1
    Launcher --> Circle2
    Launcher --> Circle3
    
    Circle1 <--> Redis
    Circle2 <--> Redis
    Circle3 <--> Redis
    
    Redis <--> Worker1
    Redis <--> Worker2
    Redis <--> Worker3
    
    Worker1 <--> Storage
    Worker2 <--> Storage
    Worker3 <--> Storage

4. Authentication and Security

4.1. Peer Authentication Flow

sequenceDiagram
    participant Client as Peer Client
    participant Circle as Circle Node
    
    Note over Client: Has secp256k1 keypair
    
    Client->>+Circle: JSON-RPC "fetch_nonce" (pubkey)
    Circle->>Circle: generate_nonce()
    Circle->>Circle: store_nonce(pubkey, nonce)
    Circle-->>-Client: nonce + expiration
    
    Client->>Client: sign(nonce, private_key)
    
    Client->>+Circle: JSON-RPC "authenticate" (pubkey, signature)
    Circle->>Circle: verify_signature(nonce, signature, pubkey)
    
    alt Signature Valid
        Circle->>Circle: Mark session as authenticated
        Circle-->>Client: {"authenticated": true}
        Note over Circle: Subsequent "play" requests include authenticated pubkey
    else Signature Invalid
        Circle-->>-Client: JSON-RPC Error: Invalid Credentials
        Circle->>Circle: Close connection
    end

4.2. Context Isolation Model

Each peer context is isolated through:

1. Filesystem Namespacing: Each peer gets a dedicated root directory
2. Environment Variables: Scripts execute with peer-specific context variables
3. Redis Queue Isolation: Each peer has dedicated task queues
4. Authentication Boundaries: Only authenticated peers can send scripts

worker_rhai_temp_db/
├── peer_a_public_key/          # Peer A's isolated namespace
│   ├── data/
│   ├── config/
│   └── logs/
├── peer_b_public_key/          # Peer B's isolated namespace
│   ├── data/
│   ├── config/
│   └── logs/
└── ...

5. Script Execution Model

5.1. Cross-Peer Script Execution

When Peer A sends a script to Peer B:

1. Authentication: Peer A authenticates with their private key
2. Script Routing: Circle routes the script to Peer B's Redis queue
3. Context Loading: Worker loads Peer B's execution context
4. Environment Setup: Worker sets context variables:

   CALLER_PUBLIC_KEY = "peer_a_public_key"
   CIRCLE_PUBLIC_KEY = "peer_b_public_key"
   DB_PATH = "/path/to/peer_b_namespace"
   

5. Script Execution: Rhai engine executes script in Peer B's context
6. Result Return: Execution result is returned to Peer A via Redis/WebSocket

5.2. Example Script Execution

// Script sent by Peer A to Peer B
print("Script running in context of: " + CIRCLE_PUBLIC_KEY);
print("Initiated by: " + CALLER_PUBLIC_KEY);

// Access Peer B's data (isolated namespace)
let peer_b_data = load_data("user_preferences.json");

// Perform operations in Peer B's context
let result = process_data(peer_b_data);

// Return result to Peer A
result

6. API Reference

6.1. JSON-RPC Methods

The Circle WebSocket API provides three core methods:

fetch_nonce

Requests a cryptographic nonce for authentication.

Parameters:

• pubkey (string): Client's secp256k1 public key

Response:

{
  "nonce": "base64_encoded_nonce",
  "expires_at": 1234567890
}

authenticate

Authenticates the client using a signed nonce.

Parameters:

• pubkey (string): Client's public key
• signature (string): Nonce signed with client's private key

Response:

{
  "authenticated": true
}

play

Executes a script in a target peer's context.

Parameters:

• script (string): Rhai script to execute

Response:

{
  "output": "script_execution_result"
}

6.2. Client Library Usage

use circle_client_ws::{CircleWsClientBuilder, CircleWsClient};

// Create authenticated client
let mut client = CircleWsClientBuilder::new("ws://localhost:8090/peer_b_pubkey".to_string())
    .with_keypair(private_key)
    .build();

// Connect and authenticate
client.connect().await?;
client.authenticate().await?;

// Send script to execute in peer B's context
let result = client.play(r#"
    print("Hello from " + CALLER_PUBLIC_KEY + " to " + CIRCLE_PUBLIC_KEY);
    "Script executed successfully"
"#.to_string()).await?;

println!("Result: {}", result.output);

7. Deployment Models

7.1. Single-Peer Worker

Each peer runs in its own dedicated worker process:

# Start worker for specific peer
./worker --circle-public-key peer_a_public_key --redis-url redis://localhost:6379

7.2. Multi-Peer Worker

Single worker serves multiple peer contexts with namespace isolation:

# Worker handles multiple peers with context switching
./worker --redis-url redis://localhost:6379

7.3. Launcher-Managed Deployment

Use the launcher to manage multiple circles:

[
  {
    "name": "OurWorld",
    "port": 8090,
    "script_path": "scripts/ourworld.rhai"
  },
  {
    "name": "Threefold", 
    "port": 8091,
    "script_path": "scripts/threefold.rhai"
  }
]

./launcher --config circles.json

8. Technical Implementation Details

8.1. Redis Protocol

Task Queue Pattern:

• Queue Key: rhailib:{peer_public_key}
• Task Details: rhailib:{task_id} (hash)
• Reply Queue: rhailib:reply:{task_id}

Task Lifecycle:

1. Client submits script → Redis queue
2. Worker picks up task → Updates status to "processing"
3. Worker executes script → Updates status to "completed"/"error"
4. Worker publishes result → Reply queue
5. Client receives result → Task cleanup (optional)

8.2. Worker Context Management

// Worker sets up peer context before script execution
let mut db_config = rhai::Map::new();
db_config.insert("DB_PATH".into(), db_path.clone().into());
db_config.insert("CALLER_PUBLIC_KEY".into(), caller_id.clone().into());
db_config.insert("CIRCLE_PUBLIC_KEY".into(), circle_public_key.clone().into());
engine.set_default_tag(Dynamic::from(db_config));

8.3. Filesystem Isolation

Each peer context gets an isolated filesystem namespace:

// Example namespace structure
let peer_namespace = format!("{}/{}", base_db_path, peer_public_key);
std::fs::create_dir_all(&peer_namespace)?;

// All file operations are relative to this namespace
let data_file = format!("{}/data/user_data.json", peer_namespace);

9. Comparison to Traditional Architectures

Feature	Traditional Backend	Circles (Peer-Centric)
User Representation	Row in database	Autonomous execution context
Request Routing	Global endpoints	Peer-scoped script execution
State Management	Shared application state	Isolated peer contexts
Communication	HTTP/REST, JSON-RPC	WebSocket + Redis coordination
Distribution	Centralized	Decentralized, per-peer scalable
Execution Model	Threaded request handlers	Context-aware script processors
Data Isolation	Application-level	Filesystem namespacing
Cross-User Operations	Database transactions	Cross-peer script execution

10. Summary

Circles implements a distributed, peer-contextual application layer runtime where:

• Each peer is an isolated execution context identified by secp256k1 public keys
• Peers can send scripts to other peers for execution within the target's context
• Workers enforce isolation through filesystem namespacing and environment variables
• The system supports both single-peer and multi-peer worker deployments
• Communication flows through WebSocket gateways coordinated via Redis queues

This architecture enables decentralized application logic where peers maintain autonomy over their execution environments while supporting secure cross-peer interactions through script-based workflows.