self/docs/development.md

Development Guide

Overview

This guide covers setting up a development environment, contributing to the Self project, and understanding the development workflow.

Development Environment Setup

Prerequisites

Required Tools

• Rust: Latest stable version (1.75+)
• Node.js: 18+ (for frontend tooling)
• Trunk: WASM build tool
• PostgreSQL: 12+ (for database development)
• Docker: For containerized development
• Git: Version control

Installation Commands

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Add WASM target
rustup target add wasm32-unknown-unknown

# Install Trunk
cargo install trunk

# Install Node.js (using nvm)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 18
nvm use 18

# Install PostgreSQL (Ubuntu/Debian)
sudo apt-get install postgresql postgresql-contrib

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

Project Setup

Clone and Build

# Clone repository
git clone https://github.com/your-org/self.git
cd self

# Install dependencies and build
cargo build

# Build WASM components
cd app
trunk build

# Start development server
cd ../server
cargo run

# In another terminal, serve frontend
cd ../app
trunk serve

Development Database Setup

# Start PostgreSQL
sudo systemctl start postgresql

# Create development database
sudo -u postgres createdb selfdb_dev
sudo -u postgres createuser selfuser
sudo -u postgres psql -c "ALTER USER selfuser WITH PASSWORD 'devpassword';"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE selfdb_dev TO selfuser;"

# Set environment variable
export DATABASE_URL="postgresql://selfuser:devpassword@localhost/selfdb_dev"

# Run migrations
cargo install sqlx-cli
sqlx migrate run

Development Configuration

Environment Variables

# .env.development
DATABASE_URL=postgresql://selfuser:devpassword@localhost/selfdb_dev
JWT_SECRET=dev-secret-key-not-for-production
SMTP_HOST=localhost
SMTP_PORT=1025
RUST_LOG=debug
BASE_URL=http://localhost:8080

VS Code Configuration

// .vscode/settings.json
{
    "rust-analyzer.cargo.features": ["dev"],
    "rust-analyzer.checkOnSave.command": "clippy",
    "rust-analyzer.checkOnSave.extraArgs": ["--", "-W", "clippy::all"],
    "files.associations": {
        "*.rs": "rust"
    },
    "editor.formatOnSave": true,
    "[rust]": {
        "editor.defaultFormatter": "rust-lang.rust-analyzer"
    }
}

// .vscode/launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "lldb",
            "request": "launch",
            "name": "Debug Server",
            "cargo": {
                "args": ["build", "--bin=server"],
                "filter": {
                    "name": "server",
                    "kind": "bin"
                }
            },
            "args": [],
            "cwd": "${workspaceFolder}/server"
        }
    ]
}

Project Structure

Directory Layout

self/
├── components/              # Reusable Yew components
│   ├── src/
│   │   ├── crypto.rs       # Cryptographic utilities
│   │   ├── vault.rs        # Vault storage system
│   │   ├── vault_manager.rs # Vault management UI
│   │   ├── login.rs        # Login component
│   │   ├── registration.rs # Registration component
│   │   ├── identity.rs     # Identity management
│   │   ├── sign.rs         # Digital signing
│   │   └── lib.rs          # Component exports
│   ├── Cargo.toml
│   └── README.md
├── app/                    # Reference application
│   ├── src/
│   │   └── lib.rs          # Main application
│   ├── index.html          # HTML template
│   ├── Trunk.toml          # Trunk configuration
│   ├── serve.sh            # Development server script
│   └── Cargo.toml
├── server/                 # Backend server
│   ├── src/
│   │   └── main.rs         # Server implementation
│   ├── migrations/         # Database migrations
│   └── Cargo.toml
├── docs/                   # Documentation
├── tests/                  # Integration tests
├── scripts/                # Development scripts
├── docker-compose.yml      # Development containers
├── Cargo.toml              # Workspace configuration
└── README.md

Component Architecture

// Component dependency graph
components/
├── crypto.rs              // Core cryptographic functions
├── vault.rs               // Low-level vault operations
├── vault_manager.rs       // Vault UI component (depends on vault)
├── registration.rs        // Registration flow (depends on crypto, vault)
├── login.rs              // Login flow (depends on crypto, vault)
├── identity.rs           // Identity display (depends on vault)
└── sign.rs               // Signing operations (depends on crypto, vault)

Development Workflow

Git Workflow

Branch Strategy

# Main branches
main                    # Production-ready code
develop                 # Integration branch for features

# Feature branches
feature/vault-system    # New feature development
bugfix/auth-issue      # Bug fixes
hotfix/security-patch  # Critical fixes for production

Commit Convention

# Commit message format
<type>(<scope>): <description>

# Types
feat:     # New feature
fix:      # Bug fix
docs:     # Documentation changes
style:    # Code style changes (formatting, etc.)
refactor: # Code refactoring
test:     # Adding or updating tests
chore:    # Maintenance tasks

# Examples
feat(vault): add multi-key storage support
fix(auth): resolve JWT token validation issue
docs(api): update authentication flow documentation

Development Commands

Common Tasks

# Format code
cargo fmt

# Run linter
cargo clippy

# Run tests
cargo test

# Run tests with coverage
cargo tarpaulin --out Html

# Check for security vulnerabilities
cargo audit

# Update dependencies
cargo update

# Build documentation
cargo doc --open

Frontend Development

# Start development server with hot reload
cd app
trunk serve --open

# Build for production
trunk build --release

# Run frontend tests
wasm-pack test --headless --firefox

Backend Development

# Start server with auto-reload
cd server
cargo watch -x run

# Run database migrations
sqlx migrate run

# Generate migration
sqlx migrate add create_users_table

# Check SQL queries at compile time
cargo sqlx prepare

Testing Strategy

Unit Tests

// Example unit test
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_key_generation() {
        let keypair = generate_keypair().unwrap();
        assert_eq!(keypair.private_key.len(), 64); // 32 bytes hex-encoded
        assert_eq!(keypair.public_key.len(), 130); // 65 bytes hex-encoded
    }

    #[test]
    fn test_encryption_roundtrip() {
        let data = "test data";
        let password = "test password";
        
        let encrypted = encrypt_private_key(data, password).unwrap();
        let decrypted = decrypt_private_key(&encrypted, password).unwrap();
        
        assert_eq!(data, decrypted);
    }
}

Integration Tests

// tests/integration_test.rs
use self_server::*;
use reqwest;
use tokio;

#[tokio::test]
async fn test_registration_flow() {
    // Start test server
    let server = start_test_server().await;
    let client = reqwest::Client::new();
    
    // Test email verification
    let response = client
        .post(&format!("{}/api/send-verification", server.url()))
        .json(&json!({"email": "test@example.com"}))
        .send()
        .await
        .unwrap();
    
    assert_eq!(response.status(), 200);
    
    // Test registration
    let keypair = generate_keypair().unwrap();
    let response = client
        .post(&format!("{}/api/register", server.url()))
        .json(&json!({
            "email": "test@example.com",
            "name": "Test User",
            "public_key": keypair.public_key
        }))
        .send()
        .await
        .unwrap();
    
    assert_eq!(response.status(), 200);
}

WASM Tests

// tests/wasm_tests.rs
use wasm_bindgen_test::*;

wasm_bindgen_test_configure!(run_in_browser);

#[wasm_bindgen_test]
fn test_crypto_in_browser() {
    let keypair = generate_keypair().unwrap();
    assert!(!keypair.private_key.is_empty());
    assert!(!keypair.public_key.is_empty());
}

End-to-End Tests

// e2e/registration.spec.js
const { test, expect } = require('@playwright/test');

test('user registration flow', async ({ page }) => {
  await page.goto('http://localhost:8000');
  
  // Fill registration form
  await page.fill('[data-testid="name-input"]', 'Test User');
  await page.fill('[data-testid="email-input"]', 'test@example.com');
  await page.click('[data-testid="send-verification"]');
  
  // Wait for verification (in test environment)
  await page.waitForSelector('[data-testid="email-verified"]');
  
  // Generate keys
  await page.click('[data-testid="generate-keys"]');
  await page.fill('[data-testid="password-input"]', 'testpassword123');
  await page.fill('[data-testid="confirm-password"]', 'testpassword123');
  
  // Complete registration
  await page.click('[data-testid="complete-registration"]');
  await expect(page.locator('[data-testid="registration-success"]')).toBeVisible();
});

Code Style and Standards

Rust Style Guide

Naming Conventions

// Use snake_case for functions and variables
fn generate_keypair() -> Result<KeyPair, String> { }
let private_key = "...";

// Use PascalCase for types and traits
struct KeyPair { }
trait VaultStorage { }

// Use SCREAMING_SNAKE_CASE for constants
const DEFAULT_ITERATIONS: u32 = 10_000;

// Use descriptive names
fn encrypt_private_key_with_password() // Good
fn encrypt() // Too generic

Error Handling

// Use Result types for fallible operations
fn risky_operation() -> Result<String, MyError> {
    // Implementation
}

// Use ? operator for error propagation
fn calling_function() -> Result<(), MyError> {
    let result = risky_operation()?;
    Ok(())
}

// Provide context for errors
fn with_context() -> Result<(), Box<dyn std::error::Error>> {
    risky_operation()
        .map_err(|e| format!("Failed to perform risky operation: {}", e))?;
    Ok(())
}

Documentation

/// Generates a new secp256k1 key pair for cryptographic operations.
/// 
/// This function uses cryptographically secure random number generation
/// to create a private key, then derives the corresponding public key.
/// 
/// # Returns
/// 
/// Returns a `Result` containing a `KeyPair` on success, or a `String`
/// error message on failure.
/// 
/// # Errors
/// 
/// This function will return an error if:
/// - Random number generation fails
/// - Key validation fails
/// - Public key derivation fails
/// 
/// # Examples
/// 
/// ```
/// use self_components::generate_keypair;
/// 
/// let keypair = generate_keypair().expect("Failed to generate keypair");
/// println!("Public key: {}", keypair.public_key);
/// ```
pub fn generate_keypair() -> Result<KeyPair, String> {
    // Implementation
}

Frontend Style Guide

Component Structure

// Component organization
pub struct MyComponent {
    // State fields
    loading: bool,
    error_message: Option<String>,
    data: Option<MyData>,
}

pub enum MyMsg {
    // User actions
    LoadData,
    UpdateField(String),
    Submit,
    
    // Async responses
    DataLoaded(MyData),
    LoadFailed(String),
    
    // UI events
    ClearError,
}

impl Component for MyComponent {
    type Message = MyMsg;
    type Properties = MyProps;

    fn create(ctx: &Context<Self>) -> Self {
        // Initialize component
    }

    fn update(&mut self, ctx: &Context<Self>, msg: Self::Message) -> bool {
        // Handle messages
    }

    fn view(&self, ctx: &Context<Self>) -> Html {
        // Render component
    }
}

HTML Structure

// Use semantic HTML and proper accessibility
fn view(&self, ctx: &Context<Self>) -> Html {
    html! {
        <main class="container" role="main">
            <h1>{"Page Title"}</h1>
            
            <form onsubmit={ctx.link().callback(|e: SubmitEvent| {
                e.prevent_default();
                MyMsg::Submit
            })}>
                <div class="form-group">
                    <label for="email-input">{"Email Address"}</label>
                    <input 
                        id="email-input"
                        type="email"
                        class="form-control"
                        required=true
                        aria-describedby="email-help"
                        value={self.email.clone()}
                        oninput={ctx.link().callback(|e: InputEvent| {
                            let input: HtmlInputElement = e.target_unchecked_into();
                            MyMsg::UpdateEmail(input.value())
                        })}
                    />
                    <small id="email-help" class="form-text text-muted">
                        {"We'll never share your email with anyone else."}
                    </small>
                </div>
                
                <button type="submit" class="btn btn-primary" disabled={self.loading}>
                    {if self.loading { "Loading..." } else { "Submit" }}
                </button>
            </form>
        </main>
    }
}

Debugging and Troubleshooting

Common Issues

WASM Build Failures

# Clear trunk cache
trunk clean

# Rebuild with verbose output
trunk build --verbose

# Check for missing dependencies
cargo check --target wasm32-unknown-unknown

Database Connection Issues

# Check PostgreSQL status
sudo systemctl status postgresql

# Test connection
psql -h localhost -U selfuser -d selfdb_dev

# Reset database
dropdb selfdb_dev && createdb selfdb_dev
sqlx migrate run

CORS Issues

// Add CORS middleware in development
use tower_http::cors::{CorsLayer, Any};

let app = Router::new()
    .layer(
        CorsLayer::new()
            .allow_origin(Any)
            .allow_methods(Any)
            .allow_headers(Any)
    );

Debugging Tools

Browser DevTools

// Enable debug logging in browser
localStorage.setItem('debug', 'self:*');

// View WASM memory usage
console.log(performance.memory);

// Inspect localStorage
console.log(localStorage.getItem('self_vault'));

Server Debugging

// Add debug logging
use tracing::{debug, info, warn, error};

debug!("Processing request: {:?}", request);
info!("User authenticated: {}", user_id);
warn!("Rate limit approaching: {}", current_rate);
error!("Database connection failed: {}", error);

Contributing Guidelines

Pull Request Process

1. Fork and Branch

   git checkout -b feature/my-new-feature
   

2. Make Changes

   • Follow code style guidelines
   • Add tests for new functionality
   • Update documentation

3. Test Changes

   cargo test
   cargo clippy
   cargo fmt --check
   

4. Submit PR

   • Write clear commit messages
   • Include description of changes
   • Reference related issues

Code Review Checklist

Functionality

• Code works as intended
• Edge cases are handled
• Error conditions are properly managed
• Performance is acceptable

Security

• Input validation is present
• No secrets in code
• Cryptographic operations are correct
• Authentication/authorization is proper

Code Quality

• Code is readable and well-documented
• Tests are comprehensive
• No code duplication
• Follows project conventions

Documentation

• Public APIs are documented
• README is updated if needed
• Breaking changes are noted
• Examples are provided

Release Process

Version Numbering

# Semantic versioning: MAJOR.MINOR.PATCH
1.0.0  # Initial release
1.0.1  # Bug fix
1.1.0  # New feature
2.0.0  # Breaking change

Release Checklist

1. Pre-release

   • All tests pass
   • Documentation updated
   • CHANGELOG.md updated
   • Version numbers bumped

2. Release

   • Create release tag
   • Build release artifacts
   • Deploy to staging
   • Run integration tests

3. Post-release

   • Deploy to production
   • Monitor for issues
   • Update documentation site
   • Announce release

This development guide provides everything needed to contribute to the Self project effectively, from initial setup through the release process.