self/docs/development.md

# Development Guide

## Overview

This guide covers setting up a development environment, contributing to the Self project, and understanding the development workflow.

## Development Environment Setup

### Prerequisites

#### Required Tools
- **Rust**: Latest stable version (1.75+)
- **Node.js**: 18+ (for frontend tooling)
- **Trunk**: WASM build tool
- **PostgreSQL**: 12+ (for database development)
- **Docker**: For containerized development
- **Git**: Version control

#### Installation Commands

```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Add WASM target
rustup target add wasm32-unknown-unknown

# Install Trunk
cargo install trunk

# Install Node.js (using nvm)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 18
nvm use 18

# Install PostgreSQL (Ubuntu/Debian)
sudo apt-get install postgresql postgresql-contrib

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
```

### Project Setup

#### Clone and Build

```bash
# Clone repository
git clone https://github.com/your-org/self.git
cd self

# Install dependencies and build
cargo build

# Build WASM components
cd app
trunk build

# Start development server
cd ../server
cargo run

# In another terminal, serve frontend
cd ../app
trunk serve
```

#### Development Database Setup

```bash
# Start PostgreSQL
sudo systemctl start postgresql

# Create development database
sudo -u postgres createdb selfdb_dev
sudo -u postgres createuser selfuser
sudo -u postgres psql -c "ALTER USER selfuser WITH PASSWORD 'devpassword';"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE selfdb_dev TO selfuser;"

# Set environment variable
export DATABASE_URL="postgresql://selfuser:devpassword@localhost/selfdb_dev"

# Run migrations
cargo install sqlx-cli
sqlx migrate run
```

### Development Configuration

#### Environment Variables

```bash
# .env.development
DATABASE_URL=postgresql://selfuser:devpassword@localhost/selfdb_dev
JWT_SECRET=dev-secret-key-not-for-production
SMTP_HOST=localhost
SMTP_PORT=1025
RUST_LOG=debug
BASE_URL=http://localhost:8080
```

#### VS Code Configuration

```json
// .vscode/settings.json
{
    "rust-analyzer.cargo.features": ["dev"],
    "rust-analyzer.checkOnSave.command": "clippy",
    "rust-analyzer.checkOnSave.extraArgs": ["--", "-W", "clippy::all"],
    "files.associations": {
        "*.rs": "rust"
    },
    "editor.formatOnSave": true,
    "[rust]": {
        "editor.defaultFormatter": "rust-lang.rust-analyzer"
    }
}
```

```json
// .vscode/launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "lldb",
            "request": "launch",
            "name": "Debug Server",
            "cargo": {
                "args": ["build", "--bin=server"],
                "filter": {
                    "name": "server",
                    "kind": "bin"
                }
            },
            "args": [],
            "cwd": "${workspaceFolder}/server"
        }
    ]
}
```

## Project Structure

### Directory Layout

```
self/
├── components/              # Reusable Yew components
│   ├── src/
│   │   ├── crypto.rs       # Cryptographic utilities
│   │   ├── vault.rs        # Vault storage system
│   │   ├── vault_manager.rs # Vault management UI
│   │   ├── login.rs        # Login component
│   │   ├── registration.rs # Registration component
│   │   ├── identity.rs     # Identity management
│   │   ├── sign.rs         # Digital signing
│   │   └── lib.rs          # Component exports
│   ├── Cargo.toml
│   └── README.md
├── app/                    # Reference application
│   ├── src/
│   │   └── lib.rs          # Main application
│   ├── index.html          # HTML template
│   ├── Trunk.toml          # Trunk configuration
│   ├── serve.sh            # Development server script
│   └── Cargo.toml
├── server/                 # Backend server
│   ├── src/
│   │   └── main.rs         # Server implementation
│   ├── migrations/         # Database migrations
│   └── Cargo.toml
├── docs/                   # Documentation
├── tests/                  # Integration tests
├── scripts/                # Development scripts
├── docker-compose.yml      # Development containers
├── Cargo.toml              # Workspace configuration
└── README.md
```

### Component Architecture

```rust
// Component dependency graph
components/
├── crypto.rs              // Core cryptographic functions
├── vault.rs               // Low-level vault operations
├── vault_manager.rs       // Vault UI component (depends on vault)
├── registration.rs        // Registration flow (depends on crypto, vault)
├── login.rs              // Login flow (depends on crypto, vault)
├── identity.rs           // Identity display (depends on vault)
└── sign.rs               // Signing operations (depends on crypto, vault)
```

## Development Workflow

### Git Workflow

#### Branch Strategy

```bash
# Main branches
main                    # Production-ready code
develop                 # Integration branch for features

# Feature branches
feature/vault-system    # New feature development
bugfix/auth-issue      # Bug fixes
hotfix/security-patch  # Critical fixes for production
```

#### Commit Convention

```bash
# Commit message format
<type>(<scope>): <description>

# Types
feat:     # New feature
fix:      # Bug fix
docs:     # Documentation changes
style:    # Code style changes (formatting, etc.)
refactor: # Code refactoring
test:     # Adding or updating tests
chore:    # Maintenance tasks

# Examples
feat(vault): add multi-key storage support
fix(auth): resolve JWT token validation issue
docs(api): update authentication flow documentation
```

### Development Commands

#### Common Tasks

```bash
# Format code
cargo fmt

# Run linter
cargo clippy

# Run tests
cargo test

# Run tests with coverage
cargo tarpaulin --out Html

# Check for security vulnerabilities
cargo audit

# Update dependencies
cargo update

# Build documentation
cargo doc --open
```

#### Frontend Development

```bash
# Start development server with hot reload
cd app
trunk serve --open

# Build for production
trunk build --release

# Run frontend tests
wasm-pack test --headless --firefox
```

#### Backend Development

```bash
# Start server with auto-reload
cd server
cargo watch -x run

# Run database migrations
sqlx migrate run

# Generate migration
sqlx migrate add create_users_table

# Check SQL queries at compile time
cargo sqlx prepare
```

## Testing Strategy

### Unit Tests

```rust
// Example unit test
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_key_generation() {
        let keypair = generate_keypair().unwrap();
        assert_eq!(keypair.private_key.len(), 64); // 32 bytes hex-encoded
        assert_eq!(keypair.public_key.len(), 130); // 65 bytes hex-encoded
    }

    #[test]
    fn test_encryption_roundtrip() {
        let data = "test data";
        let password = "test password";
        
        let encrypted = encrypt_private_key(data, password).unwrap();
        let decrypted = decrypt_private_key(&encrypted, password).unwrap();
        
        assert_eq!(data, decrypted);
    }
}
```

### Integration Tests

```rust
// tests/integration_test.rs
use self_server::*;
use reqwest;
use tokio;

#[tokio::test]
async fn test_registration_flow() {
    // Start test server
    let server = start_test_server().await;
    let client = reqwest::Client::new();
    
    // Test email verification
    let response = client
        .post(&format!("{}/api/send-verification", server.url()))
        .json(&json!({"email": "test@example.com"}))
        .send()
        .await
        .unwrap();
    
    assert_eq!(response.status(), 200);
    
    // Test registration
    let keypair = generate_keypair().unwrap();
    let response = client
        .post(&format!("{}/api/register", server.url()))
        .json(&json!({
            "email": "test@example.com",
            "name": "Test User",
            "public_key": keypair.public_key
        }))
        .send()
        .await
        .unwrap();
    
    assert_eq!(response.status(), 200);
}
```

### WASM Tests

```rust
// tests/wasm_tests.rs
use wasm_bindgen_test::*;

wasm_bindgen_test_configure!(run_in_browser);

#[wasm_bindgen_test]
fn test_crypto_in_browser() {
    let keypair = generate_keypair().unwrap();
    assert!(!keypair.private_key.is_empty());
    assert!(!keypair.public_key.is_empty());
}
```

### End-to-End Tests

```javascript
// e2e/registration.spec.js
const { test, expect } = require('@playwright/test');

test('user registration flow', async ({ page }) => {
  await page.goto('http://localhost:8000');
  
  // Fill registration form
  await page.fill('[data-testid="name-input"]', 'Test User');
  await page.fill('[data-testid="email-input"]', 'test@example.com');
  await page.click('[data-testid="send-verification"]');
  
  // Wait for verification (in test environment)
  await page.waitForSelector('[data-testid="email-verified"]');
  
  // Generate keys
  await page.click('[data-testid="generate-keys"]');
  await page.fill('[data-testid="password-input"]', 'testpassword123');
  await page.fill('[data-testid="confirm-password"]', 'testpassword123');
  
  // Complete registration
  await page.click('[data-testid="complete-registration"]');
  await expect(page.locator('[data-testid="registration-success"]')).toBeVisible();
});
```

## Code Style and Standards

### Rust Style Guide

#### Naming Conventions

```rust
// Use snake_case for functions and variables
fn generate_keypair() -> Result<KeyPair, String> { }
let private_key = "...";

// Use PascalCase for types and traits
struct KeyPair { }
trait VaultStorage { }

// Use SCREAMING_SNAKE_CASE for constants
const DEFAULT_ITERATIONS: u32 = 10_000;

// Use descriptive names
fn encrypt_private_key_with_password() // Good
fn encrypt() // Too generic
```

#### Error Handling

```rust
// Use Result types for fallible operations
fn risky_operation() -> Result<String, MyError> {
    // Implementation
}

// Use ? operator for error propagation
fn calling_function() -> Result<(), MyError> {
    let result = risky_operation()?;
    Ok(())
}

// Provide context for errors
fn with_context() -> Result<(), Box<dyn std::error::Error>> {
    risky_operation()
        .map_err(|e| format!("Failed to perform risky operation: {}", e))?;
    Ok(())
}
```

#### Documentation

```rust
/// Generates a new secp256k1 key pair for cryptographic operations.
/// 
/// This function uses cryptographically secure random number generation
/// to create a private key, then derives the corresponding public key.
/// 
/// # Returns
/// 
/// Returns a `Result` containing a `KeyPair` on success, or a `String`
/// error message on failure.
/// 
/// # Errors
/// 
/// This function will return an error if:
/// - Random number generation fails
/// - Key validation fails
/// - Public key derivation fails
/// 
/// # Examples
/// 
/// ```
/// use self_components::generate_keypair;
/// 
/// let keypair = generate_keypair().expect("Failed to generate keypair");
/// println!("Public key: {}", keypair.public_key);
/// ```
pub fn generate_keypair() -> Result<KeyPair, String> {
    // Implementation
}
```

### Frontend Style Guide

#### Component Structure

```rust
// Component organization
pub struct MyComponent {
    // State fields
    loading: bool,
    error_message: Option<String>,
    data: Option<MyData>,
}

pub enum MyMsg {
    // User actions
    LoadData,
    UpdateField(String),
    Submit,
    
    // Async responses
    DataLoaded(MyData),
    LoadFailed(String),
    
    // UI events
    ClearError,
}

impl Component for MyComponent {
    type Message = MyMsg;
    type Properties = MyProps;

    fn create(ctx: &Context<Self>) -> Self {
        // Initialize component
    }

    fn update(&mut self, ctx: &Context<Self>, msg: Self::Message) -> bool {
        // Handle messages
    }

    fn view(&self, ctx: &Context<Self>) -> Html {
        // Render component
    }
}
```

#### HTML Structure

```rust
// Use semantic HTML and proper accessibility
fn view(&self, ctx: &Context<Self>) -> Html {
    html! {
        <main class="container" role="main">
            <h1>{"Page Title"}</h1>
            
            <form onsubmit={ctx.link().callback(|e: SubmitEvent| {
                e.prevent_default();
                MyMsg::Submit
            })}>
                <div class="form-group">
                    <label for="email-input">{"Email Address"}</label>
                    <input 
                        id="email-input"
                        type="email"
                        class="form-control"
                        required=true
                        aria-describedby="email-help"
                        value={self.email.clone()}
                        oninput={ctx.link().callback(|e: InputEvent| {
                            let input: HtmlInputElement = e.target_unchecked_into();
                            MyMsg::UpdateEmail(input.value())
                        })}
                    />
                    <small id="email-help" class="form-text text-muted">
                        {"We'll never share your email with anyone else."}
                    </small>
                </div>
                
                <button type="submit" class="btn btn-primary" disabled={self.loading}>
                    {if self.loading { "Loading..." } else { "Submit" }}
                </button>
            </form>
        </main>
    }
}
```

## Debugging and Troubleshooting

### Common Issues

#### WASM Build Failures

```bash
# Clear trunk cache
trunk clean

# Rebuild with verbose output
trunk build --verbose

# Check for missing dependencies
cargo check --target wasm32-unknown-unknown
```

#### Database Connection Issues

```bash
# Check PostgreSQL status
sudo systemctl status postgresql

# Test connection
psql -h localhost -U selfuser -d selfdb_dev

# Reset database
dropdb selfdb_dev && createdb selfdb_dev
sqlx migrate run
```

#### CORS Issues

```rust
// Add CORS middleware in development
use tower_http::cors::{CorsLayer, Any};

let app = Router::new()
    .layer(
        CorsLayer::new()
            .allow_origin(Any)
            .allow_methods(Any)
            .allow_headers(Any)
    );
```

### Debugging Tools

#### Browser DevTools

```javascript
// Enable debug logging in browser
localStorage.setItem('debug', 'self:*');

// View WASM memory usage
console.log(performance.memory);

// Inspect localStorage
console.log(localStorage.getItem('self_vault'));
```

#### Server Debugging

```rust
// Add debug logging
use tracing::{debug, info, warn, error};

debug!("Processing request: {:?}", request);
info!("User authenticated: {}", user_id);
warn!("Rate limit approaching: {}", current_rate);
error!("Database connection failed: {}", error);
```

## Contributing Guidelines

### Pull Request Process

1. **Fork and Branch**
   ```bash
   git checkout -b feature/my-new-feature
   ```

2. **Make Changes**
   - Follow code style guidelines
   - Add tests for new functionality
   - Update documentation

3. **Test Changes**
   ```bash
   cargo test
   cargo clippy
   cargo fmt --check
   ```

4. **Submit PR**
   - Write clear commit messages
   - Include description of changes
   - Reference related issues

### Code Review Checklist

#### Functionality
- [ ] Code works as intended
- [ ] Edge cases are handled
- [ ] Error conditions are properly managed
- [ ] Performance is acceptable

#### Security
- [ ] Input validation is present
- [ ] No secrets in code
- [ ] Cryptographic operations are correct
- [ ] Authentication/authorization is proper

#### Code Quality
- [ ] Code is readable and well-documented
- [ ] Tests are comprehensive
- [ ] No code duplication
- [ ] Follows project conventions

#### Documentation
- [ ] Public APIs are documented
- [ ] README is updated if needed
- [ ] Breaking changes are noted
- [ ] Examples are provided

### Release Process

#### Version Numbering

```bash
# Semantic versioning: MAJOR.MINOR.PATCH
1.0.0  # Initial release
1.0.1  # Bug fix
1.1.0  # New feature
2.0.0  # Breaking change
```

#### Release Checklist

1. **Pre-release**
   - [ ] All tests pass
   - [ ] Documentation updated
   - [ ] CHANGELOG.md updated
   - [ ] Version numbers bumped

2. **Release**
   - [ ] Create release tag
   - [ ] Build release artifacts
   - [ ] Deploy to staging
   - [ ] Run integration tests

3. **Post-release**
   - [ ] Deploy to production
   - [ ] Monitor for issues
   - [ ] Update documentation site
   - [ ] Announce release

This development guide provides everything needed to contribute to the Self project effectively, from initial setup through the release process.