...

examples/README.md

@@ -0,0 +1,171 @@
+# HeroDB Tantivy Search Examples
+
+This directory contains examples demonstrating HeroDB's full-text search capabilities powered by Tantivy.
+
+## Tantivy Search Demo (Bash Script)
+
+### Overview
+The `tantivy_search_demo.sh` script provides a comprehensive demonstration of HeroDB's search functionality using Redis commands. It showcases various search scenarios including basic text search, filtering, sorting, geographic queries, and more.
+
+### Prerequisites
+1. **HeroDB Server**: The server must be running on port 6381
+2. **Redis CLI**: The `redis-cli` tool must be installed and available in your PATH
+
+### Running the Demo
+
+#### Step 1: Start HeroDB Server
+```bash
+# From the project root directory
+cargo run -- --port 6381
+```
+
+#### Step 2: Run the Demo (in a new terminal)
+```bash
+# From the project root directory
+./examples/tantivy_search_demo.sh
+```
+
+### What the Demo Covers
+
+The script demonstrates 15 different search scenarios:
+
+1. **Index Creation** - Creating a search index with various field types
+2. **Data Insertion** - Adding sample products to the index
+3. **Basic Text Search** - Simple keyword searches
+4. **Filtered Search** - Combining text search with category filters
+5. **Numeric Range Search** - Finding products within price ranges
+6. **Sorting Results** - Ordering results by different fields
+7. **Limited Results** - Pagination and result limiting
+8. **Complex Queries** - Multi-field searches with sorting
+9. **Geographic Search** - Location-based queries
+10. **Index Information** - Getting statistics about the search index
+11. **Search Comparison** - Tantivy vs simple pattern matching
+12. **Fuzzy Search** - Typo tolerance and approximate matching
+13. **Phrase Search** - Exact phrase matching
+14. **Boolean Queries** - AND, OR, NOT operators
+15. **Cleanup** - Removing test data
+
+### Sample Data
+
+The demo uses a product catalog with the following fields:
+- **title** (TEXT) - Product name with higher search weight
+- **description** (TEXT) - Detailed product description
+- **category** (TAG) - Comma-separated categories
+- **price** (NUMERIC) - Product price for range queries
+- **rating** (NUMERIC) - Customer rating for sorting
+- **location** (GEO) - Geographic coordinates for location searches
+
+### Key Redis Commands Demonstrated
+
+#### Index Management
+```bash
+# Create search index
+FT.CREATE product_catalog ON HASH PREFIX 1 product: SCHEMA title TEXT WEIGHT 2.0 SORTABLE description TEXT category TAG SEPARATOR , price NUMERIC SORTABLE rating NUMERIC SORTABLE location GEO
+
+# Get index information
+FT.INFO product_catalog
+
+# Drop index
+FT.DROPINDEX product_catalog
+```
+
+#### Search Queries
+```bash
+# Basic text search
+FT.SEARCH product_catalog wireless
+
+# Filtered search
+FT.SEARCH product_catalog 'organic @category:{food}'
+
+# Numeric range
+FT.SEARCH product_catalog '@price:[50 150]'
+
+# Sorted results
+FT.SEARCH product_catalog '@category:{electronics}' SORTBY price ASC
+
+# Geographic search
+FT.SEARCH product_catalog '@location:[37.7749 -122.4194 50 km]'
+
+# Boolean queries
+FT.SEARCH product_catalog 'wireless AND audio'
+FT.SEARCH product_catalog 'coffee OR tea'
+
+# Phrase search
+FT.SEARCH product_catalog '"noise canceling"'
+```
+
+### Interactive Features
+
+The demo script includes:
+- **Colored output** for better readability
+- **Pause between steps** to review results
+- **Error handling** with clear error messages
+- **Automatic cleanup** of test data
+- **Progress indicators** showing what each step demonstrates
+
+### Troubleshooting
+
+#### HeroDB Not Running
+```
+✗ HeroDB is not running on port 6381
+ℹ Please start HeroDB with: cargo run -- --port 6381
+```
+**Solution**: Start the HeroDB server in a separate terminal.
+
+#### Redis CLI Not Found
+```
+redis-cli: command not found
+```
+**Solution**: Install Redis tools or use an alternative Redis client.
+
+#### Connection Refused
+```
+Could not connect to Redis at localhost:6381: Connection refused
+```
+**Solution**: Ensure HeroDB is running and listening on the correct port.
+
+### Manual Testing
+
+You can also run individual commands manually:
+
+```bash
+# Connect to HeroDB
+redis-cli -h localhost -p 6381
+
+# Create a simple index
+FT.CREATE myindex ON HASH SCHEMA title TEXT description TEXT
+
+# Add a document
+HSET doc:1 title "Hello World" description "This is a test document"
+
+# Search
+FT.SEARCH myindex hello
+```
+
+### Performance Notes
+
+- **Indexing**: Documents are indexed in real-time as they're added
+- **Search Speed**: Full-text search is much faster than pattern matching on large datasets
+- **Memory Usage**: Tantivy indexes are memory-efficient and disk-backed
+- **Scalability**: Supports millions of documents with sub-second search times
+
+### Advanced Features
+
+The demo showcases advanced Tantivy features:
+- **Relevance Scoring** - Results ranked by relevance
+- **Fuzzy Matching** - Handles typos and approximate matches
+- **Field Weighting** - Title field has higher search weight
+- **Multi-field Search** - Search across multiple fields simultaneously
+- **Geographic Queries** - Distance-based location searches
+- **Numeric Ranges** - Efficient range queries on numeric fields
+- **Tag Filtering** - Fast categorical filtering
+
+### Next Steps
+
+After running the demo, explore:
+1. **Custom Schemas** - Define your own field types and configurations
+2. **Large Datasets** - Test with thousands or millions of documents
+3. **Real Applications** - Integrate search into your applications
+4. **Performance Tuning** - Optimize for your specific use case
+
+For more information, see the [search documentation](../herodb/docs/search.md).

examples/age_bash_demo.sh

@@ -0,0 +1,71 @@
+#!/bin/bash
+
+# Start the herodb server in the background
+echo "Starting herodb server..."
+cargo run -p herodb -- --dir /tmp/herodb_age_test --port 6382 --debug --encryption-key "testkey" &
+SERVER_PID=$!
+sleep 2 # Give the server a moment to start
+
+REDIS_CLI="redis-cli -p 6382"
+
+echo "--- Generating and Storing Encryption Keys ---"
+# The new AGE commands are 'AGE KEYGEN <name>' etc., based on src/cmd.rs
+# This script uses older commands like 'AGE.GENERATE_KEYPAIR alice'
+# The demo script needs to be updated to match the implemented commands.
+# Let's assume the commands in the script are what's expected for now,
+# but note this discrepancy. The new commands are AGE KEYGEN etc.
+# The script here uses a different syntax not found in src/cmd.rs like 'AGE.GENERATE_KEYPAIR'.
+# For now, I will modify the script to fit the actual implementation.
+
+echo "--- Generating and Storing Encryption Keys ---"
+$REDIS_CLI AGE KEYGEN alice
+$REDIS_CLI AGE KEYGEN bob
+
+echo "--- Encrypting and Decrypting a Message ---"
+MESSAGE="Hello, AGE encryption!"
+# The new logic stores keys internally and does not expose a command to get the public key.
+# We will encrypt by name.
+ALICE_PUBKEY_REPLY=$($REDIS_CLI AGE KEYGEN alice | head -n 2 | tail -n 1)
+echo "Alice's Public Key: $ALICE_PUBKEY_REPLY"
+
+echo "Encrypting message: '$MESSAGE' with Alice's identity..."
+# AGE.ENCRYPT recipient message. But since we use persistent keys, let's use ENCRYPTNAME
+CIPHERTEXT=$($REDIS_CLI AGE ENCRYPTNAME alice "$MESSAGE")
+echo "Ciphertext: $CIPHERTEXT"
+
+echo "Decrypting ciphertext with Alice's private key..."
+DECRYPTED_MESSAGE=$($REDIS_CLI AGE DECRYPTNAME alice "$CIPHERTEXT")
+echo "Decrypted Message: $DECRYPTED_MESSAGE"
+
+echo "--- Generating and Storing Signing Keys ---"
+$REDIS_CLI AGE SIGNKEYGEN signer1
+
+echo "--- Signing and Verifying a Message ---"
+SIGN_MESSAGE="This is a message to be signed."
+# Similar to above, we don't have GET_SIGN_PUBKEY. We will verify by name.
+
+echo "Signing message: '$SIGN_MESSAGE' with signer1's private key..."
+SIGNATURE=$($REDIS_CLI AGE SIGNNAME "$SIGN_MESSAGE" signer1)
+echo "Signature: $SIGNATURE"
+
+echo "Verifying signature with signer1's public key..."
+VERIFY_RESULT=$($REDIS_CLI AGE VERIFYNAME signer1 "$SIGN_MESSAGE" "$SIGNATURE")
+echo "Verification Result: $VERIFY_RESULT"
+
+
+# There is no DELETE_KEYPAIR command in the implementation
+echo "--- Cleaning up keys (manual in herodb) ---"
+# We would use DEL for age:key:alice, etc.
+$REDIS_CLI DEL age:key:alice
+$REDIS_CLI DEL age:privkey:alice
+$REDIS_CLI DEL age:key:bob
+$REDIS_CLI DEL age:privkey:bob
+$REDIS_CLI DEL age:signpub:signer1
+$REDIS_CLI DEL age:signpriv:signer1
+
+echo "--- Stopping herodb server ---"
+kill $SERVER_PID
+wait $SERVER_PID 2>/dev/null
+echo "Server stopped."
+
+echo "Bash demo complete."

examples/age_persist_demo.rs

@@ -0,0 +1,83 @@
+use std::io::{Read, Write};
+use std::net::TcpStream;
+
+// Minimal RESP helpers
+fn arr(parts: &[&str]) -> String {
+    let mut out = format!("*{}\r\n", parts.len());
+    for p in parts {
+        out.push_str(&format!("${}\r\n{}\r\n", p.len(), p));
+    }
+    out
+}
+fn read_reply(s: &mut TcpStream) -> String {
+    let mut buf = [0u8; 65536];
+    let n = s.read(&mut buf).unwrap();
+    String::from_utf8_lossy(&buf[..n]).to_string()
+}
+fn parse_two_bulk(reply: &str) -> Option<(String,String)> {
+    let mut lines = reply.split("\r\n");
+    if lines.next()? != "*2" { return None; }
+    let _n = lines.next()?;
+    let a = lines.next()?.to_string();
+    let _m = lines.next()?;
+    let b = lines.next()?.to_string();
+    Some((a,b))
+}
+fn parse_bulk(reply: &str) -> Option<String> {
+    let mut lines = reply.split("\r\n");
+    let hdr = lines.next()?;
+    if !hdr.starts_with('$') { return None; }
+    Some(lines.next()?.to_string())
+}
+fn parse_simple(reply: &str) -> Option<String> {
+    let mut lines = reply.split("\r\n");
+    let hdr = lines.next()?;
+    if !hdr.starts_with('+') { return None; }
+    Some(hdr[1..].to_string())
+}
+
+fn main() {
+    let mut args = std::env::args().skip(1);
+    let host = args.next().unwrap_or_else(|| "127.0.0.1".into());
+    let port = args.next().unwrap_or_else(|| "6379".into());
+    let addr = format!("{host}:{port}");
+    println!("Connecting to {addr}...");
+    let mut s = TcpStream::connect(addr).expect("connect");
+
+    // Generate & persist X25519 enc keys under name "alice"
+    s.write_all(arr(&["age","keygen","alice"]).as_bytes()).unwrap();
+    let (_alice_recip, _alice_ident) = parse_two_bulk(&read_reply(&mut s)).expect("gen enc");
+
+    // Generate & persist Ed25519 signing key under name "signer"
+    s.write_all(arr(&["age","signkeygen","signer"]).as_bytes()).unwrap();
+    let (_verify, _secret) = parse_two_bulk(&read_reply(&mut s)).expect("gen sign");
+
+    // Encrypt by name
+    let msg = "hello from persistent keys";
+    s.write_all(arr(&["age","encryptname","alice", msg]).as_bytes()).unwrap();
+    let ct_b64 = parse_bulk(&read_reply(&mut s)).expect("ct b64");
+    println!("ciphertext b64: {}", ct_b64);
+
+    // Decrypt by name
+    s.write_all(arr(&["age","decryptname","alice", &ct_b64]).as_bytes()).unwrap();
+    let pt = parse_bulk(&read_reply(&mut s)).expect("pt");
+    assert_eq!(pt, msg);
+    println!("decrypted ok");
+
+    // Sign by name
+    s.write_all(arr(&["age","signname","signer", msg]).as_bytes()).unwrap();
+    let sig_b64 = parse_bulk(&read_reply(&mut s)).expect("sig b64");
+
+    // Verify by name
+    s.write_all(arr(&["age","verifyname","signer", msg, &sig_b64]).as_bytes()).unwrap();
+    let ok = parse_simple(&read_reply(&mut s)).expect("verify");
+    assert_eq!(ok, "1");
+    println!("signature verified");
+
+    // List names
+    s.write_all(arr(&["age","list"]).as_bytes()).unwrap();
+    let list = read_reply(&mut s);
+    println!("LIST -> {list}");
+
+    println!("✔ persistent AGE workflow complete.");
+}

examples/simple_demo.sh

@@ -0,0 +1,186 @@
+#!/bin/bash
+
+# Simple HeroDB Demo - Basic Redis Commands
+# This script demonstrates basic Redis functionality that's currently implemented
+
+set -e  # Exit on any error
+
+# Configuration
+REDIS_HOST="localhost"
+REDIS_PORT="6381"
+REDIS_CLI="redis-cli -h $REDIS_HOST -p $REDIS_PORT"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+BLUE='\033[0;34m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+# Function to print colored output
+print_header() {
+    echo -e "${BLUE}=== $1 ===${NC}"
+}
+
+print_success() {
+    echo -e "${GREEN}✓ $1${NC}"
+}
+
+print_info() {
+    echo -e "${YELLOW}ℹ $1${NC}"
+}
+
+print_error() {
+    echo -e "${RED}✗ $1${NC}"
+}
+
+# Function to check if HeroDB is running
+check_herodb() {
+    print_info "Checking if HeroDB is running on port $REDIS_PORT..."
+    if ! $REDIS_CLI ping > /dev/null 2>&1; then
+        print_error "HeroDB is not running on port $REDIS_PORT"
+        print_info "Please start HeroDB with: cargo run -- --port $REDIS_PORT"
+        exit 1
+    fi
+    print_success "HeroDB is running and responding"
+}
+
+# Function to execute Redis command with error handling
+execute_cmd() {
+    local cmd="$1"
+    local description="$2"
+    
+    echo -e "${YELLOW}Command:${NC} $cmd"
+    if result=$($REDIS_CLI $cmd 2>&1); then
+        echo -e "${GREEN}Result:${NC} $result"
+        return 0
+    else
+        print_error "Failed: $description"
+        echo "Error: $result"
+        return 1
+    fi
+}
+
+# Main demo function
+main() {
+    clear
+    print_header "HeroDB Basic Functionality Demo"
+    echo "This demo shows basic Redis commands that are currently implemented"
+    echo "HeroDB runs on port $REDIS_PORT (instead of Redis default 6379)"
+    echo
+
+    # Check if HeroDB is running
+    check_herodb
+    echo
+
+    print_header "Step 1: Basic Key-Value Operations"
+    
+    execute_cmd "SET greeting 'Hello HeroDB!'" "Setting a simple key-value pair"
+    echo
+    execute_cmd "GET greeting" "Getting the value"
+    echo
+    execute_cmd "SET counter 42" "Setting a numeric value"
+    echo
+    execute_cmd "INCR counter" "Incrementing the counter"
+    echo
+    execute_cmd "GET counter" "Getting the incremented value"
+    echo
+
+    print_header "Step 2: Hash Operations"
+    
+    execute_cmd "HSET user:1 name 'John Doe' email 'john@example.com' age 30" "Setting hash fields"
+    echo
+    execute_cmd "HGET user:1 name" "Getting a specific field"
+    echo
+    execute_cmd "HGETALL user:1" "Getting all fields"
+    echo
+    execute_cmd "HLEN user:1" "Getting hash length"
+    echo
+
+    print_header "Step 3: List Operations"
+    
+    execute_cmd "LPUSH tasks 'Write code' 'Test code' 'Deploy code'" "Adding items to list"
+    echo
+    execute_cmd "LLEN tasks" "Getting list length"
+    echo
+    execute_cmd "LRANGE tasks 0 -1" "Getting all list items"
+    echo
+    execute_cmd "LPOP tasks" "Popping from left"
+    echo
+    execute_cmd "LRANGE tasks 0 -1" "Checking remaining items"
+    echo
+
+    print_header "Step 4: Key Management"
+    
+    execute_cmd "KEYS *" "Listing all keys"
+    echo
+    execute_cmd "EXISTS greeting" "Checking if key exists"
+    echo
+    execute_cmd "TYPE user:1" "Getting key type"
+    echo
+    execute_cmd "DBSIZE" "Getting database size"
+    echo
+
+    print_header "Step 5: Expiration"
+    
+    execute_cmd "SET temp_key 'temporary value'" "Setting temporary key"
+    echo
+    execute_cmd "EXPIRE temp_key 5" "Setting 5 second expiration"
+    echo
+    execute_cmd "TTL temp_key" "Checking time to live"
+    echo
+    print_info "Waiting 2 seconds..."
+    sleep 2
+    execute_cmd "TTL temp_key" "Checking TTL again"
+    echo
+
+    print_header "Step 6: Multiple Operations"
+    
+    execute_cmd "MSET key1 'value1' key2 'value2' key3 'value3'" "Setting multiple keys"
+    echo
+    execute_cmd "MGET key1 key2 key3" "Getting multiple values"
+    echo
+    execute_cmd "DEL key1 key2" "Deleting multiple keys"
+    echo
+    execute_cmd "EXISTS key1 key2 key3" "Checking existence of multiple keys"
+    echo
+
+    print_header "Step 7: Search Commands (Placeholder)"
+    print_info "Testing FT.CREATE command (currently returns placeholder response)"
+    
+    execute_cmd "FT.CREATE test_index SCHEMA title TEXT description TEXT" "Creating search index"
+    echo
+
+    print_header "Step 8: Server Information"
+    
+    execute_cmd "INFO" "Getting server information"
+    echo
+    execute_cmd "CONFIG GET dir" "Getting configuration"
+    echo
+
+    print_header "Step 9: Cleanup"
+    
+    execute_cmd "FLUSHDB" "Clearing database"
+    echo
+    execute_cmd "DBSIZE" "Confirming database is empty"
+    echo
+
+    print_header "Demo Summary"
+    echo "This demonstration showed:"
+    echo "• Basic key-value operations (GET, SET, INCR)"
+    echo "• Hash operations (HSET, HGET, HGETALL)"
+    echo "• List operations (LPUSH, LPOP, LRANGE)"
+    echo "• Key management (KEYS, EXISTS, TYPE, DEL)"
+    echo "• Expiration handling (EXPIRE, TTL)"
+    echo "• Multiple key operations (MSET, MGET)"
+    echo "• Server information commands"
+    echo
+    print_success "HeroDB basic functionality demo completed successfully!"
+    echo
+    print_info "Note: Full-text search (FT.*) commands are defined but not yet fully implemented"
+    print_info "To run HeroDB server: cargo run -- --port 6381"
+    print_info "To connect with redis-cli: redis-cli -h localhost -p 6381"
+}
+
+# Run the demo
+main "$@"