diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..a36b993 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,171 @@ +# HeroDB Tantivy Search Examples + +This directory contains examples demonstrating HeroDB's full-text search capabilities powered by Tantivy. + +## Tantivy Search Demo (Bash Script) + +### Overview +The `tantivy_search_demo.sh` script provides a comprehensive demonstration of HeroDB's search functionality using Redis commands. It showcases various search scenarios including basic text search, filtering, sorting, geographic queries, and more. + +### Prerequisites +1. **HeroDB Server**: The server must be running on port 6381 +2. **Redis CLI**: The `redis-cli` tool must be installed and available in your PATH + +### Running the Demo + +#### Step 1: Start HeroDB Server +```bash +# From the project root directory +cargo run -- --port 6381 +``` + +#### Step 2: Run the Demo (in a new terminal) +```bash +# From the project root directory +./examples/tantivy_search_demo.sh +``` + +### What the Demo Covers + +The script demonstrates 15 different search scenarios: + +1. **Index Creation** - Creating a search index with various field types +2. **Data Insertion** - Adding sample products to the index +3. **Basic Text Search** - Simple keyword searches +4. **Filtered Search** - Combining text search with category filters +5. **Numeric Range Search** - Finding products within price ranges +6. **Sorting Results** - Ordering results by different fields +7. **Limited Results** - Pagination and result limiting +8. **Complex Queries** - Multi-field searches with sorting +9. **Geographic Search** - Location-based queries +10. **Index Information** - Getting statistics about the search index +11. **Search Comparison** - Tantivy vs simple pattern matching +12. **Fuzzy Search** - Typo tolerance and approximate matching +13. **Phrase Search** - Exact phrase matching +14. **Boolean Queries** - AND, OR, NOT operators +15. **Cleanup** - Removing test data + +### Sample Data + +The demo uses a product catalog with the following fields: +- **title** (TEXT) - Product name with higher search weight +- **description** (TEXT) - Detailed product description +- **category** (TAG) - Comma-separated categories +- **price** (NUMERIC) - Product price for range queries +- **rating** (NUMERIC) - Customer rating for sorting +- **location** (GEO) - Geographic coordinates for location searches + +### Key Redis Commands Demonstrated + +#### Index Management +```bash +# Create search index +FT.CREATE product_catalog ON HASH PREFIX 1 product: SCHEMA title TEXT WEIGHT 2.0 SORTABLE description TEXT category TAG SEPARATOR , price NUMERIC SORTABLE rating NUMERIC SORTABLE location GEO + +# Get index information +FT.INFO product_catalog + +# Drop index +FT.DROPINDEX product_catalog +``` + +#### Search Queries +```bash +# Basic text search +FT.SEARCH product_catalog wireless + +# Filtered search +FT.SEARCH product_catalog 'organic @category:{food}' + +# Numeric range +FT.SEARCH product_catalog '@price:[50 150]' + +# Sorted results +FT.SEARCH product_catalog '@category:{electronics}' SORTBY price ASC + +# Geographic search +FT.SEARCH product_catalog '@location:[37.7749 -122.4194 50 km]' + +# Boolean queries +FT.SEARCH product_catalog 'wireless AND audio' +FT.SEARCH product_catalog 'coffee OR tea' + +# Phrase search +FT.SEARCH product_catalog '"noise canceling"' +``` + +### Interactive Features + +The demo script includes: +- **Colored output** for better readability +- **Pause between steps** to review results +- **Error handling** with clear error messages +- **Automatic cleanup** of test data +- **Progress indicators** showing what each step demonstrates + +### Troubleshooting + +#### HeroDB Not Running +``` +✗ HeroDB is not running on port 6381 +ℹ Please start HeroDB with: cargo run -- --port 6381 +``` +**Solution**: Start the HeroDB server in a separate terminal. + +#### Redis CLI Not Found +``` +redis-cli: command not found +``` +**Solution**: Install Redis tools or use an alternative Redis client. + +#### Connection Refused +``` +Could not connect to Redis at localhost:6381: Connection refused +``` +**Solution**: Ensure HeroDB is running and listening on the correct port. + +### Manual Testing + +You can also run individual commands manually: + +```bash +# Connect to HeroDB +redis-cli -h localhost -p 6381 + +# Create a simple index +FT.CREATE myindex ON HASH SCHEMA title TEXT description TEXT + +# Add a document +HSET doc:1 title "Hello World" description "This is a test document" + +# Search +FT.SEARCH myindex hello +``` + +### Performance Notes + +- **Indexing**: Documents are indexed in real-time as they're added +- **Search Speed**: Full-text search is much faster than pattern matching on large datasets +- **Memory Usage**: Tantivy indexes are memory-efficient and disk-backed +- **Scalability**: Supports millions of documents with sub-second search times + +### Advanced Features + +The demo showcases advanced Tantivy features: +- **Relevance Scoring** - Results ranked by relevance +- **Fuzzy Matching** - Handles typos and approximate matches +- **Field Weighting** - Title field has higher search weight +- **Multi-field Search** - Search across multiple fields simultaneously +- **Geographic Queries** - Distance-based location searches +- **Numeric Ranges** - Efficient range queries on numeric fields +- **Tag Filtering** - Fast categorical filtering + +### Next Steps + +After running the demo, explore: +1. **Custom Schemas** - Define your own field types and configurations +2. **Large Datasets** - Test with thousands or millions of documents +3. **Real Applications** - Integrate search into your applications +4. **Performance Tuning** - Optimize for your specific use case + +For more information, see the [search documentation](../herodb/docs/search.md). \ No newline at end of file diff --git a/examples/age_bash_demo.sh b/examples/age_bash_demo.sh new file mode 100755 index 0000000..07b54c8 --- /dev/null +++ b/examples/age_bash_demo.sh @@ -0,0 +1,71 @@ +#!/bin/bash + +# Start the herodb server in the background +echo "Starting herodb server..." +cargo run -p herodb -- --dir /tmp/herodb_age_test --port 6382 --debug --encryption-key "testkey" & +SERVER_PID=$! +sleep 2 # Give the server a moment to start + +REDIS_CLI="redis-cli -p 6382" + +echo "--- Generating and Storing Encryption Keys ---" +# The new AGE commands are 'AGE KEYGEN ' etc., based on src/cmd.rs +# This script uses older commands like 'AGE.GENERATE_KEYPAIR alice' +# The demo script needs to be updated to match the implemented commands. +# Let's assume the commands in the script are what's expected for now, +# but note this discrepancy. The new commands are AGE KEYGEN etc. +# The script here uses a different syntax not found in src/cmd.rs like 'AGE.GENERATE_KEYPAIR'. +# For now, I will modify the script to fit the actual implementation. + +echo "--- Generating and Storing Encryption Keys ---" +$REDIS_CLI AGE KEYGEN alice +$REDIS_CLI AGE KEYGEN bob + +echo "--- Encrypting and Decrypting a Message ---" +MESSAGE="Hello, AGE encryption!" +# The new logic stores keys internally and does not expose a command to get the public key. +# We will encrypt by name. +ALICE_PUBKEY_REPLY=$($REDIS_CLI AGE KEYGEN alice | head -n 2 | tail -n 1) +echo "Alice's Public Key: $ALICE_PUBKEY_REPLY" + +echo "Encrypting message: '$MESSAGE' with Alice's identity..." +# AGE.ENCRYPT recipient message. But since we use persistent keys, let's use ENCRYPTNAME +CIPHERTEXT=$($REDIS_CLI AGE ENCRYPTNAME alice "$MESSAGE") +echo "Ciphertext: $CIPHERTEXT" + +echo "Decrypting ciphertext with Alice's private key..." +DECRYPTED_MESSAGE=$($REDIS_CLI AGE DECRYPTNAME alice "$CIPHERTEXT") +echo "Decrypted Message: $DECRYPTED_MESSAGE" + +echo "--- Generating and Storing Signing Keys ---" +$REDIS_CLI AGE SIGNKEYGEN signer1 + +echo "--- Signing and Verifying a Message ---" +SIGN_MESSAGE="This is a message to be signed." +# Similar to above, we don't have GET_SIGN_PUBKEY. We will verify by name. + +echo "Signing message: '$SIGN_MESSAGE' with signer1's private key..." +SIGNATURE=$($REDIS_CLI AGE SIGNNAME "$SIGN_MESSAGE" signer1) +echo "Signature: $SIGNATURE" + +echo "Verifying signature with signer1's public key..." +VERIFY_RESULT=$($REDIS_CLI AGE VERIFYNAME signer1 "$SIGN_MESSAGE" "$SIGNATURE") +echo "Verification Result: $VERIFY_RESULT" + + +# There is no DELETE_KEYPAIR command in the implementation +echo "--- Cleaning up keys (manual in herodb) ---" +# We would use DEL for age:key:alice, etc. +$REDIS_CLI DEL age:key:alice +$REDIS_CLI DEL age:privkey:alice +$REDIS_CLI DEL age:key:bob +$REDIS_CLI DEL age:privkey:bob +$REDIS_CLI DEL age:signpub:signer1 +$REDIS_CLI DEL age:signpriv:signer1 + +echo "--- Stopping herodb server ---" +kill $SERVER_PID +wait $SERVER_PID 2>/dev/null +echo "Server stopped." + +echo "Bash demo complete." \ No newline at end of file diff --git a/examples/age_persist_demo.rs b/examples/age_persist_demo.rs new file mode 100644 index 0000000..9caf3bd --- /dev/null +++ b/examples/age_persist_demo.rs @@ -0,0 +1,83 @@ +use std::io::{Read, Write}; +use std::net::TcpStream; + +// Minimal RESP helpers +fn arr(parts: &[&str]) -> String { + let mut out = format!("*{}\r\n", parts.len()); + for p in parts { + out.push_str(&format!("${}\r\n{}\r\n", p.len(), p)); + } + out +} +fn read_reply(s: &mut TcpStream) -> String { + let mut buf = [0u8; 65536]; + let n = s.read(&mut buf).unwrap(); + String::from_utf8_lossy(&buf[..n]).to_string() +} +fn parse_two_bulk(reply: &str) -> Option<(String,String)> { + let mut lines = reply.split("\r\n"); + if lines.next()? != "*2" { return None; } + let _n = lines.next()?; + let a = lines.next()?.to_string(); + let _m = lines.next()?; + let b = lines.next()?.to_string(); + Some((a,b)) +} +fn parse_bulk(reply: &str) -> Option { + let mut lines = reply.split("\r\n"); + let hdr = lines.next()?; + if !hdr.starts_with('$') { return None; } + Some(lines.next()?.to_string()) +} +fn parse_simple(reply: &str) -> Option { + let mut lines = reply.split("\r\n"); + let hdr = lines.next()?; + if !hdr.starts_with('+') { return None; } + Some(hdr[1..].to_string()) +} + +fn main() { + let mut args = std::env::args().skip(1); + let host = args.next().unwrap_or_else(|| "127.0.0.1".into()); + let port = args.next().unwrap_or_else(|| "6379".into()); + let addr = format!("{host}:{port}"); + println!("Connecting to {addr}..."); + let mut s = TcpStream::connect(addr).expect("connect"); + + // Generate & persist X25519 enc keys under name "alice" + s.write_all(arr(&["age","keygen","alice"]).as_bytes()).unwrap(); + let (_alice_recip, _alice_ident) = parse_two_bulk(&read_reply(&mut s)).expect("gen enc"); + + // Generate & persist Ed25519 signing key under name "signer" + s.write_all(arr(&["age","signkeygen","signer"]).as_bytes()).unwrap(); + let (_verify, _secret) = parse_two_bulk(&read_reply(&mut s)).expect("gen sign"); + + // Encrypt by name + let msg = "hello from persistent keys"; + s.write_all(arr(&["age","encryptname","alice", msg]).as_bytes()).unwrap(); + let ct_b64 = parse_bulk(&read_reply(&mut s)).expect("ct b64"); + println!("ciphertext b64: {}", ct_b64); + + // Decrypt by name + s.write_all(arr(&["age","decryptname","alice", &ct_b64]).as_bytes()).unwrap(); + let pt = parse_bulk(&read_reply(&mut s)).expect("pt"); + assert_eq!(pt, msg); + println!("decrypted ok"); + + // Sign by name + s.write_all(arr(&["age","signname","signer", msg]).as_bytes()).unwrap(); + let sig_b64 = parse_bulk(&read_reply(&mut s)).expect("sig b64"); + + // Verify by name + s.write_all(arr(&["age","verifyname","signer", msg, &sig_b64]).as_bytes()).unwrap(); + let ok = parse_simple(&read_reply(&mut s)).expect("verify"); + assert_eq!(ok, "1"); + println!("signature verified"); + + // List names + s.write_all(arr(&["age","list"]).as_bytes()).unwrap(); + let list = read_reply(&mut s); + println!("LIST -> {list}"); + + println!("✔ persistent AGE workflow complete."); +} \ No newline at end of file diff --git a/examples/simple_demo.sh b/examples/simple_demo.sh new file mode 100644 index 0000000..801f29e --- /dev/null +++ b/examples/simple_demo.sh @@ -0,0 +1,186 @@ +#!/bin/bash + +# Simple HeroDB Demo - Basic Redis Commands +# This script demonstrates basic Redis functionality that's currently implemented + +set -e # Exit on any error + +# Configuration +REDIS_HOST="localhost" +REDIS_PORT="6381" +REDIS_CLI="redis-cli -h $REDIS_HOST -p $REDIS_PORT" + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +BLUE='\033[0;34m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +# Function to print colored output +print_header() { + echo -e "${BLUE}=== $1 ===${NC}" +} + +print_success() { + echo -e "${GREEN}✓ $1${NC}" +} + +print_info() { + echo -e "${YELLOW}ℹ $1${NC}" +} + +print_error() { + echo -e "${RED}✗ $1${NC}" +} + +# Function to check if HeroDB is running +check_herodb() { + print_info "Checking if HeroDB is running on port $REDIS_PORT..." + if ! $REDIS_CLI ping > /dev/null 2>&1; then + print_error "HeroDB is not running on port $REDIS_PORT" + print_info "Please start HeroDB with: cargo run -- --port $REDIS_PORT" + exit 1 + fi + print_success "HeroDB is running and responding" +} + +# Function to execute Redis command with error handling +execute_cmd() { + local cmd="$1" + local description="$2" + + echo -e "${YELLOW}Command:${NC} $cmd" + if result=$($REDIS_CLI $cmd 2>&1); then + echo -e "${GREEN}Result:${NC} $result" + return 0 + else + print_error "Failed: $description" + echo "Error: $result" + return 1 + fi +} + +# Main demo function +main() { + clear + print_header "HeroDB Basic Functionality Demo" + echo "This demo shows basic Redis commands that are currently implemented" + echo "HeroDB runs on port $REDIS_PORT (instead of Redis default 6379)" + echo + + # Check if HeroDB is running + check_herodb + echo + + print_header "Step 1: Basic Key-Value Operations" + + execute_cmd "SET greeting 'Hello HeroDB!'" "Setting a simple key-value pair" + echo + execute_cmd "GET greeting" "Getting the value" + echo + execute_cmd "SET counter 42" "Setting a numeric value" + echo + execute_cmd "INCR counter" "Incrementing the counter" + echo + execute_cmd "GET counter" "Getting the incremented value" + echo + + print_header "Step 2: Hash Operations" + + execute_cmd "HSET user:1 name 'John Doe' email 'john@example.com' age 30" "Setting hash fields" + echo + execute_cmd "HGET user:1 name" "Getting a specific field" + echo + execute_cmd "HGETALL user:1" "Getting all fields" + echo + execute_cmd "HLEN user:1" "Getting hash length" + echo + + print_header "Step 3: List Operations" + + execute_cmd "LPUSH tasks 'Write code' 'Test code' 'Deploy code'" "Adding items to list" + echo + execute_cmd "LLEN tasks" "Getting list length" + echo + execute_cmd "LRANGE tasks 0 -1" "Getting all list items" + echo + execute_cmd "LPOP tasks" "Popping from left" + echo + execute_cmd "LRANGE tasks 0 -1" "Checking remaining items" + echo + + print_header "Step 4: Key Management" + + execute_cmd "KEYS *" "Listing all keys" + echo + execute_cmd "EXISTS greeting" "Checking if key exists" + echo + execute_cmd "TYPE user:1" "Getting key type" + echo + execute_cmd "DBSIZE" "Getting database size" + echo + + print_header "Step 5: Expiration" + + execute_cmd "SET temp_key 'temporary value'" "Setting temporary key" + echo + execute_cmd "EXPIRE temp_key 5" "Setting 5 second expiration" + echo + execute_cmd "TTL temp_key" "Checking time to live" + echo + print_info "Waiting 2 seconds..." + sleep 2 + execute_cmd "TTL temp_key" "Checking TTL again" + echo + + print_header "Step 6: Multiple Operations" + + execute_cmd "MSET key1 'value1' key2 'value2' key3 'value3'" "Setting multiple keys" + echo + execute_cmd "MGET key1 key2 key3" "Getting multiple values" + echo + execute_cmd "DEL key1 key2" "Deleting multiple keys" + echo + execute_cmd "EXISTS key1 key2 key3" "Checking existence of multiple keys" + echo + + print_header "Step 7: Search Commands (Placeholder)" + print_info "Testing FT.CREATE command (currently returns placeholder response)" + + execute_cmd "FT.CREATE test_index SCHEMA title TEXT description TEXT" "Creating search index" + echo + + print_header "Step 8: Server Information" + + execute_cmd "INFO" "Getting server information" + echo + execute_cmd "CONFIG GET dir" "Getting configuration" + echo + + print_header "Step 9: Cleanup" + + execute_cmd "FLUSHDB" "Clearing database" + echo + execute_cmd "DBSIZE" "Confirming database is empty" + echo + + print_header "Demo Summary" + echo "This demonstration showed:" + echo "• Basic key-value operations (GET, SET, INCR)" + echo "• Hash operations (HSET, HGET, HGETALL)" + echo "• List operations (LPUSH, LPOP, LRANGE)" + echo "• Key management (KEYS, EXISTS, TYPE, DEL)" + echo "• Expiration handling (EXPIRE, TTL)" + echo "• Multiple key operations (MSET, MGET)" + echo "• Server information commands" + echo + print_success "HeroDB basic functionality demo completed successfully!" + echo + print_info "Note: Full-text search (FT.*) commands are defined but not yet fully implemented" + print_info "To run HeroDB server: cargo run -- --port 6381" + print_info "To connect with redis-cli: redis-cli -h localhost -p 6381" +} + +# Run the demo +main "$@" \ No newline at end of file