113 lines
5.7 KiB
Markdown
113 lines
5.7 KiB
Markdown
# HeroDB
|
|
|
|
HeroDB is a Redis-compatible database built with Rust, offering a flexible and secure storage solution. It supports two primary storage backends: `redb` (default) and `sled`, both with full encryption capabilities. HeroDB aims to provide a robust and performant key-value store with advanced features like data-at-rest encryption, hash operations, list operations, and cursor-based scanning.
|
|
|
|
## Purpose
|
|
|
|
The main purpose of HeroDB is to offer a lightweight, embeddable, and Redis-compatible database that prioritizes data security through transparent encryption. It's designed for applications that require fast, reliable data storage with the option for strong cryptographic protection, without the overhead of a full-fledged Redis server.
|
|
|
|
## Features
|
|
|
|
- **Redis Compatibility**: Supports a subset of Redis commands over RESP (Redis Serialization Protocol) via TCP.
|
|
- **Dual Backend Support**:
|
|
- `redb` (default): Optimized for concurrent access and high-throughput scenarios.
|
|
- `sled`: A lock-free, log-structured database, excellent for specific workloads.
|
|
- **Data-at-Rest Encryption**: Transparent encryption for both backends using the `age` encryption library.
|
|
- **Key-Value Operations**: Full support for basic string, hash, and list operations.
|
|
- **Expiration**: Time-to-live (TTL) functionality for keys.
|
|
- **Scanning**: Cursor-based iteration for keys and hash fields (`SCAN`, `HSCAN`).
|
|
- **AGE Cryptography Commands**: HeroDB-specific extensions for cryptographic operations.
|
|
- **Symmetric Encryption**: Stateless symmetric encryption using XChaCha20-Poly1305.
|
|
- **Admin Database 0**: Centralized control for database management, access control, and per-database encryption.
|
|
|
|
## Quick Start
|
|
|
|
### Building HeroDB
|
|
|
|
To build HeroDB, navigate to the project root and run:
|
|
|
|
```bash
|
|
cargo build --release
|
|
```
|
|
|
|
### Running HeroDB
|
|
|
|
Launch HeroDB with the required `--admin-secret` flag, which encrypts the admin database (DB 0) and authorizes admin access. Optional flags include `--dir` for the database directory, `--port` for the TCP port (default 6379), `--sled` for the sled backend, `--enable-rpc` to start the HTTP JSON-RPC server on a TCP port, `--enable-rpc-ipc` to start JSON-RPC over a Unix Domain Socket (non-HTTP), and `--rpc-ipc-path <path>` to specify the socket path (default: `/tmp/herodb.ipc`).
|
|
|
|
Example:
|
|
```bash
|
|
./target/release/herodb --dir /tmp/herodb --admin-secret myadminsecret --port 6379 --enable-rpc
|
|
```
|
|
|
|
To enable JSON-RPC over a Unix Domain Socket at `/tmp/herodb.sock`:
|
|
```bash
|
|
./target/release/herodb --dir /tmp/herodb --admin-secret myadminsecret --enable-rpc-ipc --rpc-ipc-path /tmp/herodb.sock
|
|
```
|
|
|
|
Test the IPC endpoint interactively with socat:
|
|
```bash
|
|
sudo socat -d -d -t 5 - UNIX-CONNECT:/tmp/herodb.sock
|
|
```
|
|
Then paste a framed JSON-RPC request (Content-Length header, blank line, then JSON body). Example:
|
|
```
|
|
Content-Length: 73
|
|
|
|
{"jsonrpc":"2.0","method":"hero_listDatabases","params":[],"id":3}
|
|
```
|
|
|
|
For a one-liner that auto-computes Content-Length and pretty-prints the JSON response, see docs/rpc_examples.md.
|
|
|
|
For detailed launch options, see [Basics](docs/basics.md).
|
|
|
|
## Usage with Redis Clients
|
|
|
|
HeroDB can be interacted with using any standard Redis client, such as `redis-cli`, `redis-py` (Python), or `ioredis` (Node.js).
|
|
|
|
### Example with `redis-cli`
|
|
|
|
Connections start with no database selected. You must SELECT a database first.
|
|
|
|
- To work in the admin database (DB 0), authenticate with the admin secret:
|
|
```bash
|
|
redis-cli -p 6379 SELECT 0 KEY myadminsecret
|
|
redis-cli -p 6379 SET mykey "Hello from HeroDB!"
|
|
redis-cli -p 6379 GET mykey
|
|
# → "Hello from HeroDB!"
|
|
```
|
|
|
|
- To use a user database, first create one via the JSON-RPC API (see docs/rpc_examples.md), then select it:
|
|
```bash
|
|
# Suppose RPC created database id 1
|
|
redis-cli -p 6379 SELECT 1
|
|
redis-cli -p 6379 HSET user:1 name "Alice" age "30"
|
|
redis-cli -p 6379 HGET user:1 name
|
|
# → "Alice"
|
|
redis-cli -p 6379 SCAN 0 MATCH user:* COUNT 10
|
|
```
|
|
|
|
## Cryptography
|
|
|
|
HeroDB supports asymmetric encryption/signatures via AGE commands (X25519 for encryption, Ed25519 for signatures) in stateless or key-managed modes, and symmetric encryption via SYM commands. Keys are persisted in the admin database (DB 0) for managed modes.
|
|
|
|
For details, see [AGE Cryptography](docs/age.md) and [Basics](docs/basics.md).
|
|
|
|
## Database Management
|
|
|
|
Databases are managed via JSON-RPC API, with metadata stored in the encrypted admin database (DB 0). Databases are public by default upon creation; use RPC to set them private, requiring access keys for SELECT operations (read or readwrite based on permissions). This includes per-database encryption keys, access control, and lifecycle management.
|
|
|
|
For examples, see [JSON-RPC Examples](docs/rpc_examples.md) and [Admin DB 0 Model](docs/admin.md).
|
|
|
|
## Documentation
|
|
|
|
For more detailed information on commands, features, and advanced usage, please refer to the documentation:
|
|
|
|
- [Basics](docs/basics.md) - Launch options, symmetric encryption, and basic usage
|
|
- [Supported Commands](docs/cmds.md) - Complete Redis command reference and backend comparison
|
|
- [AGE Cryptography](docs/age.md) - Asymmetric encryption and digital signatures
|
|
- [Admin DB 0 Model](docs/admin.md) - Database management, access control, and per-database encryption
|
|
- [JSON-RPC Examples](docs/rpc_examples.md) - Management API examples
|
|
- [Full-Text Search](docs/search.md) - Tantivy-powered search capabilities
|
|
- [Tantivy Backend](docs/tantivy.md) - Tantivy as a dedicated database backend
|
|
- [Lance Vector Store](docs/lance.md) - Vector embeddings and semantic search
|
|
- [Lance Text and Images Example](docs/lancedb_text_and_images_example.md) - End-to-end vector search examples
|
|
- [Local Embedder Tutorial](docs/local_embedder_full_example.md) - Complete embedding models tutorial |