15 Commits

Author SHA1 Message Date
Maxime Van Hees
052cf2ecdb update RPC docs 2025-09-10 16:34:01 +02:00
Maxime Van Hees
e5b844deee bump version of jsonrpsee to newest + add rpc.discover endpoint for openrpc spec 2025-09-10 16:16:10 +02:00
Maxime Van Hees
764fcb68fa update documentation with info about RPC server 2025-09-10 11:59:31 +02:00
Maxime Van Hees
271c6cb0ae fixed connection handling issue 2025-09-10 11:53:01 +02:00
Maxime Van Hees
e84f7b7e3b working created DB over RPC 2025-09-09 17:37:18 +02:00
Maxime Van Hees
7e5da9c6eb WIP2 2025-09-09 17:20:12 +02:00
Maxime Van Hees
bd77a7db48 WIP1 2025-09-09 16:31:06 +02:00
Maxime Van Hees
d931770e90 Fix test suite + update Cargo.toml file 2025-09-09 16:04:31 +02:00
Timur Gordon
a87ec4dbb5 add readme 2025-08-27 15:39:59 +02:00
Maxime Van Hees
58cb1e8d5e fixed test not running due to misaligned path 2025-08-22 16:54:43 +02:00
d3d92819cf ... 2025-08-22 16:26:04 +02:00
4fd48f8b0d ... 2025-08-22 16:16:26 +02:00
4bedf71c2d Update herodb/instructions/age_usage.md 2025-08-22 14:02:58 +00:00
b9987a027b Merge pull request 'clean workspace: remove supervisor packe + add instructions about how AGE redis commands work' (#2) from blpop into main
Reviewed-on: #2
2025-08-22 12:25:59 +00:00
f22a25f5a1 Merge pull request 'BLPOP + COMMAND + MGET/MSET + DEL/EXISTS + EXPIRE/PEXPIRE/PERSIST + HINCRBY/HINCRBYFLOAT + BRPOP + DBSIZE + EXPIREAT/PEXIREAT implementations' (#1) from blpop into main
Reviewed-on: #1
2025-08-22 11:41:37 +00:00
28 changed files with 2856 additions and 154 deletions

926
Cargo.lock generated

File diff suppressed because it is too large Load Diff

91
README.md Normal file
View File

@@ -0,0 +1,91 @@
# HeroDB
Redis-compatible database server with encryption and AGE cryptographic operations.
## Features
- Redis protocol compatibility
- String, hash, and list data types
- Key expiration and persistence
- Database encryption with ChaCha20-Poly1305
- AGE encryption/decryption operations
- Digital signatures with Ed25519
- Persistent storage using redb
## Installation
```bash
cargo build --release
```
## Usage
```bash
./target/release/herodb --dir /path/to/db --port 6379
```
## RPC Server
HeroDB includes an optional JSON-RPC 2.0 management server for database administration tasks. Enable it with the `--enable-rpc` flag and specify the port with `--rpc-port` (default: 8080).
For a complete list of available RPC commands and usage examples, see [RPC_COMMANDS.md](RPC_COMMANDS.md).
### Options
- `--dir`: Database directory (required)
- `--port`: Server port (default: 6379)
- `--debug`: Enable debug logging
- `--encrypt`: Enable database encryption
- `--encryption-key`: Master encryption key for encrypted databases
- `--enable-rpc`: Enable RPC management server
- `--rpc-port`: RPC server port (default: 8080)
### Examples
```bash
# Basic server
herodb --dir ./data
# Encrypted database
herodb --dir ./data --encrypt --encryption-key "your-key"
# Custom port with debug
herodb --dir ./data --port 7000 --debug
```
## Redis Commands
Supports standard Redis commands including:
- **Strings**: GET, SET, MGET, MSET, INCR, DEL
- **Hashes**: HGET, HSET, HGETALL, HDEL, HEXISTS
- **Lists**: LPUSH, RPUSH, LPOP, RPOP, LLEN, LRANGE
- **Keys**: KEYS, SCAN, EXISTS, EXPIRE, TTL
- **Transactions**: MULTI, EXEC, DISCARD
- **Server**: PING, ECHO, INFO, CONFIG
## AGE Commands
Extended commands for cryptographic operations:
- **Key Generation**: `AGE GENENC`, `AGE GENSIGN`, `AGE KEYGEN`
- **Encryption**: `AGE ENCRYPT`, `AGE DECRYPT`, `AGE ENCRYPTNAME`
- **Signing**: `AGE SIGN`, `AGE VERIFY`, `AGE SIGNNAME`
- **Management**: `AGE LIST`
## Client Usage
Connect using any Redis client:
```bash
redis-cli -p 6379 SET key value
redis-cli -p 6379 GET key
redis-cli -p 6379 AGE GENENC
```
## Architecture
- **Storage**: redb embedded database
- **Protocol**: Redis RESP protocol over TCP
- **Encryption**: ChaCha20-Poly1305 for data, AGE for operations
- **Concurrency**: Tokio async runtime

93
RPC_COMMANDS.md Normal file
View File

@@ -0,0 +1,93 @@
# HeroDB RPC Commands
HeroDB provides a JSON-RPC 2.0 interface for database management operations. The RPC server runs on a separate port (default 8080) and can be enabled with the `--enable-rpc` flag.
All RPC methods are prefixed with the namespace `herodb`. With the exception fo the `rpc.discover` call (using the `rpc` namespace), which returns the OpenRPC spec.
## Available Commands
### herodb_listDatabases
Lists all database indices that exist.
**Example:**
```bash
curl -X POST http://localhost:8080 \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "herodb_listDatabases", "id": 1}'
```
### herodb_createDatabase
Creates a new database at the specified index.
**Parameters:**
- `db_index` (number): Database index to create
**Example:**
```bash
curl -X POST http://localhost:8080 \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "herodb_createDatabase", "params": [1], "id": 1}'
```
### herodb_getDatabaseInfo
Retrieves detailed information about a specific database.
**Parameters:**
- `db_index` (number): Database index
**Example:**
```bash
curl -X POST http://localhost:8080 \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "herodb_getDatabaseInfo", "params": [0], "id": 1}'
```
### herodb_configureDatabase
Configures an existing database with specific settings.
**Parameters:**
- `db_index` (number): Database index
- `config` (object): Configuration object
**Example:**
```bash
curl -X POST http://localhost:8080 \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "herodb_configureDatabase", "params": [0, {"name": "test", "max_size": 1048576}], "id": 1}'
```
### herodb_setDatabaseEncryption
Sets encryption for a specific database index.
**Parameters:**
- `db_index` (number): Database index
- `encryption_key` (string): Encryption key
**Example:**
```bash
curl -X POST http://localhost:8080 \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "herodb_setDatabaseEncryption", "params": [10, "my-secret-key"], "id": 1}'
```
### herodb_deleteDatabase
Deletes a database and its files.
**Parameters:**
- `db_index` (number): Database index to delete
**Example:**
```bash
curl -X POST http://localhost:8080 \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "herodb_deleteDatabase", "params": [1], "id": 1}'
```
### herodb_getServerStats
Retrieves server statistics.
**Example:**
```bash
curl -X POST http://localhost:8080 \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "herodb_getServerStats", "id": 1}'

View File

@@ -1,8 +1,8 @@
[package]
name = "herodb"
version = "0.0.1"
authors = ["Pin Fang <fpfangpin@hotmail.com>"]
edition = "2021"
authors = ["ThreeFold Tech"]
edition = "2024"
[dependencies]
anyhow = "1.0.59"
@@ -23,6 +23,7 @@ age = "0.10"
secrecy = "0.8"
ed25519-dalek = "2"
base64 = "0.22"
jsonrpsee = { version = "0.26", features = ["http-client", "ws-client", "server", "macros"] }
[dev-dependencies]
redis = { version = "0.24", features = ["aio", "tokio-comp"] }

View File

@@ -16,33 +16,40 @@ Note: Database-at-rest encryption flags in the test harness are unrelated to AGE
## Quick start
Assuming the server is running on localhost on some PORT:
Assuming the server is running on localhost on some $PORT:
```bash
~/code/git.ourworld.tf/herocode/herodb/herodb/build.sh
~/code/git.ourworld.tf/herocode/herodb/target/release/herodb --dir /tmp/data --debug --$PORT 6381 --encryption-key 1234 --encrypt
```
```bash
export PORT=6381
# Generate an ephemeral keypair and encrypt/decrypt a message (stateless mode)
redis-cli -p PORT AGE GENENC
redis-cli -p $PORT AGE GENENC
# → returns an array: [recipient, identity]
redis-cli -p PORT AGE ENCRYPT <recipient> "hello world"
redis-cli -p $PORT AGE ENCRYPT <recipient> "hello world"
# → returns ciphertext (base64 in a bulk string)
redis-cli -p PORT AGE DECRYPT <identity> <ciphertext_b64>
redis-cli -p $PORT AGE DECRYPT <identity> <ciphertext_b64>
# → returns "hello world"
```
For keymanaged mode, generate a named key once and reference it by name afterwards:
```bash
redis-cli -p PORT AGE KEYGEN app1
redis-cli -p $PORT AGE KEYGEN app1
# → persists encryption keypair under name "app1"
redis-cli -p PORT AGE ENCRYPTNAME app1 "hello"
redis-cli -p PORT AGE DECRYPTNAME app1 <ciphertext_b64>
redis-cli -p $PORT AGE ENCRYPTNAME app1 "hello"
redis-cli -p $PORT AGE DECRYPTNAME app1 <ciphertext_b64>
```
## Stateless AGE (ephemeral)
Characteristics
- No serverside storage of keys.
- You pass the actual key material with every call.
- Not listable via AGE LIST.
@@ -53,35 +60,39 @@ Commands and examples
```bash
# Generate an ephemeral encryption keypair
redis-cli -p PORT AGE GENENC
redis-cli -p $PORT AGE GENENC
# Example output (abridged):
# 1) "age1qz..." # recipient (public)
# 2) "AGE-SECRET-KEY-1..." # identity (secret)
# 1) "age1qz..." # recipient (public key) = can be used by others e.g. to verify what I sign
# 2) "AGE-SECRET-KEY-1..." # identity (secret) = is like my private, cannot lose this one
# Encrypt with the recipient
redis-cli -p PORT AGE ENCRYPT "age1qz..." "hello world"
# → returns bulk string payload: base64 ciphertext
# Encrypt with the recipient public key
redis-cli -p $PORT AGE ENCRYPT "age1qz..." "hello world"
# Decrypt with the identity (secret)
redis-cli -p PORT AGE DECRYPT "AGE-SECRET-KEY-1..." "<ciphertext_b64>"
# → returns bulk string payload: base64 ciphertext (encrypted content)
# Decrypt with the identity (secret) in other words your private key
redis-cli -p $PORT AGE DECRYPT "AGE-SECRET-KEY-1..." "<ciphertext_b64>"
# → "hello world"
```
2) Ephemeral signing keys
> ? is this same as my private key
```bash
# Generate an ephemeral signing keypair
redis-cli -p PORT AGE GENSIGN
redis-cli -p $PORT AGE GENSIGN
# Example output:
# 1) "<verify_pub_b64>"
# 2) "<sign_secret_b64>"
# Sign a message with the secret
redis-cli -p PORT AGE SIGN "<sign_secret_b64>" "msg"
redis-cli -p $PORT AGE SIGN "<sign_secret_b64>" "msg"
# → returns "<signature_b64>"
# Verify with the public key
redis-cli -p PORT AGE VERIFY "<verify_pub_b64>" "msg" "<signature_b64>"
redis-cli -p $PORT AGE VERIFY "<verify_pub_b64>" "msg" "<signature_b64>"
# → 1 (valid) or 0 (invalid)
```
@@ -105,15 +116,17 @@ Commands and examples
```bash
# Create/persist a named encryption keypair
redis-cli -p PORT AGE KEYGEN app1
redis-cli -p $PORT AGE KEYGEN app1
# → returns [recipient, identity] but also stores them under name "app1"
> TODO: should not return identity (security, but there can be separate function to export it e.g. AGE EXPORTKEY app1)
# Encrypt using the stored public key
redis-cli -p PORT AGE ENCRYPTNAME app1 "hello"
redis-cli -p $PORT AGE ENCRYPTNAME app1 "hello"
# → returns bulk string payload: base64 ciphertext
# Decrypt using the stored secret
redis-cli -p PORT AGE DECRYPTNAME app1 "<ciphertext_b64>"
redis-cli -p $PORT AGE DECRYPTNAME app1 "<ciphertext_b64>"
# → "hello"
```
@@ -121,22 +134,24 @@ redis-cli -p PORT AGE DECRYPTNAME app1 "<ciphertext_b64>"
```bash
# Create/persist a named signing keypair
redis-cli -p PORT AGE SIGNKEYGEN app1
redis-cli -p $PORT AGE SIGNKEYGEN app1
# → returns [verify_pub_b64, sign_secret_b64] and stores under name "app1"
> TODO: should not return sign_secret_b64 (for security, but there can be separate function to export it e.g. AGE EXPORTSIGNKEY app1)
# Sign using the stored secret
redis-cli -p PORT AGE SIGNNAME app1 "msg"
redis-cli -p $PORT AGE SIGNNAME app1 "msg"
# → returns "<signature_b64>"
# Verify using the stored public key
redis-cli -p PORT AGE VERIFYNAME app1 "msg" "<signature_b64>"
redis-cli -p $PORT AGE VERIFYNAME app1 "msg" "<signature_b64>"
# → 1 (valid) or 0 (invalid)
```
3) List stored AGE keys
```bash
redis-cli -p PORT AGE LIST
redis-cli -p $PORT AGE LIST
# Example output includes labels such as "encpub" and your key names (e.g., "app1")
```

623
herodb/docs/basics.md Normal file
View File

@@ -0,0 +1,623 @@
Here's an expanded version of the cmds.md documentation to include the list commands:
# HeroDB Commands
HeroDB implements a subset of Redis commands over the Redis protocol. This document describes the available commands and their usage.
## String Commands
### PING
Ping the server to test connectivity.
```bash
redis-cli -p $PORT PING
# → PONG
```
### ECHO
Echo the given message.
```bash
redis-cli -p $PORT ECHO "hello"
# → hello
```
### SET
Set a key to hold a string value.
```bash
redis-cli -p $PORT SET key value
# → OK
```
Options:
- EX seconds: Set expiration in seconds
- PX milliseconds: Set expiration in milliseconds
- NX: Only set if key doesn't exist
- XX: Only set if key exists
- GET: Return old value
Examples:
```bash
redis-cli -p $PORT SET key value EX 60
redis-cli -p $PORT SET key value PX 1000
redis-cli -p $PORT SET key value NX
redis-cli -p $PORT SET key value XX
redis-cli -p $PORT SET key value GET
```
### GET
Get the value of a key.
```bash
redis-cli -p $PORT GET key
# → value
```
### MGET
Get values of multiple keys.
```bash
redis-cli -p $PORT MGET key1 key2 key3
# → 1) "value1"
# 2) "value2"
# 3) (nil)
```
### MSET
Set multiple key-value pairs.
```bash
redis-cli -p $PORT MSET key1 value1 key2 value2
# → OK
```
### INCR
Increment the integer value of a key by 1.
```bash
redis-cli -p $PORT SET counter 10
redis-cli -p $PORT INCR counter
# → 11
```
### DEL
Delete a key.
```bash
redis-cli -p $PORT DEL key
# → 1
```
For multiple keys:
```bash
redis-cli -p $PORT DEL key1 key2 key3
# → number of keys deleted
```
### TYPE
Determine the type of a key.
```bash
redis-cli -p $PORT TYPE key
# → string
```
### EXISTS
Check if a key exists.
```bash
redis-cli -p $PORT EXISTS key
# → 1 (exists) or 0 (doesn't exist)
```
For multiple keys:
```bash
redis-cli -p $PORT EXISTS key1 key2 key3
# → count of existing keys
```
### EXPIRE / PEXPIRE
Set expiration time for a key.
```bash
redis-cli -p $PORT EXPIRE key 60
# → 1 (timeout set) or 0 (timeout not set)
redis-cli -p $PORT PEXPIRE key 1000
# → 1 (timeout set) or 0 (timeout not set)
```
### EXPIREAT / PEXPIREAT
Set expiration timestamp for a key.
```bash
redis-cli -p $PORT EXPIREAT key 1672531200
# → 1 (timeout set) or 0 (timeout not set)
redis-cli -p $PORT PEXPIREAT key 1672531200000
# → 1 (timeout set) or 0 (timeout not set)
```
### TTL
Get the time to live for a key.
```bash
redis-cli -p $PORT TTL key
# → remaining time in seconds
```
### PERSIST
Remove expiration from a key.
```bash
redis-cli -p $PORT PERSIST key
# → 1 (timeout removed) or 0 (key has no timeout)
```
## Hash Commands
### HSET
Set field-value pairs in a hash.
```bash
redis-cli -p $PORT HSET hashkey field1 value1 field2 value2
# → number of fields added
```
### HGET
Get value of a field in a hash.
```bash
redis-cli -p $PORT HGET hashkey field1
# → value1
```
### HGETALL
Get all field-value pairs in a hash.
```bash
redis-cli -p $PORT HGETALL hashkey
# → 1) "field1"
# 2) "value1"
# 3) "field2"
# 4) "value2"
```
### HDEL
Delete fields from a hash.
```bash
redis-cli -p $PORT HDEL hashkey field1 field2
# → number of fields deleted
```
### HEXISTS
Check if a field exists in a hash.
```bash
redis-cli -p $PORT HEXISTS hashkey field1
# → 1 (exists) or 0 (doesn't exist)
```
### HKEYS
Get all field names in a hash.
```bash
redis-cli -p $PORT HKEYS hashkey
# → 1) "field1"
# 2) "field2"
```
### HVALS
Get all values in a hash.
```bash
redis-cli -p $PORT HVALS hashkey
# → 1) "value1"
# 2) "value2"
```
### HLEN
Get number of fields in a hash.
```bash
redis-cli -p $PORT HLEN hashkey
# → number of fields
```
### HMGET
Get values of multiple fields in a hash.
```bash
redis-cli -p $PORT HMGET hashkey field1 field2 field3
# → 1) "value1"
# 2) "value2"
# 3) (nil)
```
### HSETNX
Set field-value pair in hash only if field doesn't exist.
```bash
redis-cli -p $PORT HSETNX hashkey field1 value1
# → 1 (field set) or 0 (field not set)
```
### HINCRBY
Increment integer value of a field in a hash.
```bash
redis-cli -p $PORT HINCRBY hashkey field1 5
# → new value
```
### HINCRBYFLOAT
Increment float value of a field in a hash.
```bash
redis-cli -p $PORT HINCRBYFLOAT hashkey field1 3.14
# → new value
```
### HSCAN
Incrementally iterate over fields in a hash.
```bash
redis-cli -p $PORT HSCAN hashkey 0
# → 1) "next_cursor"
# 2) 1) "field1"
# 2) "value1"
# 3) "field2"
# 4) "value2"
```
Options:
- MATCH pattern: Filter fields by pattern
- COUNT number: Suggest number of fields to return
Examples:
```bash
redis-cli -p $PORT HSCAN hashkey 0 MATCH f*
redis-cli -p $PORT HSCAN hashkey 0 COUNT 10
redis-cli -p $PORT HSCAN hashkey 0 MATCH f* COUNT 10
```
## List Commands
### LPUSH
Insert elements at the head of a list.
```bash
redis-cli -p $PORT LPUSH listkey element1 element2 element3
# → number of elements in the list
```
### RPUSH
Insert elements at the tail of a list.
```bash
redis-cli -p $PORT RPUSH listkey element1 element2 element3
# → number of elements in the list
```
### LPOP
Remove and return elements from the head of a list.
```bash
redis-cli -p $PORT LPOP listkey
# → element1
```
With count:
```bash
redis-cli -p $PORT LPOP listkey 2
# → 1) "element1"
# 2) "element2"
```
### RPOP
Remove and return elements from the tail of a list.
```bash
redis-cli -p $PORT RPOP listkey
# → element3
```
With count:
```bash
redis-cli -p $PORT RPOP listkey 2
# → 1) "element3"
# 2) "element2"
```
### LLEN
Get the length of a list.
```bash
redis-cli -p $PORT LLEN listkey
# → number of elements in the list
```
### LINDEX
Get element at index in a list.
```bash
redis-cli -p $PORT LINDEX listkey 0
# → first element
```
Negative indices count from the end:
```bash
redis-cli -p $PORT LINDEX listkey -1
# → last element
```
### LRANGE
Get a range of elements from a list.
```bash
redis-cli -p $PORT LRANGE listkey 0 -1
# → 1) "element1"
# 2) "element2"
# 3) "element3"
```
### LTRIM
Trim a list to specified range.
```bash
redis-cli -p $PORT LTRIM listkey 0 1
# → OK (list now contains only first 2 elements)
```
### LREM
Remove elements from a list.
```bash
redis-cli -p $PORT LREM listkey 2 element1
# → number of elements removed
```
Count values:
- Positive: Remove from head
- Negative: Remove from tail
- Zero: Remove all
### LINSERT
Insert element before or after pivot element.
```bash
redis-cli -p $PORT LINSERT listkey BEFORE pivot newelement
# → number of elements in the list
```
### BLPOP
Blocking remove and return elements from the head of a list.
```bash
redis-cli -p $PORT BLPOP listkey1 listkey2 5
# → 1) "listkey1"
# 2) "element1"
```
If no elements are available, blocks for specified timeout (in seconds) until an element is pushed to one of the lists.
### BRPOP
Blocking remove and return elements from the tail of a list.
```bash
redis-cli -p $PORT BRPOP listkey1 listkey2 5
# → 1) "listkey1"
# 2) "element1"
```
If no elements are available, blocks for specified timeout (in seconds) until an element is pushed to one of the lists.
## Keyspace Commands
### KEYS
Get all keys matching pattern.
```bash
redis-cli -p $PORT KEYS *
# → 1) "key1"
# 2) "key2"
```
### SCAN
Incrementally iterate over keys.
```bash
redis-cli -p $PORT SCAN 0
# → 1) "next_cursor"
# 2) 1) "key1"
# 2) "key2"
```
Options:
- MATCH pattern: Filter keys by pattern
- COUNT number: Suggest number of keys to return
Examples:
```bash
redis-cli -p $PORT SCAN 0 MATCH k*
redis-cli -p $PORT SCAN 0 COUNT 10
redis-cli -p $PORT SCAN 0 MATCH k* COUNT 10
```
### DBSIZE
Get number of keys in current database.
```bash
redis-cli -p $PORT DBSIZE
# → number of keys
```
### FLUSHDB
Remove all keys from current database.
```bash
redis-cli -p $PORT FLUSHDB
# → OK
```
## Configuration Commands
### CONFIG GET
Get configuration parameter.
```bash
redis-cli -p $PORT CONFIG GET dir
# → 1) "dir"
# 2) "/path/to/db"
redis-cli -p $PORT CONFIG GET dbfilename
# → 1) "dbfilename"
# 2) "0.db"
```
## Client Commands
### CLIENT SETNAME
Set current connection name.
```bash
redis-cli -p $PORT CLIENT SETNAME myconnection
# → OK
```
### CLIENT GETNAME
Get current connection name.
```bash
redis-cli -p $PORT CLIENT GETNAME
# → myconnection
```
## Transaction Commands
### MULTI
Start a transaction block.
```bash
redis-cli -p $PORT MULTI
# → OK
```
### EXEC
Execute all commands in transaction block.
```bash
redis-cli -p $PORT MULTI
redis-cli -p $PORT SET key1 value1
redis-cli -p $PORT SET key2 value2
redis-cli -p $PORT EXEC
# → 1) OK
# 2) OK
```
### DISCARD
Discard all commands in transaction block.
```bash
redis-cli -p $PORT MULTI
redis-cli -p $PORT SET key1 value1
redis-cli -p $PORT DISCARD
# → OK
```
## AGE Commands
### AGE GENENC
Generate ephemeral encryption keypair.
```bash
redis-cli -p $PORT AGE GENENC
# → 1) "recipient_public_key"
# 2) "identity_secret_key"
```
### AGE ENCRYPT
Encrypt message with recipient public key.
```bash
redis-cli -p $PORT AGE ENCRYPT recipient_public_key "message"
# → base64_encoded_ciphertext
```
### AGE DECRYPT
Decrypt ciphertext with identity secret key.
```bash
redis-cli -p $PORT AGE DECRYPT identity_secret_key base64_encoded_ciphertext
# → decrypted_message
```
### AGE GENSIGN
Generate ephemeral signing keypair.
```bash
redis-cli -p $PORT AGE GENSIGN
# → 1) "verify_public_key"
# 2) "sign_secret_key"
```
### AGE SIGN
Sign message with signing secret key.
```bash
redis-cli -p $PORT AGE SIGN sign_secret_key "message"
# → base64_encoded_signature
```
### AGE VERIFY
Verify signature with verify public key.
```bash
redis-cli -p $PORT AGE VERIFY verify_public_key "message" base64_encoded_signature
# → 1 (valid) or 0 (invalid)
```
### AGE KEYGEN
Generate and persist named encryption keypair.
```bash
redis-cli -p $PORT AGE KEYGEN keyname
# → 1) "recipient_public_key"
# 2) "identity_secret_key"
```
### AGE SIGNKEYGEN
Generate and persist named signing keypair.
```bash
redis-cli -p $PORT AGE SIGNKEYGEN keyname
# → 1) "verify_public_key"
# 2) "sign_secret_key"
```
### AGE ENCRYPTNAME
Encrypt message with named key.
```bash
redis-cli -p $PORT AGE ENCRYPTNAME keyname "message"
# → base64_encoded_ciphertext
```
### AGE DECRYPTNAME
Decrypt ciphertext with named key.
```bash
redis-cli -p $PORT AGE DECRYPTNAME keyname base64_encoded_ciphertext
# → decrypted_message
```
### AGE SIGNNAME
Sign message with named signing key.
```bash
redis-cli -p $PORT AGE SIGNNAME keyname "message"
# → base64_encoded_signature
```
### AGE VERIFYNAME
Verify signature with named verify key.
```bash
redis-cli -p $PORT AGE VERIFYNAME keyname "message" base64_encoded_signature
# → 1 (valid) or 0 (invalid)
```
### AGE LIST
List all stored AGE keys.
```bash
redis-cli -p $PORT AGE LIST
# → 1) "keyname1"
# 2) "keyname2"
```
## Server Information Commands
### INFO
Get server information.
```bash
redis-cli -p $PORT INFO
# → Server information
```
With section:
```bash
redis-cli -p $PORT INFO replication
# → Replication information
```
### COMMAND
Get command information (stub implementation).
```bash
redis-cli -p $PORT COMMAND
# → Empty array (stub)
```
## Database Selection
### SELECT
Select database by index.
```bash
redis-cli -p $PORT SELECT 0
# → OK
```
```
This expanded documentation includes all the list commands that were implemented in the cmd.rs file:
1. LPUSH - push elements to the left (head) of a list
2. RPUSH - push elements to the right (tail) of a list
3. LPOP - pop elements from the left (head) of a list
4. RPOP - pop elements from the right (tail) of a list
5. BLPOP - blocking pop from the left with timeout
6. BRPOP - blocking pop from the right with timeout
7. LLEN - get list length
8. LREM - remove elements from list
9. LTRIM - trim list to range
10. LINDEX - get element by index
11. LRANGE - get range of elements

227
herodb/docs/cmds.md Normal file
View File

@@ -0,0 +1,227 @@
# HeroDB Redis Protocol Support: Commands & Client Usage
HeroDB is a Redis-compatible database built using the `redb` database backend.
It supports a subset of Redis commands over the standard RESP (Redis Serialization Protocol) via TCP, allowing you to interact with it using standard Redis clients like `redis-cli`, Python's `redis-py`, Node.js's `ioredis`, etc.
This document provides:
- A list of all currently supported Redis commands.
- Example usage with standard Redis clients.
- Bash and Rust test-inspired usage examples.
## Quick Start
Assuming the server is running on localhost at port `$PORT`:
```bash
# Build HeroDB
cargo build --release
# Start HeroDB server
./target/release/herodb --dir /tmp/herodb_data --port 6381 --debug
```
## Using Standard Redis Clients
### With `redis-cli`
```bash
redis-cli -p 6381 SET mykey "hello"
redis-cli -p 6381 GET mykey
```
### With Python (`redis-py`)
```python
import redis
r = redis.Redis(host='localhost', port=6381, db=0)
r.set('mykey', 'hello')
print(r.get('mykey').decode())
```
### With Node.js (`ioredis`)
```js
const Redis = require("ioredis");
const redis = new Redis({ port: 6381, host: "localhost" });
await redis.set("mykey", "hello");
const value = await redis.get("mykey");
console.log(value); // "hello"
```
## Supported Redis Commands
### String Commands
| Command | Description | Example Usage |
|---------------|------------------------------------------|-------------------------------------------|
| `SET` | Set a key to a string value | `SET name "Alice"` |
| `GET` | Get the value of a key | `GET name` |
| `DEL` | Delete one or more keys | `DEL name age` |
| `INCR` | Increment the integer value of a key | `INCR counter` |
| `DECR` | Decrement the integer value of a key | `DECR counter` |
| `INCRBY` | Increment key by a given integer | `INCRBY counter 5` |
| `DECRBY` | Decrement key by a given integer | `DECRBY counter 3` |
| `EXISTS` | Check if a key exists | `EXISTS name` |
| `TYPE` | Return the type of a key | `TYPE name` |
### Hash Commands
| Command | Description | Example Usage |
|---------------|------------------------------------------|-------------------------------------------|
| `HSET` | Set field in hash stored at key | `HSET user:1 name "Alice"` |
| `HGET` | Get value of a field in hash | `HGET user:1 name` |
| `HGETALL` | Get all fields and values in a hash | `HGETALL user:1` |
| `HDEL` | Delete one or more fields from hash | `HDEL user:1 name age` |
| `HEXISTS` | Check if field exists in hash | `HEXISTS user:1 name` |
| `HKEYS` | Get all field names in a hash | `HKEYS user:1` |
| `HVALS` | Get all values in a hash | `HVALS user:1` |
| `HLEN` | Get number of fields in a hash | `HLEN user:1` |
| `HMGET` | Get values of multiple fields | `HMGET user:1 name age` |
| `HSETNX` | Set field only if it does not exist | `HSETNX user:1 email alice@example.com` |
### List Commands
| Command | Description | Example Usage |
|---------------|------------------------------------------|-------------------------------------------|
| `LPUSH` | Insert elements at the head of a list | `LPUSH mylist "item1" "item2"` |
| `RPUSH` | Insert elements at the tail of a list | `RPUSH mylist "item3" "item4"` |
| `LPOP` | Remove and return element from head | `LPOP mylist` |
| `RPOP` | Remove and return element from tail | `RPOP mylist` |
| `BLPOP` | Blocking remove from head with timeout | `BLPOP mylist1 mylist2 5` |
| `BRPOP` | Blocking remove from tail with timeout | `BRPOP mylist1 mylist2 5` |
| `LLEN` | Get the length of a list | `LLEN mylist` |
| `LREM` | Remove elements from list | `LREM mylist 2 "item"` |
| `LTRIM` | Trim list to specified range | `LTRIM mylist 0 5` |
| `LINDEX` | Get element by index | `LINDEX mylist 0` |
| `LRANGE` | Get range of elements | `LRANGE mylist 0 -1` |
### Keys & Scanning
| Command | Description | Example Usage |
|---------------|------------------------------------------|-------------------------------------------|
| `KEYS` | Find all keys matching a pattern | `KEYS user:*` |
| `SCAN` | Incrementally iterate keys | `SCAN 0 MATCH user:* COUNT 10` |
### Expiration
| Command | Description | Example Usage |
|---------------|------------------------------------------|-------------------------------------------|
| `EXPIRE` | Set a key's time to live in seconds | `EXPIRE tempkey 60` |
| `TTL` | Get the time to live for a key | `TTL tempkey` |
| `PERSIST` | Remove the expiration from a key | `PERSIST tempkey` |
### Transactions
| Command | Description | Example Usage |
|---------------|------------------------------------------|-------------------------------------------|
| `MULTI` | Start a transaction block | `MULTI` |
| `EXEC` | Execute all commands in a transaction | `EXEC` |
| `DISCARD` | Discard all commands in a transaction | `DISCARD` |
### Configuration
| Command | Description | Example Usage |
|---------------|------------------------------------------|-------------------------------------------|
| `CONFIG GET` | Get configuration parameters | `CONFIG GET dir` |
| `CONFIG SET` | Set configuration parameters | `CONFIG SET maxmemory 100mb` |
### Info & Monitoring
| Command | Description | Example Usage |
|---------------|------------------------------------------|-------------------------------------------|
| `INFO` | Get information and statistics about server | `INFO` |
| `PING` | Ping the server | `PING` |
### AGE Cryptography Commands
| Command | Description | Example Usage |
|--------------------|-----------------------------------------------|-----------------------------------------------|
| `AGE GENENC` | Generate ephemeral encryption keypair | `AGE GENENC` |
| `AGE GENSIGN` | Generate ephemeral signing keypair | `AGE GENSIGN` |
| `AGE ENCRYPT` | Encrypt a message using a public key | `AGE ENCRYPT <recipient> "msg"` |
| `AGE DECRYPT` | Decrypt a message using a secret key | `AGE DECRYPT <identity> <ciphertext>` |
| `AGE SIGN` | Sign a message using a secret key | `AGE SIGN <sign_secret> "msg"` |
| `AGE VERIFY` | Verify a signature using a public key | `AGE VERIFY <pubkey> "msg" <signature>` |
| `AGE KEYGEN` | Create and persist a named encryption key | `AGE KEYGEN app1` |
| `AGE SIGNKEYGEN` | Create and persist a named signing key | `AGE SIGNKEYGEN app1` |
| `AGE ENCRYPTNAME` | Encrypt using a named key | `AGE ENCRYPTNAME app1 "msg"` |
| `AGE DECRYPTNAME` | Decrypt using a named key | `AGE DECRYPTNAME app1 <ciphertext>` |
| `AGE SIGNNAME` | Sign using a named key | `AGE SIGNNAME app1 "msg"` |
| `AGE VERIFYNAME` | Verify using a named key | `AGE VERIFYNAME app1 "msg" <signature>` |
| `AGE LIST` | List all persisted named keys | `AGE LIST` |
> Note: AGE commands are not part of standard Redis. They are HeroDB-specific extensions for cryptographic operations.
## Example Usage
### Basic String Operations
```bash
redis-cli -p 6381 SET greeting "Hello, HeroDB!"
redis-cli -p 6381 GET greeting
# → "Hello, HeroDB!"
redis-cli -p 6381 INCR visits
redis-cli -p 6381 INCR visits
redis-cli -p 6381 GET visits
# → "2"
```
### Hash Operations
```bash
redis-cli -p 6381 HSET user:1000 name "Alice" age "30" city "NYC"
redis-cli -p 6381 HGET user:1000 name
# → "Alice"
redis-cli -p 6381 HGETALL user:1000
# → 1) "name"
# 2) "Alice"
# 3) "age"
# 4) "30"
# 5) "city"
# 6) "NYC"
```
### Expiration
```bash
redis-cli -p 6381 SET tempkey "temporary"
redis-cli -p 6381 EXPIRE tempkey 5
redis-cli -p 6381 TTL tempkey
# → (integer) 4
# After 5 seconds:
redis-cli -p 6381 GET tempkey
# → (nil)
```
### Transactions
```bash
redis-cli -p 6381 MULTI
redis-cli -p 6381 SET txkey1 "value1"
redis-cli -p 6381 SET txkey2 "value2"
redis-cli -p 6381 INCR counter
redis-cli -p 6381 EXEC
# → 1) OK
# 2) OK
# 3) (integer) 3
```
### Scanning Keys
```bash
redis-cli -p 6381 SET scankey1 "val1"
redis-cli -p 6381 SET scankey2 "val2"
redis-cli -p 6381 HSET scanhash field1 "val1"
redis-cli -p 6381 SCAN 0 MATCH scankey*
# → 1) "0"
# 2) 1) "scankey1"
# 2) "scankey2"
```

290
herodb/docs/openrpc.json Normal file
View File

@@ -0,0 +1,290 @@
{
"openrpc": "1.2.6",
"info": {
"title": "HeroDB RPC API",
"version": "0.0.1",
"description": "Database management API for HeroDB"
},
"servers": [
{
"name": "HeroDB Server",
"url": "http://localhost:8080"
}
],
"methods": [
{
"name": "herodb_configureDatabase",
"summary": "Configure an existing database with specific settings",
"params": [
{
"name": "db_index",
"description": "Database index to configure",
"schema": {
"type": "integer",
"minimum": 0
},
"required": true
},
{
"name": "config",
"description": "Configuration object",
"schema": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"storage_path": {
"type": "string"
},
"max_size": {
"type": "integer"
},
"redis_version": {
"type": "string"
}
}
},
"required": true
}
],
"result": {
"name": "success",
"schema": {
"type": "boolean"
}
}
},
{
"name": "herodb_createDatabase",
"summary": "Create/pre-initialize a database at the specified index",
"params": [
{
"name": "db_index",
"description": "Database index to create",
"schema": {
"type": "integer",
"minimum": 0
},
"required": true
}
],
"result": {
"name": "success",
"schema": {
"type": "boolean"
}
}
},
{
"name": "herodb_setDatabaseEncryption",
"summary": "Set encryption for a specific database index",
"params": [
{
"name": "db_index",
"description": "Database index",
"schema": {
"type": "integer",
"minimum": 0
},
"required": true
},
{
"name": "encryption_key",
"description": "Encryption key (write-only)",
"schema": {
"type": "string"
},
"required": true
}
],
"result": {
"name": "success",
"schema": {
"type": "boolean"
}
}
},
{
"name": "herodb_listDatabases",
"summary": "List all database indices that exist",
"params": [],
"result": {
"name": "database_indices",
"schema": {
"type": "array",
"items": {
"type": "integer"
}
}
}
},
{
"name": "herodb_getDatabaseInfo",
"summary": "Get detailed information about a specific database",
"params": [
{
"name": "db_index",
"description": "Database index",
"schema": {
"type": "integer",
"minimum": 0
},
"required": true
}
],
"result": {
"name": "database_info",
"schema": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"name": {
"type": "string",
"nullable": true
},
"backend": {
"type": "string",
"enum": ["Redb"]
},
"encrypted": {
"type": "boolean"
},
"redis_version": {
"type": "string",
"nullable": true
},
"storage_path": {
"type": "string",
"nullable": true
},
"size_on_disk": {
"type": "integer",
"nullable": true
},
"key_count": {
"type": "integer",
"nullable": true
},
"created_at": {
"type": "integer"
},
"last_access": {
"type": "integer",
"nullable": true
}
}
}
}
},
{
"name": "herodb_deleteDatabase",
"summary": "Delete a database and its files",
"params": [
{
"name": "db_index",
"description": "Database index to delete",
"schema": {
"type": "integer",
"minimum": 0
},
"required": true
}
],
"result": {
"name": "success",
"schema": {
"type": "boolean"
}
}
},
{
"name": "herodb_getServerStats",
"summary": "Get server statistics",
"params": [],
"result": {
"name": "stats",
"schema": {
"type": "object",
"additionalProperties": {
"oneOf": [
{"type": "string"},
{"type": "integer"},
{"type": "boolean"},
{"type": "array"}
]
}
}
}
}
],
"components": {
"schemas": {
"DatabaseConfig": {
"type": "object",
"properties": {
"name": {
"type": "string",
"nullable": true
},
"storage_path": {
"type": "string",
"nullable": true
},
"max_size": {
"type": "integer",
"nullable": true
},
"redis_version": {
"type": "string",
"nullable": true
}
}
},
"DatabaseInfo": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"name": {
"type": "string",
"nullable": true
},
"backend": {
"type": "string",
"enum": ["Redb", "InMemory", "Custom"]
},
"encrypted": {
"type": "boolean"
},
"redis_version": {
"type": "string",
"nullable": true
},
"storage_path": {
"type": "string",
"nullable": true
},
"size_on_disk": {
"type": "integer",
"nullable": true
},
"key_count": {
"type": "integer",
"nullable": true
},
"created_at": {
"type": "integer"
},
"last_access": {
"type": "integer",
"nullable": true
}
}
}
}
}
}

View File

@@ -1,8 +1,11 @@
pub mod age; // NEW
pub mod age;
pub mod cmd;
pub mod crypto;
pub mod error;
pub mod options;
pub mod protocol;
pub mod rpc;
pub mod rpc_server;
pub mod server;
pub mod storage;
pub mod openrpc_spec;

View File

@@ -1,8 +1,10 @@
// #![allow(unused_imports)]
use std::sync::Arc;
use tokio::sync::Mutex;
use tokio::net::TcpListener;
use herodb::server;
use herodb::server::Server;
use herodb::rpc_server;
use clap::Parser;
@@ -30,6 +32,14 @@ struct Args {
/// Encrypt the database
#[arg(long)]
encrypt: bool,
/// Enable RPC management server
#[arg(long)]
enable_rpc: bool,
/// RPC server port (default: 8080)
#[arg(long, default_value = "8080")]
rpc_port: u16,
}
#[tokio::main]
@@ -46,7 +56,7 @@ async fn main() {
// new DB option
let option = herodb::options::DBOption {
dir: args.dir,
dir: args.dir.clone(),
port,
debug: args.debug,
encryption_key: args.encryption_key,
@@ -54,11 +64,30 @@ async fn main() {
};
// new server
let server = server::Server::new(option).await;
let server = Arc::new(Mutex::new(server::Server::new(option).await));
// Add a small delay to ensure the port is ready
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
// Start RPC server if enabled
let _rpc_handle = if args.enable_rpc {
let rpc_addr = format!("127.0.0.1:{}", args.rpc_port).parse().unwrap();
let base_dir = args.dir.clone();
match rpc_server::start_rpc_server(rpc_addr, Arc::clone(&server), base_dir).await {
Ok(handle) => {
println!("RPC management server started on port {}", args.rpc_port);
Some(handle)
}
Err(e) => {
eprintln!("Failed to start RPC server: {}", e);
None
}
}
} else {
None
};
// accept new connections
loop {
let stream = listener.accept().await;
@@ -66,9 +95,9 @@ async fn main() {
Ok((stream, _)) => {
println!("accepted new connection");
let mut sc = server.clone();
let sc = Arc::clone(&server);
tokio::spawn(async move {
if let Err(e) = sc.handle(stream).await {
if let Err(e) = Server::handle(sc, stream).await {
println!("error: {:?}, will close the connection. Bye", e);
}
});

View File

@@ -0,0 +1,2 @@
/// The OpenRPC specification for the HeroDB JSON-RPC API
pub const OPENRPC_SPEC: &str = include_str!("../docs/openrpc.json");

300
herodb/src/rpc.rs Normal file
View File

@@ -0,0 +1,300 @@
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::Mutex;
use jsonrpsee::{core::RpcResult, proc_macros::rpc};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::server::Server;
use crate::openrpc_spec::OPENRPC_SPEC;
/// Database backend types
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum BackendType {
Redb,
// Future: InMemory, Custom(String)
}
/// Database configuration
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DatabaseConfig {
pub name: Option<String>,
pub storage_path: Option<String>,
pub max_size: Option<u64>,
pub redis_version: Option<String>,
}
/// Database information returned by metadata queries
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DatabaseInfo {
pub id: u64,
pub name: Option<String>,
pub backend: BackendType,
pub encrypted: bool,
pub redis_version: Option<String>,
pub storage_path: Option<String>,
pub size_on_disk: Option<u64>,
pub key_count: Option<u64>,
pub created_at: u64,
pub last_access: Option<u64>,
}
/// RPC trait for HeroDB management
#[rpc(client, server, namespace = "herodb")]
pub trait Rpc {
/// Configure an existing database with specific settings
#[method(name = "configureDatabase")]
async fn configure_database(
&self,
db_index: u64,
config: DatabaseConfig
) -> RpcResult<bool>;
/// Create/pre-initialize a database at the specified index
#[method(name = "createDatabase")]
async fn create_database(&self, db_index: u64) -> RpcResult<bool>;
/// Set encryption for a specific database index (write-only key)
#[method(name = "setDatabaseEncryption")]
async fn set_database_encryption(&self, db_index: u64, encryption_key: String) -> RpcResult<bool>;
/// List all database indices that exist
#[method(name = "listDatabases")]
async fn list_databases(&self) -> RpcResult<Vec<u64>>;
/// Get detailed information about a specific database
#[method(name = "getDatabaseInfo")]
async fn get_database_info(&self, db_index: u64) -> RpcResult<DatabaseInfo>;
/// Delete a database and its files
#[method(name = "deleteDatabase")]
async fn delete_database(&self, db_index: u64) -> RpcResult<bool>;
/// Get server statistics
#[method(name = "getServerStats")]
async fn get_server_stats(&self) -> RpcResult<HashMap<String, serde_json::Value>>;
}
/// RPC Discovery trait for API introspection
#[rpc(client, server, namespace = "rpc", namespace_separator = ".")]
pub trait RpcDiscovery {
/// Get the OpenRPC specification for API discovery
#[method(name = "discover")]
async fn discover(&self) -> RpcResult<Value>;
}
/// RPC Server implementation
#[derive(Clone)]
pub struct RpcServerImpl {
/// Reference to the main Redis server
main_server: Arc<Mutex<Server>>,
/// Base directory for database files
base_dir: String,
}
impl RpcServerImpl {
/// Create a new RPC server instance with reference to main server
pub fn new(main_server: Arc<Mutex<Server>>, base_dir: String) -> Self {
Self {
main_server,
base_dir,
}
}
}
#[jsonrpsee::core::async_trait]
impl RpcServer for RpcServerImpl {
async fn configure_database(
&self,
db_index: u64,
config: DatabaseConfig
) -> RpcResult<bool> {
// For now, configuration is mainly informational
// In a full implementation, this could set database-specific settings
println!("Configured database {} with settings: {:?}", db_index, config);
Ok(true)
}
async fn create_database(&self, db_index: u64) -> RpcResult<bool> {
// Lock the main server to create the database
let mut server_guard = self.main_server.lock().await;
// Save the current selected_db to restore it later
let original_db = server_guard.selected_db;
// Temporarily set the selected_db to the target database
server_guard.selected_db = db_index;
// Call current_storage() which will create the database file if it doesn't exist
match server_guard.current_storage() {
Ok(_) => {
println!("Successfully created database at index {}", db_index);
// Restore the original selected_db
server_guard.selected_db = original_db;
Ok(true)
}
Err(e) => {
// Restore the original selected_db even on error
server_guard.selected_db = original_db;
Err(jsonrpsee::types::ErrorObjectOwned::owned(
-32000,
format!("Failed to create database {}: {}", db_index, e.0),
None::<()>
))
}
}
}
async fn set_database_encryption(&self, db_index: u64, _encryption_key: String) -> RpcResult<bool> {
// Note: Encryption is determined at database creation time based on db_index
// DB 0-9 are non-encrypted, DB 10+ are encrypted
// This method is mainly for documentation/configuration purposes
println!("Note: Database {} encryption is determined by index (10+ = encrypted)", db_index);
println!("Encryption key provided but not stored (write-only policy)");
Ok(db_index >= 10) // Return true if this DB would be encrypted
}
async fn list_databases(&self) -> RpcResult<Vec<u64>> {
// Scan the database directory for existing .db files
let mut db_indices = Vec::new();
if let Ok(entries) = std::fs::read_dir(&self.base_dir) {
for entry in entries.flatten() {
if let Some(file_name) = entry.file_name().to_str() {
if let Some(index_str) = file_name.strip_suffix(".db") {
if let Ok(index) = index_str.parse::<u64>() {
db_indices.push(index);
}
}
}
}
}
// Also include database 0 (default) even if file doesn't exist yet
if !db_indices.contains(&0) {
db_indices.push(0);
}
db_indices.sort();
Ok(db_indices)
}
async fn get_database_info(&self, db_index: u64) -> RpcResult<DatabaseInfo> {
// Check if database file exists
let db_path = std::path::PathBuf::from(&self.base_dir).join(format!("{}.db", db_index));
let file_exists = db_path.exists();
// If database doesn't exist, return an error
if !file_exists && db_index != 0 {
return Err(jsonrpsee::types::ErrorObjectOwned::owned(
-32000,
format!("Database {} does not exist", db_index),
None::<()>
));
}
// Get file metadata if it exists
let (size_on_disk, created_at) = if file_exists {
if let Ok(metadata) = std::fs::metadata(&db_path) {
let size = Some(metadata.len());
let created = metadata.created()
.unwrap_or(std::time::SystemTime::UNIX_EPOCH)
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs();
(size, created)
} else {
(None, 0)
}
} else {
// Database 0 might not have a file yet
(None, std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs())
};
Ok(DatabaseInfo {
id: db_index,
name: None, // Could be extended to store names
backend: BackendType::Redb,
encrypted: db_index >= 10, // Based on HeroDB's encryption rule
redis_version: Some("7.0".to_string()),
storage_path: Some(self.base_dir.clone()),
size_on_disk,
key_count: None, // Would need to open DB to count keys
created_at,
last_access: None,
})
}
async fn delete_database(&self, db_index: u64) -> RpcResult<bool> {
// Don't allow deletion of database 0 (default)
if db_index == 0 {
return Err(jsonrpsee::types::ErrorObjectOwned::owned(
-32000,
"Cannot delete default database (index 0)".to_string(),
None::<()>
));
}
let db_path = std::path::PathBuf::from(&self.base_dir).join(format!("{}.db", db_index));
if db_path.exists() {
match std::fs::remove_file(&db_path) {
Ok(_) => {
println!("Deleted database file: {}", db_path.display());
Ok(true)
}
Err(e) => {
Err(jsonrpsee::types::ErrorObjectOwned::owned(
-32000,
format!("Failed to delete database {}: {}", db_index, e),
None::<()>
))
}
}
} else {
Ok(false) // Database didn't exist
}
}
async fn get_server_stats(&self) -> RpcResult<HashMap<String, serde_json::Value>> {
let mut stats = HashMap::new();
// Get list of databases
let databases = self.list_databases().await.unwrap_or_default();
stats.insert("total_databases".to_string(), serde_json::json!(databases.len()));
stats.insert("database_indices".to_string(), serde_json::json!(databases));
stats.insert("uptime".to_string(), serde_json::json!(
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs()
));
let server_guard = self.main_server.lock().await;
stats.insert("server_port".to_string(), serde_json::json!(server_guard.option.port));
stats.insert("data_directory".to_string(), serde_json::json!(self.base_dir));
Ok(stats)
}
}
#[jsonrpsee::core::async_trait]
impl RpcDiscoveryServer for RpcServerImpl {
async fn discover(&self) -> RpcResult<Value> {
// Parse the OpenRPC spec JSON and return it
match serde_json::from_str(OPENRPC_SPEC) {
Ok(spec) => Ok(spec),
Err(e) => Err(jsonrpsee::types::ErrorObjectOwned::owned(
-32000,
format!("Failed to parse OpenRPC specification: {}", e),
None::<()>
))
}
}
}

63
herodb/src/rpc_server.rs Normal file
View File

@@ -0,0 +1,63 @@
use std::net::SocketAddr;
use jsonrpsee::server::{ServerBuilder, ServerHandle};
use jsonrpsee::RpcModule;
use std::sync::Arc;
use tokio::sync::Mutex;
use crate::rpc::{RpcServer, RpcDiscoveryServer, RpcServerImpl};
use crate::server::Server;
/// Start the RPC server on the specified address
pub async fn start_rpc_server(
addr: SocketAddr,
main_server: Arc<Mutex<Server>>,
base_dir: String
) -> Result<ServerHandle, Box<dyn std::error::Error + Send + Sync>> {
// Create the RPC server implementation
let rpc_impl = RpcServerImpl::new(main_server, base_dir);
// Create the RPC module
let mut module = RpcModule::new(());
module.merge(RpcServer::into_rpc(rpc_impl.clone()))?;
module.merge(RpcDiscoveryServer::into_rpc(rpc_impl))?;
// Build the server with both HTTP and WebSocket support
let server = ServerBuilder::default()
.build(addr)
.await?;
// Start the server
let handle = server.start(module);
println!("RPC server started on {}", addr);
Ok(handle)
}
#[cfg(test)]
mod tests {
use super::*;
use std::time::Duration;
#[tokio::test]
async fn test_rpc_server_startup() {
let addr = "127.0.0.1:0".parse().unwrap(); // Use port 0 for auto-assignment
let base_dir = "/tmp/test_rpc".to_string();
let main_server = Arc::new(Mutex::new(crate::server::Server::new(crate::options::DBOption {
dir: "/tmp".to_string(),
port: 0,
debug: false,
encryption_key: None,
encrypt: false,
}).await));
let handle = start_rpc_server(addr, main_server, base_dir).await.unwrap();
// Give the server a moment to start
tokio::time::sleep(Duration::from_millis(100)).await;
// Stop the server
handle.stop().unwrap();
handle.stopped().await;
}
}

View File

@@ -167,7 +167,7 @@ impl Server {
}
pub async fn handle(
&mut self,
server: Arc<Mutex<Server>>,
mut stream: tokio::net::TcpStream,
) -> Result<(), DBError> {
// Accumulate incoming bytes to handle partial RESP frames
@@ -205,31 +205,49 @@ impl Server {
// Advance the accumulator to the unparsed remainder
acc = remaining.to_string();
if self.option.debug {
println!("\x1b[34;1mgot command: {:?}, protocol: {:?}\x1b[0m", cmd, protocol);
} else {
println!("got command: {:?}, protocol: {:?}", cmd, protocol);
}
// Check if this is a QUIT command before processing
let is_quit = matches!(cmd, Cmd::Quit);
let res = match cmd.run(self).await {
Ok(p) => p,
Err(e) => {
if self.option.debug {
eprintln!("[run error] {:?}", e);
}
Protocol::err(&format!("ERR {}", e.0))
// Lock the server only for command execution
let (res, debug_info) = {
let mut server_guard = server.lock().await;
if server_guard.option.debug {
println!("\x1b[34;1mgot command: {:?}, protocol: {:?}\x1b[0m", cmd, protocol);
} else {
println!("got command: {:?}, protocol: {:?}", cmd, protocol);
}
let res = match cmd.run(&mut server_guard).await {
Ok(p) => p,
Err(e) => {
if server_guard.option.debug {
eprintln!("[run error] {:?}", e);
}
Protocol::err(&format!("ERR {}", e.0))
}
};
let debug_info = if server_guard.option.debug {
Some((format!("queued cmd {:?}", server_guard.queued_cmd), format!("going to send response {}", res.encode())))
} else {
Some((format!("queued cmd {:?}", server_guard.queued_cmd), format!("going to send response {}", res.encode())))
};
(res, debug_info)
};
if self.option.debug {
println!("\x1b[34;1mqueued cmd {:?}\x1b[0m", self.queued_cmd);
println!("\x1b[32;1mgoing to send response {}\x1b[0m", res.encode());
} else {
print!("queued cmd {:?}", self.queued_cmd);
println!("going to send response {}", res.encode());
// Print debug info outside the lock
if let Some((queued_info, response_info)) = debug_info {
if let Some((_, response)) = response_info.split_once("going to send response ") {
if queued_info.contains("\x1b[34;1m") {
println!("\x1b[34;1m{}\x1b[0m", queued_info);
println!("\x1b[32;1mgoing to send response {}\x1b[0m", response);
} else {
println!("{}", queued_info);
println!("going to send response {}", response);
}
}
}
_ = stream.write(res.encode().as_bytes()).await?;

View File

@@ -298,7 +298,7 @@ main() {
# Start the server
print_status "Starting HeroDB server..."
./target/release/herodb --dir "$DB_DIR" --port $PORT &
../target/release/herodb --dir "$DB_DIR" --port $PORT &
SERVER_PID=$!
# Wait for server to start

View File

@@ -1,4 +1,6 @@
use herodb::{server::Server, options::DBOption};
use std::sync::Arc;
use tokio::sync::Mutex;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
@@ -29,7 +31,7 @@ async fn debug_hset_simple() {
encryption_key: None,
};
let mut server = Server::new(option).await;
let server = Arc::new(Mutex::new(Server::new(option).await));
// Start server in background
tokio::spawn(async move {
@@ -39,7 +41,10 @@ async fn debug_hset_simple() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let server_clone = Arc::clone(&server);
tokio::spawn(async move {
let _ = Server::handle(server_clone, stream).await;
});
}
}
});

View File

@@ -3,6 +3,8 @@ use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio::time::sleep;
use std::sync::Arc;
use tokio::sync::Mutex;
#[tokio::test]
async fn debug_hset_return_value() {
@@ -20,7 +22,7 @@ async fn debug_hset_return_value() {
encryption_key: None,
};
let mut server = Server::new(option).await;
let server = Arc::new(Mutex::new(Server::new(option).await));
// Start server in background
tokio::spawn(async move {
@@ -30,7 +32,10 @@ async fn debug_hset_return_value() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let server_clone = Arc::clone(&server);
tokio::spawn(async move {
let _ = Server::handle(server_clone, stream).await;
});
}
}
});

View File

@@ -1,11 +1,13 @@
use herodb::{server::Server, options::DBOption};
use std::sync::Arc;
use tokio::sync::Mutex;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio::time::sleep;
// Helper function to start a test server
async fn start_test_server(test_name: &str) -> (Server, u16) {
async fn start_test_server(test_name: &str) -> (Arc<Mutex<Server>>, u16) {
use std::sync::atomic::{AtomicU16, Ordering};
static PORT_COUNTER: AtomicU16 = AtomicU16::new(16379);
@@ -24,7 +26,7 @@ async fn start_test_server(test_name: &str) -> (Server, u16) {
encryption_key: None,
};
let server = Server::new(option).await;
let server = Arc::new(Mutex::new(Server::new(option).await));
(server, port)
}
@@ -54,7 +56,7 @@ async fn send_command(stream: &mut TcpStream, command: &str) -> String {
#[tokio::test]
async fn test_basic_ping() {
let (mut server, port) = start_test_server("ping").await;
let (server, port) = start_test_server("ping").await;
// Start server in background
tokio::spawn(async move {
@@ -64,7 +66,7 @@ async fn test_basic_ping() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -78,7 +80,7 @@ async fn test_basic_ping() {
#[tokio::test]
async fn test_string_operations() {
let (mut server, port) = start_test_server("string").await;
let (server, port) = start_test_server("string").await;
// Start server in background
tokio::spawn(async move {
@@ -88,7 +90,7 @@ async fn test_string_operations() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -120,7 +122,7 @@ async fn test_string_operations() {
#[tokio::test]
async fn test_incr_operations() {
let (mut server, port) = start_test_server("incr").await;
let (server, port) = start_test_server("incr").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -129,7 +131,7 @@ async fn test_incr_operations() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -154,7 +156,7 @@ async fn test_incr_operations() {
#[tokio::test]
async fn test_hash_operations() {
let (mut server, port) = start_test_server("hash").await;
let (server, port) = start_test_server("hash").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -163,7 +165,7 @@ async fn test_hash_operations() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -220,7 +222,7 @@ async fn test_hash_operations() {
#[tokio::test]
async fn test_expiration() {
let (mut server, port) = start_test_server("expiration").await;
let (server, port) = start_test_server("expiration").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -229,7 +231,7 @@ async fn test_expiration() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -268,7 +270,7 @@ async fn test_expiration() {
#[tokio::test]
async fn test_scan_operations() {
let (mut server, port) = start_test_server("scan").await;
let (server, port) = start_test_server("scan").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -277,7 +279,7 @@ async fn test_scan_operations() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -304,7 +306,7 @@ async fn test_scan_operations() {
#[tokio::test]
async fn test_hscan_operations() {
let (mut server, port) = start_test_server("hscan").await;
let (server, port) = start_test_server("hscan").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -313,7 +315,7 @@ async fn test_hscan_operations() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -336,7 +338,7 @@ async fn test_hscan_operations() {
#[tokio::test]
async fn test_transaction_operations() {
let (mut server, port) = start_test_server("transaction").await;
let (server, port) = start_test_server("transaction").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -345,7 +347,7 @@ async fn test_transaction_operations() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -379,7 +381,7 @@ async fn test_transaction_operations() {
#[tokio::test]
async fn test_discard_transaction() {
let (mut server, port) = start_test_server("discard").await;
let (server, port) = start_test_server("discard").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -388,7 +390,7 @@ async fn test_discard_transaction() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -416,7 +418,7 @@ async fn test_discard_transaction() {
#[tokio::test]
async fn test_type_command() {
let (mut server, port) = start_test_server("type").await;
let (server, port) = start_test_server("type").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -425,7 +427,7 @@ async fn test_type_command() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -451,7 +453,7 @@ async fn test_type_command() {
#[tokio::test]
async fn test_config_commands() {
let (mut server, port) = start_test_server("config").await;
let (server, port) = start_test_server("config").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -460,7 +462,7 @@ async fn test_config_commands() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -482,7 +484,7 @@ async fn test_config_commands() {
#[tokio::test]
async fn test_info_command() {
let (mut server, port) = start_test_server("info").await;
let (server, port) = start_test_server("info").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -491,7 +493,7 @@ async fn test_info_command() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -511,7 +513,7 @@ async fn test_info_command() {
#[tokio::test]
async fn test_error_handling() {
let (mut server, port) = start_test_server("error").await;
let (server, port) = start_test_server("error").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -520,7 +522,7 @@ async fn test_error_handling() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -549,7 +551,7 @@ async fn test_error_handling() {
#[tokio::test]
async fn test_list_operations() {
let (mut server, port) = start_test_server("list").await;
let (server, port) = start_test_server("list").await;
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
@@ -558,7 +560,7 @@ async fn test_list_operations() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});

37
herodb/tests/rpc_tests.rs Normal file
View File

@@ -0,0 +1,37 @@
use herodb::rpc::{BackendType, DatabaseConfig};
#[tokio::test]
async fn test_rpc_server_basic() {
// This test would require starting the RPC server in a separate thread
// For now, we'll just test that the types compile correctly
// Test serialization of types
let backend = BackendType::Redb;
let config = DatabaseConfig {
name: Some("test_db".to_string()),
storage_path: Some("/tmp/test".to_string()),
max_size: Some(1024 * 1024),
redis_version: Some("7.0".to_string()),
};
let backend_json = serde_json::to_string(&backend).unwrap();
let config_json = serde_json::to_string(&config).unwrap();
assert_eq!(backend_json, "\"Redb\"");
assert!(config_json.contains("test_db"));
}
#[tokio::test]
async fn test_database_config_serialization() {
let config = DatabaseConfig {
name: Some("my_db".to_string()),
storage_path: None,
max_size: Some(1000000),
redis_version: Some("7.0".to_string()),
};
let json = serde_json::to_value(&config).unwrap();
assert_eq!(json["name"], "my_db");
assert_eq!(json["max_size"], 1000000);
assert_eq!(json["redis_version"], "7.0");
}

View File

@@ -1,11 +1,13 @@
use herodb::{server::Server, options::DBOption};
use std::sync::Arc;
use tokio::sync::Mutex;
use std::time::Duration;
use tokio::time::sleep;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
// Helper function to start a test server with clean data directory
async fn start_test_server(test_name: &str) -> (Server, u16) {
async fn start_test_server(test_name: &str) -> (Arc<Mutex<Server>>, u16) {
use std::sync::atomic::{AtomicU16, Ordering};
static PORT_COUNTER: AtomicU16 = AtomicU16::new(17000);
@@ -26,7 +28,7 @@ async fn start_test_server(test_name: &str) -> (Server, u16) {
encryption_key: None,
};
let server = Server::new(option).await;
let server = Arc::new(Mutex::new(Server::new(option).await));
(server, port)
}
@@ -42,7 +44,7 @@ async fn send_redis_command(port: u16, command: &str) -> String {
#[tokio::test]
async fn test_basic_redis_functionality() {
let (mut server, port) = start_test_server("basic").await;
let (server, port) = start_test_server("basic").await;
// Start server in background with timeout
let server_handle = tokio::spawn(async move {
@@ -53,7 +55,7 @@ async fn test_basic_redis_functionality() {
// Accept only a few connections for testing
for _ in 0..10 {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -111,7 +113,7 @@ async fn test_basic_redis_functionality() {
#[tokio::test]
async fn test_hash_operations() {
let (mut server, port) = start_test_server("hash_ops").await;
let (server, port) = start_test_server("hash_ops").await;
// Start server in background with timeout
let server_handle = tokio::spawn(async move {
@@ -122,7 +124,7 @@ async fn test_hash_operations() {
// Accept only a few connections for testing
for _ in 0..5 {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});
@@ -165,7 +167,7 @@ async fn test_hash_operations() {
#[tokio::test]
async fn test_transaction_operations() {
let (mut server, port) = start_test_server("transactions").await;
let (server, port) = start_test_server("transactions").await;
// Start server in background with timeout
let server_handle = tokio::spawn(async move {
@@ -176,7 +178,7 @@ async fn test_transaction_operations() {
// Accept only a few connections for testing
for _ in 0..5 {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let _ = Server::handle(Arc::clone(&server), stream).await;
}
}
});

View File

@@ -1,11 +1,13 @@
use herodb::{server::Server, options::DBOption};
use std::sync::Arc;
use tokio::sync::Mutex;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio::time::sleep;
// Helper function to start a test server with clean data directory
async fn start_test_server(test_name: &str) -> (Server, u16) {
async fn start_test_server(test_name: &str) -> (Arc<Mutex<Server>>, u16) {
use std::sync::atomic::{AtomicU16, Ordering};
static PORT_COUNTER: AtomicU16 = AtomicU16::new(16500);
@@ -24,7 +26,7 @@ async fn start_test_server(test_name: &str) -> (Server, u16) {
encryption_key: None,
};
let server = Server::new(option).await;
let server = Arc::new(Mutex::new(Server::new(option).await));
(server, port)
}
@@ -54,7 +56,7 @@ async fn connect_to_server(port: u16) -> TcpStream {
#[tokio::test]
async fn test_basic_ping_simple() {
let (mut server, port) = start_test_server("ping").await;
let (server, port) = start_test_server("ping").await;
// Start server in background
tokio::spawn(async move {
@@ -64,7 +66,8 @@ async fn test_basic_ping_simple() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let server_clone = Arc::clone(&server);
let _ = Server::handle(server_clone, stream).await;
}
}
});
@@ -78,7 +81,7 @@ async fn test_basic_ping_simple() {
#[tokio::test]
async fn test_hset_clean_db() {
let (mut server, port) = start_test_server("hset_clean").await;
let (server, port) = start_test_server("hset_clean").await;
// Start server in background
tokio::spawn(async move {
@@ -88,7 +91,8 @@ async fn test_hset_clean_db() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let server_clone = Arc::clone(&server);
let _ = Server::handle(server_clone, stream).await;
}
}
});
@@ -110,7 +114,7 @@ async fn test_hset_clean_db() {
#[tokio::test]
async fn test_type_command_simple() {
let (mut server, port) = start_test_server("type").await;
let (server, port) = start_test_server("type").await;
// Start server in background
tokio::spawn(async move {
@@ -120,7 +124,8 @@ async fn test_type_command_simple() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let server_clone = Arc::clone(&server);
let _ = Server::handle(server_clone, stream).await;
}
}
});
@@ -149,7 +154,7 @@ async fn test_type_command_simple() {
#[tokio::test]
async fn test_hexists_simple() {
let (mut server, port) = start_test_server("hexists").await;
let (server, port) = start_test_server("hexists").await;
// Start server in background
tokio::spawn(async move {
@@ -159,7 +164,8 @@ async fn test_hexists_simple() {
loop {
if let Ok((stream, _)) = listener.accept().await {
let _ = server.handle(stream).await;
let server_clone = Arc::clone(&server);
let _ = Server::handle(server_clone, stream).await;
}
}
});

View File

@@ -2,12 +2,14 @@ use herodb::{options::DBOption, server::Server};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio::time::{sleep, Duration};
use std::sync::Arc;
use tokio::sync::Mutex;
// =========================
// Helpers
// =========================
async fn start_test_server(test_name: &str) -> (Server, u16) {
async fn start_test_server(test_name: &str) -> (Arc<Mutex<Server>>, u16) {
use std::sync::atomic::{AtomicU16, Ordering};
static PORT_COUNTER: AtomicU16 = AtomicU16::new(17100);
let port = PORT_COUNTER.fetch_add(1, Ordering::SeqCst);
@@ -24,11 +26,11 @@ async fn start_test_server(test_name: &str) -> (Server, u16) {
encryption_key: None,
};
let server = Server::new(option).await;
let server = Arc::new(Mutex::new(Server::new(option).await));
(server, port)
}
async fn spawn_listener(server: Server, port: u16) {
async fn spawn_listener(server: Arc<Mutex<Server>>, port: u16) {
tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(format!("127.0.0.1:{}", port))
.await
@@ -36,9 +38,9 @@ async fn spawn_listener(server: Server, port: u16) {
loop {
match listener.accept().await {
Ok((stream, _)) => {
let mut s_clone = server.clone();
let server_clone = Arc::clone(&server);
tokio::spawn(async move {
let _ = s_clone.handle(stream).await;
let _ = Server::handle(server_clone, stream).await;
});
}
Err(_e) => break,
@@ -500,11 +502,11 @@ async fn test_07_age_stateless_suite() {
let mut s = connect(port).await;
// GENENC -> [recipient, identity]
let gen = send_cmd(&mut s, &["AGE", "GENENC"]).await;
let gen_result = send_cmd(&mut s, &["AGE", "GENENC"]).await;
assert!(
gen.starts_with("*2\r\n$"),
gen_result.starts_with("*2\r\n$"),
"AGE GENENC should return array [recipient, identity], got:\n{}",
gen
gen_result
);
// Parse simple RESP array of two bulk strings to extract keys
@@ -519,7 +521,7 @@ async fn test_07_age_stateless_suite() {
let ident = lines.next().unwrap_or("").to_string();
(recip, ident)
}
let (recipient, identity) = parse_two_bulk_array(&gen);
let (recipient, identity) = parse_two_bulk_array(&gen_result);
assert!(
recipient.starts_with("age1") && identity.starts_with("AGE-SECRET-KEY-1"),
"Unexpected AGE key formats.\nrecipient: {}\nidentity: {}",