...
This commit is contained in:
225
src/core/kvs/README.md
Normal file
225
src/core/kvs/README.md
Normal file
@@ -0,0 +1,225 @@
|
||||
# Key-Value Store (KVS) Module
|
||||
|
||||
This module provides a simple key-value store implementation with dual backends:
|
||||
- IndexedDB for WebAssembly applications running in browsers
|
||||
- In-memory storage for testing and non-browser environments
|
||||
|
||||
## Overview
|
||||
|
||||
The KVS module provides a simple, yet powerful interface for storing and retrieving data. In a browser environment, it uses IndexedDB as the underlying storage mechanism, which provides a robust, persistent storage solution that works offline and can handle large amounts of data. In non-browser environments, it uses an in-memory store for testing purposes.
|
||||
|
||||
## Features
|
||||
|
||||
- **Simple API**: Easy-to-use methods for common operations like get, set, delete
|
||||
- **Type Safety**: Generic methods that preserve your data types through serialization/deserialization
|
||||
- **Error Handling**: Comprehensive error types for robust error handling
|
||||
- **Async/Await**: Modern async interface for all operations
|
||||
- **Serialization**: Automatic serialization/deserialization of complex data types
|
||||
|
||||
## Core Components
|
||||
|
||||
### KvsStore
|
||||
|
||||
The main struct that provides access to the key-value store, with different implementations based on the environment:
|
||||
|
||||
```rust
|
||||
// In WebAssembly environments (browsers)
|
||||
pub struct KvsStore {
|
||||
db: Arc<Database>,
|
||||
store_name: String,
|
||||
}
|
||||
|
||||
// In non-WebAssembly environments (for testing)
|
||||
pub struct KvsStore {
|
||||
data: Arc<Mutex<HashMap<String, String>>>,
|
||||
}
|
||||
```
|
||||
|
||||
### Error Types
|
||||
|
||||
The module defines several error types to handle different failure scenarios:
|
||||
|
||||
```rust
|
||||
pub enum KvsError {
|
||||
Idb(String),
|
||||
KeyNotFound(String),
|
||||
Serialization(String),
|
||||
Deserialization(String),
|
||||
Other(String),
|
||||
}
|
||||
```
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Opening a Store
|
||||
|
||||
```rust
|
||||
let store = KvsStore::open("my_database", "my_store").await?;
|
||||
```
|
||||
|
||||
### Storing Values
|
||||
|
||||
```rust
|
||||
// Store a simple string
|
||||
store.set("string_key", &"Hello, world!").await?;
|
||||
|
||||
// Store a complex object
|
||||
let user = User {
|
||||
id: 1,
|
||||
name: "John Doe".to_string(),
|
||||
email: "john@example.com".to_string(),
|
||||
};
|
||||
store.set("user_1", &user).await?;
|
||||
```
|
||||
|
||||
### Retrieving Values
|
||||
|
||||
```rust
|
||||
// Get a string
|
||||
let value: String = store.get("string_key").await?;
|
||||
|
||||
// Get a complex object
|
||||
let user: User = store.get("user_1").await?;
|
||||
```
|
||||
|
||||
### Checking if a Key Exists
|
||||
|
||||
```rust
|
||||
if store.contains("user_1").await? {
|
||||
// Key exists
|
||||
}
|
||||
```
|
||||
|
||||
### Deleting Values
|
||||
|
||||
```rust
|
||||
store.delete("user_1").await?;
|
||||
```
|
||||
|
||||
### Listing All Keys
|
||||
|
||||
```rust
|
||||
let keys = store.keys().await?;
|
||||
for key in keys {
|
||||
println!("Found key: {}", key);
|
||||
}
|
||||
```
|
||||
|
||||
### Clearing the Store
|
||||
|
||||
```rust
|
||||
store.clear().await?;
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
The module uses a custom `Result` type that wraps `KvsError`:
|
||||
|
||||
```rust
|
||||
type Result<T> = std::result::Result<T, KvsError>;
|
||||
```
|
||||
|
||||
Example of error handling:
|
||||
|
||||
```rust
|
||||
match store.get::<User>("nonexistent_key").await {
|
||||
Ok(user) => {
|
||||
// Process user
|
||||
},
|
||||
Err(KvsError::KeyNotFound(key)) => {
|
||||
println!("Key not found: {}", key);
|
||||
},
|
||||
Err(e) => {
|
||||
println!("An error occurred: {}", e);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation Details
|
||||
|
||||
The KVS module uses:
|
||||
|
||||
- **Dual backend architecture**:
|
||||
- IndexedDB for browser environments via the `idb` crate (direct Rust implementation)
|
||||
- In-memory HashMap for testing and non-browser environments
|
||||
- **Conditional compilation** with `#[cfg(target_arch = "wasm32")]` to select the appropriate implementation
|
||||
- **Serde** for serialization/deserialization
|
||||
- **Wasm-bindgen** for JavaScript interop in browser environments
|
||||
- **Async/await** for non-blocking operations
|
||||
- **Arc and Mutex** for thread-safe access to the in-memory store
|
||||
|
||||
Note: This implementation uses the `idb` crate to interact with IndexedDB directly from Rust, eliminating the need for a JavaScript bridge file.
|
||||
|
||||
## Testing
|
||||
|
||||
The module includes comprehensive tests in `src/tests/kvs_tests.rs` that verify all functionality works as expected.
|
||||
|
||||
### Running the Tests
|
||||
|
||||
Thanks to the dual implementation, tests can be run in two ways:
|
||||
|
||||
#### Standard Rust Tests
|
||||
|
||||
The in-memory implementation allows tests to run in a standard Rust environment without requiring a browser:
|
||||
|
||||
```bash
|
||||
cargo test
|
||||
```
|
||||
|
||||
This runs all tests using the in-memory implementation, which is perfect for CI/CD pipelines and quick development testing.
|
||||
|
||||
#### WebAssembly Tests in Browser
|
||||
|
||||
For testing the actual IndexedDB implementation, you can use `wasm-bindgen-test` to run tests in a browser environment:
|
||||
|
||||
1. **Install wasm-pack if you haven't already**:
|
||||
```bash
|
||||
cargo install wasm-pack
|
||||
```
|
||||
|
||||
2. **Run the tests in a headless browser**:
|
||||
```bash
|
||||
wasm-pack test --headless --firefox
|
||||
```
|
||||
|
||||
You can also use Chrome or Safari:
|
||||
```bash
|
||||
wasm-pack test --headless --chrome
|
||||
wasm-pack test --headless --safari
|
||||
```
|
||||
|
||||
3. **Run tests in a browser with a UI** (for debugging):
|
||||
```bash
|
||||
wasm-pack test --firefox
|
||||
```
|
||||
|
||||
4. **Run specific tests**:
|
||||
```bash
|
||||
wasm-pack test --firefox -- --filter kvs_tests
|
||||
```
|
||||
|
||||
### Test Structure
|
||||
|
||||
The tests are organized to test each functionality of the KVS module:
|
||||
|
||||
1. **Basic Operations**: Tests for opening a store, setting/getting values
|
||||
2. **Complex Data**: Tests for storing and retrieving complex objects
|
||||
3. **Error Handling**: Tests for handling nonexistent keys and errors
|
||||
4. **Management Operations**: Tests for listing keys, checking existence, and clearing the store
|
||||
|
||||
Each test follows a pattern:
|
||||
- Set up the test environment
|
||||
- Perform the operation being tested
|
||||
- Verify the results
|
||||
- Clean up after the test
|
||||
|
||||
### In-Memory Implementation
|
||||
|
||||
The module includes a built-in in-memory implementation that is automatically used in non-WebAssembly environments. This implementation:
|
||||
|
||||
- Uses a `HashMap<String, String>` wrapped in `Arc<Mutex<>>` for thread safety
|
||||
- Provides the same API as the IndexedDB implementation
|
||||
- Automatically serializes/deserializes values using serde_json
|
||||
- Makes testing much easier by eliminating the need for a browser environment
|
||||
|
||||
This dual implementation approach means you don't need to create separate mocks for testing - the module handles this automatically through conditional compilation.
|
||||
47
src/core/kvs/error.rs
Normal file
47
src/core/kvs/error.rs
Normal file
@@ -0,0 +1,47 @@
|
||||
//! Error types for the key-value store.
|
||||
|
||||
use std::fmt;
|
||||
|
||||
/// Errors that can occur when using the key-value store.
|
||||
#[derive(Debug)]
|
||||
pub enum KvsError {
|
||||
/// Error from the idb crate
|
||||
Idb(String),
|
||||
/// Key not found
|
||||
KeyNotFound(String),
|
||||
/// Serialization error
|
||||
Serialization(String),
|
||||
/// Deserialization error
|
||||
Deserialization(String),
|
||||
/// Other error
|
||||
Other(String),
|
||||
}
|
||||
|
||||
impl fmt::Display for KvsError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
KvsError::Idb(msg) => write!(f, "IndexedDB error: {}", msg),
|
||||
KvsError::KeyNotFound(key) => write!(f, "Key not found: {}", key),
|
||||
KvsError::Serialization(msg) => write!(f, "Serialization error: {}", msg),
|
||||
KvsError::Deserialization(msg) => write!(f, "Deserialization error: {}", msg),
|
||||
KvsError::Other(msg) => write!(f, "Error: {}", msg),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for KvsError {}
|
||||
|
||||
impl From<idb::Error> for KvsError {
|
||||
fn from(err: idb::Error) -> Self {
|
||||
KvsError::Idb(err.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
impl From<serde_json::Error> for KvsError {
|
||||
fn from(err: serde_json::Error) -> Self {
|
||||
KvsError::Serialization(err.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
/// Result type for key-value store operations.
|
||||
pub type Result<T> = std::result::Result<T, KvsError>;
|
||||
7
src/core/kvs/mod.rs
Normal file
7
src/core/kvs/mod.rs
Normal file
@@ -0,0 +1,7 @@
|
||||
//! A simple key-value store implementation using IndexedDB.
|
||||
|
||||
pub mod error;
|
||||
pub mod store;
|
||||
|
||||
pub use error::{KvsError, Result};
|
||||
pub use store::KvsStore;
|
||||
317
src/core/kvs/store.rs
Normal file
317
src/core/kvs/store.rs
Normal file
@@ -0,0 +1,317 @@
|
||||
//! Implementation of a simple key-value store using IndexedDB for WebAssembly
|
||||
//! and an in-memory store for testing.
|
||||
|
||||
use crate::core::kvs::error::{KvsError, Result};
|
||||
use serde::{de::DeserializeOwned, Serialize};
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
use {
|
||||
idb::{Database, DatabaseEvent, Factory, TransactionMode},
|
||||
js_sys::Promise,
|
||||
wasm_bindgen::prelude::*,
|
||||
wasm_bindgen_futures::JsFuture,
|
||||
};
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
impl From<JsValue> for KvsError {
|
||||
fn from(err: JsValue) -> Self {
|
||||
KvsError::Other(format!("JavaScript error: {:?}", err))
|
||||
}
|
||||
}
|
||||
|
||||
/// A simple key-value store.
|
||||
///
|
||||
/// In WebAssembly environments, this uses IndexedDB.
|
||||
/// In non-WebAssembly environments, this uses an in-memory store for testing.
|
||||
#[derive(Clone)]
|
||||
pub struct KvsStore {
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
data: Arc<Mutex<HashMap<String, String>>>,
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
db: Arc<Database>,
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
store_name: String,
|
||||
}
|
||||
|
||||
impl KvsStore {
|
||||
/// Opens a new key-value store with the given name.
|
||||
///
|
||||
/// # Arguments
|
||||
///
|
||||
/// * `db_name` - The name of the database
|
||||
/// * `store_name` - The name of the object store
|
||||
///
|
||||
/// # Returns
|
||||
///
|
||||
/// A new `KvsStore` instance
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub async fn open(_db_name: &str, _store_name: &str) -> Result<Self> {
|
||||
// In non-WASM environments, use an in-memory store for testing
|
||||
Ok(Self {
|
||||
data: Arc::new(Mutex::new(HashMap::new())),
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
pub async fn open(db_name: &str, store_name: &str) -> Result<Self> {
|
||||
let factory = Factory::new()?;
|
||||
let mut db_req = factory.open(db_name, Some(1))?;
|
||||
|
||||
db_req.on_upgrade_needed(|event| {
|
||||
let db = event.database()?;
|
||||
if !db.object_store_names().includes(&JsValue::from_str(store_name)) {
|
||||
db.create_object_store(store_name, None)?;
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
|
||||
let db = Arc::new(db_req.await?);
|
||||
|
||||
Ok(Self {
|
||||
db,
|
||||
store_name: store_name.to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Stores a value with the given key.
|
||||
///
|
||||
/// # Arguments
|
||||
///
|
||||
/// * `key` - The key to store the value under
|
||||
/// * `value` - The value to store
|
||||
///
|
||||
/// # Returns
|
||||
///
|
||||
/// `Ok(())` if the operation was successful
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub async fn set<K, V>(&self, key: K, value: &V) -> Result<()>
|
||||
where
|
||||
K: ToString,
|
||||
V: Serialize,
|
||||
{
|
||||
let key_str = key.to_string();
|
||||
let serialized = serde_json::to_string(value)?;
|
||||
|
||||
let mut data = self.data.lock().unwrap();
|
||||
data.insert(key_str, serialized);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
pub async fn set<K, V>(&self, key: K, value: &V) -> Result<()>
|
||||
where
|
||||
K: ToString + Into<JsValue>,
|
||||
V: Serialize,
|
||||
{
|
||||
let tx = self.db.transaction(&[&self.store_name], TransactionMode::ReadWrite)?;
|
||||
let store = tx.object_store(&self.store_name)?;
|
||||
|
||||
let serialized = serde_json::to_string(value)?;
|
||||
store.put_with_key(&JsValue::from_str(&serialized), &key.into())?;
|
||||
|
||||
JsFuture::from(tx.done()).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Retrieves a value for the given key.
|
||||
///
|
||||
/// # Arguments
|
||||
///
|
||||
/// * `key` - The key to retrieve the value for
|
||||
///
|
||||
/// # Returns
|
||||
///
|
||||
/// The value if found, or `Err(KvsError::KeyNotFound)` if not found
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub async fn get<K, V>(&self, key: K) -> Result<V>
|
||||
where
|
||||
K: ToString,
|
||||
V: DeserializeOwned,
|
||||
{
|
||||
let key_str = key.to_string();
|
||||
let data = self.data.lock().unwrap();
|
||||
|
||||
match data.get(&key_str) {
|
||||
Some(serialized) => {
|
||||
let value = serde_json::from_str(serialized)?;
|
||||
Ok(value)
|
||||
},
|
||||
None => Err(KvsError::KeyNotFound(key_str)),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
pub async fn get<K, V>(&self, key: K) -> Result<V>
|
||||
where
|
||||
K: ToString + Into<JsValue>,
|
||||
V: DeserializeOwned,
|
||||
{
|
||||
let tx = self.db.transaction(&[&self.store_name], TransactionMode::ReadOnly)?;
|
||||
let store = tx.object_store(&self.store_name)?;
|
||||
|
||||
let request = store.get(&key.into())?;
|
||||
let promise = Promise::from(request);
|
||||
let result = JsFuture::from(promise).await?;
|
||||
|
||||
if result.is_undefined() {
|
||||
return Err(KvsError::KeyNotFound(key.to_string()));
|
||||
}
|
||||
|
||||
let value_str = result.as_string().ok_or_else(|| {
|
||||
KvsError::Deserialization("Failed to convert value to string".to_string())
|
||||
})?;
|
||||
|
||||
let value = serde_json::from_str(&value_str)?;
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
/// Deletes a value for the given key.
|
||||
///
|
||||
/// # Arguments
|
||||
///
|
||||
/// * `key` - The key to delete
|
||||
///
|
||||
/// # Returns
|
||||
///
|
||||
/// `Ok(())` if the operation was successful
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub async fn delete<K>(&self, key: K) -> Result<()>
|
||||
where
|
||||
K: ToString,
|
||||
{
|
||||
let key_str = key.to_string();
|
||||
let mut data = self.data.lock().unwrap();
|
||||
|
||||
if data.remove(&key_str).is_some() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(KvsError::KeyNotFound(key_str))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
pub async fn delete<K>(&self, key: K) -> Result<()>
|
||||
where
|
||||
K: ToString + Into<JsValue>,
|
||||
{
|
||||
let tx = self.db.transaction(&[&self.store_name], TransactionMode::ReadWrite)?;
|
||||
let store = tx.object_store(&self.store_name)?;
|
||||
|
||||
// First check if the key exists
|
||||
let request = store.count_with_key(&key.into())?;
|
||||
let promise = Promise::from(request);
|
||||
let result = JsFuture::from(promise).await?;
|
||||
|
||||
let count = result.as_f64().unwrap_or(0.0);
|
||||
if count <= 0.0 {
|
||||
return Err(KvsError::KeyNotFound(key.to_string()));
|
||||
}
|
||||
|
||||
store.delete(&key.into())?;
|
||||
|
||||
JsFuture::from(tx.done()).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Checks if a key exists in the store.
|
||||
///
|
||||
/// # Arguments
|
||||
///
|
||||
/// * `key` - The key to check
|
||||
///
|
||||
/// # Returns
|
||||
///
|
||||
/// `true` if the key exists, `false` otherwise
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub async fn contains<K>(&self, key: K) -> Result<bool>
|
||||
where
|
||||
K: ToString,
|
||||
{
|
||||
let key_str = key.to_string();
|
||||
let data = self.data.lock().unwrap();
|
||||
|
||||
Ok(data.contains_key(&key_str))
|
||||
}
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
pub async fn contains<K>(&self, key: K) -> Result<bool>
|
||||
where
|
||||
K: ToString + Into<JsValue>,
|
||||
{
|
||||
let tx = self.db.transaction(&[&self.store_name], TransactionMode::ReadOnly)?;
|
||||
let store = tx.object_store(&self.store_name)?;
|
||||
|
||||
let request = store.count_with_key(&key.into())?;
|
||||
let promise = Promise::from(request);
|
||||
let result = JsFuture::from(promise).await?;
|
||||
|
||||
let count = result.as_f64().unwrap_or(0.0);
|
||||
Ok(count > 0.0)
|
||||
}
|
||||
|
||||
/// Lists all keys in the store.
|
||||
///
|
||||
/// # Returns
|
||||
///
|
||||
/// A vector of keys as strings
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub async fn keys(&self) -> Result<Vec<String>> {
|
||||
let data = self.data.lock().unwrap();
|
||||
|
||||
Ok(data.keys().cloned().collect())
|
||||
}
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
pub async fn keys(&self) -> Result<Vec<String>> {
|
||||
let tx = self.db.transaction(&[&self.store_name], TransactionMode::ReadOnly)?;
|
||||
let store = tx.object_store(&self.store_name)?;
|
||||
|
||||
let request = store.get_all_keys(None)?;
|
||||
let promise = Promise::from(request);
|
||||
let result = JsFuture::from(promise).await?;
|
||||
|
||||
let keys_array = js_sys::Array::from(&result);
|
||||
let mut keys = Vec::new();
|
||||
|
||||
for i in 0..keys_array.length() {
|
||||
let key = keys_array.get(i);
|
||||
if let Some(key_str) = key.as_string() {
|
||||
keys.push(key_str);
|
||||
} else {
|
||||
// Try to convert non-string keys to string
|
||||
keys.push(format!("{:?}", key));
|
||||
}
|
||||
}
|
||||
|
||||
Ok(keys)
|
||||
}
|
||||
|
||||
/// Clears all key-value pairs from the store.
|
||||
///
|
||||
/// # Returns
|
||||
///
|
||||
/// `Ok(())` if the operation was successful
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub async fn clear(&self) -> Result<()> {
|
||||
let mut data = self.data.lock().unwrap();
|
||||
data.clear();
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
pub async fn clear(&self) -> Result<()> {
|
||||
let tx = self.db.transaction(&[&self.store_name], TransactionMode::ReadWrite)?;
|
||||
let store = tx.object_store(&self.store_name)?;
|
||||
|
||||
store.clear()?;
|
||||
|
||||
JsFuture::from(tx.done()).await?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
@@ -4,8 +4,10 @@ pub mod error;
|
||||
pub mod keypair;
|
||||
pub mod symmetric;
|
||||
pub mod ethereum;
|
||||
pub mod kvs;
|
||||
|
||||
// Re-export commonly used items for internal use
|
||||
// (Keeping this even though it's currently unused, as it's good practice for internal modules)
|
||||
#[allow(unused_imports)]
|
||||
pub use error::CryptoError;
|
||||
pub use error::CryptoError;
|
||||
pub use kvs::{KvsStore as KvStore, KvsError as KvError, Result as KvResult};
|
||||
Reference in New Issue
Block a user