first commit

This commit is contained in:
Timur Gordon
2025-10-20 22:24:25 +02:00
commit 097360ad12
48 changed files with 6712 additions and 0 deletions

125
README.md Normal file
View File

@@ -0,0 +1,125 @@
# OSIRIS
**Object Storage, Indexing & Retrieval Intelligent System**
OSIRIS is a Rust-native object storage and retrieval layer built on top of HeroDB, providing structured storage with metadata, field indexing, and search capabilities.
## Features
- **Object Storage**: Store structured objects with metadata (title, tags, MIME type, timestamps)
- **Namespace Management**: Organize objects into isolated namespaces
- **Field Indexing**: Fast filtering by tags and metadata fields
- **Text Search**: Simple keyword-based search across object content
- **CLI Interface**: Command-line tools for object management and search
- **9P Filesystem**: Mount OSIRIS as a filesystem (future)
## Quick Start
### Prerequisites
Start HeroDB:
```bash
cd /path/to/herodb
cargo run --release -- --dir ./data --admin-secret mysecret --port 6379
```
### Installation
```bash
cd /path/to/osiris
cargo build --release
```
### Initialize
```bash
# Create configuration
mkdir -p ~/.config/osiris
cat > ~/.config/osiris/config.toml <<EOF
[herodb]
url = "redis://localhost:6379"
[namespaces.notes]
db_id = 1
EOF
# Initialize OSIRIS
./target/release/osiris init --herodb redis://localhost:6379
# Create a namespace
./target/release/osiris ns create notes
```
### Usage
```bash
# Add objects
./target/release/osiris put notes/my-note.md ./my-note.md --tags topic=rust,project=osiris
# Get objects
./target/release/osiris get notes/my-note.md
# Search
./target/release/osiris find --ns notes --filter topic=rust
./target/release/osiris find "retrieval" --ns notes
# Delete objects
./target/release/osiris del notes/my-note.md
# Show statistics
./target/release/osiris stats --ns notes
```
## Architecture
```
HeroDB (unmodified)
├── KV store + encryption
└── RESP protocol
└── OSIRIS
├── store/ object schema + persistence
├── index/ field index & keyword scanning
├── retrieve/ query planner + filtering
├── interfaces/ CLI, 9P
└── config/ namespaces + settings
```
## Data Model
Objects are stored with metadata:
- **ID**: Unique identifier (UUID or user-assigned)
- **Namespace**: Logical grouping (e.g., "notes", "calendar")
- **Title**: Optional human-readable title
- **MIME Type**: Content type
- **Tags**: Key-value pairs for categorization
- **Timestamps**: Created and updated times
- **Text Content**: Optional plain text content
## Keyspace Design
```
meta:<id> → serialized OsirisObject
field:<field>:<val> → Set of IDs (for equality filtering)
scan:index → list of IDs for text scan
```
Example:
```
field:tag:project=osiris → {note_1, note_2}
field:mime:text/markdown → {note_1, note_3}
```
## Future Enhancements
- Content-addressable deduplication
- Vector embeddings for semantic search
- Relation graphs
- Full-text search with Tantivy
- 9P filesystem interface
## License
See LICENSE file.