Hero Books - Document Management System

A Rust-based document collection management system with CLI, library, and web interfaces for processing markdown-based documentation with support for cross-collection references, link validation, AI processing, vector search, and export to self-contained directories.

# Source your environment variables
source ~/.config/env.sh   # or wherever you keep your secrets

# Build, sync 3 libraries from Forge repos, and run
make run

# Or run the offline demo (7 local example books)
#make demo

# See all available commands
make help

Architecture

Workspace Layout

hero_books/
├── Cargo.toml                       # workspace root
├── Makefile                         # root build automation
├── scripts/buildenv.sh              # project configuration
├── scripts/build_lib.sh             # shared build functions
├── src/                             # root library + server binary
├── templates/                       # Askama HTML templates (dark theme)
├── crates/
│   ├── hero_books_sdk/              # library: generated OpenRPC client (openrpc_client! macro)
│   ├── hero_books_server/           # binary: OpenRPC daemon (Unix socket only)
│   ├── hero_books_admin/            # binary: Axum+HTML admin panel (uses SDK)
│   ├── hero_books_ui/               # binary: user-facing web interface (browsing, search)
│   └── hero_books_examples/         # examples: SDK usage demos

Dependency Graph

hero_books_sdk  (no internal deps)
     ↑         ↑          ↑
     |         |          |
  server     CLI         UI

Sockets

All services communicate via Unix domain sockets only. No TCP ports are used.

Component	Transport	Address
Server (OpenRPC)	Unix socket	~/hero/var/sockets/hero_books_server.sock
Admin (Admin panel)	Unix socket	~/hero/var/sockets/hero_books_admin.sock
UI (User-facing)	Unix socket	~/hero/var/sockets/hero_books_ui.sock
Hero Embedder (dependency)	Unix socket	~/hero/var/sockets/hero_embedder.sock

Binaries

Binary	Source	Purpose
hero_books	src/main.rs	Legacy combined server (being phased out)
hero_books_server	crates/hero_books_server/	OpenRPC daemon (Unix socket only)
hero_books_admin	crates/hero_books_admin/	Axum+HTML admin panel (uses SDK)
hero_books_ui	crates/hero_books_ui/	User-facing web interface (browsing, search)
hero_books_client	src/bin/books.rs	CLI client (uses SDK via Unix socket)

All binaries are installed to ~/hero/bin/.

Features

• Collection scanning: Automatically discover collections marked with .collection files
• Cross-collection references: Link between pages in different collections using collection:page syntax
• Include directives: Embed content from other pages with !!include collection:page
• Link validation: Detect broken links to pages, images, and files
• AI processing: Q&A extraction, semantic search, ontology classification
• Vector search: Embeddings-powered semantic search via hero_embedder
• Export: Generate self-contained directories with all dependencies
• Access control: Group-based ACL via .group files
• Admin dashboard: Web-based admin panel with settings, logs, and import management
• PDF generation: Convert books to PDF via Chrome
• MCP support: Model Context Protocol for AI tool integration

Installation

Install from Binaries

Download the pre-built binary from the Forge package registry:

mkdir -p ~/hero/bin
curl -fsSL -o ~/hero/bin/hero_books \
  "https://forge.ourworld.tf/api/packages/lhumina_code/generic/hero_books/dev/hero_books-linux-amd64"
chmod +x ~/hero/bin/hero_books

Build from Source

git clone https://forge.ourworld.tf/lhumina_code/hero_books
cd hero_books
make build

Install to ~/hero/bin/:

make install

Run Different Documentation Sets

make run              # 3 libraries from Forge repos (36 books, needs network on first run)
make demo             # Local example books (7 books, offline, fast)
make stop             # Stop all services
make status           # Check service status
make logs             # View server logs
make logs-ui          # View UI logs

Environment Variables

This project follows the env_secrets convention. Source your env file before running.

source ~/.config/env.sh
make run

Variables used by Hero Books

Variable	Required	Purpose
GROQ_API_KEY	Yes	Groq API for Q&A extraction and AI summary
OPENROUTER_API_KEY	Yes	OpenRouter API for LLM fallback
SAMBANOVA_API_KEY	No	SambaNova API for additional LLM provider
GIT_TOKEN	No	Personal access token for cloning private repos
HEROBOOKS_PASSWORD	No	HTTP Basic Auth password (username: hero)
HERO_EMBEDDER_URL	No	Override embedder endpoint (default: Unix socket)
HERO_SECRET	No	JWT shared secret for hero_auth integration
HERO_AUTH_URL	No	Hero Auth server URL for admin login

CLI Usage

The hero_books_client CLI talks to a running server via OpenRPC.

# Start server
make run

# List all collections
hero_books_client list

# Scan for collections
hero_books_client scan --path /path/to/docs

# Process for Q&A and embeddings
hero_books_client process my-collection

# Check server health
hero_books_client health

# View OpenRPC schema
hero_books_client discover

# Import from git
hero_books_client import --git-url https://forge.ourworld.tf/mycelium/docs --namespace mycelium

Development

make build        # Build release binaries
make check        # Fast code check
make test         # Run unit tests
make test-all     # Full test suite (fmt, clippy, test, build, smoke, e2e)
make lint         # Run clippy
make fmt          # Format code
make rundev       # Run in debug mode (uses zinit for process management)
make demo         # Run with local example books
make install      # Install release binaries to ~/hero/bin
make installdev   # Install debug binaries to ~/hero/bin

License

Apache-2.0