Merge pull request #217 from Incubaid/development-docs

Enhance AI Prompts Files

aiprompts/README.md

@@ -0,0 +1,77 @@
+# HeroLib AI Prompts (`aiprompts/`)
+
+This directory contains AI-oriented instructions and manuals for working with the Hero tool and the `herolib` codebase.
+
+It is the **entry point for AI agents** that generate or modify code/docs in this repository.
+
+## Scope
+
+- **Global rules for AI and V/Hero usage**  
+  See:
+  - `herolib_start_here.md`
+  - `vlang_herolib_core.md`
+- **Herolib core modules**  
+  See:
+  - `herolib_core/` (core HeroLib modules)
+  - `herolib_advanced/` (advanced topics)
+- **Docusaurus & Site module (Hero docs)**  
+  See:
+  - `docusaurus/docusaurus_ebook_manual.md`
+  - `lib/web/docusaurus/README.md` (authoritative module doc)
+  - `lib/web/site/ai_instructions.md` and `lib/web/site/readme.md`
+- **HeroModels / HeroDB**  
+  See:
+  - `ai_instructions_hero_models.md`
+  - `heromodel_instruct.md`
+- **V language & web server docs** (upstream-style, mostly language-level)  
+  See:
+  - `v_core/`, `v_advanced/`  
+  - `v_veb_webserver/`
+
+## Sources of Truth
+
+For any domain, **code and module-level docs are authoritative**:
+
+- Core install & usage: `herolib/README.md`, scripts under `scripts/`
+- Site module: `lib/web/site/ai_instructions.md`, `lib/web/site/readme.md`
+- Docusaurus module: `lib/web/docusaurus/README.md`, `lib/web/docusaurus/*.v`
+- Atlas client: `lib/data/atlas/client/README.md`
+- HeroModels: `lib/hero/heromodels/*.v` + tests
+
+`aiprompts/` files **must not contradict** these. When in doubt, follow the code / module docs first and treat prompts as guidance.
+
+## Directory Overview
+
+- `herolib_start_here.md` / `vlang_herolib_core.md`  
+  Global AI rules and V/Hero basics.
+- `herolib_core/` & `herolib_advanced/`  
+  Per-module instructions for core/advanced HeroLib features.
+- `docusaurus/`  
+  AI manual for building Hero docs/ebooks with the Docusaurus + Site + Atlas pipeline.
+- `instructions/`  
+  Active, higher-level instructions (e.g. HeroDB base filesystem).
+- `instructions_archive/`  
+  **Legacy / historical** prompt material. See `instructions_archive/README.md`.
+- `todo/`  
+  Meta design/refactor notes (not up-to-date instructions for normal usage).
+- `v_core/`, `v_advanced/`, `v_veb_webserver/`  
+  V language and web framework references used when generating V code.
+- `bizmodel/`, `unpolly/`, `doctree/`, `documentor/`  
+  Domain-specific or feature-specific instructions.
+
+## How to Treat Legacy Material
+
+- Content under `instructions_archive/` is **kept for reference** and may describe older flows (e.g. older documentation or prompt pipelines).  
+  Do **not** use it as a primary source for new work unless explicitly requested.
+- Some prompts mention **Doctree**; the current default docs pipeline uses **Atlas**. Doctree/`doctreeclient` is an alternative/legacy backend.
+
+## Guidelines for AI Agents
+
+- Always:
+  - Respect global rules in `herolib_start_here.md` and `vlang_herolib_core.md`.
+  - Prefer module docs under `lib/` when behavior or parameters differ.
+  - Avoid modifying generated files (e.g. `*_ .v` or other generated artifacts) as instructed.
+- When instructions conflict, resolve as:
+  1. **Code & module docs in `lib/`**
+  2. **AI instructions in `aiprompts/`**
+  3. **Archived docs (`instructions_archive/`) only when explicitly needed**.

aiprompts/ai_instructions_hero_models.md

@@ -4,7 +4,7 @@
 
 This document provides clear instructions for AI agents to create new HeroDB models similar to `message.v`.
 These models are used to store structured data in Redis using the HeroDB system.
-The message.v can be found in `lib/hero/heromodels/message.v`.s
+The `message.v` example can be found in `lib/hero/heromodels/message.v`.
 
 ## Key Concepts
 
@@ -108,7 +108,7 @@ Add your model to the ModelsFactory struct in `factory.v`:
 ```v
 pub struct ModelsFactory {
 pub mut:
- messages DBCalendar
+ calendar DBCalendar
  // ... other models
 }
 ```

aiprompts/doctree/doctree_specs.md

@@ -1,3 +1,5 @@
+> NOTE: Atlas is the default document collections/export pipeline used by the current Docusaurus integration (see `lib/data/atlas/client`). The Doctree module described here is an alternative/legacy export mechanism that still exists but is not the primary path. Use Atlas by default unless you explicitly need Doctree.
+
 # Doctree Export Specification
 
 ## Overview

aiprompts/docusaurus/docusaurus_ebook_manual.md

@@ -8,7 +8,7 @@ To effectively create ebooks with HeroLib, it's crucial to understand the interp
 
 * **HeroScript**: A concise scripting language used to define the structure, configuration, and content flow of your Docusaurus site. It acts as the declarative interface for the entire process.
 * **Docusaurus**: A popular open-source static site generator. HeroLib uses Docusaurus as the underlying framework to render your ebook content into a navigable website.
-* **Doctree**: HeroLib's content management system. Doctree organizes your markdown files into "collections" and "pages," allowing for structured content retrieval and reuse across multiple projects.
+* **Atlas (and Doctree)**: HeroLib's document collection layer. In the current pipeline, Atlas exports markdown "collections" and "pages" that Docusaurus consumes via the Atlas client. Doctree and `doctreeclient` are legacy/alternative ways to provide the same collections.
 
 ## 2. Setting Up a Docusaurus Project with HeroLib
 
@@ -22,18 +22,26 @@ The `docusaurus.define` HeroScript directive configures the global settings for
 
 ```heroscript
 !!docusaurus.define
+    name:"my_ebook"                  // must match the site name from !!site.config
     path_build: "/tmp/my_ebook_build"
     path_publish: "/tmp/my_ebook_publish"
-    production: true
+    reset: true                      // clean build dir before building (optional)
-    update: true
+    install: true                    // run bun install if needed (optional)
+    template_update: true            // update the Docusaurus template (optional)
+    atlas_dir: "/tmp/atlas_export"   // where Atlas exports collections
+    use_atlas: true                  // use Atlas as content backend
 ```
 
 **Arguments:**
 
+* `name` (string, required): The site/factory name. Must match the `name` used in `!!site.config` so Docusaurus can find the corresponding site definition.
 * `path_build` (string, optional): The local path where the Docusaurus site will be built. Defaults to `~/hero/var/docusaurus/build`.
 * `path_publish` (string, optional): The local path where the final Docusaurus site will be published (e.g., for deployment). Defaults to `~/hero/var/docusaurus/publish`.
-* `production` (boolean, optional): If `true`, the site will be built for production (optimized). Default is `false`.
+* `reset` (boolean, optional): If `true`, clean the build directory before starting.
-* `update` (boolean, optional): If `true`, the Docusaurus template and dependencies will be updated. Default is `false`.
+* `install` (boolean, optional): If `true`, run dependency installation (e.g., `bun install`).
+* `template_update` (boolean, optional): If `true`, update the Docusaurus template.
+* `atlas_dir` (string, optional): Directory where Atlas exports collections (used by the Atlas client in `lib/data/atlas/client`).
+* `use_atlas` (boolean, optional): If `true`, use the Atlas client as the content backend (default behavior).
 
 ### 2.2. Adding a Docusaurus Site (`docusaurus.add`)
 
@@ -53,7 +61,7 @@ The `docusaurus.add` directive defines an individual Docusaurus site (your ebook
 ```heroscript
 !!docusaurus.add
     name:"tfgrid_tech_ebook"
-    git_url:"https://git.threefold.info/tfgrid/docs_tfgrid4/src/branch/main/ebooks/tech"
+    git_url:"https://git.ourworld.tf/tfgrid/docs_tfgrid4/src/branch/main/ebooks/tech"
     git_reset:true // Reset Git repository before pulling
     git_pull:true // Pull latest changes
     git_root:"/tmp/git_clones" // Optional: specify a root directory for git clones
@@ -190,18 +198,18 @@ Configure the footer section of your Docusaurus site.
   * `href` (string, optional): External URL for the link.
   * `to` (string, optional): Internal Docusaurus path.
 
-### 3.4. Build Destinations (`site.build_dest`, `site.build_dest_dev`)
+### 3.4. Publish Destinations (`site.publish`, `site.publish_dev`)
 
 Specify where the built Docusaurus site should be deployed. This typically involves an SSH connection defined elsewhere (e.g., `!!site.ssh_connection`).
 
 **HeroScript Example:**
 
 ```heroscript
-!!site.build_dest
+!!site.publish
     ssh_name:"production_server" // Name of a pre-defined SSH connection
-    path:"/var/www/my-ebook" // Remote path on the server
+    path:"/var/www/my-ebook"     // Remote path on the server
 
-!!site.build_dest_dev
+!!site.publish_dev
     ssh_name:"dev_server"
     path:"/tmp/dev-ebook"
 ```
@@ -219,7 +227,7 @@ This powerful feature allows you to pull markdown content and assets from other
 
 ```heroscript
 !!site.import
-    url:'https://git.threefold.info/tfgrid/docs_tfgrid4/src/branch/main/collections/cloud_reinvented'
+    url:'https://git.ourworld.tf/tfgrid/docs_tfgrid4/src/branch/main/collections/cloud_reinvented'
     dest:'cloud_reinvented' // Destination subdirectory within your Docusaurus docs folder
     replace:'NAME:MyName, URGENCY:red' // Optional: comma-separated key:value pairs for text replacement
 ```
@@ -265,22 +273,29 @@ This is where you define the actual content pages and how they are organized int
   * `draft` (boolean, optional): If `true`, the page will be marked as a draft and not included in production builds.
   * `title_nr` (int, optional): If set, HeroLib will re-number the markdown headings (e.g., `title_nr:3` will make `# Heading` become `### Heading`). Useful for consistent heading levels across imported content.
 
-### 3.7. Doctree Integration Details
+### 3.7. Collections and Atlas/Doctree Integration
 
-The `site.page` directive's `src` parameter (`collection_name:page_name`) is the bridge to your Doctree content.
+The `site.page` directive's `src` parameter (`collection_name:page_name`) is the bridge to your content collections.
 
-**How Doctree Works:**
+**Current default: Atlas export**
+
+1. **Collections**: Atlas exports markdown files into collections under an `export_dir` (see `lib/data/atlas/client`).
+2. **Export step**: A separate process (Atlas) writes the collections into `atlas_dir` (e.g., `/tmp/atlas_export`), following the `content/` + `meta/` structure.
+3. **Docusaurus consumption**: The Docusaurus module uses the Atlas client (`atlas_client`) to resolve `collection_name:page_name` into markdown content and assets when generating docs.
+
+**Alternative: Doctree/`doctreeclient`**
+
+In older setups, or when explicitly configured, Doctree and `doctreeclient` can still be used to provide the same `collection:page` model:
 
 1. **Collections**: Doctree organizes markdown files into logical groups called "collections." A collection is typically a directory containing markdown files and an empty `.collection` file.
-2. **Scanning**: You define which collections Doctree should scan using `!!doctree.scan` in a HeroScript file (e.g., `doctree.heroscript`).
+2. **Scanning**: You define which collections Doctree should scan using `!!doctree.scan` in a HeroScript file (e.g., `doctree.heroscript`):
-    **Example `doctree.heroscript`:**
 
     ```heroscript
-    !!doctree.scan git_url:"https://git.threefold.info/tfgrid/docs_tfgrid4/src/branch/main/collections"
+    !!doctree.scan git_url:"https://git.ourworld.tf/tfgrid/docs_tfgrid4/src/branch/main/collections"
     ```
 
     This will pull the `collections` directory from the specified Git URL and make its contents available to Doctree.
-3. **Page Retrieval**: When `site.page` references `src:"my_collection:my_page"`, HeroLib's `doctreeclient` fetches the content of `my_page.md` from the `my_collection` collection that Doctree has scanned.
+3. **Page Retrieval**: When `site.page` references `src:"my_collection:my_page"`, the client (`atlas_client` or `doctreeclient`, depending on configuration) fetches the content of `my_page.md` from the `my_collection` collection.
 
 ## 4. Building and Developing Your Ebook
 

aiprompts/herolib_core/core_heroscript_basics.md

@@ -35,11 +35,11 @@ pub fn play(mut plbook PlayBook) ! {
  if plbook.exists_once(filter: 'docusaurus.define') {
   mut action := plbook.get(filter: 'docusaurus.define')!
   mut p := action.params
-  //example how we get parameters from the action see core_params.md for more details
+  //example how we get parameters from the action see aiprompts/herolib_core/core_params.md for more details
-  ds = new(
+  path_build := p.get_default('path_build', '')!
-   path: p.get_default('path_publish', '')!
+  path_publish := p.get_default('path_publish', '')!
-   production:   p.get_default_false('production')
+  reset := p.get_default_false('reset')
-  )!
+  use_atlas := p.get_default_false('use_atlas')
  }
 
  // Process 'docusaurus.add' actions to configure individual Docusaurus sites
@@ -51,4 +51,4 @@ pub fn play(mut plbook PlayBook) ! {
 }
 ```
 
-For detailed information on parameter retrieval methods (e.g., `p.get()`, `p.get_int()`, `p.get_default_true()`), refer to `aiprompts/ai_core/core_params.md`.
+For detailed information on parameter retrieval methods (e.g., `p.get()`, `p.get_int()`, `p.get_default_true()`), refer to `aiprompts/herolib_core/core_params.md`.

aiprompts/instructions/herodb_base_fs.md

@@ -1,3 +1,5 @@
+> NOTE: This document is an example snapshot of a developer's filesystem layout for HeroDB/HeroModels. Paths under `/Users/despiegk/...` are illustrative only. For the current, authoritative structure always use the live repository tree (this checkout) and the modules under `lib/hero/heromodels` and `lib/hero/db`.
+
 <file_map>
 /Users/despiegk/code/github/incubaid/herolib
 ├── .github

aiprompts/instructions_archive/README.md

@@ -0,0 +1,15 @@
+# Instructions Archive (Legacy Prompts)
+
+This directory contains **archived / legacy AI prompt material** for `herolib`.
+
+- Files here may describe **older workflows** (e.g. previous documentation generation or model pipelines).
+- They are kept for **historical reference** and to help understand how things evolved.
+- They are **not** guaranteed to match the current `herolib` implementation.
+
+## Usage Guidelines
+
+- Do **not** use these files as the primary source for new features or refactors.
+- When generating code or documentation, prefer:
+  1. Code and module docs under `lib/` (e.g. `lib/web/site/ai_instructions.md`, `lib/web/docusaurus/README.md`).
+  2. Up-to-date AI instructions under `aiprompts/` (outside of `instructions_archive/`).
+- Only consult this directory when you explicitly need to understand **historical behavior** or migrate old flows.

aiprompts/vlang_herolib_core.md

@@ -122,12 +122,12 @@ pub fn play(mut plbook PlayBook) ! {
  if plbook.exists_once(filter: 'docusaurus.define') {
   mut action := plbook.get(filter: 'docusaurus.define')!
   mut p := action.params
-  //example how we get parameters from the action see core_params.md for more details
+  //example how we get parameters from the action see aiprompts/herolib_core/core_params.md for more details
-  ds = new(
+  path_build := p.get_default('path_build', '')!
-   path: p.get_default('path_publish', '')!
+  path_publish := p.get_default('path_publish', '')!
-   production:   p.get_default_false('production')
+  reset := p.get_default_false('reset')
-  )!
+  use_atlas := p.get_default_false('use_atlas')
- }
+  }
 
  // Process 'docusaurus.add' actions to configure individual Docusaurus sites
  actions := plbook.find(filter: 'docusaurus.add')!
@@ -138,7 +138,7 @@ pub fn play(mut plbook PlayBook) ! {
 }
 ```
 
-For detailed information on parameter retrieval methods (e.g., `p.get()`, `p.get_int()`, `p.get_default_true()`), refer to `aiprompts/ai_core/core_params.md`.
+For detailed information on parameter retrieval methods (e.g., `p.get()`, `p.get_int()`, `p.get_default_true()`), refer to `aiprompts/herolib_core/core_params.md`.
 
 # PlayBook, process heroscripts