MahmoudEmad/herolib
1
0
Fork 0
Code Issues 55 Pull Requests 1 Actions Packages Projects Releases 26 Wiki Activity

...

Browse Source
This commit is contained in:
despiegk
2025-10-25 09:44:19 +04:00
parent 9d2dedb2b6
commit e574bcbc50
8 changed files with 1185 additions and 427 deletions
Download Patch File Download Diff File Expand all files Collapse all files

371
aiprompts/unpolly/unpolly_api.md Normal file
View File

@@ -0,0 +1,371 @@
# Unpoly API – Complete Reference & Operational Guide for AI Coding Agents
## Introduction
This document serves as a complete, self-contained reference manual for the **Unpoly JavaScript API**. It is written explicitly for AI coding agents that need to generate, validate, or refactor Unpoly-enhanced HTML and JavaScript code.
Unpoly is a front-end framework for progressive enhancement. It allows HTML pages to update fragments instead of reloading the full document, enabling seamless transitions, modals, caching, and navigation while maintaining server-side rendering.
An AI agent should use this manual to:
- Understand each Unpoly module and its purpose.
- Generate correct `up-*` attributes and JavaScript calls.
- Detect configuration errors and provide correction suggestions.
- Apply Unpoly in automated scaffolding, validation, and optimization workflows.
---
## 1. The up.link Module
### Purpose
Handles Unpoly-enhanced navigation. Converts normal links into AJAX-based fragment updates rather than full-page reloads.
### Core Concepts
When a user clicks a link with certain attributes, Unpoly intercepts the event and fetches the new page in the background. It then replaces specified fragments in the current document with matching elements from the response.
### Common Attributes
| Attribute | Description |
| --------------- | -------------------------------------------------------- |
| `up-follow` | Marks the link as handled by Unpoly. Usually implied. |
| `up-target` | CSS selector identifying which fragment(s) to replace. |
| `up-method` | Overrides HTTP method (e.g. `GET`, `POST`). |
| `up-params` | Adds query parameters to the request. |
| `up-headers` | Adds or overrides HTTP headers. |
| `up-layer` | Determines which layer (page, overlay, modal) to update. |
| `up-transition` | Defines animation during fragment replacement. |
| `up-cache` | Enables caching of the response. |
| `up-history` | Controls browser history behavior. |
### JavaScript API Methods
- `up.link.isFollowable(element)` – Returns true if Unpoly will intercept the link.
- `up.link.follow(element, options)` – Programmatically follow the link via Unpoly.
- `up.link.preload(element, options)` – Preload the linked resource into the cache.
### Agent Reasoning & Validation
- Ensure that every `up-follow` element has a valid `up-target` selector.
- Validate that target elements exist in both the current DOM and the server response.
- Recommend `up-cache` for commonly visited links to improve performance.
- Prevent using `target="_blank"` or `download` attributes with Unpoly links.
### Example
```html
<a href="/profile" up-target="#main" up-transition="fade">View Profile</a>
```
---
## 2. The up.form Module
### Purpose
Handles progressive enhancement for forms. Submissions happen via AJAX and update only specific fragments.
### Core Attributes
| Attribute | Description |
| ---------------- | --------------------------------------- |
| `up-submit` | Marks form to be submitted via Unpoly. |
| `up-target` | Fragment selector to update on success. |
| `up-fail-target` | Selector to update if submission fails. |
| `up-validate` | Enables live field validation. |
| `up-autosubmit` | Submits automatically on change. |
| `up-disable-for` | Disables fields during request. |
| `up-enable-for` | Enables fields after request completes. |
### JavaScript API
- `up.form.submit(form, options)` – Submit programmatically.
- `up.validate(field, options)` – Trigger server validation.
- `up.form.fields(form)` – Returns all input fields.
### Agent Reasoning
- Always ensure form has both `action` and `method` attributes.
- Match `up-target` to an element existing in the rendered HTML.
- For validation, ensure server supports `X-Up-Validate` header.
- When generating forms, add `up-fail-target` to handle errors gracefully.
### Example
```html
<form action="/update" method="POST" up-submit up-target="#user-info" up-fail-target="#form-errors">
<input name="email" up-validate required>
<button type="submit">Save</button>
</form>
```
---
## 3. The up.layer Module
### Purpose
Manages overlays, modals, and stacked layers of navigation.
### Attributes
| Attribute | Description |
| ---------------- | -------------------------------------------------- |
| `up-layer="new"` | Opens content in a new overlay. |
| `up-size` | Controls modal size (e.g., `small`, `large`). |
| `up-dismissable` | Allows overlay to close by clicking outside. |
| `up-history` | Determines if the overlay updates browser history. |
| `up-title` | Sets overlay title. |
### JavaScript API
- `up.layer.open(options)` – Opens a new layer.
- `up.layer.close(layer)` – Closes a given layer.
- `up.layer.on(event, callback)` – Hooks into lifecycle events.
### Agent Notes
- Ensure `up-layer="new"` only used with valid targets.
- For overlays, set `up-history="false"` unless explicitly required.
- Auto-generate dismiss buttons with `up-layer-close`.
### Example
```html
<a href="/settings" up-layer="new" up-size="large" up-target=".modal-content">Open Settings</a>
```
---
## 4. The up.fragment Module
### Purpose
Handles low-level fragment rendering, preserving, replacing, and merging.
### JavaScript API
- `up.render(options)` – Replace fragment(s) with new content.
- `up.fragment.config` – Configure defaults for rendering.
- `up.fragment.get(target)` – Retrieve a fragment.
### Example
```js
up.render({ target: '#main', url: '/dashboard', transition: 'fade' })
```
### Agent Notes
- Ensure only fragment HTML is sent from server (not full document).
- Use `preserve` for elements like forms where input state matters.
---
## 5. The up.network Module
### Purpose
Handles network requests, caching, and aborting background loads.
### JavaScript API
- `up.network.loadPage(url, options)` – Load a page via Unpoly.
- `up.network.abort()` – Abort ongoing requests.
- `up.network.config.timeout` – Default timeout setting.
### Agent Tasks
- Preload probable links (`up.link.preload`).
- Use caching for frequent calls.
- Handle `up:network:late` event to show spinners.
---
## 6. The up.event Module
### Purpose
Manages custom events fired throughout Unpoly’s lifecycle.
### Common Events
- `up:link:follow`
- `up:form:submit`
- `up:layer:open`
- `up:layer:close`
- `up:rendered`
- `up:network:late`
### Example
```js
up.on('up:layer:close', (event) => {
console.log('Overlay closed');
});
```
### Agent Actions
- Register listeners for key events.
- Prevent duplicate bindings.
- Offer analytics hooks for `up:rendered` or `up:location:changed`.
---
## 7. The up.motion Module
Handles animations and transitions.
### API
- `up.motion()` – Animate elements.
- `up.animate(element, keyframes, options)` – Custom animation.
### Agent Notes
- Suggest `up-transition="fade"` or similar for fragment changes.
- Avoid heavy animations for performance-sensitive devices.
---
## 8. The up.radio Module
Handles broadcasting and receiving cross-fragment events.
### Example
```js
up.radio.emit('user:updated', { id: 5 })
up.radio.on('user:updated', (data) => console.log(data))
```
### Agent Tasks
- Use for coordinating multiple fragments.
- Ensure channel names are namespaced (e.g., `form:valid`, `modal:open`).
---
## 9. The up.history Module
### Purpose
Manages URL history, titles, and restoration.
### API
- `up.history.push(url, options)` – Push new history entry.
- `up.history.restore()` – Restore previous state.
### Agent Guidance
- Disable history (`up-history="false"`) for temporary overlays.
- Ensure proper title update via `up-title`.
---
## 10. The up.viewport Module
### Purpose
Manages scrolling, focusing, and viewport restoration.
### API
- `up.viewport.scroll(element)` – Scroll to element.
- `up.viewport.restoreScroll()` – Restore previous position.
### Agent Tasks
- Restore scroll after fragment updates.
- Manage focus for accessibility after `up.render()`.
---
## 11. The up.protocol Module
Handles request headers and special HTTP status codes.
### Key Concepts
- Adds `X-Up-Target` and `X-Up-Validate` headers.
- Interprets redirect and validation responses.
### Agent Notes
- Ensure backend supports Unpoly headers.
- Warn if 204 or redirect status used without fragments.
---
## 12. The up.util Module
Contains helpers for arrays, strings, and timing.
### Examples
```js
up.util.escapeHTML("<script>") // "&lt;script&gt;"
up.util.copy(text)
```
### Agent Tasks
- Use helpers for DOM-safe generation.
- Avoid reimplementing basic utilities.
---
## 13. The up.framework and up.log Modules
- `up.framework.init()` – Initializes Unpoly.
- `up.log.enable()` – Enables debug logging.
### Agent Tasks
- Always call `up.framework.init()` once.
- Enable logging during debug sessions.
---
Good addition — including **Unpoly from a CDN** is essential for setup and automation.
Here’s the section you can append to the Markdown file (I can also regenerate the full file with it included if you prefer):
---
## 14. Including Unpoly from CDN
### Purpose
Before any Unpoly features or attributes (`up-*`) can work, the library must be loaded into the page.
For AI agents generating HTML pages or templates, always include Unpoly’s **CSS** and **JavaScript** assets early in the `<head>` section.
### Basic Setup (Latest Stable Version)
```html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>My Unpoly Page</title>
<!-- Unpoly Core Styles -->
<link rel="stylesheet" href="https://unpkg.com/unpoly@3.7.2/unpoly.min.css">
<!-- Optional Bootstrap integration -->
<link rel="stylesheet" href="https://unpkg.com/unpoly@3.7.2/unpoly-bootstrap3.min.css">
<!-- Unpoly Core Script -->
<script src="https://unpkg.com/unpoly@3.7.2/unpoly.min.js"></script>
<!-- Initialize framework -->
<script>
up.framework.init()
</script>
</head>
<body>
<!-- Page content -->
</body>
</html>
```
### AI Agent Notes
* Always include **`unpoly.min.css`** before any custom styles to ensure default animations and transitions function properly.
* **Never** load Unpoly twice; check if the page already has a `<script src="...unpoly.min.js">`.
* After loading, the agent may immediately use Unpoly JavaScript APIs (`up.render`, `up.link.follow`, etc.) or rely on HTML attributes.
* If dynamically injecting HTML pages, the agent should re-run `up.framework.init()` **only once globally**, not after every fragment load.
### Recommended CDN Sources
* `https://unpkg.com/unpoly@3.x/`
* `https://cdn.jsdelivr.net/npm/unpoly@3.x/`
### Offline Use
For fully offline or embedded environments, the agent can download both `.js` and `.css` files and reference them locally:
```html
<link rel="stylesheet" href="/assets/unpoly.min.css">
<script src="/assets/unpoly.min.js"></script>
```
---
## Agent Validation Checklist
1. Verify `up-*` attributes match existing fragments.
2. Check backend returns valid fragment markup.
3. Ensure forms use `up-submit` and `up-fail-target`.
4. Overlay layers must have dismissable controls.
5. Use caching wisely (`up-cache`, `up.link.preload`).
6. Handle network and render events gracefully.
7. Log events (`up.log`) for debugging.
8. Confirm scroll/focus restoration after renders.
9. Gracefully degrade if JavaScript disabled.
10. Document reasoning and configuration.

647
aiprompts/unpolly/unpolly_core.md Normal file
View File

@@ -0,0 +1,647 @@
# Unpoly Quick Reference for AI Agents
## Installation
Include Unpoly from CDN in your HTML `<head>`:
```html
<script src="https://unpoly.com/unpoly.min.js"></script>
<link rel="stylesheet" href="https://unpoly.com/unpoly.min.css">
```
## Core Concept
Unpoly updates page fragments without full page reloads. Users click links/submit forms → server responds with HTML → Unpoly extracts and swaps matching fragments.
---
## 1. Following Links (Fragment Updates)
### Basic Link Following
```html
<a href="/users/5" up-follow>View User</a>
```
Updates the `<main>` element (or `<body>` if no main exists) with content from `/users/5`.
### Target Specific Fragment
```html
<a href="/users/5" up-target=".user-details">View User</a>
<div class="user-details">
<!-- Content replaced here -->
</div>
```
### Multiple Fragments
```html
<a href="/users/5" up-target=".profile, .activity">View User</a>
```
Updates both `.profile` and `.activity` from single response.
### Append/Prepend Content
```html
<!-- Append to list -->
<a href="/items?page=2" up-target=".items:after">Load More</a>
<!-- Prepend to list -->
<a href="/latest" up-target=".items:before">Show Latest</a>
```
### Handle All Links Automatically
```js
up.link.config.followSelectors.push('a[href]')
```
Now all links update fragments by default.
---
## 2. Submitting Forms
### Basic Form Submission
```html
<form action="/users" method="post" up-submit>
<input name="email">
<button type="submit">Create</button>
</form>
```
Submits via AJAX and updates `<main>` with response.
### Target Specific Fragment
```html
<form action="/search" up-submit up-target=".results">
<input name="query">
<button>Search</button>
</form>
<div class="results">
<!-- Search results appear here -->
</div>
```
### Handle Success vs. Error Responses
```html
<form action="/users" method="post" up-submit
up-target="#success"
up-fail-target="form">
<input name="email">
<button>Create</button>
</form>
<div id="success">Success message here</div>
```
- **Success (2xx status)**: Updates `#success`
- **Error (4xx/5xx status)**: Re-renders `form` with validation errors
**Server must return HTTP 422** (or similar error code) for validation failures.
---
## 3. Opening Overlays (Modal, Drawer, Popup)
### Modal Dialog
```html
<a href="/details" up-layer="new">Open Modal</a>
```
Opens `/details` in a modal overlay.
### Drawer (Sidebar)
```html
<a href="/menu" up-layer="new drawer">Open Drawer</a>
```
### Popup (Anchored to Link)
```html
<a href="/help" up-layer="new popup">Help</a>
```
### Close Overlay When Condition Met
```html
<a href="/users/new"
up-layer="new"
up-accept-location="/users/$id"
up-on-accepted="console.log('Created user:', value.id)">
New User
</a>
```
Overlay auto-closes when URL matches `/users/123`, passes `{ id: 123 }` to callback.
### Local Content (No Server Request)
```html
<a up-layer="new popup" up-content="<p>Help text here</p>">Help</a>
```
---
## 4. Validation
### Validate on Field Change
```html
<form action="/users" method="post">
<input name="email" up-validate>
<input name="password" up-validate>
<button type="submit">Register</button>
</form>
```
When field loses focus → submits form with `X-Up-Validate: email` header → server re-renders form → Unpoly updates the field's parent `<fieldset>` (or closest form group).
**Server must return HTTP 422** for validation errors.
### Validate While Typing
```html
<input name="email" up-validate
up-watch-event="input"
up-watch-delay="300">
```
Validates 300ms after user stops typing.
---
## 5. Lazy Loading & Polling
### Load When Element Appears in DOM
```html
<div id="menu" up-defer up-href="/menu">
Loading menu...
</div>
```
Immediately loads `/menu` when placeholder renders.
### Load When Scrolled Into View
```html
<div id="comments" up-defer="reveal" up-href="/comments">
Loading comments...
</div>
```
Loads when element scrolls into viewport.
### Auto-Refresh (Polling)
```html
<div class="status" up-poll up-interval="5000">
Current status
</div>
```
Reloads fragment every 5 seconds from original URL.
---
## 6. Caching & Revalidation
### Enable Caching
```html
<a href="/users" up-cache="true">Users</a>
```
Caches response, instantly shows cached content, then revalidates with server.
### Disable Caching
```html
<a href="/stock" up-cache="false">Live Prices</a>
```
### Conditional Requests (Server-Side)
Server sends:
```http
HTTP/1.1 200 OK
ETag: "abc123"
<div class="data">Content</div>
```
Next reload, Unpoly sends:
```http
GET /path
If-None-Match: "abc123"
```
Server responds `304 Not Modified` if unchanged → saves bandwidth.
---
## 7. Navigation Bar (Current Link Highlighting)
```html
<nav>
<a href="/home">Home</a>
<a href="/about">About</a>
</nav>
```
Current page link gets `.up-current` class automatically.
**Style it:**
```css
.up-current {
font-weight: bold;
color: blue;
}
```
---
## 8. Loading State
### Feedback Classes
Automatically applied:
- `.up-active` on clicked link/button
- `.up-loading` on targeted fragment
**Style them:**
```css
.up-active { opacity: 0.6; }
.up-loading { opacity: 0.8; }
```
### Disable Form While Submitting
```html
<form up-submit up-disable>
<input name="email">
<button>Submit</button>
</form>
```
All fields disabled during submission.
### Show Placeholder While Loading
```html
<a href="/data" up-target=".data"
up-placeholder="<p>Loading...</p>">
Load Data
</a>
```
---
## 9. Preloading
### Preload on Hover
```html
<a href="/users/5" up-preload>User Profile</a>
```
Starts loading when user hovers (90ms delay by default).
### Preload Immediately
```html
<a href="/menu" up-preload="insert">Menu</a>
```
Loads as soon as link appears in DOM.
---
## 10. Templates (Client-Side HTML)
### Define Template
```html
<template id="user-card">
<div class="card">
<h3>{{name}}</h3>
<p>{{email}}</p>
</div>
</template>
```
### Use Template
```html
<a up-fragment="#user-card"
up-use-data="{ name: 'Alice', email: 'alice@example.com' }">
Show User
</a>
```
**Process variables with compiler:**
```js
up.compiler('.card', function(element, data) {
element.innerHTML = element.innerHTML
.replace(/{{name}}/g, data.name)
.replace(/{{email}}/g, data.email)
})
```
---
## 11. JavaScript API
### Render Fragment
```js
up.render({
url: '/users/5',
target: '.user-details'
})
```
### Navigate (Updates History)
```js
up.navigate({
url: '/users',
target: 'main'
})
```
### Submit Form
```js
let form = document.querySelector('form')
up.submit(form)
```
### Open Overlay
```js
up.layer.open({
url: '/users/new',
onAccepted: (event) => {
console.log('User created:', event.value)
}
})
```
### Close Overlay with Value
```js
up.layer.accept({ id: 123, name: 'Alice' })
```
### Reload Fragment
```js
up.reload('.status')
```
---
## 12. Request Headers (Server Protocol)
Unpoly sends these headers with requests:
| Header | Value | Purpose |
| --------------- | -------- | ------------------------------- |
| `X-Up-Version` | `1.0.0` | Identifies Unpoly request |
| `X-Up-Target` | `.users` | Fragment selector being updated |
| `X-Up-Mode` | `modal` | Current layer mode |
| `X-Up-Validate` | `email` | Field being validated |
**Server can respond with:**
| Header | Effect |
| ------------------------ | ------------------------ |
| `X-Up-Target: .other` | Changes target selector |
| `X-Up-Accept-Layer: {}` | Closes overlay (success) |
| `X-Up-Dismiss-Layer: {}` | Closes overlay (cancel) |
---
## 13. Common Patterns
### Infinite Scrolling
```html
<div id="items">
<div>Item 1</div>
<div>Item 2</div>
</div>
<a id="next" href="/items?page=2"
up-defer="reveal"
up-target="#items:after, #next">
Load More
</a>
```
### Dependent Form Fields
```html
<form action="/order">
<!-- Changing country updates city select -->
<select name="country" up-validate="#city">
<option>USA</option>
<option>Canada</option>
</select>
<select name="city" id="city">
<option>New York</option>
</select>
</form>
```
### Confirm Before Action
```html
<a href="/delete" up-method="delete"
up-confirm="Really delete?">
Delete
</a>
```
### Auto-Submit on Change
```html
<form action="/search" up-autosubmit>
<input name="query">
</form>
```
Submits form when any field changes.
---
## 14. Error Handling
### Handle Network Errors
```js
up.on('up:fragment:offline', function(event) {
if (confirm('You are offline. Retry?')) {
event.retry()
}
})
```
### Handle Failed Responses
```js
try {
await up.render({ url: '/path', target: '.data' })
} catch (error) {
if (error instanceof up.RenderResult) {
console.log('Server error:', error)
}
}
```
---
## 15. Compilers (Enhance Elements)
### Basic Compiler
```js
up.compiler('.current-time', function(element) {
element.textContent = new Date().toString()
})
```
Runs when `.current-time` is inserted (initial load OR fragment update).
### Compiler with Cleanup
```js
up.compiler('.auto-refresh', function(element) {
let timer = setInterval(() => {
element.textContent = new Date().toString()
}, 1000)
// Return destructor function
return () => clearInterval(timer)
})
```
Destructor called when element is removed from DOM.
---
## Quick Reference Table
| Task | HTML | JavaScript |
| --------------- | ---------------------------- | -------------------------- |
| Follow link | `<a href="/path" up-follow>` | `up.follow(link)` |
| Submit form | `<form up-submit>` | `up.submit(form)` |
| Target fragment | `up-target=".foo"` | `{ target: '.foo' }` |
| Open modal | `up-layer="new"` | `up.layer.open({ url })` |
| Validate field | `up-validate` | `up.validate(field)` |
| Lazy load | `up-defer` | — |
| Poll fragment | `up-poll` | — |
| Preload link | `up-preload` | `up.link.preload(link)` |
| Local content | `up-content="<p>Hi</p>"` | `{ content: '<p>Hi</p>' }` |
| Append content | `up-target=".list:after"` | — |
| Confirm action | `up-confirm="Sure?"` | `{ confirm: 'Sure?' }` |
---
## Key Defaults
- **Target**: Updates `<main>` (or `<body>`) if no `up-target` specified
- **Caching**: Auto-enabled for GET requests during navigation
- **History**: Auto-updated when rendering `<main>` or major fragments
- **Scrolling**: Auto-scrolls to top when updating `<main>`
- **Focus**: Auto-focuses new fragment
- **Validation**: Targets field's parent `<fieldset>` or form group
---
## Best Practices for AI Agents
1. **Always provide HTTP error codes**: Return 422 for validation errors, 404 for not found, etc.
2. **Send full HTML responses**: Include entire page structure; Unpoly extracts needed fragments
3. **Use semantic HTML**: `<main>`, `<nav>`, `<form>` elements work best
4. **Set IDs on fragments**: Makes targeting easier (e.g., `<div id="user-123">`)
5. **Return consistent selectors**: If request targets `.users`, response must contain `.users`
---
## Common Mistakes to Avoid
❌ **Don't**: Return only partial HTML without wrapper
```html
<h1>Title</h1>
<p>Content</p>
```
✅ **Do**: Wrap in target selector
```html
<div class="content">
<h1>Title</h1>
<p>Content</p>
</div>
```
❌ **Don't**: Return 200 OK for validation errors
✅ **Do**: Return 422 Unprocessable Entity
❌ **Don't**: Use `onclick="up.follow(this)"`
✅ **Do**: Use `up-follow` attribute (handles keyboard, accessibility)
---
## Server Response Examples
### Successful Form Submission
```http
HTTP/1.1 200 OK
<div id="success">
User created successfully!
</div>
```
### Validation Error
```http
HTTP/1.1 422 Unprocessable Entity
<form action="/users" method="post" up-submit>
<input name="email" value="invalid">
<div class="error">Email is invalid</div>
<button>Submit</button>
</form>
```
### Partial Response (Optimized)
```http
HTTP/1.1 200 OK
Vary: X-Up-Target
<div class="user-details">
<!-- Only the targeted fragment -->
</div>
```

3
lib/data/atlas/atlas.v
View File

@@ -73,7 +73,7 @@ pub mut:
}
// Add a collection to the Atlas
pub fn (mut a Atlas) add_collection(args AddCollectionArgs) ! {
pub fn (mut a Atlas) add_collection(args AddCollectionArgs) !&Collection {
name := texttools.name_fix(args.name)
console.print_item('Known collections: ${a.collections.keys()}')
console.print_item("Adding collection '${name}' to Atlas '${a.name}' at path '${args.path}'")
@@ -85,6 +85,7 @@ pub fn (mut a Atlas) add_collection(args AddCollectionArgs) ! {
col.scan()!
a.collections[name] = &col
return &col
}
// Scan a path for collections

19
lib/data/atlas/collection.v
View File

@@ -375,3 +375,22 @@ pub fn (c Collection) can_write(session Session) bool {
return false
}
// scan_groups scans the collection's directory for .group files and loads them into memory.
pub fn (mut c Collection) scan_groups() ! {
if c.name != 'groups' {
return error('scan_groups only works on "groups" collection')
}
mut entries := c.path.list(recursive: false)!
for mut entry in entries.paths {
if entry.extension_lower() == 'group' {
filename := entry.name_fix_no_ext()
mut visited := map[string]bool{}
mut group := parse_group_file(filename, c.path.path, mut visited)!
c.atlas.group_add(mut group)!
}
}
}

114
lib/data/atlas/group.v
View File

@@ -1,62 +1,104 @@
module atlas
import incubaid.herolib.core.texttools
import incubaid.herolib.core.pathlib
import os
@[heap]
pub struct Group {
pub mut:
name string // normalized to lowercase
patterns []string // email patterns, normalized to lowercase
name string // normalized to lowercase
patterns []string // email patterns, normalized to lowercase
}
@[params]
pub struct GroupNewArgs {
pub mut:
name string @[required]
patterns []string @[required]
name string @[required]
patterns []string @[required]
}
// Create a new Group
pub fn new_group(args GroupNewArgs) !Group {
mut name := texttools.name_fix(args.name)
mut patterns := args.patterns.map(it.to_lower())
return Group{
name: name
patterns: patterns
}
mut name := texttools.name_fix(args.name)
mut patterns := args.patterns.map(it.to_lower())
return Group{
name: name
patterns: patterns
}
}
// Check if email matches any pattern in this group
pub fn (g Group) matches(email string) bool {
email_lower := email.to_lower()
for pattern in g.patterns {
if matches_pattern(email_lower, pattern) {
return true
}
}
return false
email_lower := email.to_lower()
for pattern in g.patterns {
if matches_pattern(email_lower, pattern) {
return true
}
}
return false
}
// Helper: match email against wildcard pattern
// '*@domain.com' matches 'user@domain.com'
// 'exact@email.com' matches only 'exact@email.com'
fn matches_pattern(email string, pattern string) bool {
if pattern == '*' {
return true
}
if !pattern.contains('*') {
return email == pattern
}
// Handle wildcard patterns like '*@domain.com'
if pattern.starts_with('*') {
suffix := pattern[1..] // Remove the '*'
return email.ends_with(suffix)
}
// Could add more complex patterns here if needed
return false
}
if pattern == '*' {
return true
}
if !pattern.contains('*') {
return email == pattern
}
// Handle wildcard patterns like '*@domain.com'
if pattern.starts_with('*') {
suffix := pattern[1..] // Remove the '*'
return email.ends_with(suffix)
}
// Could add more complex patterns here if needed
return false
}
// parse_group_file parses a single .group file, resolving includes recursively.
fn parse_group_file(filename string, base_path string, mut visited map[string]bool) !Group {
if filename in visited {
return error('Circular include detected: ${filename}')
}
visited[filename] = true
mut group := Group{
name: texttools.name_fix(filename)
patterns: []string{}
}
mut file_path := pathlib.get_file(path: '${base_path}/${filename}.group')!
content := file_path.read()!
for line_orig in content.split_into_lines() {
line := line_orig.trim_space()
if line.len == 0 || line.starts_with('//') {
continue
}
if line.starts_with('include:') {
mut included_name := line.trim_string_left('include:').trim_space()
included_name = included_name.replace('.group', '') // Remove .group if present
include_path := '${base_path}/${included_name}.group'
if !os.exists(include_path) {
return error('Included group file not found: ${included_name}.group')
}
included_group := parse_group_file(included_name, base_path, mut visited)!
group.patterns << included_group.patterns
} else {
group.patterns << line.to_lower()
}
}
return group
}

15
lib/data/atlas/instruction.md Normal file
View File

@@ -0,0 +1,15 @@
in atlas/
check format of groups
see content/groups
now the groups end with .group
check how the include works, so we can include another group in the group as defined, only works in same folder
in the scan function in atlas, now make scan_groups function, find groups, only do this for collection as named groups
do not add collection groups to atlas, this is a system collection
make the groups and add them to atlas
give clear instructions for coding agent how to write the code

438
lib/data/atlas/readme.md
View File

@@ -416,266 +416,59 @@ println('Logo image: ${img_path}') // Output: img/logo.png
```
## Atlas Save/Load Functionality
This document describes the save/load functionality for Atlas collections, which allows you to persist collection metadata to JSON files and load them in both V and Python.
## Overview
The Atlas module now supports:
- **Saving collections** to `.collection.json` files
- **Loading collections** from `.collection.json` files in V
- **Loading collections** from `.collection.json` files in Python
## Saving Collections (Beta)
This enables:
1. Persistence of collection metadata (pages, images, files, errors)
2. Cross-language access to Atlas data
3. Faster loading without re-scanning directories
**Status:** Basic save functionality is implemented. Load functionality is work-in-progress.
## V Implementation
### Saving to JSON
### Saving Collections
Save collection metadata to JSON files for archival or cross-tool compatibility:
```v
import incubaid.herolib.data.atlas
// Create and scan atlas
mut a := atlas.new(name: 'my_docs')!
a.scan(path: './docs')!
// Save all collections (creates .collection.json in each collection dir)
a.save_all()!
// Or save a single collection
col := a.get_collection('guides')!
col.save()!
```
### Loading Collections
```v
import incubaid.herolib.data.atlas
// Load single collection
mut a := atlas.new(name: 'loaded')!
mut col := a.load_collection('/path/to/collection')!
println('Pages: ${col.pages.len}')
// Load all collections from directory tree
mut a2 := atlas.new(name: 'all_docs')!
a2.load_from_directory('./docs')!
println('Loaded ${a2.collections.len} collections')
// Save all collections to a specified directory
// Creates: ${save_path}/${collection_name}.json
a.save('./metadata')!
```
### What Gets Saved
The `.collection.json` file contains:
- Collection name and path
- All pages (name, path, collection_name)
- All images (name, ext, path, ftype)
- All files (name, ext, path, ftype)
Each `.json` file contains:
- Collection metadata (name, path, git URL, git branch)
- All pages (with paths and collection references)
- All images and files (with paths and types)
- All errors (category, page_key, message, file)
**Note:** Circular references (`atlas` and `collection` pointers) are automatically skipped using the `[skip]` attribute and reconstructed during load.
## Python Implementation
### Installation
The Python loader is a standalone script with no external dependencies (uses only Python stdlib):
```bash
# No installation needed - just use the script
python3 lib/data/atlas/atlas_loader.py
```
### Loading Collections
```python
from atlas_loader import Atlas
# Load single collection
atlas = Atlas.load_collection('/path/to/collection')
# Or load all collections from directory tree
atlas = Atlas.load_from_directory('/path/to/docs')
# Access collections
col = atlas.get_collection('guides')
print(f"Pages: {len(col.pages)}")
# Access pages
page = atlas.page_get('guides:intro')
if page:
content = page.content()
print(content)
# Check for errors
if atlas.has_errors():
atlas.print_all_errors()
```
### Python API
#### Atlas Class
- `Atlas.load_collection(path, name='default')` - Load single collection
- `Atlas.load_from_directory(path, name='default')` - Load all collections from directory tree
- `atlas.get_collection(name)` - Get collection by name
- `atlas.page_get(key)` - Get page using 'collection:page' format
- `atlas.image_get(key)` - Get image using 'collection:image' format
- `atlas.file_get(key)` - Get file using 'collection:file' format
- `atlas.list_collections()` - List all collection names
- `atlas.list_pages()` - List all pages grouped by collection
- `atlas.has_errors()` - Check if any collection has errors
- `atlas.print_all_errors()` - Print errors from all collections
#### Collection Class
- `collection.page_get(name)` - Get page by name
- `collection.image_get(name)` - Get image by name
- `collection.file_get(name)` - Get file by name
- `collection.has_errors()` - Check if collection has errors
- `collection.error_summary()` - Get error count by category
- `collection.print_errors()` - Print all errors
#### Page Class
- `page.key()` - Get page key in format 'collection:page'
- `page.content()` - Read page content from file
#### File Class
- `file.file_name` - Get full filename with extension
- `file.is_image()` - Check if file is an image
- `file.read()` - Read file content as bytes
## Workflow
### 1. V: Create and Save
```v
#!/usr/bin/env -S v -n -w -cg -gc none -cc tcc -d use_openssl -enable-globals run
import incubaid.herolib.data.atlas
// Create atlas and scan
mut a := atlas.new(name: 'my_docs')!
a.scan(path: './docs')!
// Validate
a.validate_links()!
// Save all collections (creates .collection.json in each collection dir)
a.save_all()!
println('Saved ${a.collections.len} collections')
```
### 2. V: Load and Use
```v
#!/usr/bin/env -S v -n -w -cg -gc none -cc tcc -d use_openssl -enable-globals run
import incubaid.herolib.data.atlas
// Load single collection
mut a := atlas.new(name: 'loaded')!
mut col := a.load_collection('/path/to/collection')!
println('Pages: ${col.pages.len}')
// Load all from directory
mut a2 := atlas.new(name: 'all_docs')!
a2.load_from_directory('./docs')!
println('Loaded ${a2.collections.len} collections')
```
### 3. Python: Load and Use
```python
#!/usr/bin/env python3
from atlas_loader import Atlas
# Load single collection
atlas = Atlas.load_collection('/path/to/collection')
# Or load all collections
atlas = Atlas.load_from_directory('/path/to/docs')
# Access pages
page = atlas.page_get('guides:intro')
if page:
content = page.content()
print(content)
# Check errors
if atlas.has_errors():
atlas.print_all_errors()
```
## File Structure
After saving, each collection directory will contain:
### Storage Location
```
collection_dir/
├── .collection # Original collection config
├── .collection.json # Saved collection metadata (NEW)
├── page1.md
├── page2.md
└── img/
└── image1.png
```
## Error Handling
Errors are preserved during save/load:
```v
// V: Errors are saved
mut a := atlas.new()!
a.scan(path: './docs')!
a.validate_links()! // May generate errors
a.save_all()! // Errors are saved to .collection.json
// V: Errors are loaded
mut a2 := atlas.new()!
a2.load_from_directory('./docs')!
col := a2.get_collection('guides')!
if col.has_errors() {
col.print_errors()
}
```
```python
# Python: Access errors
atlas = Atlas.load_from_directory('./docs')
if atlas.has_errors():
atlas.print_all_errors()
# Get error summary
col = atlas.get_collection('guides')
if col.has_errors():
summary = col.error_summary()
for category, count in summary.items():
print(f"{category}: {count}")
save_path/
├── collection1.json
├── collection2.json
└── collection3.json
```
**Note:** Not in the collection directories themselves - saved to a separate location you specify.
### Limitations
- Load-from-JSON functionality is not yet implemented
- Python loader is planned but not yet available
- Currently, collections must be rescanned from source files
## HeroScript Integration
Atlas integrates with HeroScript, allowing you to define Atlas operations in `.vsh` or playbook files.
### Available Actions
#### 1. `atlas.scan` - Scan Directory for Collections
#### `atlas.scan` - Scan Directory for Collections
Scan a directory tree to find and load collections marked with `.collection` files.
@@ -683,163 +476,31 @@ Scan a directory tree to find and load collections marked with `.collection` fil
!!atlas.scan
name: 'main'
path: './docs'
git_url: 'https://github.com/org/repo.git' # optional
git_root: '~/code' # optional, default: ~/code
meta_path: './metadata' # optional, saves metadata here
ignore: ['private', 'draft'] # optional, directories to skip
```
**Parameters:**
- `name` (optional, default: 'main') - Atlas instance name
- `path` (required) - Directory path to scan
- `path` (required when git_url not provided) - Directory path to scan
- `git_url` (alternative to path) - Git repository URL to clone/checkout
- `git_root` (optional when using git_url, default: ~/code) - Base directory for cloning
- `meta_path` (optional) - Directory to save collection metadata JSON
- `ignore` (optional) - List of directory names to skip during scan
#### 2. `atlas.load` - Load from Saved Collections
Load collections from `.collection.json` files (previously saved with `atlas.save`).
### Real Workflow Example: Scan and Export
```heroscript
!!atlas.load
name: 'main'
path: './docs'
```
**Parameters:**
- `name` (optional, default: 'main') - Atlas instance name
- `path` (required) - Directory path containing `.collection.json` files
#### 3. `atlas.validate` - Validate All Links
Validate all markdown links in all collections.
```heroscript
!!atlas.validate
name: 'main'
```
**Parameters:**
- `name` (optional, default: 'main') - Atlas instance name
#### 4. `atlas.fix_links` - Fix All Links
Automatically rewrite all local links with correct relative paths.
```heroscript
!!atlas.fix_links
name: 'main'
```
**Parameters:**
- `name` (optional, default: 'main') - Atlas instance name
#### 5. `atlas.save` - Save Collections
Save all collections to `.collection.json` files in their respective directories.
```heroscript
!!atlas.save
name: 'main'
```
**Parameters:**
- `name` (optional, default: 'main') - Atlas instance name
#### 6. `atlas.export` - Export Collections
Export collections to a destination directory.
```heroscript
!!atlas.export
name: 'main'
destination: './output'
reset: true
include: true
redis: true
```
**Parameters:**
- `name` (optional, default: 'main') - Atlas instance name
- `destination` (required) - Export destination path
- `reset` (optional, default: true) - Clear destination before export
- `include` (optional, default: true) - Process `!!include` actions
- `redis` (optional, default: true) - Store metadata in Redis
### Complete Workflow Examples
#### Example 1: Scan, Validate, and Export
```heroscript
# Scan for collections
!!atlas.scan
path: '~/docs/myproject'
meta_path: '~/docs/metadata'
# Validate all links
!!atlas.validate
# Export to output directory
!!atlas.export
destination: '~/docs/output'
include: true
```
#### Example 2: Load, Fix Links, and Export
```heroscript
# Load from saved collections
!!atlas.load
path: '~/docs/myproject'
# Fix all broken links
!!atlas.fix_links
# Save updated collections
!!atlas.save
# Export
!!atlas.export
destination: '~/docs/output'
```
#### Example 3: Multiple Atlas Instances
```heroscript
# Main documentation
!!atlas.scan
name: 'docs'
path: '~/docs'
# API reference
!!atlas.scan
name: 'api'
path: '~/api-docs'
# Export docs
!!atlas.export
name: 'docs'
destination: '~/output/docs'
# Export API
!!atlas.export
name: 'api'
destination: '~/output/api'
```
#### Example 4: Development Workflow
```heroscript
# Scan collections
!!atlas.scan
path: './docs'
# Validate links (errors will be reported)
!!atlas.validate
# Fix links automatically
!!atlas.fix_links
# Save updated collections
!!atlas.save
# Export final version
!!atlas.export
destination: './public'
include: true
redis: true
redis: false
```
### Using in V Scripts
@@ -857,8 +518,6 @@ heroscript := "
!!atlas.scan
path: './docs'
!!atlas.validate
!!atlas.export
destination: './output'
include: true
@@ -882,12 +541,6 @@ Create a `docs.play` file:
name: 'main'
path: '~/code/docs'
!!atlas.validate
!!atlas.fix_links
!!atlas.save
!!atlas.export
destination: '~/code/output'
reset: true
@@ -922,8 +575,6 @@ Errors are automatically collected and reported:
!!atlas.scan
path: './docs'
!!atlas.validate
# Errors will be printed during export
!!atlas.export
destination: './output'
@@ -939,14 +590,23 @@ Collection guides - Errors (2)
### Auto-Export Behavior
If you use `!!atlas.scan` or `!!atlas.load` **without** an explicit `!!atlas.export`, Atlas will automatically export to the default location (current directory).
If you use `!!atlas.scan` **without** an explicit `!!atlas.export`, Atlas will automatically export to the default location (current directory).
To disable auto-export, include an explicit (empty) export action or simply don't include any scan/load actions.
To disable auto-export, include an explicit (empty) export action or simply don't include any scan actions.
### Best Practices
1. **Always validate before export**: Use `!!atlas.validate` to catch broken links early
2. **Save after fixing**: Use `!!atlas.save` after `!!atlas.fix_links` to persist changes
3. **Use named instances**: When working with multiple documentation sets, use the `name` parameter
4. **Enable Redis for production**: Use `redis: true` for web deployments to enable fast lookups
5. **Process includes during export**: Keep `include: true` to embed referenced content in exported files
2. **Use named instances**: When working with multiple documentation sets, use the `name` parameter
3. **Enable Redis for production**: Use `redis: true` for web deployments to enable fast lookups
4. **Process includes during export**: Keep `include: true` to embed referenced content in exported files
## Roadmap - Not Yet Implemented
The following features are planned but not yet available:
- [ ] Load collections from `.collection.json` files
- [ ] Python API for reading collections
- [ ] `atlas.validate` playbook action
- [ ] `atlas.fix_links` playbook action
- [ ] Auto-save on collection modifications
- [ ] Collection version control

5
lib/data/atlas/scan.v
View File

@@ -22,7 +22,10 @@ fn (mut a Atlas) scan_directory(mut dir pathlib.Path, ignore_ []string) ! {
if collection_name.to_lower() in ignore {
return
}
a.add_collection(path: dir.path, name: collection_name)!
mut col := a.add_collection(path: dir.path, name: collection_name)!
if collection_name == 'groups' {
col.scan_groups()!
}
return
}