MahmoudEmad/herolib
1
0
Fork 0
Code Issues 55 Pull Requests 1 Actions Packages Projects Releases 26 Wiki Activity

Merge branch 'development_heropods' into development_nile_installers

Browse Source 
* development_heropods: (21 commits)
  test: Ignore virt/heropods/network_test.v in CI
  feat: implement container keep-alive feature
  test: Add comprehensive heropods network and container tests
  refactor: Refactor Mycelium configuration and dependencies
  feat: Add Mycelium IPv6 overlay networking
  test: Replace hero binary checks with network test
  feat: Add iptables FORWARD rules for bridge
  Revert "feat: Add `pods` command for container management"
  feat: Add `pods` command for container management
  chore: Enable execution of cmd_run
  feat: Add `run` command for Heroscript execution
  feat: Separate initialization and configuration
  refactor: Remove hero binary installation from rootfs
  refactor: Integrate logger and refactor network operations
  feat: Implement container networking and improve lifecycle
  feat: Auto-install hero binary in containers
  feat: Add container management actions for heropods
  feat: Add heropods library to plbook
  refactor: Rename heropods variable and method
  refactor: Rename container factory to heropods
  ...
This commit is contained in:
despiegk
2025-11-25 18:39:08 +01:00
parent 253e26aec6 9f75a454fa
commit 3547f04a79
38 changed files with 4595 additions and 549 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/gittools/test_check.hero
View File

@@ -1 +0,0 @@
!!git.check filter:'herolib'

11
examples/installers/virt/crun.vsh Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env -S v -n -w -gc none -cc tcc -d use_openssl -enable-globals run
import incubaid.herolib.installers.virt.crun_installer
mut crun := crun_installer.get()!
// To install
crun.install()!
// To remove
crun.destroy()!

169
examples/virt/heropods/README.md Normal file
View File

@@ -0,0 +1,169 @@
# HeroPods Examples
This directory contains example HeroScript files demonstrating different HeroPods use cases.
## Prerequisites
- **Linux system** (HeroPods requires Linux-specific tools: ip, iptables, nsenter, crun)
- **Root/sudo access** (required for network configuration and container management)
- **Podman** (optional but recommended for image management)
- **Hero CLI** installed and configured
## Example Scripts
### 1. simple_container.heroscript
**Purpose**: Demonstrate basic container lifecycle management
**What it does**:
- Creates a HeroPods instance
- Creates an Alpine Linux container
- Starts the container
- Executes basic commands inside the container (uname, ls, cat, ps, env)
- Stops the container
- Deletes the container
**Run it**:
```bash
hero run examples/virt/heropods/simple_container.heroscript
```
**Use this when**: You want to learn the basic container operations without networking complexity.
---
### 2. ipv4_connection.heroscript
**Purpose**: Demonstrate IPv4 networking and internet connectivity
**What it does**:
- Creates a HeroPods instance with bridge networking
- Creates an Alpine Linux container
- Starts the container with IPv4 networking
- Verifies network configuration (interfaces, routes, DNS)
- Tests DNS resolution
- Tests HTTP/HTTPS connectivity to the internet
- Stops and deletes the container
**Run it**:
```bash
hero run examples/virt/heropods/ipv4_connection.heroscript
```
**Use this when**: You want to verify that IPv4 bridge networking and internet access work correctly.
---
### 3. container_mycelium.heroscript
**Purpose**: Demonstrate Mycelium IPv6 overlay networking
**What it does**:
- Creates a HeroPods instance
- Enables Mycelium IPv6 overlay network with all required configuration
- Creates an Alpine Linux container
- Starts the container with both IPv4 and IPv6 (Mycelium) networking
- Verifies IPv6 configuration
- Tests Mycelium IPv6 connectivity to public nodes
- Verifies dual-stack networking (IPv4 + IPv6)
- Stops and deletes the container
**Run it**:
```bash
hero run examples/virt/heropods/container_mycelium.heroscript
```
**Use this when**: You want to test Mycelium IPv6 overlay networking for encrypted peer-to-peer connectivity.
**Note**: Requires Mycelium to be installed and configured on the host system.
---
### 4. demo.heroscript
**Purpose**: Quick demonstration of HeroPods with both IPv4 and IPv6 networking
**What it does**:
- Combines IPv4 and Mycelium IPv6 networking in a single demo
- Shows a complete workflow from configuration to cleanup
- Serves as a quick reference for common operations
**Run it**:
```bash
hero run examples/virt/heropods/demo.heroscript
```
**Use this when**: You want a quick overview of HeroPods capabilities.
---
## Common Issues
### Permission Denied for ping/ping6
Alpine Linux containers don't have `CAP_NET_RAW` capability by default, which is required for ICMP packets (ping).
**Solution**: Use `wget`, `curl`, or `nc` for connectivity testing instead of ping.
### Mycelium Not Found
If you get errors about Mycelium not being installed:
**Solution**: The HeroPods Mycelium integration will automatically install Mycelium when you run `heropods.enable_mycelium`. Make sure you have internet connectivity and the required permissions.
### Container Already Exists
If you get errors about containers already existing:
**Solution**: Either delete the existing container manually or set `reset:true` in the `heropods.configure` action.
---
## Learning Path
We recommend running the examples in this order:
1. **simple_container.heroscript** - Learn basic container operations
2. **ipv4_connection.heroscript** - Understand IPv4 networking
3. **container_mycelium.heroscript** - Explore IPv6 overlay networking
4. **demo.heroscript** - See everything together
---
## Customization
Feel free to modify these scripts to:
- Use different container images (Ubuntu, custom images, etc.)
- Test different network configurations
- Add your own commands and tests
- Experiment with multiple containers
---
## Documentation
For more information, see:
- [HeroPods Main README](../../../lib/virt/heropods/readme.md)
- [Mycelium Integration Guide](../../../lib/virt/heropods/MYCELIUM_README.md)
- [Production Readiness Review](../../../lib/virt/heropods/PRODUCTION_READINESS_REVIEW.md)
---
## Support
If you encounter issues:
1. Check the logs in `~/.containers/logs/`
2. Verify your system meets the prerequisites
3. Review the error messages carefully
4. Consult the documentation linked above

114
examples/virt/heropods/container_mycelium.heroscript Normal file
View File

@@ -0,0 +1,114 @@
#!/usr/bin/env hero
// ============================================================================
// HeroPods Example: Mycelium IPv6 Overlay Networking
// ============================================================================
//
// This script demonstrates Mycelium IPv6 overlay networking:
// - End-to-end encrypted IPv6 connectivity
// - Peer-to-peer routing through public relay nodes
// - Container IPv6 address assignment from host's /64 prefix
// - Connectivity to other Mycelium nodes across the internet
//
// Mycelium provides each container with an IPv6 address in the 400::/7 range
// and enables encrypted communication with other Mycelium nodes.
// ============================================================================
// Step 1: Configure HeroPods instance
// This creates a HeroPods instance with default IPv4 networking
!!heropods.configure
name:'mycelium_demo'
reset:false
use_podman:true
// Step 2: Enable Mycelium IPv6 overlay network
// All parameters are required for Mycelium configuration
!!heropods.enable_mycelium
heropods:'mycelium_demo'
version:'v0.5.6'
ipv6_range:'400::/7'
key_path:'~/hero/cfg/priv_key.bin'
peers:'tcp://185.69.166.8:9651,quic://[2a02:1802:5e:0:ec4:7aff:fe51:e36b]:9651,tcp://65.109.18.113:9651,quic://[2a01:4f9:5a:1042::2]:9651,tcp://5.78.122.16:9651,quic://[2a01:4ff:1f0:8859::1]:9651,tcp://5.223.43.251:9651,quic://[2a01:4ff:2f0:3621::1]:9651,tcp://142.93.217.194:9651,quic://[2400:6180:100:d0::841:2001]:9651'
// Step 3: Create a new Alpine Linux container
// Alpine includes basic IPv6 networking tools
!!heropods.container_new
name:'mycelium_container'
image:'custom'
custom_image_name:'alpine_3_20'
docker_url:'docker.io/library/alpine:3.20'
// Step 4: Start the container
// This sets up both IPv4 and IPv6 (Mycelium) networking
!!heropods.container_start
name:'mycelium_container'
// Step 5: Verify IPv6 network configuration
// Show all network interfaces (including IPv6 addresses)
!!heropods.container_exec
name:'mycelium_container'
cmd:'ip addr show'
stdout:true
// Show IPv6 addresses specifically
!!heropods.container_exec
name:'mycelium_container'
cmd:'ip -6 addr show'
stdout:true
// Show IPv6 routing table
!!heropods.container_exec
name:'mycelium_container'
cmd:'ip -6 route show'
stdout:true
// Step 6: Test Mycelium IPv6 connectivity
// Ping a known public Mycelium node to verify connectivity
// Note: This requires the container to have CAP_NET_RAW capability for ping6
// If ping6 fails with permission denied, this is expected behavior in Alpine
!!heropods.container_exec
name:'mycelium_container'
cmd:'ping6 -c 3 400:8f3a:8d0e:3503:db8e:6a02:2e9:83dd'
stdout:true
// Alternative: Test IPv6 connectivity using nc (netcat) if available
// This doesn't require special capabilities
!!heropods.container_exec
name:'mycelium_container'
cmd:'nc -6 -zv -w 3 400:8f3a:8d0e:3503:db8e:6a02:2e9:83dd 80 2>&1 || echo nc test completed'
stdout:true
// Step 7: Show Mycelium-specific information
// Display the container's Mycelium IPv6 address
!!heropods.container_exec
name:'mycelium_container'
cmd:'ip -6 addr show | grep 400: || echo No Mycelium IPv6 address found'
stdout:true
// Show IPv6 neighbors (if any)
!!heropods.container_exec
name:'mycelium_container'
cmd:'ip -6 neigh show'
stdout:true
// Step 8: Verify dual-stack networking (IPv4 + IPv6)
// The container should have both IPv4 and IPv6 connectivity
// Test IPv4 connectivity
!!heropods.container_exec
name:'mycelium_container'
cmd:'wget -O- http://google.com --timeout=5 2>&1 | head -n 5'
stdout:true
// Step 9: Stop the container
// This cleans up both IPv4 and IPv6 (Mycelium) networking
!!heropods.container_stop
name:'mycelium_container'
// Step 10: Delete the container
// This removes the container and all associated resources
!!heropods.container_delete
name:'mycelium_container'

75
examples/virt/heropods/hello_world_keepalive.heroscript Normal file
View File

@@ -0,0 +1,75 @@
#!/usr/bin/env hero
// ============================================================================
// HeroPods Keep-Alive Feature Test - Alpine Container
// ============================================================================
//
// This script demonstrates the keep_alive feature with an Alpine container.
//
// Test Scenario:
// Alpine's default CMD is /bin/sh, which exits immediately when run
// non-interactively (no stdin). This makes it perfect for testing keep_alive:
//
// 1. Container starts with CMD=["/bin/sh"]
// 2. /bin/sh exits immediately (exit code 0)
// 3. HeroPods detects the successful exit
// 4. HeroPods recreates the container with keep-alive command
// 5. Container remains running and accepts exec commands
//
// This demonstrates the core keep_alive functionality:
// - Detecting when a container's entrypoint/cmd exits
// - Checking the exit code
// - Injecting a keep-alive process on successful exit
// - Allowing subsequent exec commands
//
// ============================================================================
// Step 1: Configure HeroPods instance
!!heropods.configure
name:'hello_world'
reset:true
use_podman:true
// Step 2: Create a container with Alpine 3.20 image
// Using custom image type to automatically download from Docker Hub
!!heropods.container_new
name:'alpine_test_keepalive'
image:'custom'
custom_image_name:'alpine_test'
docker_url:'docker.io/library/alpine:3.20'
// Step 3: Start the container with keep_alive enabled
// Alpine's CMD is /bin/sh which exits immediately when run non-interactively.
// With keep_alive:true, HeroPods will:
// 1. Start the container with /bin/sh
// 2. Wait for /bin/sh to exit (which happens immediately)
// 3. Detect the successful exit (exit code 0)
// 4. Recreate the container with a keep-alive command (tail -f /dev/null)
// 5. The container will then remain running and accept exec commands
!!heropods.container_start
name:'alpine_test_keepalive'
keep_alive:true
// Step 4: Execute a simple hello world command
!!heropods.container_exec
name:'alpine_test_keepalive'
cmd:'echo Hello World from HeroPods'
stdout:true
// Step 5: Display OS information
!!heropods.container_exec
name:'alpine_test_keepalive'
cmd:'cat /etc/os-release'
stdout:true
// Step 6: Show running processes
!!heropods.container_exec
name:'alpine_test_keepalive'
cmd:'ps aux'
stdout:true
// Step 7: Verify Alpine version
!!heropods.container_exec
name:'alpine_test_keepalive'
cmd:'cat /etc/alpine-release'
stdout:true

27
examples/virt/heropods/herobin.heroscript Normal file
View File

@@ -0,0 +1,27 @@
#!/usr/bin/env hero
// Step 1: Configure HeroPods instance
!!heropods.configure
name:'simple_demo'
reset:false
use_podman:true
// Step 2: Create a container with hero binary
!!heropods.container_new
name:'simple_container'
image:'custom'
custom_image_name:'hero_container'
docker_url:'docker.io/threefolddev/hero-container:latest'
// Step 3: Start the container with keep_alive enabled
// This will run the entrypoint, wait for it to complete, then inject a keep-alive process
!!heropods.container_start
name:'simple_container'
keep_alive:true
// Step 4: Execute hero command inside the container
!!heropods.container_exec
name:'simple_container'
cmd:'hero -help'
stdout:true

12
examples/virt/heropods/heropods.vsh
View File

@@ -2,17 +2,17 @@
import incubaid.herolib.virt.heropods
// Initialize factory
mut factory := heropods.new(
// Initialize heropods
mut heropods_ := heropods.new(
reset: false
use_podman: true
) or { panic('Failed to init ContainerFactory: ${err}') }
) or { panic('Failed to init HeroPods: ${err}') }
println('=== HeroPods Refactored API Demo ===')
// Step 1: factory.new() now only creates a container definition/handle
// Step 1: heropods_.new() now only creates a container definition/handle
// It does NOT create the actual container in the backend yet
mut container := factory.new(
mut container := heropods_.container_new(
name: 'demo_alpine'
image: .custom
custom_image_name: 'alpine_3_20'
@@ -56,7 +56,7 @@ println('✓ Container deleted successfully')
println('\n=== Demo completed! ===')
println('The refactored API now works as expected:')
println('- factory.new() creates definition only')
println('- heropods_.new() creates definition only')
println('- container.start() is idempotent')
println('- container.exec() works and returns results')
println('- container.delete() works on instances')

96
examples/virt/heropods/ipv4_connection.heroscript Normal file
View File

@@ -0,0 +1,96 @@
#!/usr/bin/env hero
// ============================================================================
// HeroPods Example: IPv4 Networking and Internet Connectivity
// ============================================================================
//
// This script demonstrates IPv4 networking functionality:
// - Bridge networking with automatic IP allocation
// - NAT for outbound internet access
// - DNS resolution
// - HTTP connectivity testing
//
// The container gets an IP address from the bridge subnet (default: 10.10.0.0/24)
// and can access the internet through NAT.
// ============================================================================
// Step 1: Configure HeroPods instance with IPv4 networking
// This creates a HeroPods instance with bridge networking enabled
!!heropods.configure
name:'ipv4_demo'
reset:false
use_podman:true
bridge_name:'heropods0'
subnet:'10.10.0.0/24'
gateway_ip:'10.10.0.1'
dns_servers:['8.8.8.8', '8.8.4.4']
// Step 2: Create a new Alpine Linux container
// Alpine is lightweight and includes basic networking tools
!!heropods.container_new
name:'ipv4_container'
image:'custom'
custom_image_name:'alpine_3_20'
docker_url:'docker.io/library/alpine:3.20'
// Step 3: Start the container
// This sets up the veth pair and configures IPv4 networking
!!heropods.container_start
name:'ipv4_container'
// Step 4: Verify network configuration inside the container
// Show network interfaces and IP addresses
!!heropods.container_exec
name:'ipv4_container'
cmd:'ip addr show'
stdout:true
// Show routing table
!!heropods.container_exec
name:'ipv4_container'
cmd:'ip route show'
stdout:true
// Show DNS configuration
!!heropods.container_exec
name:'ipv4_container'
cmd:'cat /etc/resolv.conf'
stdout:true
// Step 5: Test DNS resolution
// Verify that DNS queries work correctly
!!heropods.container_exec
name:'ipv4_container'
cmd:'nslookup google.com'
stdout:true
// Step 6: Test HTTP connectivity
// Use wget to verify internet access (ping requires CAP_NET_RAW capability)
!!heropods.container_exec
name:'ipv4_container'
cmd:'wget -O- http://google.com --timeout=5 2>&1 | head -n 10'
stdout:true
// Test another website to confirm connectivity
!!heropods.container_exec
name:'ipv4_container'
cmd:'wget -O- http://example.com --timeout=5 2>&1 | head -n 10'
stdout:true
// Step 7: Test HTTPS connectivity (if wget supports it)
!!heropods.container_exec
name:'ipv4_container'
cmd:'wget -O- https://www.google.com --timeout=5 --no-check-certificate 2>&1 | head -n 10'
stdout:true
// Step 8: Stop the container
// This removes the veth pair and cleans up network configuration
!!heropods.container_stop
name:'ipv4_container'
// Step 9: Delete the container
// This removes the container and all associated resources
!!heropods.container_delete
name:'ipv4_container'

6
examples/virt/heropods/runcommands.vsh Normal file → Executable file
View File

@@ -2,12 +2,12 @@
import incubaid.herolib.virt.heropods
mut factory := heropods.new(
mut heropods_ := heropods.new(
reset: false
use_podman: true
) or { panic('Failed to init ContainerFactory: ${err}') }
) or { panic('Failed to init HeroPods: ${err}') }
mut container := factory.new(
mut container := heropods_.container_new(
name: 'alpine_demo'
image: .custom
custom_image_name: 'alpine_3_20'

79
examples/virt/heropods/simple_container.heroscript Normal file
View File

@@ -0,0 +1,79 @@
#!/usr/bin/env hero
// ============================================================================
// HeroPods Example: Simple Container Lifecycle Management
// ============================================================================
//
// This script demonstrates the basic container lifecycle operations:
// - Creating a container
// - Starting a container
// - Executing commands inside the container
// - Stopping a container
// - Deleting a container
//
// No networking tests - just basic container operations.
// ============================================================================
// Step 1: Configure HeroPods instance
// This creates a HeroPods instance named 'simple_demo' with default settings
!!heropods.configure
name:'simple_demo'
reset:false
use_podman:true
// Step 2: Create a new Alpine Linux container
// This pulls the Alpine 3.20 image from Docker Hub and prepares it for use
!!heropods.container_new
name:'simple_container'
image:'custom'
custom_image_name:'alpine_3_20'
docker_url:'docker.io/library/alpine:3.20'
// Step 3: Start the container
// This starts the container using crun (OCI runtime)
!!heropods.container_start
name:'simple_container'
// Step 4: Execute basic commands inside the container
// These commands demonstrate that the container is running and functional
// Show kernel information
!!heropods.container_exec
name:'simple_container'
cmd:'uname -a'
stdout:true
// List root directory contents
!!heropods.container_exec
name:'simple_container'
cmd:'ls -la /'
stdout:true
// Show OS release information
!!heropods.container_exec
name:'simple_container'
cmd:'cat /etc/os-release'
stdout:true
// Show current processes
!!heropods.container_exec
name:'simple_container'
cmd:'ps aux'
stdout:true
// Show environment variables
!!heropods.container_exec
name:'simple_container'
cmd:'env'
stdout:true
// Step 5: Stop the container
// This gracefully stops the container (SIGTERM, then SIGKILL if needed)
!!heropods.container_stop
name:'simple_container'
// Step 6: Delete the container
// This removes the container and cleans up all associated resources
!!heropods.container_delete
name:'simple_container'

10
lib/core/playcmds/factory.v
View File

@@ -6,6 +6,7 @@ import incubaid.herolib.biz.bizmodel
import incubaid.herolib.threefold.incatokens
import incubaid.herolib.web.site
import incubaid.herolib.virt.hetznermanager
import incubaid.herolib.virt.heropods
import incubaid.herolib.web.docusaurus
import incubaid.herolib.clients.openai
import incubaid.herolib.clients.giteaclient
@@ -18,6 +19,9 @@ import incubaid.herolib.installers.horus.supervisor
import incubaid.herolib.installers.horus.herorunner
import incubaid.herolib.installers.horus.osirisrunner
import incubaid.herolib.installers.horus.salrunner
import incubaid.herolib.installers.virt.podman
import incubaid.herolib.installers.infra.gitea
import incubaid.herolib.builder
// -------------------------------------------------------------------
// run – entry point for all HeroScript play‑commands
@@ -53,6 +57,9 @@ pub fn run(args_ PlayArgs) ! {
// Tmux actions
tmux.play(mut plbook)!
// Builder actions (nodes and commands)
builder.play(mut plbook)!
// Business model (e.g. currency, bizmodel)
bizmodel.play(mut plbook)!
@@ -67,10 +74,13 @@ pub fn run(args_ PlayArgs) ! {
docusaurus.play(mut plbook)!
hetznermanager.play(mut plbook)!
hetznermanager.play2(mut plbook)!
heropods.play(mut plbook)!
base.play(mut plbook)!
herolib.play(mut plbook)!
vlang.play(mut plbook)!
podman.play(mut plbook)!
gitea.play(mut plbook)!
giteaclient.play(mut plbook)!

10
lib/installers/virt/herorunner/.heroscript → lib/installers/virt/crun_installer/.heroscript
View File

@@ -1,13 +1,13 @@
!!hero_code.generate_installer
name:'herorunner'
classname:'HeroRunner'
name:'crun_installer'
classname:'CrunInstaller'
singleton:0
templates:0
default:1
title:''
title:'crun container runtime installer'
supported_platforms:''
reset:0
startupmanager:0
hasconfig:0
build:0
hasconfig:1
build:1

77
lib/installers/virt/crun_installer/crun_installer_actions.v Normal file
View File

@@ -0,0 +1,77 @@
module crun_installer
import incubaid.herolib.osal.core as osal
import incubaid.herolib.ui.console
import incubaid.herolib.core
import incubaid.herolib.installers.ulist
import os
//////////////////// following actions are not specific to instance of the object
// checks if crun is installed
pub fn (self &CrunInstaller) installed() !bool {
res := os.execute('${osal.profile_path_source_and()!} crun --version')
if res.exit_code != 0 {
return false
}
return true
}
// get the Upload List of the files
fn ulist_get() !ulist.UList {
return ulist.UList{}
}
// uploads to S3 server if configured
fn upload() ! {
}
@[params]
pub struct InstallArgs {
pub mut:
reset bool
}
pub fn (mut self CrunInstaller) install(args InstallArgs) ! {
console.print_header('install crun')
// Check platform support
pl := core.platform()!
if pl == .ubuntu || pl == .arch {
console.print_debug('installing crun via package manager')
osal.package_install('crun')!
console.print_header('crun is installed')
return
}
if pl == .osx {
return error('crun is not available on macOS - it is a Linux-only container runtime. On macOS, use Docker Desktop or Podman Desktop instead.')
}
return error('unsupported platform for crun installation')
}
pub fn (mut self CrunInstaller) destroy() ! {
console.print_header('destroy crun')
if !self.installed()! {
console.print_debug('crun is not installed')
return
}
pl := core.platform()!
if pl == .ubuntu || pl == .arch {
console.print_debug('removing crun via package manager')
osal.package_remove('crun')!
console.print_header('crun has been removed')
return
}
if pl == .osx {
return error('crun is not available on macOS')
}
return error('unsupported platform for crun removal')
}

170
lib/installers/virt/crun_installer/crun_installer_factory_.v Normal file
View File

@@ -0,0 +1,170 @@
module crun_installer
import incubaid.herolib.core.base
import incubaid.herolib.core.playbook { PlayBook }
import incubaid.herolib.ui.console
import json
__global (
crun_installer_global map[string]&CrunInstaller
crun_installer_default string
)
/////////FACTORY
@[params]
pub struct ArgsGet {
pub mut:
name string = 'default'
fromdb bool // will load from filesystem
create bool // default will not create if not exist
}
pub fn new(args ArgsGet) !&CrunInstaller {
mut obj := CrunInstaller{
name: args.name
}
set(obj)!
return get(name: args.name)!
}
pub fn get(args ArgsGet) !&CrunInstaller {
mut context := base.context()!
crun_installer_default = args.name
if args.fromdb || args.name !in crun_installer_global {
mut r := context.redis()!
if r.hexists('context:crun_installer', args.name)! {
data := r.hget('context:crun_installer', args.name)!
if data.len == 0 {
print_backtrace()
return error('CrunInstaller with name: ${args.name} does not exist, prob bug.')
}
mut obj := json.decode(CrunInstaller, data)!
set_in_mem(obj)!
} else {
if args.create {
new(args)!
} else {
print_backtrace()
return error("CrunInstaller with name '${args.name}' does not exist")
}
}
return get(name: args.name)! // no longer from db nor create
}
return crun_installer_global[args.name] or {
print_backtrace()
return error('could not get config for crun_installer with name:${args.name}')
}
}
// register the config for the future
pub fn set(o CrunInstaller) ! {
mut o2 := set_in_mem(o)!
crun_installer_default = o2.name
mut context := base.context()!
mut r := context.redis()!
r.hset('context:crun_installer', o2.name, json.encode(o2))!
}
// does the config exists?
pub fn exists(args ArgsGet) !bool {
mut context := base.context()!
mut r := context.redis()!
return r.hexists('context:crun_installer', args.name)!
}
pub fn delete(args ArgsGet) ! {
mut context := base.context()!
mut r := context.redis()!
r.hdel('context:crun_installer', args.name)!
}
@[params]
pub struct ArgsList {
pub mut:
fromdb bool // will load from filesystem
}
// if fromdb set: load from filesystem, and not from mem, will also reset what is in mem
pub fn list(args ArgsList) ![]&CrunInstaller {
mut res := []&CrunInstaller{}
mut context := base.context()!
if args.fromdb {
// reset what is in mem
crun_installer_global = map[string]&CrunInstaller{}
crun_installer_default = ''
}
if args.fromdb {
mut r := context.redis()!
mut l := r.hkeys('context:crun_installer')!
for name in l {
res << get(name: name, fromdb: true)!
}
return res
} else {
// load from memory
for _, client in crun_installer_global {
res << client
}
}
return res
}
// only sets in mem, does not set as config
fn set_in_mem(o CrunInstaller) !CrunInstaller {
mut o2 := obj_init(o)!
crun_installer_global[o2.name] = &o2
crun_installer_default = o2.name
return o2
}
pub fn play(mut plbook PlayBook) ! {
if !plbook.exists(filter: 'crun_installer.') {
return
}
mut install_actions := plbook.find(filter: 'crun_installer.configure')!
if install_actions.len > 0 {
for mut install_action in install_actions {
heroscript := install_action.heroscript()
mut obj2 := heroscript_loads(heroscript)!
set(obj2)!
install_action.done = true
}
}
mut other_actions := plbook.find(filter: 'crun_installer.')!
for mut other_action in other_actions {
if other_action.name in ['destroy', 'install', 'build'] {
mut p := other_action.params
name := p.get_default('name', 'default')!
reset := p.get_default_false('reset')
mut crun_installer_obj := get(name: name)!
console.print_debug('action object:\n${crun_installer_obj}')
if other_action.name == 'destroy' || reset {
console.print_debug('install action crun_installer.destroy')
crun_installer_obj.destroy()!
}
if other_action.name == 'install' {
console.print_debug('install action crun_installer.install')
crun_installer_obj.install(reset: reset)!
}
}
other_action.done = true
}
}
////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////# LIVE CYCLE MANAGEMENT FOR INSTALLERS ///////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////////////
// load from disk and make sure is properly intialized
pub fn (mut self CrunInstaller) reload() ! {
switch(self.name)
self = obj_init(self)!
}
// switch instance to be used for crun_installer
pub fn switch(name string) {
crun_installer_default = name
}

32
lib/installers/virt/crun_installer/crun_installer_model.v Normal file
View File

@@ -0,0 +1,32 @@
module crun_installer
import incubaid.herolib.data.encoderhero
pub const version = '0.0.0'
const singleton = false
const default = true
// CrunInstaller manages the installation of the crun container runtime
@[heap]
pub struct CrunInstaller {
pub mut:
name string = 'default'
}
// Initialize the installer object
fn obj_init(mycfg_ CrunInstaller) !CrunInstaller {
mut mycfg := mycfg_
return mycfg
}
// Configure is called before installation if needed
fn configure() ! {
// No configuration needed for crun installer
}
/////////////NORMALLY NO NEED TO TOUCH
pub fn heroscript_loads(heroscript string) !CrunInstaller {
mut obj := encoderhero.decode[CrunInstaller](heroscript)!
return obj
}

53
lib/installers/virt/crun_installer/readme.md Normal file
View File

@@ -0,0 +1,53 @@
# crun_installer
Installer for the crun container runtime - a fast and lightweight OCI runtime written in C.
## Features
- **Simple Package Installation**: Installs crun via system package manager
- **Cross-Platform Support**: Works on Ubuntu, Arch Linux, and macOS
- **Clean Uninstall**: Removes crun cleanly from the system
## Quick Start
### Using V Code
```v
import incubaid.herolib.installers.virt.crun_installer
mut crun := crun_installer.get()!
// Install crun
crun.install()!
// Check if installed
if crun.installed()! {
println('crun is installed')
}
// Uninstall crun
crun.destroy()!
```
### Using Heroscript
```hero
!!crun_installer.install
!!crun_installer.destroy
```
## Platform Support
- **Ubuntu/Debian**: Installs via `apt`
- **Arch Linux**: Installs via `pacman`
- **macOS**: ⚠️ Not supported - crun is Linux-only. Use Docker Desktop or Podman Desktop on macOS instead.
## What is crun?
crun is a fast and low-memory footprint OCI Container Runtime fully written in C. It is designed to be a drop-in replacement for runc and is used by container engines like Podman.
## See Also
- **crun client**: `lib/virt/crun` - V client for interacting with crun
- **podman installer**: `lib/installers/virt/podman` - Podman installer (includes crun)

67
lib/installers/virt/herorunner/herorunner_actions.v
View File

@@ -1,67 +0,0 @@
module herorunner
import incubaid.herolib.osal.core as osal
import incubaid.herolib.ui.console
import incubaid.herolib.core.texttools
import incubaid.herolib.core.pathlib
import incubaid.herolib.installers.ulist
import os
//////////////////// following actions are not specific to instance of the object
fn installed() !bool {
return false
}
// get the Upload List of the files
fn ulist_get() !ulist.UList {
return ulist.UList{}
}
fn upload() ! {
}
fn install() ! {
console.print_header('install herorunner')
osal.package_install('crun')!
// osal.exec(
// cmd: '
// '
// stdout: true
// name: 'herorunner_install'
// )!
}
fn destroy() ! {
// mut systemdfactory := systemd.new()!
// systemdfactory.destroy("zinit")!
// osal.process_kill_recursive(name:'zinit')!
// osal.cmd_delete('zinit')!
// osal.package_remove('
// podman
// conmon
// buildah
// skopeo
// runc
// ')!
// //will remove all paths where go/bin is found
// osal.profile_path_add_remove(paths2delete:"go/bin")!
// osal.rm("
// podman
// conmon
// buildah
// skopeo
// runc
// /var/lib/containers
// /var/lib/podman
// /var/lib/buildah
// /tmp/podman
// /tmp/conmon
// ")!
}

80
lib/installers/virt/herorunner/herorunner_factory_.v
View File

@@ -1,80 +0,0 @@
module herorunner
import incubaid.herolib.core.playbook { PlayBook }
import incubaid.herolib.ui.console
import json
import incubaid.herolib.osal.startupmanager
__global (
herorunner_global map[string]&HeroRunner
herorunner_default string
)
/////////FACTORY
@[params]
pub struct ArgsGet {
pub mut:
name string = 'default'
}
pub fn new(args ArgsGet) !&HeroRunner {
return &HeroRunner{}
}
pub fn get(args ArgsGet) !&HeroRunner {
return new(args)!
}
pub fn play(mut plbook PlayBook) ! {
if !plbook.exists(filter: 'herorunner.') {
return
}
mut install_actions := plbook.find(filter: 'herorunner.configure')!
if install_actions.len > 0 {
return error("can't configure herorunner, because no configuration allowed for this installer.")
}
mut other_actions := plbook.find(filter: 'herorunner.')!
for mut other_action in other_actions {
if other_action.name in ['destroy', 'install', 'build'] {
mut p := other_action.params
reset := p.get_default_false('reset')
if other_action.name == 'destroy' || reset {
console.print_debug('install action herorunner.destroy')
destroy()!
}
if other_action.name == 'install' {
console.print_debug('install action herorunner.install')
install()!
}
}
other_action.done = true
}
}
////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////# LIVE CYCLE MANAGEMENT FOR INSTALLERS ///////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////////////
@[params]
pub struct InstallArgs {
pub mut:
reset bool
}
pub fn (mut self HeroRunner) install(args InstallArgs) ! {
switch(self.name)
if args.reset || (!installed()!) {
install()!
}
}
pub fn (mut self HeroRunner) destroy() ! {
switch(self.name)
destroy()!
}
// switch instance to be used for herorunner
pub fn switch(name string) {
herorunner_default = name
}

34
lib/installers/virt/herorunner/herorunner_model.v
View File

@@ -1,34 +0,0 @@
module herorunner
import incubaid.herolib.data.paramsparser
import incubaid.herolib.data.encoderhero
import os
pub const version = '0.0.0'
const singleton = false
const default = true
// THIS THE THE SOURCE OF THE INFORMATION OF THIS FILE, HERE WE HAVE THE CONFIG OBJECT CONFIGURED AND MODELLED
@[heap]
pub struct HeroRunner {
pub mut:
name string = 'default'
}
// your checking & initialization code if needed
fn obj_init(mycfg_ HeroRunner) !HeroRunner {
mut mycfg := mycfg_
return mycfg
}
// called before start if done
fn configure() ! {
// mut installer := get()!
}
/////////////NORMALLY NO NEED TO TOUCH
pub fn heroscript_loads(heroscript string) !HeroRunner {
mut obj := encoderhero.decode[HeroRunner](heroscript)!
return obj
}

40
lib/installers/virt/herorunner/readme.md
View File

@@ -1,40 +0,0 @@
# herorunner
To get started
```vlang
import incubaid.herolib.installers.something.herorunner as herorunner_installer
heroscript:="
!!herorunner.configure name:'test'
password: '1234'
port: 7701
!!herorunner.start name:'test' reset:1
"
herorunner_installer.play(heroscript=heroscript)!
//or we can call the default and do a start with reset
//mut installer:= herorunner_installer.get()!
//installer.start(reset:true)!
```
## example heroscript
```hero
!!herorunner.configure
homedir: '/home/user/herorunner'
username: 'admin'
password: 'secretpassword'
title: 'Some Title'
host: 'localhost'
port: 8888
```

44
lib/virt/crun/crun_test.v
View File

@@ -1,6 +1,6 @@
module crun
import json
import x.json2
fn test_factory_creation() {
mut configs := map[string]&CrunConfig{}
@@ -15,21 +15,26 @@ fn test_json_generation() {
json_str := config.to_json()!
// Parse back to verify structure
parsed := json.decode(map[string]json.Any, json_str)!
parsed := json2.decode[json2.Any](json_str)!
parsed_map := parsed.as_map()
assert parsed['ociVersion']! as string == '1.0.2'
oci_version := parsed_map['ociVersion']!
assert oci_version.str() == '1.0.2'
process := parsed['process']! as map[string]json.Any
assert process['terminal']! as bool == true
process := parsed_map['process']!
process_map := process.as_map()
terminal := process_map['terminal']!
assert terminal.bool() == true
}
fn test_configuration_methods() {
mut configs := map[string]&CrunConfig{}
mut config := new(mut configs, name: 'test')!
// Set configuration (methods don't return self for chaining)
config.set_command(['/bin/echo', 'hello'])
.set_working_dir('/tmp')
.set_hostname('test-host')
config.set_working_dir('/tmp')
config.set_hostname('test-host')
assert config.spec.process.args == ['/bin/echo', 'hello']
assert config.spec.process.cwd == '/tmp'
@@ -58,17 +63,24 @@ fn test_heropods_compatibility() {
// The default config should match heropods template structure
json_str := config.to_json()!
parsed := json.decode(map[string]json.Any, json_str)!
parsed := json2.decode[json2.Any](json_str)!
parsed_map := parsed.as_map()
// Check key fields match template
assert parsed['ociVersion']! as string == '1.0.2'
oci_version := parsed_map['ociVersion']!
assert oci_version.str() == '1.0.2'
process := parsed['process']! as map[string]json.Any
assert process['noNewPrivileges']! as bool == true
process := parsed_map['process']!
process_map := process.as_map()
no_new_privs := process_map['noNewPrivileges']!
assert no_new_privs.bool() == true
capabilities := process['capabilities']! as map[string]json.Any
bounding := capabilities['bounding']! as []json.Any
assert 'CAP_AUDIT_WRITE' in bounding.map(it as string)
assert 'CAP_KILL' in bounding.map(it as string)
assert 'CAP_NET_BIND_SERVICE' in bounding.map(it as string)
capabilities := process_map['capabilities']!
capabilities_map := capabilities.as_map()
bounding := capabilities_map['bounding']!
bounding_array := bounding.arr()
bounding_strings := bounding_array.map(it.str())
assert 'CAP_AUDIT_WRITE' in bounding_strings
assert 'CAP_KILL' in bounding_strings
assert 'CAP_NET_BIND_SERVICE' in bounding_strings
}

7
lib/virt/heropods/.heroscript Normal file
View File

@@ -0,0 +1,7 @@
!!hero_code.generate_client
name:''
classname:'HeroPods'
singleton:0
default:1
hasconfig:1

219
lib/virt/heropods/MYCELIUM.md Normal file
View File

@@ -0,0 +1,219 @@
# Mycelium IPv6 Overlay Network Integration for HeroPods
## Prerequisites
**Mycelium must be installed on your system before using this feature.** HeroPods does not install Mycelium automatically.
### Installing Mycelium
Download and install Mycelium from the official repository:
- **GitHub**: <https://github.com/threefoldtech/mycelium>
- **Releases**: <https://github.com/threefoldtech/mycelium/releases>
For detailed installation instructions, see the [Mycelium documentation](https://github.com/threefoldtech/mycelium/tree/master/docs).
After installation, verify that the `mycelium` command is available:
```bash
mycelium -V
```
## Overview
HeroPods now supports Mycelium IPv6 overlay networking, providing end-to-end encrypted IPv6 connectivity for containers across the internet.
## What is Mycelium?
Mycelium is an IPv6 overlay network that provides:
- **End-to-end encrypted** connectivity in the `400::/7` address range
- **Peer-to-peer routing** through public relay nodes
- **Automatic address assignment** based on cryptographic keys
- **NAT traversal** for containers behind firewalls
## Architecture
### Components
1. **mycelium.v** - Core Mycelium integration logic
- Service management (start/stop)
- Container IPv6 configuration
- veth pair creation for IPv6 routing
2. **heropods_model.v** - Configuration struct
- `MyceliumConfig` struct with enable flag, peers, key path
3. **container.v** - Lifecycle integration
- Mycelium setup during container start
- Mycelium cleanup during container stop/delete
### How It Works
1. **Host Setup**:
- Mycelium service runs on the host
- Connects to public peer nodes for routing
- Gets a unique IPv6 address in `400::/7` range
2. **Container Setup**:
- Creates a veth pair (`vmy-HASH` ↔ `vmyh-HASH`)
- Assigns container IPv6 from host's `/64` prefix
- Configures routing through host's Mycelium interface
3. **Connectivity**:
- Container can reach other Mycelium nodes via IPv6
- Traffic is encrypted end-to-end
- Works across NAT and firewalls
## Configuration
### Enable Mycelium
All parameters are **required** when enabling Mycelium:
```heroscript
!!heropods.configure
name:'demo'
!!heropods.enable_mycelium
heropods:'demo'
version:'v0.5.6'
ipv6_range:'400::/7'
key_path:'~/hero/cfg/priv_key.bin'
peers:'tcp://185.69.166.8:9651,quic://[2a02:1802:5e:0:ec4:7aff:fe51:e36b]:9651,tcp://65.109.18.113:9651'
```
### Configuration Parameters
All parameters are **required**:
- `version` (string): Mycelium version to install (e.g., 'v0.5.6')
- `ipv6_range` (string): Mycelium IPv6 address range (e.g., '400::/7')
- `key_path` (string): Path to Mycelium private key (e.g., '~/hero/cfg/priv_key.bin')
- `peers` (string): Comma-separated list of Mycelium peer addresses (e.g., 'tcp://185.69.166.8:9651,quic://[2a02:1802:5e:0:ec4:7aff:fe51:e36b]:9651')
### Default Public Peers
You can use these public Mycelium peers:
```text
tcp://185.69.166.8:9651
quic://[2a02:1802:5e:0:ec4:7aff:fe51:e36b]:9651
tcp://65.109.18.113:9651
quic://[2a01:4f9:5a:1042::2]:9651
tcp://5.78.122.16:9651
quic://[2a01:4ff:1f0:8859::1]:9651
tcp://5.223.43.251:9651
quic://[2a01:4ff:2f0:3621::1]:9651
tcp://142.93.217.194:9651
quic://[2400:6180:100:d0::841:2001]:9651
```
## Usage Example
See `examples/virt/heropods/container_mycelium.heroscript` for a complete example:
**Basic example:**
```heroscript
// Configure HeroPods
!!heropods.configure
name:'mycelium_demo'
// Enable Mycelium with all required parameters
!!heropods.enable_mycelium
heropods:'mycelium_demo'
version:'v0.5.6'
ipv6_range:'400::/7'
key_path:'~/hero/cfg/priv_key.bin'
peers:'tcp://185.69.166.8:9651,quic://[2a02:1802:5e:0:ec4:7aff:fe51:e36b]:9651'
// Create and start container
!!heropods.container_new
name:'my_container'
image:'alpine_3_20'
!!heropods.container_start
name:'my_container'
// Test Mycelium connectivity
!!heropods.container_exec
name:'my_container'
cmd:'ip -6 addr show'
stdout:true
```
**Run the complete example:**
```bash
hero run examples/virt/heropods/container_mycelium.heroscript
```
## Network Details
### IPv6 Address Assignment
- Host gets address like: `400:1234:5678::1`
- Container gets address like: `400:1234:5678::2`
- Uses `/64` prefix from host's Mycelium address
### Routing
- Container → Host: via veth pair link-local addresses
- Host → Mycelium network: via Mycelium TUN interface
- End-to-end encryption handled by Mycelium
### Interface Names
- Container side: `vmy-HASH` (6-char hash of container name)
- Host side: `vmyh-HASH`
- Mycelium TUN: `mycelium0` (configurable)
## Troubleshooting
### Check Mycelium Status
```bash
mycelium inspect --key-file ~/hero/cfg/priv_key.bin --json
```
### Verify Container IPv6
```bash
# Inside container
ip -6 addr show
ip -6 route show
```
### Test Connectivity
```bash
# Ping a public Mycelium node
ping6 -c 3 400:8f3a:8d0e:3503:db8e:6a02:2e9:83dd
```
### Common Issues
1. **Mycelium service not running**: Check with `ps aux | grep mycelium`
2. **No IPv6 connectivity**: Verify IPv6 forwarding is enabled: `sysctl net.ipv6.conf.all.forwarding`
3. **Container can't reach Mycelium network**: Check routes with `ip -6 route show`
## Security
- All Mycelium traffic is end-to-end encrypted
- Each node has a unique cryptographic identity
- Private key stored at `~/hero/cfg/priv_key.bin` (configurable)
- Container inherits host's Mycelium identity
## Performance
- Minimal overhead for local routing
- Peer-to-peer routing for optimal paths
- Automatic failover between peer nodes
## Future Enhancements
- Per-container Mycelium identities
- Custom routing policies
- IPv6 firewall rules
- Mycelium network isolation

661
lib/virt/heropods/container.v
View File

@@ -1,48 +1,93 @@
module heropods
import incubaid.herolib.ui.console
import incubaid.herolib.osal.tmux
import incubaid.herolib.osal.core as osal
import incubaid.herolib.virt.crun
import time
import incubaid.herolib.builder
import json
import os
// Container lifecycle timeout constants
const cleanup_retry_delay_ms = 500 // Time to wait for filesystem cleanup to complete
const sigterm_timeout_ms = 1000 // Time to wait for graceful shutdown (1 second) - reduced from 5s for faster tests
const sigkill_wait_ms = 500 // Time to wait after SIGKILL
const stop_check_interval_ms = 200 // Interval to check if container stopped - reduced from 500ms for faster response
// Container represents a running or stopped OCI container managed by crun
//
// Thread Safety:
// Container operations that interact with network configuration (start, stop, delete)
// are thread-safe because they delegate to HeroPods.network_* methods which use
// the network_mutex for protection.
@[heap]
pub struct Container {
pub mut:
name string
node ?&builder.Node
tmux_pane ?&tmux.Pane
crun_config ?&crun.CrunConfig
factory &ContainerFactory
name string // Unique container name
node ?&builder.Node // Builder node for executing commands inside container
tmux_pane ?&tmux.Pane // Optional tmux pane for interactive access
crun_config ?&crun.CrunConfig // OCI runtime configuration
factory &HeroPods // Reference to parent HeroPods instance
}
// Struct to parse JSON output of `crun state`
// CrunState represents the JSON output of `crun state` command
struct CrunState {
id string
status string
pid int
bundle string
created string
id string // Container ID
status string // Container status (running, stopped, paused)
pid int // PID of container init process
bundle string // Path to OCI bundle
created string // Creation timestamp
}
pub fn (mut self Container) start() ! {
// ContainerStartArgs defines parameters for starting a container
@[params]
pub struct ContainerStartArgs {
pub:
keep_alive bool // If true, keep container alive after entrypoint exits successfully
}
// Start the container
//
// This method handles the complete container startup lifecycle:
// 1. Creates the container in crun if it doesn't exist
// 2. Handles leftover state cleanup if creation fails
// 3. Starts the container process
// 4. Sets up networking (thread-safe via network_mutex)
// 5. If keep_alive=true, waits for entrypoint to exit and injects keep-alive process
//
// Parameters:
// - args.keep_alive: If true, the container will be kept alive after its entrypoint exits successfully.
// The entrypoint runs first, and if it exits with code 0, a keep-alive process
// (tail -f /dev/null) is injected to prevent the container from stopping.
// If the entrypoint fails (non-zero exit), the container is allowed to stop.
// Default: false
//
// Thread Safety:
// Network setup is thread-safe via HeroPods.network_setup_container()
pub fn (mut self Container) start(args ContainerStartArgs) ! {
// Check if container exists in crun
container_exists := self.container_exists_in_crun()!
if !container_exists {
// Container doesn't exist, create it first
console.print_debug('Container ${self.name} does not exist, creating it...')
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} does not exist, creating it...'
logtype: .stdout
) or {}
// Try to create the container, if it fails with "File exists" error,
// try to force delete any leftover state and retry
crun_root := '${self.factory.base_dir}/runtime'
create_result := osal.exec(
_ := osal.exec(
cmd: 'crun --root ${crun_root} create --bundle ${self.factory.base_dir}/configs/${self.name} ${self.name}'
stdout: true
) or {
if err.msg().contains('File exists') {
console.print_debug('Container creation failed with "File exists", attempting to clean up leftover state...')
self.factory.logger.log(
cat: 'container'
log: 'Container creation failed with "File exists", attempting to clean up leftover state...'
logtype: .stdout
) or {}
// Force delete any leftover state - try multiple cleanup approaches
osal.exec(cmd: 'crun --root ${crun_root} delete ${self.name}', stdout: false) or {}
osal.exec(cmd: 'crun delete ${self.name}', stdout: false) or {} // Also try default root
@@ -50,7 +95,7 @@ pub fn (mut self Container) start() ! {
osal.exec(cmd: 'rm -rf ${crun_root}/${self.name}', stdout: false) or {}
osal.exec(cmd: 'rm -rf /run/crun/${self.name}', stdout: false) or {}
// Wait a moment for cleanup to complete
time.sleep(500 * time.millisecond)
time.sleep(cleanup_retry_delay_ms * time.millisecond)
// Retry creation
osal.exec(
cmd: 'crun --root ${crun_root} create --bundle ${self.factory.base_dir}/configs/${self.name} ${self.name}'
@@ -60,69 +105,421 @@ pub fn (mut self Container) start() ! {
return err
}
}
console.print_debug('Container ${self.name} created')
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} created'
logtype: .stdout
) or {}
}
status := self.status()!
if status == .running {
console.print_debug('Container ${self.name} is already running')
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} is already running'
logtype: .stdout
) or {}
return
}
// If container exists but is stopped, we need to delete and recreate it
// because crun doesn't allow restarting a stopped container
if container_exists && status != .running {
console.print_debug('Container ${self.name} exists but is stopped, recreating...')
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} exists but is stopped, recreating...'
logtype: .stdout
) or {}
crun_root := '${self.factory.base_dir}/runtime'
osal.exec(cmd: 'crun --root ${crun_root} delete ${self.name}', stdout: false) or {}
osal.exec(
cmd: 'crun --root ${crun_root} create --bundle ${self.factory.base_dir}/configs/${self.name} ${self.name}'
stdout: true
)!
console.print_debug('Container ${self.name} recreated')
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} recreated'
logtype: .stdout
) or {}
}
// start the container (crun start doesn't have --detach flag)
crun_root := '${self.factory.base_dir}/runtime'
osal.exec(cmd: 'crun --root ${crun_root} start ${self.name}', stdout: true)!
console.print_green('Container ${self.name} started')
self.factory.logger.log(
cat: 'container'
log: 'Starting container ${self.name} with crun...'
logtype: .stdout
) or {}
osal.exec(cmd: 'crun --root ${crun_root} start ${self.name}', stdout: false)!
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} start command completed'
logtype: .stdout
) or {}
// Handle keep_alive logic if requested
// This allows the entrypoint to run and complete, then injects a keep-alive process
if args.keep_alive {
self.factory.logger.log(
cat: 'container'
log: 'keep_alive=true: Monitoring entrypoint execution...'
logtype: .stdout
) or {}
// Wait for the entrypoint to complete and handle keep-alive
// This will recreate the container with a keep-alive command
self.handle_keep_alive()!
// After keep-alive injection, the container is recreated and started
// Now we need to wait for it to be ready and setup network
self.factory.logger.log(
cat: 'container'
log: 'Keep-alive injected, waiting for process to be ready...'
logtype: .stdout
) or {}
} else {
self.factory.logger.log(
cat: 'container'
log: 'Waiting for process to be ready...'
logtype: .stdout
) or {}
}
// Wait for container process to be fully ready before setting up network
// Poll for the PID and verify /proc/<pid>/ns/net exists
self.wait_for_process_ready()!
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} process is ready, setting up network...'
logtype: .stdout
) or {}
// Setup network for the container (thread-safe)
// If this fails, stop the container to clean up
self.setup_network() or {
self.factory.logger.log(
cat: 'container'
log: 'Network setup failed, stopping container: ${err}'
logtype: .error
) or {}
// Use stop() method to properly clean up (kills process, cleans network, etc.)
// Ignore errors from stop since we're already in an error path
self.stop() or {
self.factory.logger.log(
cat: 'container'
log: 'Failed to stop container during cleanup: ${err}'
logtype: .error
) or {}
}
return error('Failed to setup network for container: ${err}')
}
// Setup Mycelium IPv6 overlay network if enabled
if self.factory.mycelium_enabled {
container_pid := self.pid()!
self.factory.mycelium_setup_container(self.name, container_pid) or {
self.factory.logger.log(
cat: 'container'
log: 'Mycelium setup failed, stopping container: ${err}'
logtype: .error
) or {}
// Stop container to clean up
self.stop() or {
self.factory.logger.log(
cat: 'container'
log: 'Failed to stop container during Mycelium cleanup: ${err}'
logtype: .error
) or {}
}
return error('Failed to setup Mycelium for container: ${err}')
}
}
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} started'
logtype: .stdout
) or {}
}
// handle_keep_alive waits for the container's entrypoint to exit, then injects a keep-alive process
//
// This method:
// 1. Waits for the container process to exit (entrypoint completion)
// 2. Checks the exit code of the entrypoint
// 3. If exit code is 0 (success), recreates the container with a keep-alive command
// 4. If exit code is non-zero (failure), leaves the container stopped
//
// The keep-alive process is 'tail -f /dev/null' which runs indefinitely and allows
// subsequent exec commands to work.
fn (mut self Container) handle_keep_alive() ! {
crun_root := '${self.factory.base_dir}/runtime'
self.factory.logger.log(
cat: 'container'
log: 'Waiting for entrypoint to complete...'
logtype: .stdout
) or {}
// Poll for container to exit (entrypoint completion)
// We check every 100ms for up to 5 minutes (3000 iterations)
mut entrypoint_exit_code := -1
for i in 0 .. 3000 {
status := self.status() or {
// If we can't get status, container might be gone
time.sleep(100 * time.millisecond)
continue
}
if status == .stopped {
// Container stopped - get the exit code
_ := osal.exec(
cmd: 'crun --root ${crun_root} state ${self.name}'
stdout: false
) or { return error('Failed to get container state after entrypoint exit: ${err}') }
// Parse state to get exit code (if available)
// Note: crun state doesn't always provide exit code, so we'll assume success if we can't get it
entrypoint_exit_code = 0 // Default to success
self.factory.logger.log(
cat: 'container'
log: 'Entrypoint completed with exit code ${entrypoint_exit_code}'
logtype: .stdout
) or {}
break
}
// Log progress every 10 seconds
if i > 0 && i % 100 == 0 {
self.factory.logger.log(
cat: 'container'
log: 'Still waiting for entrypoint to complete (${i / 10} seconds elapsed)...'
logtype: .stdout
) or {}
}
time.sleep(100 * time.millisecond)
}
// Check if we timed out
if entrypoint_exit_code == -1 {
return error('Timeout waiting for entrypoint to complete (5 minutes)')
}
// If entrypoint failed, don't inject keep-alive
if entrypoint_exit_code != 0 {
self.factory.logger.log(
cat: 'container'
log: 'Entrypoint failed with exit code ${entrypoint_exit_code}, not injecting keep-alive'
logtype: .error
) or {}
return error('Entrypoint failed with exit code ${entrypoint_exit_code}')
}
// Entrypoint succeeded - inject keep-alive process
self.factory.logger.log(
cat: 'container'
log: 'Entrypoint succeeded, injecting keep-alive process...'
logtype: .stdout
) or {}
// Delete the stopped container
osal.exec(cmd: 'crun --root ${crun_root} delete ${self.name}', stdout: false)!
// Recreate the container config with keep-alive command
// Get the existing crun config from the container
mut config := self.crun_config or { return error('Container has no crun config') }
// Update the command to use keep-alive
config.set_command(['tail', '-f', '/dev/null'])
// Save the updated config
config_path := '${self.factory.base_dir}/configs/${self.name}/config.json'
config.save_to_file(config_path)!
self.factory.logger.log(
cat: 'container'
log: 'Updated container config with keep-alive command'
logtype: .stdout
) or {}
// Create the new container with keep-alive
osal.exec(
cmd: 'crun --root ${crun_root} create --bundle ${self.factory.base_dir}/configs/${self.name} ${self.name}'
stdout: false
)!
// Start the keep-alive container
osal.exec(cmd: 'crun --root ${crun_root} start ${self.name}', stdout: false)!
// Wait for the keep-alive process to be ready
self.wait_for_process_ready()!
self.factory.logger.log(
cat: 'container'
log: 'Keep-alive process injected successfully'
logtype: .stdout
) or {}
}
// Stop the container gracefully (SIGTERM) or forcefully (SIGKILL)
//
// This method:
// 1. Sends SIGTERM for graceful shutdown
// 2. Waits up to sigterm_timeout_ms for graceful stop
// 3. Sends SIGKILL if still running after timeout
// 4. Cleans up network resources (thread-safe)
//
// Thread Safety:
// Network cleanup is thread-safe via HeroPods.network_cleanup_container()
pub fn (mut self Container) stop() ! {
status := self.status()!
if status == .stopped {
console.print_debug('Container ${self.name} is already stopped')
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} is already stopped'
logtype: .stdout
) or {}
return
}
crun_root := '${self.factory.base_dir}/runtime'
osal.exec(cmd: 'crun --root ${crun_root} kill ${self.name} SIGTERM', stdout: false) or {}
time.sleep(2 * time.second)
// Force kill if still running
if self.status()! == .running {
osal.exec(cmd: 'crun --root ${crun_root} kill ${self.name} SIGKILL', stdout: false) or {}
// Send SIGTERM for graceful shutdown
osal.exec(cmd: 'crun --root ${crun_root} kill ${self.name} SIGTERM', stdout: false) or {
self.factory.logger.log(
cat: 'container'
log: 'Failed to send SIGTERM (container may already be stopped): ${err}'
logtype: .stdout
) or {}
}
console.print_green('Container ${self.name} stopped')
// Wait up to sigterm_timeout_ms for graceful shutdown
mut attempts := 0
max_attempts := sigterm_timeout_ms / stop_check_interval_ms
for attempts < max_attempts {
time.sleep(stop_check_interval_ms * time.millisecond)
current_status := self.status() or {
// If we can't get status, assume it's stopped (container may have been deleted)
ContainerStatus.stopped
}
if current_status == .stopped {
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} stopped gracefully'
logtype: .stdout
) or {}
self.cleanup_network()! // Thread-safe network cleanup
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} stopped'
logtype: .stdout
) or {}
return
}
attempts++
}
// Force kill if still running after timeout
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} did not stop gracefully, force killing'
logtype: .stdout
) or {}
osal.exec(cmd: 'crun --root ${crun_root} kill ${self.name} SIGKILL', stdout: false) or {
self.factory.logger.log(
cat: 'container'
log: 'Failed to send SIGKILL: ${err}'
logtype: .error
) or {}
}
// Wait for SIGKILL to take effect
time.sleep(sigkill_wait_ms * time.millisecond)
// Verify it's actually stopped
final_status := self.status() or {
// If we can't get status, assume it's stopped (container may have been deleted)
ContainerStatus.stopped
}
if final_status != .stopped {
return error('Failed to stop container ${self.name} - status: ${final_status}')
}
// Cleanup network resources (thread-safe)
self.cleanup_network()!
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} stopped'
logtype: .stdout
) or {}
}
// Delete the container
//
// This method:
// 1. Checks if container exists in crun
// 2. Stops the container (which cleans up network)
// 3. Deletes the container from crun
// 4. Removes from factory's container cache
//
// Thread Safety:
// Network cleanup is thread-safe via stop() -> cleanup_network()
pub fn (mut self Container) delete() ! {
// Check if container exists before trying to delete
if !self.container_exists_in_crun()! {
console.print_debug('Container ${self.name} does not exist, nothing to delete')
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} does not exist in crun'
logtype: .stdout
) or {}
// Still cleanup network resources in case they exist (thread-safe)
self.cleanup_network() or {
self.factory.logger.log(
cat: 'container'
log: 'Network cleanup failed (may not exist): ${err}'
logtype: .stdout
) or {}
}
// Remove from factory's container cache only after all cleanup is done
if self.name in self.factory.containers {
self.factory.containers.delete(self.name)
}
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} removed from cache'
logtype: .stdout
) or {}
return
}
// Stop the container (this will cleanup network via stop())
self.stop()!
crun_root := '${self.factory.base_dir}/runtime'
osal.exec(cmd: 'crun --root ${crun_root} delete ${self.name}', stdout: false) or {}
// Remove from factory's container cache
// Delete the container from crun
crun_root := '${self.factory.base_dir}/runtime'
osal.exec(cmd: 'crun --root ${crun_root} delete ${self.name}', stdout: false) or {
self.factory.logger.log(
cat: 'container'
log: 'Failed to delete container from crun: ${err}'
logtype: .error
) or {}
}
// Remove from factory's container cache only after all cleanup is complete
if self.name in self.factory.containers {
self.factory.containers.delete(self.name)
}
console.print_green('Container ${self.name} deleted')
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} deleted'
logtype: .stdout
) or {}
}
// Execute command inside the container
@@ -134,24 +531,191 @@ pub fn (mut self Container) exec(cmd_ osal.Command) !string {
// Use the builder node to execute inside container
mut node := self.node()!
console.print_debug('Executing command in container ${self.name}: ${cmd_.cmd}')
return node.exec(cmd: cmd_.cmd, stdout: cmd_.stdout)
self.factory.logger.log(
cat: 'container'
log: 'Executing command in container ${self.name}: ${cmd_.cmd}'
logtype: .stdout
) or {}
// Execute and provide better error context
return node.exec(cmd: cmd_.cmd, stdout: cmd_.stdout) or {
// Check if container still exists to provide better error message
if !self.container_exists_in_crun()! {
return error('Container ${self.name} was deleted during command execution')
}
return error('Command execution failed in container ${self.name}: ${err}')
}
}
pub fn (self Container) status() !ContainerStatus {
crun_root := '${self.factory.base_dir}/runtime'
result := osal.exec(cmd: 'crun --root ${crun_root} state ${self.name}', stdout: false) or {
return .unknown
// Container doesn't exist - this is expected in some cases (e.g., before creation)
// Check error message to distinguish between "not found" and real errors
err_msg := err.msg().to_lower()
if err_msg.contains('does not exist') || err_msg.contains('not found')
|| err_msg.contains('no such') {
return .stopped
}
// Real error (permissions, crun not installed, etc.) - propagate it
return error('Failed to get container status: ${err}')
}
// Parse JSON output from crun state
state := json.decode(CrunState, result.output) or { return .unknown }
state := json.decode(CrunState, result.output) or {
return error('Failed to parse container state JSON: ${err}')
}
return match state.status {
'running' { .running }
'stopped' { .stopped }
'paused' { .paused }
else { .unknown }
status_result := match state.status {
'running' {
ContainerStatus.running
}
'stopped' {
ContainerStatus.stopped
}
'paused' {
ContainerStatus.paused
}
else {
// Unknown status - return unknown (can't log here as function is immutable)
ContainerStatus.unknown
}
}
return status_result
}
// Get the PID of the container's init process
pub fn (self Container) pid() !int {
crun_root := '${self.factory.base_dir}/runtime'
result := osal.exec(
cmd: 'crun --root ${crun_root} state ${self.name}'
stdout: false
)!
// Parse JSON output from crun state
state := json.decode(CrunState, result.output)!
if state.pid == 0 {
return error('Container ${self.name} has no PID (not running?)')
}
return state.pid
}
// Wait for container process to be fully ready
//
// After `crun start` returns, the container process may not be fully initialized yet.
// This method polls for the container's PID and verifies that /proc/<pid>/ns/net exists
// before returning. This ensures network setup can proceed without errors.
//
// The method uses exponential backoff polling (no sleep delays) to minimize wait time.
fn (mut self Container) wait_for_process_ready() ! {
crun_root := '${self.factory.base_dir}/runtime'
// Poll for up to 100 iterations (very fast, no sleep)
// Most containers will be ready within the first few iterations
for i in 0 .. 100 {
// Try to get the container state
result := osal.exec(
cmd: 'crun --root ${crun_root} state ${self.name}'
stdout: false
) or {
// Container state not ready yet, continue polling
if i % 20 == 0 {
self.factory.logger.log(
cat: 'container'
log: 'Waiting for container ${self.name} state (attempt ${i})...'
logtype: .stdout
) or {}
}
continue
}
// Parse the state to get PID
state := json.decode(CrunState, result.output) or {
// JSON not ready yet, continue polling
if i % 20 == 0 {
self.factory.logger.log(
cat: 'container'
log: 'Waiting for container ${self.name} state JSON to be valid (attempt ${i})...'
logtype: .stdout
) or {}
}
continue
}
// Check if we have a valid PID
if state.pid == 0 {
if i % 20 == 0 {
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} state has PID=0, waiting (attempt ${i})...'
logtype: .stdout
) or {}
}
continue
}
// Verify that /proc/<pid>/ns/net exists (this is what nsenter needs)
ns_net_path := '/proc/${state.pid}/ns/net'
if os.exists(ns_net_path) {
// Process is ready!
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} process ready with PID ${state.pid}'
logtype: .stdout
) or {}
return
}
if i % 20 == 0 {
self.factory.logger.log(
cat: 'container'
log: 'Container ${self.name} has PID ${state.pid} but /proc/${state.pid}/ns/net does not exist yet (attempt ${i})...'
logtype: .stdout
) or {}
}
// If we've tried many times, add a tiny yield to avoid busy-waiting
if i > 50 && i % 10 == 0 {
time.sleep(1 * time.millisecond)
}
}
return error('Container process did not become ready in time')
}
// Setup network for this container (thread-safe)
//
// Delegates to HeroPods.network_setup_container() which uses network_mutex
// for thread-safe IP allocation and network configuration.
fn (mut self Container) setup_network() ! {
// Get container PID
container_pid := self.pid()!
// Delegate to factory's network setup (thread-safe)
mut factory := self.factory
factory.network_setup_container(self.name, container_pid)!
}
// Cleanup network for this container (thread-safe)
//
// Delegates to HeroPods.network_cleanup_container() which uses network_mutex
// for thread-safe IP deallocation and network cleanup.
// Also cleans up Mycelium IPv6 overlay network if enabled.
fn (mut self Container) cleanup_network() ! {
mut factory := self.factory
factory.network_cleanup_container(self.name)!
// Cleanup Mycelium IPv6 overlay network if enabled
if factory.mycelium_enabled {
factory.mycelium_cleanup_container(self.name) or {
factory.logger.log(
cat: 'container'
log: 'Warning: Failed to cleanup Mycelium for container ${self.name}: ${err}'
logtype: .error
) or {}
}
}
}
@@ -167,11 +731,12 @@ fn (self Container) container_exists_in_crun() !bool {
return result.exit_code == 0
}
// ContainerStatus represents the current state of a container
pub enum ContainerStatus {
running
stopped
paused
unknown
running // Container is running
stopped // Container is stopped or doesn't exist
paused // Container is paused
unknown // Unknown status (error case)
}
// Get CPU usage in percentage

269
lib/virt/heropods/container_create.v
View File

@@ -1,30 +1,71 @@
module heropods
import incubaid.herolib.ui.console
import incubaid.herolib.osal.core as osal
import incubaid.herolib.virt.crun
import incubaid.herolib.installers.virt.herorunner as herorunner_installer
import incubaid.herolib.installers.virt.crun_installer
import os
import json
// Updated enum to be more flexible
pub enum ContainerImageType {
alpine_3_20
ubuntu_24_04
ubuntu_25_04
custom // For custom images downloaded via podman
// Image metadata structures for podman inspect
// These structures map to the JSON output of `podman inspect <image>`
// All fields are optional since different images may have different configurations
struct ImageInspectResult {
config ImageConfig @[json: 'Config']
}
struct ImageConfig {
pub mut:
entrypoint []string @[json: 'Entrypoint'; omitempty]
cmd []string @[json: 'Cmd'; omitempty]
env []string @[json: 'Env'; omitempty]
working_dir string @[json: 'WorkingDir'; omitempty]
}
// ContainerImageType defines the available container base images
pub enum ContainerImageType {
alpine_3_20 // Alpine Linux 3.20
ubuntu_24_04 // Ubuntu 24.04 LTS
ubuntu_25_04 // Ubuntu 25.04
custom // Custom image downloaded via podman
}
// ContainerNewArgs defines parameters for creating a new container
@[params]
pub struct ContainerNewArgs {
pub:
name string @[required]
image ContainerImageType = .alpine_3_20
name string @[required] // Unique container name
image ContainerImageType = .alpine_3_20 // Base image type
custom_image_name string // Used when image = .custom
docker_url string // Docker image URL for new images
reset bool
reset bool // Reset if container already exists
}
pub fn (mut self ContainerFactory) new(args ContainerNewArgs) !&Container {
// CrunConfigArgs defines parameters for creating crun configuration
@[params]
pub struct CrunConfigArgs {
pub:
container_name string @[required] // Container name
rootfs_path string @[required] // Path to container rootfs
}
// Create a new container
//
// This method:
// 1. Validates the container name
// 2. Determines the image to use (built-in or custom)
// 3. Creates crun configuration
// 4. Configures DNS in rootfs
//
// Note: The actual container creation in crun happens when start() is called.
// This method only prepares the configuration and rootfs.
//
// Thread Safety:
// This method doesn't interact with network_config, so no mutex is needed.
// Network setup happens later in container.start().
pub fn (mut self HeroPods) container_new(args ContainerNewArgs) !&Container {
// Validate container name to prevent shell injection and path traversal
validate_container_name(args.name) or { return error('Invalid container name: ${err}') }
if args.name in self.containers && !args.reset {
return self.containers[args.name] or { panic('bug: container should exist') }
}
@@ -55,7 +96,11 @@ pub fn (mut self ContainerFactory) new(args ContainerNewArgs) !&Container {
// If image not yet extracted, pull and unpack it
if !os.is_dir(rootfs_path) && args.docker_url != '' {
console.print_debug('Pulling image ${args.docker_url} with podman...')
self.logger.log(
cat: 'images'
log: 'Pulling image ${args.docker_url} with podman...'
logtype: .stdout
) or {}
self.podman_pull_and_export(args.docker_url, image_name, rootfs_path)!
}
}
@@ -67,12 +112,15 @@ pub fn (mut self ContainerFactory) new(args ContainerNewArgs) !&Container {
}
// Create crun configuration using the crun module
mut crun_config := self.create_crun_config(args.name, rootfs_path)!
mut crun_config := self.create_crun_config(
container_name: args.name
rootfs_path: rootfs_path
)!
// Ensure crun is installed on host
if !osal.cmd_exists('crun') {
mut herorunner := herorunner_installer.new()!
herorunner.install()!
mut crun_inst := crun_installer.get()!
crun_inst.install(reset: false)!
}
// Create container struct but don't create the actual container in crun yet
@@ -84,33 +132,138 @@ pub fn (mut self ContainerFactory) new(args ContainerNewArgs) !&Container {
}
self.containers[args.name] = container
// Configure DNS in container rootfs (uses network_config but doesn't modify it)
self.network_configure_dns(args.name, rootfs_path)!
return container
}
// Create crun configuration using the crun module
fn (mut self ContainerFactory) create_crun_config(container_name string, rootfs_path string) !&crun.CrunConfig {
// Create crun configuration for a container
//
// This creates an OCI-compliant runtime configuration that respects the image's
// ENTRYPOINT and CMD according to the OCI standard:
// - If image metadata exists (from podman inspect), use ENTRYPOINT + CMD
// - Otherwise, use a default shell command
// - Apply environment variables and working directory from image metadata
// - No terminal (background container)
// - Standard resource limits
fn (mut self HeroPods) create_crun_config(args CrunConfigArgs) !&crun.CrunConfig {
// Create crun configuration using the factory pattern
mut config := crun.new(mut self.crun_configs, name: container_name)!
mut config := crun.new(mut self.crun_configs, name: args.container_name)!
// Configure for heropods use case - disable terminal for background containers
config.set_terminal(false)
config.set_command(['/bin/sh', '-c', 'while true; do sleep 30; done'])
config.set_working_dir('/')
config.set_user(0, 0, [])
config.add_env('PATH', '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin')
config.add_env('TERM', 'xterm')
config.set_rootfs(rootfs_path, false)
config.set_rootfs(args.rootfs_path, false)
config.set_hostname('container')
config.set_no_new_privileges(true)
// Add the specific rlimit for file descriptors
// Check if image metadata exists (from podman inspect)
image_dir := os.dir(args.rootfs_path)
metadata_path := '${image_dir}/image_metadata.json'
if os.exists(metadata_path) {
// Load and apply OCI image metadata
self.logger.log(
cat: 'container'
log: 'Loading image metadata from ${metadata_path}'
logtype: .stdout
) or {}
metadata_json := os.read_file(metadata_path)!
image_config := json.decode(ImageConfig, metadata_json) or {
return error('Failed to parse image metadata: ${err}')
}
// Build command according to OCI spec:
// - If ENTRYPOINT exists: final_command = ENTRYPOINT + CMD
// - Else if CMD exists: final_command = CMD
// - Else: use default shell
//
// Note: We respect the image's original ENTRYPOINT and CMD without modification.
// If keep_alive is needed, it will be injected after the entrypoint completes.
mut final_command := []string{}
if image_config.entrypoint.len > 0 {
// ENTRYPOINT exists - combine with CMD
final_command << image_config.entrypoint
if image_config.cmd.len > 0 {
final_command << image_config.cmd
}
self.logger.log(
cat: 'container'
log: 'Using ENTRYPOINT + CMD: ${final_command}'
logtype: .stdout
) or {}
} else if image_config.cmd.len > 0 {
// Only CMD exists
final_command = image_config.cmd.clone()
// Warn if CMD is a bare shell that will exit immediately
if final_command.len == 1
&& final_command[0] in ['/bin/sh', '/bin/bash', '/bin/ash', '/bin/dash'] {
self.logger.log(
cat: 'container'
log: 'WARNING: CMD is a bare shell (${final_command[0]}) which will exit immediately when run non-interactively. Consider using keep_alive:true when starting this container.'
logtype: .stdout
) or {}
}
self.logger.log(
cat: 'container'
log: 'Using CMD: ${final_command}'
logtype: .stdout
) or {}
} else {
// No ENTRYPOINT or CMD - use default shell with keep-alive
// Since there's no entrypoint to run, we start with keep-alive directly
final_command = ['tail', '-f', '/dev/null']
self.logger.log(
cat: 'container'
log: 'No ENTRYPOINT or CMD found, using keep-alive: ${final_command}'
logtype: .stdout
) or {}
}
config.set_command(final_command)
// Apply environment variables from image
for env_var in image_config.env {
parts := env_var.split_nth('=', 2)
if parts.len == 2 {
config.add_env(parts[0], parts[1])
}
}
// Apply working directory from image
if image_config.working_dir != '' {
config.set_working_dir(image_config.working_dir)
} else {
config.set_working_dir('/')
}
} else {
// No metadata - use default configuration for built-in images
self.logger.log(
cat: 'container'
log: 'No image metadata found, using default shell configuration'
logtype: .stdout
) or {}
config.set_command(['/bin/sh'])
config.set_working_dir('/')
config.add_env('PATH', '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin')
config.add_env('TERM', 'xterm')
}
// Add resource limits
config.add_rlimit(.rlimit_nofile, 1024, 1024)
// Validate the configuration
config.validate()!
// Create config directory and save JSON
config_dir := '${self.base_dir}/configs/${container_name}'
config_dir := '${self.base_dir}/configs/${args.container_name}'
osal.exec(cmd: 'mkdir -p ${config_dir}', stdout: false)!
config_path := '${config_dir}/config.json'
@@ -119,14 +272,59 @@ fn (mut self ContainerFactory) create_crun_config(container_name string, rootfs_
return config
}
// Use podman to pull image and extract rootfs
fn (self ContainerFactory) podman_pull_and_export(docker_url string, image_name string, rootfs_path string) ! {
// Pull a Docker image using podman and extract its rootfs and metadata
//
// This method:
// 1. Pulls the image from Docker registry
// 2. Extracts image metadata (ENTRYPOINT, CMD, ENV, WorkingDir) via podman inspect
// 3. Saves metadata to image_metadata.json for later use
// 4. Creates a temporary container from the image
// 5. Exports the container filesystem to rootfs_path
// 6. Cleans up the temporary container
fn (mut self HeroPods) podman_pull_and_export(docker_url string, image_name string, rootfs_path string) ! {
// Pull image
osal.exec(
cmd: 'podman pull ${docker_url}'
stdout: true
)!
// Extract image metadata (ENTRYPOINT, CMD, ENV, WorkingDir)
// This is critical for OCI-compliant behavior - we need to respect the image's configuration
image_dir := os.dir(rootfs_path)
metadata_path := '${image_dir}/image_metadata.json'
self.logger.log(
cat: 'images'
log: 'Extracting image metadata from ${docker_url}...'
logtype: .stdout
) or {}
inspect_result := osal.exec(
cmd: 'podman inspect ${docker_url}'
stdout: false
)!
// Parse the inspect output (it's a JSON array with one element)
inspect_data := json.decode([]ImageInspectResult, inspect_result.output) or {
return error('Failed to parse podman inspect output: ${err}')
}
if inspect_data.len == 0 {
return error('podman inspect returned empty result for ${docker_url}')
}
// Create image directory if it doesn't exist
osal.exec(cmd: 'mkdir -p ${image_dir}', stdout: false)!
// Save the metadata for later use in create_crun_config
os.write_file(metadata_path, json.encode(inspect_data[0].config))!
self.logger.log(
cat: 'images'
log: 'Saved image metadata to ${metadata_path}'
logtype: .stdout
) or {}
// Create temp container
temp_name := 'tmp_${image_name}_${os.getpid()}'
osal.exec(
@@ -139,11 +337,24 @@ fn (self ContainerFactory) podman_pull_and_export(docker_url string, image_name
cmd: 'mkdir -p ${rootfs_path}'
stdout: false
)!
self.logger.log(
cat: 'images'
log: 'Exporting container filesystem to ${rootfs_path}...'
logtype: .stdout
) or {}
osal.exec(
cmd: 'podman export ${temp_name} | tar -C ${rootfs_path} -xf -'
stdout: true
stdout: false
)!
self.logger.log(
cat: 'images'
log: 'Container filesystem exported successfully'
logtype: .stdout
) or {}
// Cleanup temp container
osal.exec(
cmd: 'podman rm ${temp_name}'

142
lib/virt/heropods/container_image.v
View File

@@ -1,47 +1,61 @@
module heropods
import incubaid.herolib.ui.console
import incubaid.herolib.osal.core as osal
import incubaid.herolib.core.pathlib
import incubaid.herolib.core.texttools
import os
import json
// ContainerImage represents a container base image with its rootfs
//
// Thread Safety:
// Image operations are filesystem-based and don't interact with network_config,
// so no special thread safety considerations are needed.
@[heap]
pub struct ContainerImage {
pub mut:
image_name string @[required] // image is located in ${self.factory.base_dir}/images/<image_name>/rootfs
docker_url string // optional docker image URL
rootfs_path string // path to the extracted rootfs
size_mb f64 // size in MB
created_at string // creation timestamp
factory &ContainerFactory @[skip; str: skip]
image_name string @[required] // Image name (located in ${self.factory.base_dir}/images/<image_name>/rootfs)
docker_url string // Optional Docker registry URL
rootfs_path string // Path to the extracted rootfs
size_mb f64 // Size in MB
created_at string // Creation timestamp
factory &HeroPods @[skip; str: skip] // Reference to parent HeroPods instance
}
// ContainerImageArgs defines parameters for creating/managing container images
@[params]
pub struct ContainerImageArgs {
pub mut:
image_name string @[required] // image is located in ${self.factory.base_dir}/images/<image_name>/rootfs
docker_url string // docker image URL like "alpine:3.20" or "ubuntu:24.04"
reset bool
image_name string @[required] // Unique image name (located in ${self.factory.base_dir}/images/<image_name>/rootfs)
docker_url string // Docker image URL like "alpine:3.20" or "ubuntu:24.04"
reset bool // Reset if image already exists
}
// ImageExportArgs defines parameters for exporting an image
@[params]
pub struct ImageExportArgs {
pub mut:
dest_path string @[required] // destination .tgz file path
compress_level int = 6 // compression level 1-9
dest_path string @[required] // Destination .tgz file path
compress_level int = 6 // Compression level 1-9
}
// ImageImportArgs defines parameters for importing an image
@[params]
pub struct ImageImportArgs {
pub mut:
source_path string @[required] // source .tgz file path
reset bool // overwrite if exists
source_path string @[required] // Source .tgz file path
reset bool // Overwrite if exists
}
// Create new image or get existing
pub fn (mut self ContainerFactory) image_new(args ContainerImageArgs) !&ContainerImage {
// Create a new image or get existing image
//
// This method:
// 1. Normalizes the image name
// 2. Returns existing image if found (unless reset=true)
// 3. Downloads image from Docker registry if docker_url provided
// 4. Creates image metadata and stores in cache
//
// Thread Safety:
// Image operations are filesystem-based and don't interact with network_config.
pub fn (mut self HeroPods) image_new(args ContainerImageArgs) !&ContainerImage {
mut image_name := texttools.name_fix(args.image_name)
rootfs_path := '${self.base_dir}/images/${image_name}/rootfs'
@@ -79,9 +93,19 @@ pub fn (mut self ContainerFactory) image_new(args ContainerImageArgs) !&Containe
return image
}
// Download image from docker registry using podman
// Download image from Docker registry using podman
//
// This method:
// 1. Pulls the image from Docker registry
// 2. Creates a temporary container
// 3. Exports the rootfs to the images directory
// 4. Cleans up the temporary container
fn (mut self ContainerImage) download_from_docker(docker_url string, reset bool) ! {
console.print_header('Downloading image: ${docker_url}')
self.factory.logger.log(
cat: 'images'
log: 'Downloading image: ${docker_url}'
logtype: .stdout
) or {}
// Clean image name for local storage
image_dir := '${self.factory.base_dir}/images/${self.image_name}'
@@ -95,7 +119,11 @@ fn (mut self ContainerImage) download_from_docker(docker_url string, reset bool)
osal.exec(cmd: 'mkdir -p ${image_dir}', stdout: false)!
// Pull image using podman
console.print_debug('Pulling image: ${docker_url}')
self.factory.logger.log(
cat: 'images'
log: 'Pulling image: ${docker_url}'
logtype: .stdout
) or {}
osal.exec(cmd: 'podman pull ${docker_url}', stdout: true)!
// Create container from image (without running it)
@@ -117,16 +145,22 @@ fn (mut self ContainerImage) download_from_docker(docker_url string, reset bool)
// Remove the pulled image from podman to save space (optional)
osal.exec(cmd: 'podman rmi ${docker_url}', stdout: false) or {}
console.print_green('Image ${docker_url} extracted to ${self.rootfs_path}')
self.factory.logger.log(
cat: 'images'
log: 'Image ${docker_url} extracted to ${self.rootfs_path}'
logtype: .stdout
) or {}
}
// Update image metadata (size, creation time, etc.)
//
// Calculates the rootfs size and records creation timestamp
fn (mut self ContainerImage) update_metadata() ! {
if !os.is_dir(self.rootfs_path) {
return error('Rootfs path does not exist: ${self.rootfs_path}')
}
// Calculate size
// Calculate size in MB
result := osal.exec(cmd: 'du -sm ${self.rootfs_path}', stdout: false)!
result_parts := result.output.split_by_space()[0] or { panic('bug') }
size_str := result_parts.trim_space()
@@ -134,11 +168,13 @@ fn (mut self ContainerImage) update_metadata() ! {
// Get creation time
info := os.stat(self.rootfs_path) or { return error('stat failed: ${err}') }
self.created_at = info.ctime.str() // or mtime.str(), depending on what you want
self.created_at = info.ctime.str()
}
// List all available images
pub fn (mut self ContainerFactory) images_list() ![]&ContainerImage {
//
// Scans the images directory and returns all found images with metadata
pub fn (mut self HeroPods) images_list() ![]&ContainerImage {
mut images := []&ContainerImage{}
images_base_dir := '${self.base_dir}/images'
@@ -161,7 +197,11 @@ pub fn (mut self ContainerFactory) images_list() ![]&ContainerImage {
factory: &self
}
image.update_metadata() or {
console.print_stderr('Failed to update metadata for image ${dir}: ${err}')
self.logger.log(
cat: 'images'
log: 'Failed to update metadata for image ${dir}: ${err}'
logtype: .error
) or {}
continue
}
self.images[dir] = image
@@ -175,12 +215,18 @@ pub fn (mut self ContainerFactory) images_list() ![]&ContainerImage {
}
// Export image to .tgz file
//
// Creates a compressed tarball of the image rootfs
pub fn (mut self ContainerImage) export(args ImageExportArgs) ! {
if !os.is_dir(self.rootfs_path) {
return error('Image rootfs not found: ${self.rootfs_path}')
}
console.print_header('Exporting image ${self.image_name} to ${args.dest_path}')
self.factory.logger.log(
cat: 'images'
log: 'Exporting image ${self.image_name} to ${args.dest_path}'
logtype: .stdout
) or {}
// Ensure destination directory exists
dest_dir := os.dir(args.dest_path)
@@ -190,11 +236,17 @@ pub fn (mut self ContainerImage) export(args ImageExportArgs) ! {
cmd := 'tar -czf ${args.dest_path} -C ${os.dir(self.rootfs_path)} ${os.base(self.rootfs_path)}'
osal.exec(cmd: cmd, stdout: true)!
console.print_green('Image exported successfully to ${args.dest_path}')
self.factory.logger.log(
cat: 'images'
log: 'Image exported successfully to ${args.dest_path}'
logtype: .stdout
) or {}
}
// Import image from .tgz file
pub fn (mut self ContainerFactory) image_import(args ImageImportArgs) !&ContainerImage {
//
// Extracts a compressed tarball into the images directory and creates image metadata
pub fn (mut self HeroPods) image_import(args ImageImportArgs) !&ContainerImage {
if !os.exists(args.source_path) {
return error('Source file not found: ${args.source_path}')
}
@@ -204,7 +256,11 @@ pub fn (mut self ContainerFactory) image_import(args ImageImportArgs) !&Containe
image_name := filename.replace('.tgz', '').replace('.tar.gz', '')
image_name_clean := texttools.name_fix(image_name)
console.print_header('Importing image from ${args.source_path}')
self.logger.log(
cat: 'images'
log: 'Importing image from ${args.source_path}'
logtype: .stdout
) or {}
image_dir := '${self.base_dir}/images/${image_name_clean}'
rootfs_path := '${image_dir}/rootfs'
@@ -235,13 +291,23 @@ pub fn (mut self ContainerFactory) image_import(args ImageImportArgs) !&Containe
image.update_metadata()!
self.images[image_name_clean] = image
console.print_green('Image imported successfully: ${image_name_clean}')
self.logger.log(
cat: 'images'
log: 'Image imported successfully: ${image_name_clean}'
logtype: .stdout
) or {}
return image
}
// Delete image
//
// Removes the image directory and removes from factory cache
pub fn (mut self ContainerImage) delete() ! {
console.print_header('Deleting image: ${self.image_name}')
self.factory.logger.log(
cat: 'images'
log: 'Deleting image: ${self.image_name}'
logtype: .stdout
) or {}
image_dir := os.dir(self.rootfs_path)
if os.is_dir(image_dir) {
@@ -253,10 +319,16 @@ pub fn (mut self ContainerImage) delete() ! {
self.factory.images.delete(self.image_name)
}
console.print_green('Image ${self.image_name} deleted successfully')
self.factory.logger.log(
cat: 'images'
log: 'Image ${self.image_name} deleted successfully'
logtype: .stdout
) or {}
}
// Get image info as map
//
// Returns image metadata as a string map for display/serialization
pub fn (self ContainerImage) info() map[string]string {
return {
'name': self.image_name
@@ -267,7 +339,9 @@ pub fn (self ContainerImage) info() map[string]string {
}
}
// List available docker images that can be downloaded
// List available Docker images that can be downloaded
//
// Returns a curated list of commonly used Docker images
pub fn list_available_docker_images() []string {
return [
'alpine:3.20',

175
lib/virt/heropods/factory.v
View File

@@ -1,175 +0,0 @@
module heropods
import incubaid.herolib.ui.console
import incubaid.herolib.osal.core as osal
import incubaid.herolib.virt.crun
import os
@[heap]
pub struct ContainerFactory {
pub mut:
tmux_session string
containers map[string]&Container
images map[string]&ContainerImage
crun_configs map[string]&crun.CrunConfig
base_dir string
}
@[params]
pub struct FactoryInitArgs {
pub:
reset bool
use_podman bool = true
}
pub fn new(args FactoryInitArgs) !ContainerFactory {
mut f := ContainerFactory{}
f.init(args)!
return f
}
fn (mut self ContainerFactory) init(args FactoryInitArgs) ! {
// Ensure base directories exist
self.base_dir = os.getenv_opt('CONTAINERS_DIR') or { os.home_dir() + '/.containers' }
osal.exec(
cmd: 'mkdir -p ${self.base_dir}/images ${self.base_dir}/configs ${self.base_dir}/runtime'
stdout: false
)!
if args.use_podman {
if !osal.cmd_exists('podman') {
console.print_stderr('Warning: podman not found. Install podman for better image management.')
console.print_debug('Install with: apt install podman (Ubuntu) or brew install podman (macOS)')
} else {
console.print_debug('Using podman for image management')
}
}
// Clean up any leftover crun state if reset is requested
if args.reset {
self.cleanup_crun_state()!
}
// Load existing images into cache
self.load_existing_images()!
// Setup default images if not using podman
if !args.use_podman {
self.setup_default_images(args.reset)!
}
}
fn (mut self ContainerFactory) setup_default_images(reset bool) ! {
console.print_header('Setting up default images...')
default_images := [ContainerImageType.alpine_3_20, .ubuntu_24_04, .ubuntu_25_04]
for img in default_images {
mut args := ContainerImageArgs{
image_name: img.str()
reset: reset
}
if img.str() !in self.images || reset {
console.print_debug('Preparing default image: ${img.str()}')
_ = self.image_new(args)!
}
}
}
// Load existing images from filesystem into cache
fn (mut self ContainerFactory) load_existing_images() ! {
images_base_dir := '${self.base_dir}/containers/images'
if !os.is_dir(images_base_dir) {
return
}
dirs := os.ls(images_base_dir) or { return }
for dir in dirs {
full_path := '${images_base_dir}/${dir}'
if os.is_dir(full_path) {
rootfs_path := '${full_path}/rootfs'
if os.is_dir(rootfs_path) {
mut image := &ContainerImage{
image_name: dir
rootfs_path: rootfs_path
factory: &self
}
image.update_metadata() or {
console.print_stderr('⚠️ Failed to update metadata for image ${dir}: ${err}')
continue
}
self.images[dir] = image
console.print_debug('Loaded existing image: ${dir}')
}
}
}
}
pub fn (mut self ContainerFactory) get(args ContainerNewArgs) !&Container {
if args.name !in self.containers {
return error('Container "${args.name}" does not exist. Use factory.new() to create it first.')
}
return self.containers[args.name] or { panic('bug: container should exist') }
}
// Get image by name
pub fn (mut self ContainerFactory) image_get(name string) !&ContainerImage {
if name !in self.images {
return error('Image "${name}" not found in cache. Try importing or downloading it.')
}
return self.images[name] or { panic('bug: image should exist') }
}
// List all containers currently managed by crun
pub fn (self ContainerFactory) list() ![]Container {
mut containers := []Container{}
result := osal.exec(cmd: 'crun list --format json', stdout: false)!
// Parse crun list output (tab-separated)
lines := result.output.split_into_lines()
for line in lines {
if line.trim_space() == '' || line.starts_with('ID') {
continue
}
parts := line.split('\t')
if parts.len > 0 {
containers << Container{
name: parts[0]
factory: &self
}
}
}
return containers
}
// Clean up any leftover crun state
fn (mut self ContainerFactory) cleanup_crun_state() ! {
console.print_debug('Cleaning up leftover crun state...')
crun_root := '${self.base_dir}/runtime'
// Stop and delete all containers in our custom root
result := osal.exec(cmd: 'crun --root ${crun_root} list -q', stdout: false) or { return }
for container_name in result.output.split_into_lines() {
if container_name.trim_space() != '' {
console.print_debug('Cleaning up container: ${container_name}')
osal.exec(cmd: 'crun --root ${crun_root} kill ${container_name} SIGKILL', stdout: false) or {}
osal.exec(cmd: 'crun --root ${crun_root} delete ${container_name}', stdout: false) or {}
}
}
// Also clean up any containers in the default root that might be ours
result2 := osal.exec(cmd: 'crun list -q', stdout: false) or { return }
for container_name in result2.output.split_into_lines() {
if container_name.trim_space() != '' && container_name in self.containers {
console.print_debug('Cleaning up container from default root: ${container_name}')
osal.exec(cmd: 'crun kill ${container_name} SIGKILL', stdout: false) or {}
osal.exec(cmd: 'crun delete ${container_name}', stdout: false) or {}
}
}
// Clean up runtime directories
osal.exec(cmd: 'rm -rf ${crun_root}/*', stdout: false) or {}
osal.exec(cmd: 'find /run/crun -name "*" -type d -exec rm -rf {} + 2>/dev/null', stdout: false) or {}
}

321
lib/virt/heropods/heropods_factory_.v Normal file
View File

@@ -0,0 +1,321 @@
module heropods
import incubaid.herolib.core.base
import incubaid.herolib.core.playbook { PlayBook }
import json
// Global state for HeroPods instances
//
// Thread Safety Note:
// heropods_global is not marked as `shared` because it would break compile-time
// reflection in paramsparser. The map operations are generally safe for concurrent
// read access. For write operations, the Redis backend provides the source of truth
// and synchronization. Each HeroPods instance has its own network_mutex for
// protecting network operations.
__global (
heropods_global map[string]&HeroPods
heropods_default string
)
/////////FACTORY
@[params]
pub struct ArgsGet {
pub mut:
name string = 'default' // name of the heropods
fromdb bool // will load from filesystem
create bool // default will not create if not exist
reset bool // will reset the heropods
use_podman bool = true // will use podman for image management
// Network configuration
bridge_name string = 'heropods0'
subnet string = '10.10.0.0/24'
gateway_ip string = '10.10.0.1'
dns_servers []string = ['8.8.8.8', '8.8.4.4']
// Mycelium IPv6 overlay network configuration
enable_mycelium bool // Enable Mycelium IPv6 overlay network
mycelium_version string // Mycelium version to install (default: 'v0.5.6')
mycelium_ipv6_range string // Mycelium IPv6 address range (default: '400::/7')
mycelium_peers []string // Mycelium peer addresses (default: use public nodes)
mycelium_key_path string = '~/hero/cfg/priv_key.bin' // Path to Mycelium private key
}
pub fn new(args ArgsGet) !&HeroPods {
mut obj := HeroPods{
name: args.name
reset: args.reset
use_podman: args.use_podman
network_config: NetworkConfig{
bridge_name: args.bridge_name
subnet: args.subnet
gateway_ip: args.gateway_ip
dns_servers: args.dns_servers
}
mycelium_enabled: args.enable_mycelium
mycelium_version: args.mycelium_version
mycelium_ipv6_range: args.mycelium_ipv6_range
mycelium_peers: args.mycelium_peers
mycelium_key_path: args.mycelium_key_path
}
set(obj)!
return get(name: args.name)!
}
// Get a HeroPods instance by name
// If fromdb is true, loads from Redis; otherwise returns from memory cache
pub fn get(args ArgsGet) !&HeroPods {
mut context := base.context()!
heropods_default = args.name
if args.fromdb || args.name !in heropods_global {
mut r := context.redis()!
if r.hexists('context:heropods', args.name)! {
data := r.hget('context:heropods', args.name)!
if data.len == 0 {
print_backtrace()
return error('HeroPods with name: ${args.name} does not exist, prob bug.')
}
mut obj := json.decode(HeroPods, data)!
set_in_mem(obj)!
} else {
if args.create {
new(args)!
} else {
print_backtrace()
return error("HeroPods with name '${args.name}' does not exist")
}
}
return get(args)! // Recursive call with fromdb=false
}
return heropods_global[args.name] or {
print_backtrace()
return error('could not get config for heropods with name:${args.name}')
}
}
// Register a HeroPods instance (saves to both memory and Redis)
pub fn set(o HeroPods) ! {
mut o2 := set_in_mem(o)!
heropods_default = o2.name
mut context := base.context()!
mut r := context.redis()!
r.hset('context:heropods', o2.name, json.encode(o2))!
}
// Check if a HeroPods instance exists in Redis
pub fn exists(args ArgsGet) !bool {
mut context := base.context()!
mut r := context.redis()!
return r.hexists('context:heropods', args.name)!
}
// Delete a HeroPods instance from Redis (does not affect memory cache)
pub fn delete(args ArgsGet) ! {
mut context := base.context()!
mut r := context.redis()!
r.hdel('context:heropods', args.name)!
}
@[params]
pub struct ArgsList {
pub mut:
fromdb bool // will load from filesystem
}
// List all HeroPods instances
// If fromdb is true, loads from Redis and resets memory cache
// If fromdb is false, returns from memory cache
pub fn list(args ArgsList) ![]&HeroPods {
mut res := []&HeroPods{}
mut context := base.context()!
if args.fromdb {
// Reset memory cache and load from Redis
heropods_global = map[string]&HeroPods{}
heropods_default = ''
mut r := context.redis()!
mut l := r.hkeys('context:heropods')!
for name in l {
res << get(name: name, fromdb: true)!
}
} else {
// Load from memory cache
for _, client in heropods_global {
res << client
}
}
return res
}
// Set a HeroPods instance in memory cache only (does not persist to Redis)
// Performs lightweight validation via obj_init, then heavy initialization
fn set_in_mem(o HeroPods) !HeroPods {
mut o2 := obj_init(o)!
o2.initialize()! // Perform heavy initialization after validation
heropods_global[o2.name] = &o2
heropods_default = o2.name
return o2
}
pub fn play(mut plbook PlayBook) ! {
if !plbook.exists(filter: 'heropods.') {
return
}
// Process heropods.configure actions
for mut action in plbook.find(filter: 'heropods.configure')! {
heroscript := action.heroscript()
mut obj := heroscript_loads(heroscript)!
set(obj)!
action.done = true
}
// Process heropods.enable_mycelium actions
for mut action in plbook.find(filter: 'heropods.enable_mycelium')! {
mut p := action.params
heropods_name := p.get_default('heropods', heropods_default)!
mut hp := get(name: heropods_name)!
// Validate required parameters
mycelium_version := p.get('version') or {
return error('heropods.enable_mycelium: "version" is required (e.g., version:\'v0.5.6\')')
}
mycelium_ipv6_range := p.get('ipv6_range') or {
return error('heropods.enable_mycelium: "ipv6_range" is required (e.g., ipv6_range:\'400::/7\')')
}
mycelium_key_path := p.get('key_path') or {
return error('heropods.enable_mycelium: "key_path" is required (e.g., key_path:\'~/hero/cfg/priv_key.bin\')')
}
mycelium_peers_str := p.get('peers') or {
return error('heropods.enable_mycelium: "peers" is required. Provide comma-separated list of peer addresses (e.g., peers:\'tcp://185.69.166.8:9651,quic://[2a02:1802:5e:0:ec4:7aff:fe51:e36b]:9651\')')
}
// Parse and validate peers list
peers_array := mycelium_peers_str.split(',').map(it.trim_space()).filter(it.len > 0)
if peers_array.len == 0 {
return error('heropods.enable_mycelium: "peers" cannot be empty. Provide at least one peer address.')
}
// Update Mycelium configuration
hp.mycelium_enabled = true
hp.mycelium_version = mycelium_version
hp.mycelium_ipv6_range = mycelium_ipv6_range
hp.mycelium_key_path = mycelium_key_path
hp.mycelium_peers = peers_array
// Initialize Mycelium if not already done
hp.mycelium_init()!
// Save updated configuration
set(hp)!
action.done = true
}
// Process heropods.container_new actions
for mut action in plbook.find(filter: 'heropods.container_new')! {
mut p := action.params
heropods_name := p.get_default('heropods', heropods_default)!
mut hp := get(name: heropods_name)!
container_name := p.get('name')!
image_str := p.get_default('image', 'alpine_3_20')!
custom_image_name := p.get_default('custom_image_name', '')!
docker_url := p.get_default('docker_url', '')!
reset := p.get_default_false('reset')
image_type := match image_str {
'alpine_3_20' { ContainerImageType.alpine_3_20 }
'ubuntu_24_04' { ContainerImageType.ubuntu_24_04 }
'ubuntu_25_04' { ContainerImageType.ubuntu_25_04 }
'custom' { ContainerImageType.custom }
else { ContainerImageType.alpine_3_20 }
}
hp.container_new(
name: container_name
image: image_type
custom_image_name: custom_image_name
docker_url: docker_url
reset: reset
)!
action.done = true
}
// Process heropods.container_start actions
for mut action in plbook.find(filter: 'heropods.container_start')! {
mut p := action.params
heropods_name := p.get_default('heropods', heropods_default)!
mut hp := get(name: heropods_name)!
container_name := p.get('name')!
keep_alive := p.get_default_false('keep_alive')
mut container := hp.get(name: container_name)!
container.start(
keep_alive: keep_alive
)!
action.done = true
}
// Process heropods.container_exec actions
for mut action in plbook.find(filter: 'heropods.container_exec')! {
mut p := action.params
heropods_name := p.get_default('heropods', heropods_default)!
mut hp := get(name: heropods_name)!
container_name := p.get('name')!
cmd := p.get('cmd')!
stdout := p.get_default_true('stdout')
mut container := hp.get(name: container_name)!
result := container.exec(cmd: cmd, stdout: stdout)!
if stdout {
println(result)
}
action.done = true
}
// Process heropods.container_stop actions
for mut action in plbook.find(filter: 'heropods.container_stop')! {
mut p := action.params
heropods_name := p.get_default('heropods', heropods_default)!
mut hp := get(name: heropods_name)!
container_name := p.get('name')!
mut container := hp.get(name: container_name)!
container.stop()!
action.done = true
}
// Process heropods.container_delete actions
for mut action in plbook.find(filter: 'heropods.container_delete')! {
mut p := action.params
heropods_name := p.get_default('heropods', heropods_default)!
mut hp := get(name: heropods_name)!
container_name := p.get('name')!
mut container := hp.get(name: container_name)!
container.delete()!
action.done = true
}
}
// Switch the default HeroPods instance
//
// Thread Safety Note:
// String assignment is atomic on most platforms, so no explicit locking is needed.
// If strict thread safety is required in the future, this could be wrapped in a lock.
pub fn switch(name string) {
heropods_default = name
}

284
lib/virt/heropods/heropods_model.v Normal file
View File

@@ -0,0 +1,284 @@
module heropods
import incubaid.herolib.data.encoderhero
import incubaid.herolib.osal.core as osal
import incubaid.herolib.virt.crun
import incubaid.herolib.core.logger
import incubaid.herolib.core
import os
import sync
pub const version = '0.0.0'
const singleton = false
const default = true
// MyceliumConfig holds Mycelium IPv6 overlay network configuration (flattened into HeroPods struct)
// Note: These fields are flattened to avoid nested struct serialization issues with encoderhero
// HeroPods factory for managing containers
//
// Thread Safety:
// The network_config field is protected by network_mutex for thread-safe concurrent access.
// We use a separate mutex instead of marking network_config as `shared` because V's
// compile-time reflection (used by paramsparser) cannot handle shared fields.
@[heap]
pub struct HeroPods {
pub mut:
tmux_session string // tmux session name
containers map[string]&Container // name -> container mapping
images map[string]&ContainerImage // name -> image mapping
crun_configs map[string]&crun.CrunConfig // name -> crun config mapping
base_dir string // base directory for all container data
reset bool // will reset the heropods
use_podman bool = true // will use podman for image management
name string // name of the heropods
network_config NetworkConfig @[skip; str: skip] // network configuration (automatically initialized, not serialized)
network_mutex sync.Mutex @[skip; str: skip] // protects network_config for thread-safe concurrent access
// Mycelium IPv6 overlay network configuration (flattened fields)
mycelium_enabled bool // Whether Mycelium is enabled
mycelium_version string // Mycelium version to install (e.g., 'v0.5.6')
mycelium_ipv6_range string // Mycelium IPv6 address range (e.g., '400::/7')
mycelium_peers []string // Mycelium peer addresses
mycelium_key_path string // Path to Mycelium private key
mycelium_ip6 string // Host's Mycelium IPv6 address (cached)
mycelium_interface_name string // Mycelium TUN interface name (e.g., "mycelium0")
logger logger.Logger @[skip; str: skip] // logger instance for debugging (not serialized)
}
// obj_init performs lightweight validation and field normalization only
// Heavy initialization is done in the initialize() method
fn obj_init(mycfg_ HeroPods) !HeroPods {
mut mycfg := mycfg_
// Normalize base_dir from environment variable if not set
if mycfg.base_dir == '' {
mycfg.base_dir = os.getenv_opt('CONTAINERS_DIR') or { os.home_dir() + '/.heropods/default' }
}
// Validate: warn if podman is requested but not available
if mycfg.use_podman && !osal.cmd_exists('podman') {
eprintln('Warning: podman not found. Install podman for better image management.')
eprintln('Install with: apt install podman (Ubuntu) or brew install podman (macOS)')
}
// Preserve network_config from input, set defaults only if empty
if mycfg.network_config.bridge_name == '' {
mycfg.network_config.bridge_name = 'heropods0'
}
if mycfg.network_config.subnet == '' {
mycfg.network_config.subnet = '10.10.0.0/24'
}
if mycfg.network_config.gateway_ip == '' {
mycfg.network_config.gateway_ip = '10.10.0.1'
}
if mycfg.network_config.dns_servers.len == 0 {
mycfg.network_config.dns_servers = ['8.8.8.8', '8.8.4.4']
}
// Ensure allocated_ips map is initialized
if mycfg.network_config.allocated_ips.len == 0 {
mycfg.network_config.allocated_ips = map[string]string{}
}
// Initialize Mycelium configuration defaults (only for non-required fields)
if mycfg.mycelium_interface_name == '' {
mycfg.mycelium_interface_name = 'mycelium0'
}
return mycfg
}
// initialize performs heavy initialization operations
// This should be called after obj_init in the factory pattern
fn (mut self HeroPods) initialize() ! {
// Check platform - HeroPods requires Linux
if core.is_osx()! {
return error('HeroPods requires Linux. It uses Linux-specific tools (ip, iptables, nsenter, crun) that are not available on macOS. Please run HeroPods on a Linux system or use Docker/Podman directly on macOS.')
}
// Create base directories
osal.exec(
cmd: 'mkdir -p ${self.base_dir}/images ${self.base_dir}/configs ${self.base_dir}/runtime'
stdout: false
)!
// Initialize logger
self.logger = logger.new(
path: '${self.base_dir}/logs'
console_output: true
) or {
eprintln('Warning: Failed to create logger: ${err}')
logger.Logger{} // Use empty logger as fallback
}
// Clean up any leftover crun state if reset is requested
if self.reset {
self.cleanup_crun_state()!
self.network_cleanup_all(false)! // Keep bridge for reuse
}
// Initialize network layer
self.network_init()!
// Initialize Mycelium IPv6 overlay network if enabled
if self.mycelium_enabled {
self.mycelium_init()!
}
// Load existing images into cache
self.load_existing_images()!
// Setup default images if not using podman
if !self.use_podman {
self.setup_default_images(self.reset)!
}
}
/////////////NORMALLY NO NEED TO TOUCH
pub fn heroscript_loads(heroscript string) !HeroPods {
mut obj := encoderhero.decode[HeroPods](heroscript)!
return obj
}
fn (mut self HeroPods) setup_default_images(reset bool) ! {
self.logger.log(
cat: 'images'
log: 'Setting up default images...'
logtype: .stdout
) or {}
default_images := [ContainerImageType.alpine_3_20, .ubuntu_24_04, .ubuntu_25_04]
for img in default_images {
mut args := ContainerImageArgs{
image_name: img.str()
reset: reset
}
if img.str() !in self.images || reset {
self.logger.log(
cat: 'images'
log: 'Preparing default image: ${img.str()}'
logtype: .stdout
) or {}
self.image_new(args)!
}
}
}
// Load existing images from filesystem into cache
fn (mut self HeroPods) load_existing_images() ! {
images_base_dir := '${self.base_dir}/containers/images'
if !os.is_dir(images_base_dir) {
return
}
dirs := os.ls(images_base_dir) or { return }
for dir in dirs {
full_path := '${images_base_dir}/${dir}'
if os.is_dir(full_path) {
rootfs_path := '${full_path}/rootfs'
if os.is_dir(rootfs_path) {
mut image := &ContainerImage{
image_name: dir
rootfs_path: rootfs_path
factory: &self
}
image.update_metadata() or {
self.logger.log(
cat: 'images'
log: 'Failed to update metadata for image ${dir}: ${err}'
logtype: .error
) or {}
continue
}
self.images[dir] = image
self.logger.log(
cat: 'images'
log: 'Loaded existing image: ${dir}'
logtype: .stdout
) or {}
}
}
}
}
pub fn (mut self HeroPods) get(args ContainerNewArgs) !&Container {
if args.name !in self.containers {
return error('Container "${args.name}" does not exist. Use factory.new() to create it first.')
}
return self.containers[args.name] or { panic('bug: container should exist') }
}
// Get image by name
pub fn (mut self HeroPods) image_get(name string) !&ContainerImage {
if name !in self.images {
return error('Image "${name}" not found in cache. Try importing or downloading it.')
}
return self.images[name] or { panic('bug: image should exist') }
}
// List all containers currently managed by crun
pub fn (self HeroPods) list() ![]Container {
mut containers := []Container{}
result := osal.exec(cmd: 'crun list --format json', stdout: false)!
// Parse crun list output (tab-separated)
lines := result.output.split_into_lines()
for line in lines {
if line.trim_space() == '' || line.starts_with('ID') {
continue
}
parts := line.split('\t')
if parts.len > 0 {
containers << Container{
name: parts[0]
factory: &self
}
}
}
return containers
}
// Clean up any leftover crun state
fn (mut self HeroPods) cleanup_crun_state() ! {
self.logger.log(
cat: 'cleanup'
log: 'Cleaning up leftover crun state...'
logtype: .stdout
) or {}
crun_root := '${self.base_dir}/runtime'
// Stop and delete all containers in our custom root
result := osal.exec(cmd: 'crun --root ${crun_root} list -q', stdout: false) or { return }
for container_name in result.output.split_into_lines() {
if container_name.trim_space() != '' {
self.logger.log(
cat: 'cleanup'
log: 'Cleaning up container: ${container_name}'
logtype: .stdout
) or {}
osal.exec(cmd: 'crun --root ${crun_root} kill ${container_name} SIGKILL', stdout: false) or {}
osal.exec(cmd: 'crun --root ${crun_root} delete ${container_name}', stdout: false) or {}
}
}
// Also clean up any containers in the default root that might be ours
result2 := osal.exec(cmd: 'crun list -q', stdout: false) or { return }
for container_name in result2.output.split_into_lines() {
if container_name.trim_space() != '' && container_name in self.containers {
self.logger.log(
cat: 'cleanup'
log: 'Cleaning up container from default root: ${container_name}'
logtype: .stdout
) or {}
osal.exec(cmd: 'crun kill ${container_name} SIGKILL', stdout: false) or {}
osal.exec(cmd: 'crun delete ${container_name}', stdout: false) or {}
}
}
// Clean up runtime directories
osal.exec(cmd: 'rm -rf ${crun_root}/*', stdout: false) or {}
osal.exec(cmd: 'find /run/crun -name "*" -type d -exec rm -rf {} + 2>/dev/null', stdout: false) or {}
}

349
lib/virt/heropods/heropods_test.v Normal file
View File

@@ -0,0 +1,349 @@
module heropods
import incubaid.herolib.core
import os
// Simplified test suite for HeroPods container management
//
// These tests use real Docker images (Alpine Linux) for reliability
// Prerequisites: Linux, crun, podman, ip, iptables, nsenter
// Helper function to check if we're on Linux
fn is_linux_platform() bool {
return core.is_linux() or { false }
}
// Helper function to skip test if not on Linux
fn skip_if_not_linux() {
if !is_linux_platform() {
eprintln('SKIP: Test requires Linux (crun, ip, iptables)')
exit(0)
}
}
// Cleanup helper for tests - stops and deletes all containers
fn cleanup_test_heropods(name string) {
mut hp := get(name: name) or { return }
// Stop and delete all containers
for container_name, mut container in hp.containers {
container.stop() or {}
container.delete() or {}
}
// Cleanup network - don't delete the bridge (false) - tests run in parallel
hp.network_cleanup_all(false) or {}
// Delete from factory
delete(name: name) or {}
}
// Test 1: HeroPods initialization and configuration
fn test_heropods_initialization() ! {
skip_if_not_linux()
test_name := 'test_init_${os.getpid()}'
defer {
cleanup_test_heropods(test_name)
}
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true // Skip default image setup in tests
)!
assert hp.base_dir != ''
assert hp.network_config.bridge_name == 'heropods0'
assert hp.network_config.subnet == '10.10.0.0/24'
assert hp.network_config.gateway_ip == '10.10.0.1'
assert hp.network_config.dns_servers.len > 0
assert hp.name == test_name
println('✓ HeroPods initialization test passed')
}
// Test 2: Custom network configuration
fn test_custom_network_config() ! {
skip_if_not_linux()
test_name := 'test_custom_net_${os.getpid()}'
defer { cleanup_test_heropods(test_name) }
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true // Skip default image setup in tests
bridge_name: 'testbr0'
subnet: '192.168.100.0/24'
gateway_ip: '192.168.100.1'
dns_servers: ['1.1.1.1', '1.0.0.1']
)!
assert hp.network_config.bridge_name == 'testbr0'
assert hp.network_config.subnet == '192.168.100.0/24'
assert hp.network_config.gateway_ip == '192.168.100.1'
assert hp.network_config.dns_servers == ['1.1.1.1', '1.0.0.1']
println('✓ Custom network configuration test passed')
}
// Test 3: Pull Docker image and create container
fn test_container_creation_with_docker_image() ! {
skip_if_not_linux()
test_name := 'test_docker_${os.getpid()}'
defer {
cleanup_test_heropods(test_name)
}
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true
)!
container_name := 'alpine_${os.getpid()}'
// Pull Alpine Linux image from Docker Hub (very small, ~7MB)
mut container := hp.container_new(
name: container_name
image: .custom
custom_image_name: 'alpine_test'
docker_url: 'docker.io/library/alpine:3.20'
)!
assert container.name == container_name
assert container.factory.name == test_name
assert container_name in hp.containers
// Verify rootfs was extracted
rootfs_path := '${hp.base_dir}/images/alpine_test/rootfs'
assert os.is_dir(rootfs_path)
// Alpine uses busybox, check for bin directory and basic structure
assert os.is_dir('${rootfs_path}/bin')
assert os.is_dir('${rootfs_path}/etc')
println('✓ Docker image pull and container creation test passed')
}
// Test 4: Container lifecycle with real Docker image (start, status, stop, delete)
fn test_container_lifecycle() ! {
skip_if_not_linux()
test_name := 'test_lifecycle_${os.getpid()}'
defer {
cleanup_test_heropods(test_name)
}
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true
)!
container_name := 'lifecycle_${os.getpid()}'
mut container := hp.container_new(
name: container_name
image: .custom
custom_image_name: 'alpine_lifecycle'
docker_url: 'docker.io/library/alpine:3.20'
)!
// Test start with keep_alive to prevent Alpine's /bin/sh from exiting immediately
container.start(keep_alive: true)!
status := container.status()!
assert status == .running
// Verify container has a PID
pid := container.pid()!
assert pid > 0
// Test stop
container.stop()!
status_after_stop := container.status()!
assert status_after_stop == .stopped
// Test delete
container.delete()!
exists := container.container_exists_in_crun()!
assert !exists
println('✓ Container lifecycle test passed')
}
// Test 5: Container command execution with real Alpine image
fn test_container_exec() ! {
skip_if_not_linux()
test_name := 'test_exec_${os.getpid()}'
defer {
cleanup_test_heropods(test_name)
}
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true
)!
container_name := 'exec_${os.getpid()}'
mut container := hp.container_new(
name: container_name
image: .custom
custom_image_name: 'alpine_exec'
docker_url: 'docker.io/library/alpine:3.20'
)!
// Start with keep_alive to prevent Alpine's /bin/sh from exiting immediately
container.start(keep_alive: true)!
defer {
container.stop() or {}
container.delete() or {}
}
// Execute simple echo command
result := container.exec(cmd: 'echo "test123"')!
assert result.contains('test123')
// Execute pwd command
result2 := container.exec(cmd: 'pwd')!
assert result2.contains('/')
// Execute ls command (Alpine has busybox ls)
result3 := container.exec(cmd: 'ls /')!
assert result3.contains('bin')
assert result3.contains('etc')
println('✓ Container exec test passed')
}
// Test 6: Network IP allocation (without starting containers)
fn test_network_ip_allocation() ! {
skip_if_not_linux()
test_name := 'test_ip_alloc_${os.getpid()}'
defer {
cleanup_test_heropods(test_name)
}
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true
)!
// Allocate IPs for multiple containers (without starting them)
ip1 := hp.network_allocate_ip('container1')!
ip2 := hp.network_allocate_ip('container2')!
ip3 := hp.network_allocate_ip('container3')!
// Verify IPs are different
assert ip1 != ip2
assert ip2 != ip3
assert ip1 != ip3
// Verify IPs are in correct subnet
assert ip1.starts_with('10.10.0.')
assert ip2.starts_with('10.10.0.')
assert ip3.starts_with('10.10.0.')
// Verify IPs are tracked
assert 'container1' in hp.network_config.allocated_ips
assert 'container2' in hp.network_config.allocated_ips
assert 'container3' in hp.network_config.allocated_ips
println('✓ Network IP allocation test passed')
}
// Test 7: IPv4 connectivity test with real Alpine container
fn test_ipv4_connectivity() ! {
skip_if_not_linux()
test_name := 'test_ipv4_${os.getpid()}'
defer {
cleanup_test_heropods(test_name)
}
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true
)!
container_name := 'ipv4_${os.getpid()}'
mut container := hp.container_new(
name: container_name
image: .custom
custom_image_name: 'alpine_ipv4'
docker_url: 'docker.io/library/alpine:3.20'
)!
// Start with keep_alive to prevent Alpine's /bin/sh from exiting immediately
container.start(keep_alive: true)!
defer {
container.stop() or {}
container.delete() or {}
}
// Check container has an IP address
container_ip := hp.network_config.allocated_ips[container_name] or {
return error('Container should have allocated IP')
}
assert container_ip.starts_with('10.10.0.')
// Test IPv4 connectivity by checking the container's IP configuration
result := container.exec(cmd: 'ip addr show eth0')!
assert result.contains(container_ip)
assert result.contains('eth0')
// Test that default route exists
route_result := container.exec(cmd: 'ip route')!
assert route_result.contains('default')
assert route_result.contains('10.10.0.1')
println('✓ IPv4 connectivity test passed')
}
// Test 8: Container deletion and IP cleanup
fn test_container_deletion() ! {
skip_if_not_linux()
test_name := 'test_delete_${os.getpid()}'
defer {
cleanup_test_heropods(test_name)
}
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true
)!
container_name := 'delete_${os.getpid()}'
mut container := hp.container_new(
name: container_name
image: .custom
custom_image_name: 'alpine_delete'
docker_url: 'docker.io/library/alpine:3.20'
)!
// Start container with keep_alive to prevent Alpine's /bin/sh from exiting immediately
container.start(keep_alive: true)!
// Verify IP is allocated
assert container_name in hp.network_config.allocated_ips
// Stop and delete container
container.stop()!
container.delete()!
// Verify container is deleted from crun
exists := container.container_exists_in_crun()!
assert !exists
// Verify IP is freed
assert container_name !in hp.network_config.allocated_ips
println('✓ Container deletion and IP cleanup test passed')
}

5
lib/virt/heropods/instructions.md
View File

@@ -1,5 +0,0 @@
- use builder... for remote execution inside the container
- make an executor like we have for SSH but then for the container, so we can use this to execute commands inside the container
-

362
lib/virt/heropods/mycelium.v Normal file
View File

@@ -0,0 +1,362 @@
module heropods
import incubaid.herolib.osal.core as osal
import incubaid.herolib.clients.mycelium
import crypto.sha256
// Initialize Mycelium for HeroPods
//
// This method:
// 1. Validates required configuration
// 2. Checks that Mycelium binary is installed
// 3. Checks that Mycelium service is running
// 4. Retrieves the host's Mycelium IPv6 address
//
// Prerequisites:
// - Mycelium must be installed on the system
// - Mycelium service must be running
//
// Thread Safety:
// This is called during HeroPods initialization, before any concurrent operations.
fn (mut self HeroPods) mycelium_init() ! {
if !self.mycelium_enabled {
return
}
// Validate required configuration
if self.mycelium_version == '' {
return error('Mycelium configuration error: "version" is required. Use heropods.enable_mycelium to configure.')
}
if self.mycelium_ipv6_range == '' {
return error('Mycelium configuration error: "ipv6_range" is required. Use heropods.enable_mycelium to configure.')
}
if self.mycelium_key_path == '' {
return error('Mycelium configuration error: "key_path" is required. Use heropods.enable_mycelium to configure.')
}
if self.mycelium_peers.len == 0 {
return error('Mycelium configuration error: "peers" is required. Use heropods.enable_mycelium to configure.')
}
self.logger.log(
cat: 'mycelium'
log: 'START mycelium_init() - Initializing Mycelium IPv6 overlay network'
) or {}
// Check if Mycelium is installed - it's a prerequisite
if !self.mycelium_check_installed()! {
return error('Mycelium is not installed. Please install Mycelium first. See: https://github.com/threefoldtech/mycelium')
}
self.logger.log(
cat: 'mycelium'
log: 'Mycelium binary found'
logtype: .stdout
) or {}
// Check if Mycelium service is running - it's a prerequisite
if !self.mycelium_check_running()! {
return error('Mycelium service is not running. Please start Mycelium service first (e.g., mycelium --key-file ${self.mycelium_key_path} --peers <peers>)')
}
self.logger.log(
cat: 'mycelium'
log: 'Mycelium service is running'
logtype: .stdout
) or {}
// Get and cache the host's Mycelium IPv6 address
self.mycelium_get_host_address()!
self.logger.log(
cat: 'mycelium'
log: 'END mycelium_init() - Mycelium initialized successfully with address ${self.mycelium_ip6}'
logtype: .stdout
) or {}
}
// Check if Mycelium binary is installed
fn (mut self HeroPods) mycelium_check_installed() !bool {
return osal.cmd_exists('mycelium')
}
// Check if Mycelium service is running
fn (mut self HeroPods) mycelium_check_running() !bool {
// Try to inspect Mycelium - if it succeeds, it's running
mycelium.inspect(key_file_path: self.mycelium_key_path) or { return false }
return true
}
// Get the host's Mycelium IPv6 address
fn (mut self HeroPods) mycelium_get_host_address() ! {
self.logger.log(
cat: 'mycelium'
log: 'Retrieving host Mycelium IPv6 address...'
logtype: .stdout
) or {}
// Use mycelium inspect to get the address
inspect_result := mycelium.inspect(key_file_path: self.mycelium_key_path)!
if inspect_result.address == '' {
return error('Failed to get Mycelium IPv6 address from inspect')
}
self.mycelium_ip6 = inspect_result.address
self.logger.log(
cat: 'mycelium'
log: 'Host Mycelium IPv6 address: ${self.mycelium_ip6}'
logtype: .stdout
) or {}
}
// Setup Mycelium IPv6 networking for a container
//
// This method:
// 1. Creates a veth pair for Mycelium connectivity
// 2. Moves one end into the container's network namespace
// 3. Assigns a Mycelium IPv6 address to the container
// 4. Configures IPv6 forwarding and routing
//
// Thread Safety:
// This is called from container.start() which is already serialized per container.
// Multiple containers can be started concurrently, each with their own veth pair.
fn (mut self HeroPods) mycelium_setup_container(container_name string, container_pid int) ! {
if !self.mycelium_enabled {
return
}
self.logger.log(
cat: 'mycelium'
log: 'Setting up Mycelium IPv6 for container ${container_name} (PID: ${container_pid})'
logtype: .stdout
) or {}
// Create unique veth pair names using hash (same pattern as IPv4 networking)
short_hash := sha256.hexhash(container_name)[..6]
veth_container := 'vmy-${short_hash}'
veth_host := 'vmyh-${short_hash}'
// Delete veth pair if it already exists (cleanup from previous run)
osal.exec(cmd: 'ip link delete ${veth_container} 2>/dev/null', stdout: false) or {}
osal.exec(cmd: 'ip link delete ${veth_host} 2>/dev/null', stdout: false) or {}
// Create veth pair
self.logger.log(
cat: 'mycelium'
log: 'Creating veth pair: ${veth_container} <-> ${veth_host}'
logtype: .stdout
) or {}
osal.exec(
cmd: 'ip link add ${veth_container} type veth peer name ${veth_host}'
stdout: false
)!
// Bring up host end
osal.exec(
cmd: 'ip link set ${veth_host} up'
stdout: false
)!
// Move container end into container's network namespace
self.logger.log(
cat: 'mycelium'
log: 'Moving ${veth_container} into container namespace'
logtype: .stdout
) or {}
osal.exec(
cmd: 'ip link set ${veth_container} netns ${container_pid}'
stdout: false
)!
// Configure container end inside the namespace
// Bring up the interface
osal.exec(
cmd: 'nsenter -t ${container_pid} -n ip link set ${veth_container} up'
stdout: false
)!
// Get the Mycelium IPv6 prefix from the host
// Extract the prefix from the full address (e.g., "400:1234:5678::/64" from "400:1234:5678::1")
mycelium_prefix := self.mycelium_get_ipv6_prefix()!
// Assign IPv6 address to container (use ::1 in the subnet)
container_ip6 := '${mycelium_prefix}::1/64'
self.logger.log(
cat: 'mycelium'
log: 'Assigning IPv6 address ${container_ip6} to container'
logtype: .stdout
) or {}
osal.exec(
cmd: 'nsenter -t ${container_pid} -n ip addr add ${container_ip6} dev ${veth_container}'
stdout: false
)!
// Enable IPv6 forwarding on the host
self.logger.log(
cat: 'mycelium'
log: 'Enabling IPv6 forwarding'
logtype: .stdout
) or {}
osal.exec(
cmd: 'sysctl -w net.ipv6.conf.all.forwarding=1'
stdout: false
) or {
self.logger.log(
cat: 'mycelium'
log: 'Warning: Failed to enable IPv6 forwarding: ${err}'
logtype: .error
) or {}
}
// Get the link-local address of the host end of the veth pair
veth_host_ll := self.mycelium_get_link_local_address(veth_host)!
// Add route in container for Mycelium traffic (400::/7 via link-local)
self.logger.log(
cat: 'mycelium'
log: 'Adding route for ${self.mycelium_ipv6_range} via ${veth_host_ll}'
logtype: .stdout
) or {}
osal.exec(
cmd: 'nsenter -t ${container_pid} -n ip route add ${self.mycelium_ipv6_range} via ${veth_host_ll} dev ${veth_container}'
stdout: false
)!
// Add route on host for container's IPv6 address
self.logger.log(
cat: 'mycelium'
log: 'Adding host route for ${mycelium_prefix}::1/128'
logtype: .stdout
) or {}
osal.exec(
cmd: 'ip route add ${mycelium_prefix}::1/128 dev ${veth_host}'
stdout: false
)!
self.logger.log(
cat: 'mycelium'
log: 'Mycelium IPv6 setup complete for container ${container_name}'
logtype: .stdout
) or {}
}
// Get the IPv6 prefix from the host's Mycelium address
//
// Extracts the /64 prefix from the full IPv6 address
// Example: "400:1234:5678::1" -> "400:1234:5678:"
fn (mut self HeroPods) mycelium_get_ipv6_prefix() !string {
if self.mycelium_ip6 == '' {
return error('Mycelium IPv6 address not set')
}
// Split the address by ':' and take the first 3 parts for /64 prefix
parts := self.mycelium_ip6.split(':')
if parts.len < 3 {
return error('Invalid Mycelium IPv6 address format: ${self.mycelium_ip6}')
}
// Reconstruct the prefix (first 3 parts)
prefix := '${parts[0]}:${parts[1]}:${parts[2]}'
return prefix
}
// Get the link-local IPv6 address of an interface
//
// Link-local addresses are used for routing within the same network segment
// They start with fe80::
fn (mut self HeroPods) mycelium_get_link_local_address(interface_name string) !string {
self.logger.log(
cat: 'mycelium'
log: 'Getting link-local address for interface ${interface_name}'
logtype: .stdout
) or {}
// Get IPv6 addresses for the interface
cmd := "ip -6 addr show dev ${interface_name} | grep 'inet6 fe80' | awk '{print \$2}' | cut -d'/' -f1"
result := osal.exec(
cmd: cmd
stdout: false
)!
link_local := result.output.trim_space()
if link_local == '' {
return error('Failed to get link-local address for interface ${interface_name}')
}
self.logger.log(
cat: 'mycelium'
log: 'Link-local address for ${interface_name}: ${link_local}'
logtype: .stdout
) or {}
return link_local
}
// Cleanup Mycelium networking for a container
//
// This method:
// 1. Removes the veth pair
// 2. Removes routes
//
// Thread Safety:
// This is called from container.stop() and container.delete() which are serialized per container.
fn (mut self HeroPods) mycelium_cleanup_container(container_name string) ! {
if !self.mycelium_enabled {
return
}
self.logger.log(
cat: 'mycelium'
log: 'Cleaning up Mycelium IPv6 for container ${container_name}'
logtype: .stdout
) or {}
// Remove veth interfaces (they should be auto-removed when container stops, but cleanup anyway)
short_hash := sha256.hexhash(container_name)[..6]
veth_host := 'vmyh-${short_hash}'
osal.exec(
cmd: 'ip link delete ${veth_host} 2>/dev/null'
stdout: false
) or {}
// Remove host route (if it exists)
mycelium_prefix := self.mycelium_get_ipv6_prefix() or {
self.logger.log(
cat: 'mycelium'
log: 'Warning: Could not get Mycelium prefix for cleanup: ${err}'
logtype: .error
) or {}
return
}
osal.exec(
cmd: 'ip route del ${mycelium_prefix}::1/128 2>/dev/null'
stdout: false
) or {}
self.logger.log(
cat: 'mycelium'
log: 'Mycelium IPv6 cleanup complete for container ${container_name}'
logtype: .stdout
) or {}
}
// Inspect Mycelium status and return information
//
// Returns the public key and IPv6 address of the Mycelium node
pub fn (mut self HeroPods) mycelium_inspect() !mycelium.MyceliumInspectResult {
if !self.mycelium_enabled {
return error('Mycelium is not enabled')
}
return mycelium.inspect(key_file_path: self.mycelium_key_path)!
}

594
lib/virt/heropods/network.v Normal file
View File

@@ -0,0 +1,594 @@
module heropods
import incubaid.herolib.osal.core as osal
import os
import crypto.sha256
// Network configuration for HeroPods
//
// This module provides container networking similar to Docker/Podman:
// - Bridge networking with automatic IP allocation
// - NAT for outbound internet access
// - DNS configuration
// - veth pair management
//
// Thread Safety:
// All network_config operations are protected by HeroPods.network_mutex.
// The struct is not marked as `shared` to maintain compatibility with
// paramsparser's compile-time reflection.
//
// Future extension possibilities:
// - IPv6 support
// - Custom per-container DNS servers
// - iptables isolation (firewall per container)
// - Multiple bridges for isolated networks
// - Port forwarding/mapping
// - Network policies and traffic shaping
// NetworkConfig holds network configuration for HeroPods containers
struct NetworkConfig {
pub mut:
bridge_name string // Name of the bridge (e.g., "heropods0")
subnet string // Subnet for the bridge (e.g., "10.10.0.0/24")
gateway_ip string // Gateway IP for the bridge
dns_servers []string // List of DNS servers
allocated_ips map[string]string // container_name -> IP address
freed_ip_pool []int // Pool of freed IP offsets for reuse (e.g., [15, 23, 42])
next_ip_offset int = 10 // Start allocating from 10.10.0.10 (only used when pool is empty)
}
// Initialize network configuration in HeroPods factory
fn (mut self HeroPods) network_init() ! {
self.logger.log(
cat: 'network'
log: 'START network_init() - Initializing HeroPods network layer'
) or {}
// Setup host bridge if it doesn't exist
self.logger.log(
cat: 'network'
log: 'Calling network_setup_bridge()...'
logtype: .stdout
) or {}
self.network_setup_bridge()!
self.logger.log(
cat: 'network'
log: 'END network_init() - HeroPods network layer initialized successfully'
logtype: .stdout
) or {}
}
// Setup the host bridge network (one-time setup, idempotent)
fn (mut self HeroPods) network_setup_bridge() ! {
bridge_name := self.network_config.bridge_name
gateway_ip := '${self.network_config.gateway_ip}/${self.network_config.subnet.split('/')[1]}'
subnet := self.network_config.subnet
self.logger.log(
cat: 'network'
log: 'START network_setup_bridge() - bridge=${bridge_name}, gateway=${gateway_ip}, subnet=${subnet}'
logtype: .stdout
) or {}
// Check if bridge already exists using os.execute (more reliable than osal.exec)
self.logger.log(
cat: 'network'
log: 'Checking if bridge ${bridge_name} exists (running: ip link show ${bridge_name})...'
logtype: .stdout
) or {}
check_result := os.execute('ip link show ${bridge_name} 2>/dev/null')
self.logger.log(
cat: 'network'
log: 'Bridge check result: exit_code=${check_result.exit_code}'
logtype: .stdout
) or {}
if check_result.exit_code == 0 {
self.logger.log(
cat: 'network'
log: 'Bridge ${bridge_name} already exists - skipping creation'
logtype: .stdout
) or {}
return
}
self.logger.log(
cat: 'network'
log: 'Bridge ${bridge_name} does not exist - creating new bridge'
logtype: .stdout
) or {}
// Create bridge
self.logger.log(
cat: 'network'
log: 'Step 1: Creating bridge (running: ip link add name ${bridge_name} type bridge)...'
logtype: .stdout
) or {}
osal.exec(
cmd: 'ip link add name ${bridge_name} type bridge'
stdout: false
)!
self.logger.log(
cat: 'network'
log: 'Step 1: Bridge created successfully'
logtype: .stdout
) or {}
// Assign IP to bridge
self.logger.log(
cat: 'network'
log: 'Step 2: Assigning IP to bridge (running: ip addr add ${gateway_ip} dev ${bridge_name})...'
logtype: .stdout
) or {}
osal.exec(
cmd: 'ip addr add ${gateway_ip} dev ${bridge_name}'
stdout: false
)!
self.logger.log(
cat: 'network'
log: 'Step 2: IP assigned successfully'
logtype: .stdout
) or {}
// Bring bridge up
self.logger.log(
cat: 'network'
log: 'Step 3: Bringing bridge up (running: ip link set ${bridge_name} up)...'
logtype: .stdout
) or {}
osal.exec(
cmd: 'ip link set ${bridge_name} up'
stdout: false
)!
self.logger.log(
cat: 'network'
log: 'Step 3: Bridge brought up successfully'
logtype: .stdout
) or {}
// Enable IP forwarding
self.logger.log(
cat: 'network'
log: 'Step 4: Enabling IP forwarding (running: sysctl -w net.ipv4.ip_forward=1)...'
logtype: .stdout
) or {}
forward_result := os.execute('sysctl -w net.ipv4.ip_forward=1 2>/dev/null')
if forward_result.exit_code != 0 {
self.logger.log(
cat: 'network'
log: 'Step 4: WARNING - Failed to enable IPv4 forwarding (exit_code=${forward_result.exit_code})'
logtype: .error
) or {}
} else {
self.logger.log(
cat: 'network'
log: 'Step 4: IP forwarding enabled successfully'
logtype: .stdout
) or {}
}
// Get primary network interface for NAT
self.logger.log(
cat: 'network'
log: 'Step 5: Detecting primary network interface...'
logtype: .stdout
) or {}
primary_iface := self.network_get_primary_interface() or {
self.logger.log(
cat: 'network'
log: 'Step 5: WARNING - Could not detect primary interface: ${err}, using fallback eth0'
logtype: .error
) or {}
'eth0' // fallback
}
self.logger.log(
cat: 'network'
log: 'Step 5: Primary interface detected: ${primary_iface}'
logtype: .stdout
) or {}
// Setup NAT for outbound traffic
self.logger.log(
cat: 'network'
log: 'Step 6: Setting up NAT rules for ${primary_iface} (running iptables command)...'
logtype: .stdout
) or {}
nat_result := os.execute('iptables -t nat -C POSTROUTING -s ${subnet} -o ${primary_iface} -j MASQUERADE 2>/dev/null || iptables -t nat -A POSTROUTING -s ${subnet} -o ${primary_iface} -j MASQUERADE')
if nat_result.exit_code != 0 {
self.logger.log(
cat: 'network'
log: 'Step 6: WARNING - Failed to setup NAT rules (exit_code=${nat_result.exit_code})'
logtype: .error
) or {}
} else {
self.logger.log(
cat: 'network'
log: 'Step 6: NAT rules configured successfully'
logtype: .stdout
) or {}
}
// Setup FORWARD rules to allow traffic from/to the bridge
self.logger.log(
cat: 'network'
log: 'Step 7: Setting up FORWARD rules for ${bridge_name}...'
logtype: .stdout
) or {}
// Allow forwarding from bridge to external interface
forward_out_result := os.execute('iptables -C FORWARD -i ${bridge_name} -o ${primary_iface} -j ACCEPT 2>/dev/null || iptables -A FORWARD -i ${bridge_name} -o ${primary_iface} -j ACCEPT')
if forward_out_result.exit_code != 0 {
self.logger.log(
cat: 'network'
log: 'Step 7: WARNING - Failed to setup FORWARD rule (bridge -> external) (exit_code=${forward_out_result.exit_code})'
logtype: .error
) or {}
}
// Allow forwarding from external interface to bridge (for established connections)
forward_in_result := os.execute('iptables -C FORWARD -i ${primary_iface} -o ${bridge_name} -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null || iptables -A FORWARD -i ${primary_iface} -o ${bridge_name} -m state --state RELATED,ESTABLISHED -j ACCEPT')
if forward_in_result.exit_code != 0 {
self.logger.log(
cat: 'network'
log: 'Step 7: WARNING - Failed to setup FORWARD rule (external -> bridge) (exit_code=${forward_in_result.exit_code})'
logtype: .error
) or {}
}
// Allow forwarding between containers on the same bridge
forward_bridge_result := os.execute('iptables -C FORWARD -i ${bridge_name} -o ${bridge_name} -j ACCEPT 2>/dev/null || iptables -A FORWARD -i ${bridge_name} -o ${bridge_name} -j ACCEPT')
if forward_bridge_result.exit_code != 0 {
self.logger.log(
cat: 'network'
log: 'Step 7: WARNING - Failed to setup FORWARD rule (bridge -> bridge) (exit_code=${forward_bridge_result.exit_code})'
logtype: .error
) or {}
}
self.logger.log(
cat: 'network'
log: 'Step 7: FORWARD rules configured successfully'
logtype: .stdout
) or {}
self.logger.log(
cat: 'network'
log: 'END network_setup_bridge() - Bridge ${bridge_name} created and configured successfully'
logtype: .stdout
) or {}
}
// Get the primary network interface for NAT
fn (mut self HeroPods) network_get_primary_interface() !string {
self.logger.log(
cat: 'network'
log: 'START network_get_primary_interface() - Detecting primary interface'
logtype: .stdout
) or {}
// Try to get the default route interface
cmd := "ip route | grep default | awk '{print \$5}' | head -n1"
self.logger.log(
cat: 'network'
log: 'Running command: ${cmd}'
logtype: .stdout
) or {}
result := osal.exec(
cmd: cmd
stdout: false
)!
self.logger.log(
cat: 'network'
log: 'Command completed, output: "${result.output.trim_space()}"'
logtype: .stdout
) or {}
iface := result.output.trim_space()
if iface == '' {
self.logger.log(
cat: 'network'
log: 'ERROR: Could not determine primary network interface (empty output)'
logtype: .error
) or {}
return error('Could not determine primary network interface')
}
self.logger.log(
cat: 'network'
log: 'END network_get_primary_interface() - Detected interface: ${iface}'
logtype: .stdout
) or {}
return iface
}
// Allocate an IP address for a container (thread-safe)
//
// IP REUSE STRATEGY:
// 1. First, try to reuse an IP from the freed_ip_pool (recycled IPs from deleted containers)
// 2. If pool is empty, allocate a new IP by incrementing next_ip_offset
// 3. This prevents IP exhaustion in a /24 subnet (254 usable IPs)
//
// Thread Safety:
// This function uses network_mutex to ensure atomic IP allocation.
// Multiple concurrent container starts will be serialized at the IP allocation step,
// preventing race conditions where two containers could receive the same IP.
fn (mut self HeroPods) network_allocate_ip(container_name string) !string {
self.logger.log(
cat: 'network'
log: 'START network_allocate_ip() for container: ${container_name}'
logtype: .stdout
) or {}
self.logger.log(
cat: 'network'
log: 'Acquiring network_mutex lock...'
logtype: .stdout
) or {}
self.network_mutex.@lock()
self.logger.log(
cat: 'network'
log: 'network_mutex lock acquired'
logtype: .stdout
) or {}
defer {
self.logger.log(
cat: 'network'
log: 'Releasing network_mutex lock...'
logtype: .stdout
) or {}
self.network_mutex.unlock()
self.logger.log(
cat: 'network'
log: 'network_mutex lock released'
logtype: .stdout
) or {}
}
// Check if already allocated
if container_name in self.network_config.allocated_ips {
existing_ip := self.network_config.allocated_ips[container_name]
self.logger.log(
cat: 'network'
log: 'Container ${container_name} already has IP: ${existing_ip}'
logtype: .stdout
) or {}
return existing_ip
}
// Extract base IP from subnet (e.g., "10.10.0.0/24" -> "10.10.0")
subnet_parts := self.network_config.subnet.split('/')
base_ip_parts := subnet_parts[0].split('.')
base_ip := '${base_ip_parts[0]}.${base_ip_parts[1]}.${base_ip_parts[2]}'
// Determine IP offset: reuse from pool first, then increment
mut ip_offset := 0
if self.network_config.freed_ip_pool.len > 0 {
// Reuse a freed IP from the pool (LIFO - pop from end)
ip_offset = self.network_config.freed_ip_pool.last()
self.network_config.freed_ip_pool.delete_last()
self.logger.log(
cat: 'network'
log: 'Reusing IP offset ${ip_offset} from freed pool (pool size: ${self.network_config.freed_ip_pool.len})'
logtype: .stdout
) or {}
} else {
// No freed IPs available, allocate a new one
// This increment is atomic within the mutex lock
ip_offset = self.network_config.next_ip_offset
self.network_config.next_ip_offset++
// Check if we're approaching the subnet limit (254 usable IPs in /24)
if ip_offset > 254 {
return error('IP address pool exhausted: subnet ${self.network_config.subnet} has no more available IPs. Consider using a larger subnet or multiple bridges.')
}
self.logger.log(
cat: 'network'
log: 'Allocated new IP offset ${ip_offset} (next: ${self.network_config.next_ip_offset})'
logtype: .stdout
) or {}
}
// Build the full IP address
ip := '${base_ip}.${ip_offset}'
self.network_config.allocated_ips[container_name] = ip
self.logger.log(
cat: 'network'
log: 'Allocated IP ${ip} to container ${container_name}'
logtype: .stdout
) or {}
return ip
}
// Setup network for a container (creates veth pair, assigns IP, configures routing)
fn (mut self HeroPods) network_setup_container(container_name string, container_pid int) ! {
// Allocate IP address (thread-safe)
container_ip := self.network_allocate_ip(container_name)!
bridge_name := self.network_config.bridge_name
subnet_mask := self.network_config.subnet.split('/')[1]
gateway_ip := self.network_config.gateway_ip
// Create veth pair with unique names using hash to avoid collisions
// Interface names are limited to 15 chars, so we use a hash suffix
short_hash := sha256.hexhash(container_name)[..6]
veth_container_short := 'veth-${short_hash}'
veth_bridge_short := 'vbr-${short_hash}'
// Delete veth pair if it already exists (cleanup from previous run)
osal.exec(cmd: 'ip link delete ${veth_container_short} 2>/dev/null', stdout: false) or {}
osal.exec(cmd: 'ip link delete ${veth_bridge_short} 2>/dev/null', stdout: false) or {}
// Create veth pair
osal.exec(
cmd: 'ip link add ${veth_container_short} type veth peer name ${veth_bridge_short}'
stdout: false
)!
// Attach bridge end to bridge
osal.exec(
cmd: 'ip link set ${veth_bridge_short} master ${bridge_name}'
stdout: false
)!
osal.exec(
cmd: 'ip link set ${veth_bridge_short} up'
stdout: false
)!
// Move container end into container's network namespace
osal.exec(
cmd: 'ip link set ${veth_container_short} netns ${container_pid}'
stdout: false
)!
// Configure network inside container
// Rename veth to eth0 inside container for consistency
osal.exec(
cmd: 'nsenter -t ${container_pid} -n ip link set ${veth_container_short} name eth0'
stdout: false
)!
// Assign IP address
osal.exec(
cmd: 'nsenter -t ${container_pid} -n ip addr add ${container_ip}/${subnet_mask} dev eth0'
stdout: false
)!
// Bring interface up
osal.exec(
cmd: 'nsenter -t ${container_pid} -n ip link set dev eth0 up'
stdout: false
)!
// Add default route using gateway IP
osal.exec(
cmd: 'nsenter -t ${container_pid} -n ip route add default via ${gateway_ip}'
stdout: false
)!
}
// Configure DNS inside container by writing resolv.conf
fn (self HeroPods) network_configure_dns(container_name string, rootfs_path string) ! {
resolv_conf_path := '${rootfs_path}/etc/resolv.conf'
// Ensure /etc directory exists
etc_dir := '${rootfs_path}/etc'
if !os.exists(etc_dir) {
os.mkdir_all(etc_dir)!
}
// Build DNS configuration from configured DNS servers
mut dns_lines := []string{}
for dns_server in self.network_config.dns_servers {
dns_lines << 'nameserver ${dns_server}'
}
dns_content := dns_lines.join('\n') + '\n'
os.write_file(resolv_conf_path, dns_content)!
}
// Cleanup network for a container (removes veth pair and deallocates IP)
//
// Thread Safety:
// IP deallocation is protected by network_mutex to prevent race conditions
// when multiple containers are being deleted concurrently.
fn (mut self HeroPods) network_cleanup_container(container_name string) ! {
// Remove veth interfaces (they should be auto-removed when container stops, but cleanup anyway)
// Use same hash logic as setup to ensure we delete the correct interface
short_hash := sha256.hexhash(container_name)[..6]
veth_bridge_short := 'vbr-${short_hash}'
osal.exec(
cmd: 'ip link delete ${veth_bridge_short} 2>/dev/null'
stdout: false
) or {}
// Deallocate IP address and return it to the freed pool for reuse (thread-safe)
self.network_mutex.@lock()
defer {
self.network_mutex.unlock()
}
if container_name in self.network_config.allocated_ips {
ip := self.network_config.allocated_ips[container_name]
// Extract the IP offset from the full IP address (e.g., "10.10.0.42" -> 42)
ip_parts := ip.split('.')
if ip_parts.len == 4 {
ip_offset := ip_parts[3].int()
// Add to freed pool for reuse (avoid duplicates)
if ip_offset !in self.network_config.freed_ip_pool {
self.network_config.freed_ip_pool << ip_offset
}
}
// Remove from allocated IPs
self.network_config.allocated_ips.delete(container_name)
}
}
// Cleanup all network resources (called on reset)
//
// Parameters:
// - full: if true, also removes the bridge (for complete teardown)
// if false, keeps the bridge for reuse (default)
//
// Thread Safety:
// Uses separate lock/unlock calls for read and write operations to minimize
// lock contention. The container cleanup loop runs without holding the lock.
fn (mut self HeroPods) network_cleanup_all(full bool) ! {
// Get list of containers to cleanup (thread-safe read)
self.network_mutex.@lock()
container_names := self.network_config.allocated_ips.keys()
self.network_mutex.unlock()
// Remove all veth interfaces (no lock needed - operates on local copy)
for container_name in container_names {
self.network_cleanup_container(container_name) or {
}
}
// Clear allocated IPs and freed pool (thread-safe write)
self.network_mutex.@lock()
self.network_config.allocated_ips.clear()
self.network_config.freed_ip_pool.clear()
self.network_config.next_ip_offset = 10
self.network_mutex.unlock()
// Optionally remove the bridge for full cleanup
if full {
bridge_name := self.network_config.bridge_name
osal.exec(
cmd: 'ip link delete ${bridge_name}'
stdout: false
) or {}
}
}

299
lib/virt/heropods/network_test.v Normal file
View File

@@ -0,0 +1,299 @@
module heropods
import incubaid.herolib.core
import incubaid.herolib.osal.core as osal
import os
// Network-specific tests for HeroPods
//
// These tests verify bridge setup, IP allocation, NAT rules, and network cleanup
// Helper function to check if we're on Linux
fn is_linux_platform() bool {
return core.is_linux() or { false }
}
// Helper function to skip test if not on Linux
fn skip_if_not_linux() {
if !is_linux_platform() {
eprintln('SKIP: Test requires Linux (crun, ip, iptables)')
exit(0)
}
}
// Setup minimal test rootfs for testing
fn setup_test_rootfs() ! {
rootfs_path := os.home_dir() + '/.containers/images/alpine/rootfs'
// Skip if already exists and has valid binaries
if os.is_dir(rootfs_path) && os.is_file('${rootfs_path}/bin/sh') {
// Check if sh is a real binary (> 1KB)
sh_info := os.stat('${rootfs_path}/bin/sh') or { return }
if sh_info.size > 1024 {
return
}
}
// Remove old rootfs if it exists
if os.is_dir(rootfs_path) {
os.rmdir_all(rootfs_path) or {}
}
// Create minimal rootfs structure
os.mkdir_all(rootfs_path)!
os.mkdir_all('${rootfs_path}/bin')!
os.mkdir_all('${rootfs_path}/etc')!
os.mkdir_all('${rootfs_path}/dev')!
os.mkdir_all('${rootfs_path}/proc')!
os.mkdir_all('${rootfs_path}/sys')!
os.mkdir_all('${rootfs_path}/tmp')!
os.mkdir_all('${rootfs_path}/usr/bin')!
os.mkdir_all('${rootfs_path}/usr/local/bin')!
os.mkdir_all('${rootfs_path}/lib/x86_64-linux-gnu')!
os.mkdir_all('${rootfs_path}/lib64')!
// Copy essential binaries from host
// Use dash (smaller than bash) and sleep
if os.exists('/bin/dash') {
os.execute('cp -L /bin/dash ${rootfs_path}/bin/sh')
os.chmod('${rootfs_path}/bin/sh', 0o755)!
} else if os.exists('/bin/sh') {
os.execute('cp -L /bin/sh ${rootfs_path}/bin/sh')
os.chmod('${rootfs_path}/bin/sh', 0o755)!
}
// Copy common utilities
for cmd in ['sleep', 'echo', 'cat', 'ls', 'pwd', 'true', 'false'] {
if os.exists('/bin/${cmd}') {
os.execute('cp -L /bin/${cmd} ${rootfs_path}/bin/${cmd}')
os.chmod('${rootfs_path}/bin/${cmd}', 0o755) or {}
} else if os.exists('/usr/bin/${cmd}') {
os.execute('cp -L /usr/bin/${cmd} ${rootfs_path}/bin/${cmd}')
os.chmod('${rootfs_path}/bin/${cmd}', 0o755) or {}
}
}
// Copy required libraries for dash/sh
// Copy from /lib/x86_64-linux-gnu to the same path in rootfs
if os.is_dir('/lib/x86_64-linux-gnu') {
os.execute('cp -a /lib/x86_64-linux-gnu/libc.so.6 ${rootfs_path}/lib/x86_64-linux-gnu/')
os.execute('cp -a /lib/x86_64-linux-gnu/libc-*.so ${rootfs_path}/lib/x86_64-linux-gnu/ 2>/dev/null || true')
// Copy dynamic linker (actual file, not symlink)
os.execute('cp -L /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 ${rootfs_path}/lib/x86_64-linux-gnu/')
}
// Create symlink in /lib64 pointing to the actual file
if os.is_dir('${rootfs_path}/lib64') {
os.execute('ln -sf ../lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 ${rootfs_path}/lib64/ld-linux-x86-64.so.2')
}
// Create /etc/resolv.conf
os.write_file('${rootfs_path}/etc/resolv.conf', 'nameserver 8.8.8.8\n')!
}
// Cleanup helper for tests
fn cleanup_test_heropods(name string) {
mut hp := get(name: name) or { return }
for container_name, mut container in hp.containers {
container.stop() or {}
container.delete() or {}
}
// Don't delete the bridge (false) - tests run in parallel and share the same bridge
// Only clean up containers and IPs
hp.network_cleanup_all(false) or {}
delete(name: name) or {}
}
// Test 1: Bridge network setup
fn test_network_bridge_setup() ! {
skip_if_not_linux()
test_name := 'test_bridge_${os.getpid()}'
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true // Skip default image setup in tests
)!
bridge_name := hp.network_config.bridge_name
// Verify bridge exists
job := osal.exec(cmd: 'ip link show ${bridge_name}')!
assert job.output.contains(bridge_name)
// Verify bridge is UP
assert job.output.contains('UP') || job.output.contains('state UP')
// Verify IP is assigned to bridge
job2 := osal.exec(cmd: 'ip addr show ${bridge_name}')!
assert job2.output.contains(hp.network_config.gateway_ip)
// Cleanup after test
cleanup_test_heropods(test_name)
println('✓ Bridge network setup test passed')
}
// Test 2: NAT rules verification
fn test_network_nat_rules() ! {
skip_if_not_linux()
test_name := 'test_nat_${os.getpid()}'
defer { cleanup_test_heropods(test_name) }
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true // Skip default image setup in tests
)!
// Verify NAT rules exist for the subnet
job := osal.exec(cmd: 'iptables -t nat -L POSTROUTING -n')!
assert job.output.contains(hp.network_config.subnet) || job.output.contains('MASQUERADE')
println('✓ NAT rules test passed')
}
// Test 3: IP allocation sequential
fn test_ip_allocation_sequential() ! {
skip_if_not_linux()
test_name := 'test_ip_seq_${os.getpid()}'
defer { cleanup_test_heropods(test_name) }
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true // Skip default image setup in tests
)!
// Allocate multiple IPs
mut allocated_ips := []string{}
for i in 0 .. 10 {
ip := hp.network_allocate_ip('container_${i}')!
allocated_ips << ip
}
// Verify all IPs are unique
for i, ip1 in allocated_ips {
for j, ip2 in allocated_ips {
if i != j {
assert ip1 != ip2, 'IPs should be unique: ${ip1} == ${ip2}'
}
}
}
// Verify all IPs are in correct subnet
for ip in allocated_ips {
assert ip.starts_with('10.10.0.')
}
println('✓ IP allocation sequential test passed')
}
// Test 4: IP pool management with container lifecycle
fn test_ip_pool_management() ! {
skip_if_not_linux()
setup_test_rootfs()!
test_name := 'test_ip_pool_${os.getpid()}'
defer { cleanup_test_heropods(test_name) }
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true // Skip default image setup in tests
)!
// Create and start 3 containers with custom Alpine image
mut container1 := hp.container_new(
name: 'pool_test1_${os.getpid()}'
image: .custom
custom_image_name: 'alpine_pool1'
docker_url: 'docker.io/library/alpine:3.20'
)!
mut container2 := hp.container_new(
name: 'pool_test2_${os.getpid()}'
image: .custom
custom_image_name: 'alpine_pool2'
docker_url: 'docker.io/library/alpine:3.20'
)!
mut container3 := hp.container_new(
name: 'pool_test3_${os.getpid()}'
image: .custom
custom_image_name: 'alpine_pool3'
docker_url: 'docker.io/library/alpine:3.20'
)!
// Start with keep_alive to prevent Alpine's /bin/sh from exiting immediately
container1.start(keep_alive: true)!
container2.start(keep_alive: true)!
container3.start(keep_alive: true)!
// Get allocated IPs
ip1 := hp.network_config.allocated_ips[container1.name]
ip2 := hp.network_config.allocated_ips[container2.name]
ip3 := hp.network_config.allocated_ips[container3.name]
// Delete middle container (frees IP2)
container2.stop()!
container2.delete()!
// Verify IP2 is freed
assert container2.name !in hp.network_config.allocated_ips
// Create new container - should reuse freed IP2
mut container4 := hp.container_new(
name: 'pool_test4_${os.getpid()}'
image: .custom
custom_image_name: 'alpine_pool4'
docker_url: 'docker.io/library/alpine:3.20'
)!
container4.start(keep_alive: true)!
ip4 := hp.network_config.allocated_ips[container4.name]
assert ip4 == ip2, 'Should reuse freed IP: ${ip2} vs ${ip4}'
// Cleanup
container1.stop()!
container1.delete()!
container3.stop()!
container3.delete()!
container4.stop()!
container4.delete()!
println('✓ IP pool management test passed')
}
// Test 5: Custom bridge configuration
fn test_custom_bridge_config() ! {
skip_if_not_linux()
test_name := 'test_custom_br_${os.getpid()}'
custom_bridge := 'custombr_${os.getpid()}'
defer {
cleanup_test_heropods(test_name)
// Cleanup custom bridge
osal.exec(cmd: 'ip link delete ${custom_bridge}') or {}
}
mut hp := new(
name: test_name
reset: false // Don't reset to avoid race conditions with parallel tests
use_podman: true // Skip default image setup in tests
bridge_name: custom_bridge
subnet: '172.20.0.0/24'
gateway_ip: '172.20.0.1'
)!
// Verify custom bridge exists
job := osal.exec(cmd: 'ip link show ${custom_bridge}')!
assert job.output.contains(custom_bridge)
// Verify custom IP
job2 := osal.exec(cmd: 'ip addr show ${custom_bridge}')!
assert job2.output.contains('172.20.0.1')
println('✓ Custom bridge configuration test passed')
}

205
lib/virt/heropods/readme.md
View File

@@ -0,0 +1,205 @@
# HeroPods
HeroPods is a lightweight container management system built on crun (OCI runtime), providing Docker-like functionality with bridge networking, automatic IP allocation, and image management via Podman.
## Requirements
**Platform:** Linux only
HeroPods requires Linux-specific tools and will not work on macOS or Windows:
- `crun` (OCI runtime)
- `ip` (iproute2 package)
- `iptables` (for NAT)
- `nsenter` (for network namespace management)
- `podman` (optional, for image management)
On macOS/Windows, please use Docker or Podman directly instead of HeroPods.
## Quick Start
### Basic Usage
```v
import incubaid.herolib.virt.heropods
// Initialize HeroPods
mut hp := heropods.new(
reset: false
use_podman: true
)!
// Create a container (definition only, not yet created in backend)
mut container := hp.container_new(
name: 'my_alpine'
image: .custom
custom_image_name: 'alpine_3_20'
docker_url: 'docker.io/library/alpine:3.20'
)!
// Start the container (creates and starts it)
// Use keep_alive for containers with short-lived entrypoints
container.start(keep_alive: true)!
// Execute commands
result := container.exec(cmd: 'ls -la /')!
println(result)
// Stop and delete
container.stop()!
container.delete()!
```
### Custom Network Configuration
Configure bridge name, subnet, gateway, and DNS servers:
```v
import incubaid.herolib.virt.heropods
// Initialize with custom network settings
mut hp := heropods.new(
reset: false
use_podman: true
bridge_name: 'mybr0'
subnet: '192.168.100.0/24'
gateway_ip: '192.168.100.1'
dns_servers: ['1.1.1.1', '1.0.0.1']
)!
// Containers will use the custom network configuration
mut container := hp.container_new(
name: 'custom_net_container'
image: .alpine_3_20
)!
container.start(keep_alive: true)!
```
### Using HeroScript
```heroscript
!!heropods.configure
name:'my_heropods'
reset:false
use_podman:true
!!heropods.container_new
name:'my_container'
image:'custom'
custom_image_name:'alpine_3_20'
docker_url:'docker.io/library/alpine:3.20'
!!heropods.container_start
name:'my_container'
keep_alive:true
!!heropods.container_exec
name:'my_container'
cmd:'echo "Hello from HeroPods!"'
stdout:true
!!heropods.container_stop
name:'my_container'
!!heropods.container_delete
name:'my_container'
```
### Mycelium IPv6 Overlay Network
HeroPods supports Mycelium for end-to-end encrypted IPv6 connectivity:
```heroscript
!!heropods.configure
name:'mycelium_demo'
reset:false
use_podman:true
!!heropods.enable_mycelium
heropods:'mycelium_demo'
version:'v0.5.6'
ipv6_range:'400::/7'
key_path:'~/hero/cfg/priv_key.bin'
peers:'tcp://185.69.166.8:9651,quic://[2a02:1802:5e:0:ec4:7aff:fe51:e36b]:9651'
!!heropods.container_new
name:'ipv6_container'
image:'alpine_3_20'
!!heropods.container_start
name:'ipv6_container'
keep_alive:true
// Container now has both IPv4 and IPv6 (Mycelium) connectivity
```
See [MYCELIUM.md](./MYCELIUM.md) for detailed Mycelium configuration.
### Keep-Alive Feature
The `keep_alive` parameter keeps containers running after their entrypoint exits successfully. This is useful for:
- **Short-lived entrypoints**: Containers whose entrypoint performs initialization then exits (e.g., Alpine's `/bin/sh`)
- **Interactive containers**: Containers you want to exec into after startup
- **Service containers**: Containers that need to stay alive for background tasks
**How it works**:
1. Container starts with its original ENTRYPOINT and CMD (OCI-compliant)
2. HeroPods waits for the entrypoint to complete
3. If entrypoint exits with code 0 (success), a keep-alive process is injected
4. If entrypoint fails (non-zero exit), container stops and error is returned
**Example**:
```v
// Alpine's default CMD is /bin/sh which exits immediately
mut container := hp.container_new(
name: 'my_alpine'
image: .custom
custom_image_name: 'alpine_3_20'
docker_url: 'docker.io/library/alpine:3.20'
)!
// Without keep_alive: container would exit immediately
// With keep_alive: container stays running for exec commands
container.start(keep_alive: true)!
// Now you can exec into the container
result := container.exec(cmd: 'echo "Hello!"')!
```
**Note**: If you see a warning about "bare shell CMD", use `keep_alive: true` when starting the container.
## Features
- **Container Lifecycle**: create, start, stop, delete, exec
- **Keep-Alive Support**: Keep containers running after entrypoint exits
- **IPv4 Bridge Networking**: Automatic IP allocation with NAT
- **IPv6 Mycelium Overlay**: End-to-end encrypted peer-to-peer networking
- **Image Management**: Pull Docker images via Podman or use built-in images
- **Resource Monitoring**: CPU and memory usage tracking
- **Thread-Safe**: Concurrent container operations supported
- **Configurable**: Custom network settings, DNS, resource limits
## Examples
See `examples/virt/heropods/` for complete working examples:
### HeroScript Examples
- **simple_container.heroscript** - Basic container lifecycle management
- **ipv4_connection.heroscript** - IPv4 networking and internet connectivity
- **container_mycelium.heroscript** - Mycelium IPv6 overlay networking
### V Language Examples
- **heropods.vsh** - Complete API demonstration
- **runcommands.vsh** - Simple command execution
Each example is fully documented and can be run independently. See [examples/virt/heropods/README.md](../../../examples/virt/heropods/README.md) for details.
## Documentation
- **[MYCELIUM.md](./MYCELIUM.md)** - Mycelium IPv6 overlay network integration guide
- **[PRODUCTION_READINESS_REVIEW.md](./PRODUCTION_READINESS_REVIEW.md)** - Production readiness assessment
- **[ACTIONABLE_RECOMMENDATIONS.md](./ACTIONABLE_RECOMMENDATIONS.md)** - Code quality recommendations

35
lib/virt/heropods/utils.v Normal file
View File

@@ -0,0 +1,35 @@
module heropods
// Validate container name to prevent shell injection and path traversal
//
// Security validation that ensures container names:
// - Are not empty and not too long (max 64 chars)
// - Contain only alphanumeric characters, dashes, and underscores
// - Don't start with dash or underscore
// - Don't contain path traversal sequences
//
// This is critical for preventing command injection attacks since container
// names are used in shell commands throughout the module.
fn validate_container_name(name string) ! {
if name == '' {
return error('Container name cannot be empty')
}
if name.len > 64 {
return error('Container name too long (max 64 characters)')
}
// Check if name contains only allowed characters: alphanumeric, dash, underscore
allowed_chars := 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_'
if !name.contains_only(allowed_chars) {
return error('Container name "${name}" contains invalid characters. Only alphanumeric characters, dashes, and underscores are allowed.')
}
if name.starts_with('-') || name.starts_with('_') {
return error('Container name cannot start with dash or underscore')
}
// Prevent path traversal (redundant check but explicit for security)
if name.contains('..') || name.contains('/') || name.contains('\\') {
return error('Container name cannot contain path separators or ".."')
}
}

5
lib/virt/herorun2/executor.v
View File

@@ -3,6 +3,7 @@ module herorun2
import incubaid.herolib.osal.tmux
import incubaid.herolib.osal.sshagent
import incubaid.herolib.osal.core as osal
import incubaid.herolib.core.texttools
import time
import os
@@ -49,9 +50,6 @@ pub fn new_executor(args ExecutorArgs) !Executor {
// Initialize tmux properly
mut t := tmux.new(sessionid: args.container_id)!
// Initialize Hetzner manager properly
mut hetzner := hetznermanager.get() or { hetznermanager.new()! }
return Executor{
node: node
container_id: args.container_id
@@ -61,7 +59,6 @@ pub fn new_executor(args ExecutorArgs) !Executor {
session_name: args.container_id
window_name: 'main'
agent: agent
hetzner: hetzner
}
}

5
test_basic.vsh
View File

@@ -170,8 +170,8 @@ lib/clients
lib/core
lib/develop
lib/hero/heromodels
// lib/vfs The vfs folder is not exists on the development branch, so we need to uncomment it after merging this PR https://github.com/incubaid/herolib/pull/68
// lib/crypt
lib/virt/heropods
lib/virt/crun
'
// the following tests have no prio and can be ignored
@@ -201,6 +201,7 @@ virt/kubernetes/
if in_github_actions() {
println('**** WE ARE IN GITHUB ACTION')
tests_ignore += '\nosal/tmux\n'
tests_ignore += '\nvirt/heropods\n' // Requires root for network bridge operations (ip link add)
}
tests_error := '