...
This commit is contained in:
@@ -1,104 +1,232 @@
|
||||
Here's the **adjusted README.md** to align with the current implementation, including:
|
||||
|
||||
- Clear structure by **root objects**
|
||||
- An **index field** added to each root object table
|
||||
- Only **root objects** (`Node`, `NodeGroup`) have index fields
|
||||
- All fields and structs are documented
|
||||
- Instructions on how to use the model
|
||||
|
||||
---
|
||||
|
||||
# Grid4 Data Model
|
||||
|
||||
This module defines data models for nodes, groups, slices in a cloud/grid infrastructure.
|
||||
This module defines data models for nodes, groups, and slices in a cloud/grid infrastructure. Each root object is marked with `@[heap]` and can be indexed for efficient querying.
|
||||
|
||||
## Root Objects Overview
|
||||
|
||||
| Object | Description | Index Fields |
|
||||
| ----------- | --------------------------------------------- | ------------------------------ |
|
||||
| `Node` | Represents a single node in the grid | `id`, `nodegroupid`, `country` |
|
||||
| `NodeGroup` | Represents a group of nodes owned by a farmer | `id`, `farmerid` |
|
||||
|
||||
---
|
||||
|
||||
## Node
|
||||
|
||||
Represents a single node in the grid with slices, devices, and capacity.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------- | -------------- | -------------------------------------------- |
|
||||
| id | int | Unique node ID |
|
||||
| nodegroupid | int | ID of the owning node group |
|
||||
| uptime | int | Uptime percentage (0-100) |
|
||||
| computeslices | []ComputeSlice | List of compute slices |
|
||||
| storageslices | []StorageSlice | List of storage slices |
|
||||
| devices | DeviceInfo | Hardware device info (storage, memory, etc.) |
|
||||
| country | string | 2-letter country code |
|
||||
| capacity | NodeCapacity | Aggregated hardware capacity |
|
||||
| provisiontime | u32 | Provisioning time (simple/compatible format) |
|
||||
| Field | Type | Description | Indexed |
|
||||
| --------------- | ---------------- | -------------------------------------------- | ------- |
|
||||
| `id` | `int` | Unique node ID | ✅ |
|
||||
| `nodegroupid` | `int` | ID of the owning node group | ✅ |
|
||||
| `uptime` | `int` | Uptime percentage (0-100) | ❌ |
|
||||
| `computeslices` | `[]ComputeSlice` | List of compute slices | ❌ |
|
||||
| `storageslices` | `[]StorageSlice` | List of storage slices | ❌ |
|
||||
| `devices` | `DeviceInfo` | Hardware device info (storage, memory, etc.) | ❌ |
|
||||
| `country` | `string` | 2-letter country code | ✅ |
|
||||
| `capacity` | `NodeCapacity` | Aggregated hardware capacity | ❌ |
|
||||
| `provisiontime` | `u32` | Provisioning time (simple/compatible format) | ❌ |
|
||||
|
||||
---
|
||||
|
||||
## NodeGroup
|
||||
|
||||
Represents a group of nodes owned by a farmer, with policies.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ----------------------------------- | ------------- | ---------------------------------------------- |
|
||||
| id | u32 | Unique group ID |
|
||||
| farmerid | u32 | Farmer/user ID |
|
||||
| secret | string | Encrypted secret for booting nodes |
|
||||
| description | string | Group description |
|
||||
| slapolicy | SLAPolicy | SLA policy details |
|
||||
| pricingpolicy | PricingPolicy | Pricing policy details |
|
||||
| compute_slice_normalized_pricing_cc | f64 | Pricing per 2GB compute slice in cloud credits |
|
||||
| storage_slice_normalized_pricing_cc | f64 | Pricing per 1GB storage slice in cloud credits |
|
||||
| reputation | int | Reputation (0-100) |
|
||||
| uptime | int | Uptime (0-100) |
|
||||
| Field | Type | Description | Indexed |
|
||||
| ------------------------------------- | --------------- | ---------------------------------------------- | ------- |
|
||||
| `id` | `u32` | Unique group ID | ✅ |
|
||||
| `farmerid` | `u32` | Farmer/user ID | ✅ |
|
||||
| `secret` | `string` | Encrypted secret for booting nodes | ❌ |
|
||||
| `description` | `string` | Group description | ❌ |
|
||||
| `slapolicy` | `SLAPolicy` | SLA policy details | ❌ |
|
||||
| `pricingpolicy` | `PricingPolicy` | Pricing policy details | ❌ |
|
||||
| `compute_slice_normalized_pricing_cc` | `f64` | Pricing per 2GB compute slice in cloud credits | ❌ |
|
||||
| `storage_slice_normalized_pricing_cc` | `f64` | Pricing per 1GB storage slice in cloud credits | ❌ |
|
||||
| `reputation` | `int` | Reputation (0-100) | ❌ |
|
||||
| `uptime` | `int` | Uptime (0-100) | ❌ |
|
||||
|
||||
## NodeSim
|
||||
|
||||
Extends Node for simulation purposes with cost.
|
||||
|
||||
| Field | Type | Description |
|
||||
| --------------- | ---- | --------------------------------------- |
|
||||
| ( Embeds Node ) | - | All Node fields |
|
||||
| cost | f64 | Simulation cost (free in some contexts) |
|
||||
---
|
||||
|
||||
## ComputeSlice
|
||||
|
||||
Represents a compute slice (e.g., 1GB memory unit).
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------------------ | ------------- | -------------------------------- |
|
||||
| nodeid | u32 | Owning node ID |
|
||||
| id | int | Slice ID in node |
|
||||
| mem_gb | f64 | Memory in GB |
|
||||
| storage_gb | f64 | Storage in GB |
|
||||
| passmark | int | Passmark score |
|
||||
| vcores | int | Virtual cores |
|
||||
| cpu_oversubscription | int | CPU oversubscription ratio |
|
||||
| storage_oversubscription | int | Storage oversubscription ratio |
|
||||
| price_range | []f64 | Price range [min, max] |
|
||||
| gpus | u8 | Number of GPUs |
|
||||
| price_cc | f64 | Price per slice in cloud credits |
|
||||
| pricing_policy | PricingPolicy | Pricing policy |
|
||||
| sla_policy | SLAPolicy | SLA policy |
|
||||
| -------------------------- | --------------- | -------------------------------- |
|
||||
| `nodeid` | `u32` | Owning node ID |
|
||||
| `id` | `int` | Slice ID in node |
|
||||
| `mem_gb` | `f64` | Memory in GB |
|
||||
| `storage_gb` | `f64` | Storage in GB |
|
||||
| `passmark` | `int` | Passmark score |
|
||||
| `vcores` | `int` | Virtual cores |
|
||||
| `cpu_oversubscription` | `int` | CPU oversubscription ratio |
|
||||
| `storage_oversubscription` | `int` | Storage oversubscription ratio |
|
||||
| `price_range` | `[]f64` | Price range [min, max] |
|
||||
| `gpus` | `u8` | Number of GPUs |
|
||||
| `price_cc` | `f64` | Price per slice in cloud credits |
|
||||
| `pricing_policy` | `PricingPolicy` | Pricing policy |
|
||||
| `sla_policy` | `SLAPolicy` | SLA policy |
|
||||
|
||||
---
|
||||
|
||||
## StorageSlice
|
||||
|
||||
Represents a 1GB storage slice.
|
||||
|
||||
| Field | Type | Description |
|
||||
| -------------- | ------------- | -------------------------------- |
|
||||
| nodeid | u32 | Owning node ID |
|
||||
| id | int | Slice ID in node |
|
||||
| price_cc | f64 | Price per slice in cloud credits |
|
||||
| pricing_policy | PricingPolicy | Pricing policy |
|
||||
| sla_policy | SLAPolicy | SLA policy |
|
||||
| ---------------- | --------------- | -------------------------------- |
|
||||
| `nodeid` | `u32` | Owning node ID |
|
||||
| `id` | `int` | Slice ID in node |
|
||||
| `price_cc` | `f64` | Price per slice in cloud credits |
|
||||
| `pricing_policy` | `PricingPolicy` | Pricing policy |
|
||||
| `sla_policy` | `SLAPolicy` | SLA policy |
|
||||
|
||||
---
|
||||
|
||||
## DeviceInfo
|
||||
|
||||
Hardware device information for a node.
|
||||
|
||||
| Field | Type | Description |
|
||||
| --------- | ----------------- | ----------------------- |
|
||||
| `vendor` | `string` | Vendor of the node |
|
||||
| `storage` | `[]StorageDevice` | List of storage devices |
|
||||
| `memory` | `[]MemoryDevice` | List of memory devices |
|
||||
| `cpu` | `[]CPUDevice` | List of CPU devices |
|
||||
| `gpu` | `[]GPUDevice` | List of GPU devices |
|
||||
| `network` | `[]NetworkDevice` | List of network devices |
|
||||
|
||||
---
|
||||
|
||||
## StorageDevice
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------- | -------- | --------------------- |
|
||||
| `id` | `string` | Unique ID for device |
|
||||
| `size_gb` | `f64` | Size in GB |
|
||||
| `description` | `string` | Description of device |
|
||||
|
||||
---
|
||||
|
||||
## MemoryDevice
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------- | -------- | --------------------- |
|
||||
| `id` | `string` | Unique ID for device |
|
||||
| `size_gb` | `f64` | Size in GB |
|
||||
| `description` | `string` | Description of device |
|
||||
|
||||
---
|
||||
|
||||
## CPUDevice
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------- | -------- | ------------------------ |
|
||||
| `id` | `string` | Unique ID for device |
|
||||
| `cores` | `int` | Number of CPU cores |
|
||||
| `passmark` | `int` | Passmark benchmark score |
|
||||
| `description` | `string` | Description of device |
|
||||
| `cpu_brand` | `string` | Brand of the CPU |
|
||||
| `cpu_version` | `string` | Version of the CPU |
|
||||
|
||||
---
|
||||
|
||||
## GPUDevice
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------- | -------- | --------------------- |
|
||||
| `id` | `string` | Unique ID for device |
|
||||
| `cores` | `int` | Number of GPU cores |
|
||||
| `memory_gb` | `f64` | GPU memory in GB |
|
||||
| `description` | `string` | Description of device |
|
||||
| `gpu_brand` | `string` | Brand of the GPU |
|
||||
| `gpu_version` | `string` | Version of the GPU |
|
||||
|
||||
---
|
||||
|
||||
## NetworkDevice
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------- | -------- | --------------------- |
|
||||
| `id` | `string` | Unique ID for device |
|
||||
| `speed_mbps` | `int` | Network speed in Mbps |
|
||||
| `description` | `string` | Description of device |
|
||||
|
||||
---
|
||||
|
||||
## NodeCapacity
|
||||
|
||||
Aggregated hardware capacity for a node.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------- | ---- | ---------------------- |
|
||||
| storage_gb | f64 | Total storage in GB |
|
||||
| mem_gb | f64 | Total memory in GB |
|
||||
| mem_gb_gpu | f64 | Total GPU memory in GB |
|
||||
| passmark | int | Total passmark score |
|
||||
| vcores | int | Total virtual cores |
|
||||
| ------------ | ----- | ---------------------- |
|
||||
| `storage_gb` | `f64` | Total storage in GB |
|
||||
| `mem_gb` | `f64` | Total memory in GB |
|
||||
| `mem_gb_gpu` | `f64` | Total GPU memory in GB |
|
||||
| `passmark` | `int` | Total passmark score |
|
||||
| `vcores` | `int` | Total virtual cores |
|
||||
|
||||
## NodeInfo
|
||||
---
|
||||
|
||||
Descriptive info for a node (used in aggregations).
|
||||
## SLAPolicy
|
||||
|
||||
Service Level Agreement policy for slices or node groups.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ----------- | ------ | ------------------ |
|
||||
| cpu_brand | string | CPU brand |
|
||||
| cpu_version | string | CPU version |
|
||||
| mem | string | Memory description |
|
||||
| hdd | string | HDD description |
|
||||
| ssd | string | SSD description |
|
||||
| url | string | Info URL |
|
||||
| continent | string | Continent code |
|
||||
| country | string | Country code |
|
||||
| -------------------- | ----- | --------------------------------------- |
|
||||
| `sla_uptime` | `int` | Required uptime % (e.g., 90) |
|
||||
| `sla_bandwidth_mbit` | `int` | Guaranteed bandwidth in Mbps (0 = none) |
|
||||
| `sla_penalty` | `int` | Penalty % if SLA is breached (0-100) |
|
||||
|
||||
---
|
||||
|
||||
## PricingPolicy
|
||||
|
||||
Pricing policy for slices or node groups.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------------------------- | ------- | --------------------------------------------------------- |
|
||||
| `marketplace_year_discounts` | `[]int` | Discounts for 1Y, 2Y, 3Y prepaid usage (e.g. [30,40,50]) |
|
||||
| `volume_discounts` | `[]int` | Volume discounts based on purchase size (e.g. [10,20,30]) |
|
||||
|
||||
---
|
||||
|
||||
## Usage Instructions
|
||||
|
||||
### Loading Nodes from JSON Files
|
||||
|
||||
Use the `load` function to read all `.json` files from a directory and decode them into `Node` structs:
|
||||
|
||||
```v
|
||||
import freeflowuniverse.herolib.threefold.grid4.datamodel
|
||||
|
||||
nodes := datamodel.load('/path/to/nodes')!
|
||||
```
|
||||
|
||||
### Defining Nodes and Slices
|
||||
|
||||
Nodes and slices are defined using structured data. Each node contains compute and storage slices, and detailed device information.
|
||||
|
||||
### Indexing
|
||||
|
||||
Only root objects (`Node`, `NodeGroup`) support indexing. The following fields are indexed for fast lookup:
|
||||
|
||||
- `Node`: `id`, `nodegroupid`, `country`
|
||||
- `NodeGroup`: `id`, `farmerid`
|
||||
|
||||
Use these fields in queries for efficient filtering and retrieval.
|
||||
|
||||
---
|
||||
|
||||
Let me know if you'd like this exported to a file or formatted differently.
|
||||
@@ -1,64 +0,0 @@
|
||||
module datamodel
|
||||
|
||||
import time
|
||||
|
||||
// NodeTotalSim represents the aggregated data for a node simulation, including hardware specs, pricing, and location.
|
||||
pub struct NodeTotalSim {
|
||||
pub mut:
|
||||
id int // Unique node ID
|
||||
cost f64 // Total cost of the node
|
||||
deliverytime time.Time // Expected delivery time
|
||||
inca_reward int // Incentive reward
|
||||
reputation int // Reputation score (0-100)
|
||||
uptime int // Uptime percentage (0-100)
|
||||
price_simulation f64 // Simulated price for the node
|
||||
capacity NodeCapacity // Aggregated hardware capacity
|
||||
}
|
||||
|
||||
// total aggregates totals for a single NodeSim (e.g., storage, memory, price) from its slices and devices.
|
||||
pub fn (n NodeSim) total() !NodeTotalSim {
|
||||
if n.computeslices.len == 0 && n.storageslices.len == 0 {
|
||||
return error('Node has no slices to aggregate')
|
||||
}
|
||||
|
||||
mut total := NodeTotalSim{
|
||||
id: n.id
|
||||
cost: n.cost
|
||||
deliverytime: time.now() // Placeholder; replace with actual logic if needed
|
||||
inca_reward: 0 // Placeholder; compute from policy if available
|
||||
reputation: 50 // Default; compute from uptime/history
|
||||
uptime: n.uptime
|
||||
price_simulation: 0.0
|
||||
capacity: NodeCapacity{}
|
||||
}
|
||||
|
||||
// Aggregate from compute slices
|
||||
for slice in n.computeslices {
|
||||
total.capacity.storage_gb += slice.storage_gb
|
||||
total.capacity.mem_gb += slice.mem_gb
|
||||
total.capacity.mem_gb_gpu += 0 // Add GPU logic if GPUs are in slices
|
||||
total.capacity.passmark += slice.passmark
|
||||
total.capacity.vcores += slice.vcores
|
||||
total.price_simulation += slice.price_cc
|
||||
}
|
||||
|
||||
// Aggregate from storage slices (focus on storage/price)
|
||||
for slice in n.storageslices {
|
||||
total.capacity.storage_gb += 1.0 // Assuming 1GB per storage slice as per model_slices.v
|
||||
total.price_simulation += slice.price_cc
|
||||
}
|
||||
|
||||
// Aggregate passmark/vcores from devices (e.g., CPUs)
|
||||
for cpu in n.devices.cpu {
|
||||
total.capacity.passmark += cpu.passmark
|
||||
total.capacity.vcores += cpu.cores
|
||||
}
|
||||
|
||||
// Additional aggregations (e.g., from GPUs if present)
|
||||
for gpu in n.devices.gpu {
|
||||
total.capacity.mem_gb_gpu += gpu.memory_gb
|
||||
total.capacity.vcores += gpu.cores // If GPUs contribute to vcores
|
||||
}
|
||||
|
||||
return total
|
||||
}
|
||||
@@ -75,13 +75,35 @@ pub mut:
|
||||
vcores int // Total virtual cores
|
||||
}
|
||||
|
||||
pub struct NodeGrant {
|
||||
// typically 1GB of memory, but can be adjusted based based on size of machine
|
||||
pub struct ComputeSlice {
|
||||
pub mut:
|
||||
grant_month_usd string
|
||||
grant_month_inca string
|
||||
grant_max_nrnodes int
|
||||
nodeid u32 // the node in the grid, there is an object describing the node
|
||||
id int // the id of the slice in the node
|
||||
mem_gb f64
|
||||
storage_gb f64
|
||||
passmark int
|
||||
vcores int
|
||||
cpu_oversubscription int
|
||||
storage_oversubscription int
|
||||
price_range []f64 = [0.0, 0.0]
|
||||
gpus u8 // nr of GPU's see node to know what GPU's are
|
||||
price_cc f64 // price per slice (even if the grouped one)
|
||||
pricing_policy PricingPolicy
|
||||
sla_policy SLAPolicy
|
||||
}
|
||||
|
||||
fn (mut n Node) check() ! {
|
||||
// todo
|
||||
// 1GB of storage
|
||||
pub struct StorageSlice {
|
||||
pub mut:
|
||||
nodeid u32 // the node in the grid
|
||||
id int // the id of the slice in the node, are tracked in the node itself
|
||||
price_cc f64 // price per slice (even if the grouped one)
|
||||
pricing_policy PricingPolicy
|
||||
sla_policy SLAPolicy
|
||||
}
|
||||
|
||||
|
||||
fn (mut n Node) check() ! {
|
||||
// todo calculate NodeCapacity out of the devices on the Node
|
||||
}
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
module datamodel
|
||||
|
||||
// typically 1GB of memory, but can be adjusted based based on size of machine
|
||||
pub struct ComputeSlice {
|
||||
pub mut:
|
||||
nodeid u32 // the node in the grid, there is an object describing the node
|
||||
id int // the id of the slice in the node
|
||||
mem_gb f64
|
||||
storage_gb f64
|
||||
passmark int
|
||||
vcores int
|
||||
cpu_oversubscription int
|
||||
storage_oversubscription int
|
||||
price_range []f64 = [0.0, 0.0]
|
||||
gpus u8 // nr of GPU's see node to know what GPU's are
|
||||
price_cc f64 // price per slice (even if the grouped one)
|
||||
pricing_policy PricingPolicy
|
||||
sla_policy SLAPolicy
|
||||
}
|
||||
|
||||
// 1GB of storage
|
||||
pub struct StorageSlice {
|
||||
pub mut:
|
||||
nodeid u32 // the node in the grid
|
||||
id int // the id of the slice in the node, are tracked in the node itself
|
||||
price_cc f64 // price per slice (even if the grouped one)
|
||||
pricing_policy PricingPolicy
|
||||
sla_policy SLAPolicy
|
||||
}
|
||||
Reference in New Issue
Block a user