docs: add developer docs for gpu and cloud
This commit is contained in:
mik-tf
181
docs/mycelium_cloud_for_devs.md
Normal file
181
docs/mycelium_cloud_for_devs.md
Normal file
@@ -0,0 +1,181 @@
|
|||||||
|
# Mycelium Cloud for Developers
|
||||||
|
|
||||||
|
*Developer Guide to Decentralized Cloud Computing*
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
Mycelium Cloud provides a comprehensive platform for deploying and managing Kubernetes clusters on the decentralized ThreeFold Grid infrastructure. This guide covers the core features and capabilities available to developers.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Decentralized Infrastructure
|
||||||
|
Mycelium Cloud deploys on ThreeFold Grid's distributed network, providing:
|
||||||
|
- **Direct Node Access**: Each node gets a unique Mycelium IP address
|
||||||
|
- **Cross-Node Communication**: Services communicate across nodes using Mycelium networking
|
||||||
|
- **Secure Communication**: All traffic is encrypted through the Mycelium network
|
||||||
|
- **No Public IPs Required**: Services accessible via Mycelium IPs
|
||||||
|
|
||||||
|
### Network Flow
|
||||||
|
User Machine → Mycelium Network → Cluster Node → Service
|
||||||
|
|
||||||
|
### Kubernetes Management
|
||||||
|
- **K3s Support**: Full K3s cluster deployment and management
|
||||||
|
- **IPv6 Networking**: Mycelium peer-to-peer networking
|
||||||
|
- **High Availability**: Multi-master cluster support
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Features
|
||||||
|
|
||||||
|
### Mycelium Networking
|
||||||
|
Ultra-fast, decentralized networking inspired by nature. Mycelium Networking forms a resilient, adaptive mesh that routes around failures and optimizes for speed and security.
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
- End-to-end encrypted
|
||||||
|
- Nature-inspired
|
||||||
|
- Direct node communication
|
||||||
|
- Secure peer-to-peer connections
|
||||||
|
|
||||||
|
### Zero-Image Technology
|
||||||
|
Metadata-driven zero-images cut artifacts up to 100x, slashing bandwidth and deployment overhead.
|
||||||
|
|
||||||
|
**Benefits:**
|
||||||
|
- **Deterministic Deployments**: Cryptographic verification ensures every workload deploys exactly as intended—no tampering, no drift
|
||||||
|
- **Multi-Workload Compatibility with Secure Boot**: Run containers, VMs, and Linux workloads anywhere with stateless secure boot and continuous verification
|
||||||
|
- **Smart Contract-Based Deployment**: Cryptographically signed contracts orchestrate every workload with transparent, tamper-proof execution
|
||||||
|
|
||||||
|
### Quantum-Safe Storage (QSS)
|
||||||
|
Quantum-resistant encryption secures data beyond the app layer so ownership and control stay yours.
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
- **Self-Healing Storage System**: Autonomous recovery heals failures or corruption instantly, preserving integrity without human intervention
|
||||||
|
- **Multi-Protocol Data Access**: Serve the same data via IPFS, S3, WebDAV, HTTP, and native file systems for seamless integration
|
||||||
|
- **Geo-Aware Data Placement & Replication**: Define residency, redundancy, and distribution per workload while zone-to-zone replication hardens resilience
|
||||||
|
|
||||||
|
### Multi-Master Clusters
|
||||||
|
High-availability Kubernetes clusters with multiple control plane nodes. Automatic failover, leader election, and zero-downtime upgrades built-in.
|
||||||
|
|
||||||
|
**Capabilities:**
|
||||||
|
- **HA Control Plane**
|
||||||
|
- **Automatic Failover**
|
||||||
|
- **Zero-downtime Upgrades**
|
||||||
|
|
||||||
|
### Effortless Load Balancing & Scaling
|
||||||
|
Mycelium Cloud automatically balances traffic and scales your services up or down based on demand. Enjoy high availability and optimal performance with zero manual intervention.
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
- **Auto-scaling**
|
||||||
|
- **Built-in load balancing**
|
||||||
|
- **High availability**
|
||||||
|
|
||||||
|
### Simple Web Gateway Access
|
||||||
|
Expose any service to the public web with a simple Kubernetes resource. No complex Ingress controllers. Domain and prefix-based routing is built-in.
|
||||||
|
|
||||||
|
**Benefits:**
|
||||||
|
- **Simple configuration**
|
||||||
|
- **Built-in routing**
|
||||||
|
- **No ingress controllers**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
### 1. Account Setup
|
||||||
|
- **Sign Up**: Create your account from signup page
|
||||||
|
- **Verify Email**: Check your email and verify your account
|
||||||
|
- **Add Funds**: Navigate to your dashboard and add credits to your account
|
||||||
|
- **Add SSH Key**: Navigate to Add SSH card and upload your public SSH key
|
||||||
|
|
||||||
|
### 2. Deploy Your First Cluster
|
||||||
|
- **Access Deploy**: Click "Deploy Cluster" from your dashboard
|
||||||
|
- **Configure VMs**: Define your virtual machines:
|
||||||
|
- Choose CPU, memory, and storage requirements
|
||||||
|
- Select the number of master and worker nodes
|
||||||
|
- **Select Nodes**: Choose ThreeFold Grid nodes for deployment
|
||||||
|
- **Review & Deploy**: Confirm your configuration and deploy
|
||||||
|
|
||||||
|
### 3. Access Your Cluster
|
||||||
|
|
||||||
|
#### Download Kubeconfig
|
||||||
|
- Go to dashboard → Clusters → Click download icon (⬇️)
|
||||||
|
- Set kubeconfig: `export KUBECONFIG=/path/to/config`
|
||||||
|
- Test: `kubectl get nodes`
|
||||||
|
|
||||||
|
#### SSH Access
|
||||||
|
- **Find Mycelium IPs**: Check cluster details page for node IPs
|
||||||
|
- **Download Mycelium Binary**:
|
||||||
|
```bash
|
||||||
|
wget https://github.com/threefoldtech/mycelium/releases/latest/download/mycelium-private-x86_64-unknown-linux-musl.tar.gz
|
||||||
|
tar -xzf mycelium-private-x86_64-unknown-linux-musl.tar.gz
|
||||||
|
sudo chmod +x mycelium-private
|
||||||
|
sudo mv mycelium-private /usr/local/bin/mycelium
|
||||||
|
```
|
||||||
|
- **Start Mycelium**:
|
||||||
|
```bash
|
||||||
|
sudo mycelium --peers tcp://188.40.132.242:9651 tcp://136.243.47.186:9651 tcp://185.69.166.7:9651 tcp://185.69.166.8:9651 tcp://65.21.231.58:9651 tcp://65.109.18.113:9651 tcp://209.159.146.190:9651 tcp://5.78.122.16:9651 tcp://5.223.43.251:9651 tcp://142.93.217.194:9651
|
||||||
|
```
|
||||||
|
- **SSH to nodes**: `ssh root@<mycelium-ip>`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Use Cases
|
||||||
|
|
||||||
|
### AI/ML Training
|
||||||
|
Run GPU-accelerated workloads for deep learning and data science on demand.
|
||||||
|
- **GPU Acceleration**
|
||||||
|
- **Scalable Compute**
|
||||||
|
- **Cost Optimization**
|
||||||
|
|
||||||
|
### Enterprise Kubernetes
|
||||||
|
Deploy production-grade clusters with full control and no vendor lock-in.
|
||||||
|
- **High Availability**
|
||||||
|
- **Security**
|
||||||
|
- **Compliance**
|
||||||
|
|
||||||
|
### Edge & IoT
|
||||||
|
Leverage global nodes for low-latency edge computing and IoT applications.
|
||||||
|
- **Low Latency**
|
||||||
|
- **Global Distribution**
|
||||||
|
- **Real-time Processing**
|
||||||
|
|
||||||
|
### DigitalMe Example
|
||||||
|
Example of what can be done with Mycelium Cloud. With a very simple execution, the following can be deployed on k8s cluster and served on Mycelium IPs:
|
||||||
|
|
||||||
|
- **Cryptpad**: Encrypted document collaboration
|
||||||
|
- **Elements**: Matrix chat (the Rust one)
|
||||||
|
- **Stallwart**: Mail/calendar/contacts
|
||||||
|
- **Gitea**: Git hosting and code collaboration
|
||||||
|
- **Nextcloud**: File storage and synchronization
|
||||||
|
- **LiveKit or Jitsi**: Video conferencing integrated with elements/nextcloud
|
||||||
|
- **SSO**: Single Sign-On (can this be gitea as backend - later)
|
||||||
|
|
||||||
|
This demonstrates the platform's capability to host complex, integrated applications with minimal configuration.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Differentiators
|
||||||
|
|
||||||
|
### Sovereign by Design
|
||||||
|
Control jurisdiction, residency, and governance for every workload with transparent, verifiable operations.
|
||||||
|
|
||||||
|
### Secure by Default
|
||||||
|
Cryptographic verification, secure boot, and zero-image delivery protect the entire lifecycle automatically.
|
||||||
|
|
||||||
|
### Ready to Scale
|
||||||
|
Autonomous orchestration keeps the platform elastic, cost-efficient, and always available across the globe.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security Architecture
|
||||||
|
|
||||||
|
### Provable Sovereignty
|
||||||
|
Assign workloads to trusted zones, verify state with cryptographic proofs, and maintain full lineage for every byte.
|
||||||
|
|
||||||
|
### Autonomous Zero-Trust
|
||||||
|
Identity, policy, and attestation are enforced continuously—no manual keys, no hidden backdoors, no shared control.
|
||||||
|
|
||||||
|
### Planetary-Scale Resilience
|
||||||
|
Mesh-connected infrastructure routes around failure, keeping applications responsive even when regions go dark.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Mycelium Cloud - Comprehensive platform for decentralized Kubernetes deployment.*
|
||||||
157
docs/mycelium_gpu_for_devs.md
Normal file
157
docs/mycelium_gpu_for_devs.md
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
# Mycelium GPU for Developers
|
||||||
|
|
||||||
|
*The Energy Behind Intelligence*
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Mycelium GPU provides unified access to distributed GPU acceleration across the ThreeFold Grid. It transforms fragmented GPU resources into a single sovereign fabric for running AI, ML, and rendering workloads.
|
||||||
|
|
||||||
|
## Core Concept
|
||||||
|
|
||||||
|
Mycelium GPU unifies distributed acceleration into a single sovereign fabric — turning fragmented hardware into one adaptive intelligence layer. Run AI, ML, and rendering workloads anywhere, from edge to core, with deterministic performance and transparent cost.
|
||||||
|
|
||||||
|
### Key Principles
|
||||||
|
- **No Silos**: All GPU resources accessible through single interface
|
||||||
|
- **No Intermediaries**: Direct access to GPU resources
|
||||||
|
- **Raw, Verifiable Power**: Every GPU cycle cryptographically verified
|
||||||
|
- **Orchestrated Through Code**: GPU resources managed through APIs and smart contracts
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Use Cases
|
||||||
|
|
||||||
|
### AI/ML Training
|
||||||
|
Run GPU-accelerated workloads for deep learning and data science on demand.
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
- **GPU Acceleration**: High-performance computing for machine learning
|
||||||
|
- **Scalable Compute**: Scale training across multiple GPU resources
|
||||||
|
- **Cost Optimization**: Pay only for actual GPU usage
|
||||||
|
|
||||||
|
### Rendering & Visualization
|
||||||
|
Run high-performance graphics processing workloads.
|
||||||
|
|
||||||
|
**Applications:**
|
||||||
|
- **3D Rendering**: Distributed rendering for film, games, and architecture
|
||||||
|
- **Scientific Visualization**: Complex data visualization and analysis
|
||||||
|
- **Virtual Reality**: Real-time VR/AR processing
|
||||||
|
- **Digital Twins**: Real-time simulation and modeling
|
||||||
|
|
||||||
|
### General GPU Computing
|
||||||
|
High-performance computing for various computational workloads.
|
||||||
|
|
||||||
|
**Applications:**
|
||||||
|
- **Scientific Simulations**: Physics, chemistry, climate modeling
|
||||||
|
- **Financial Modeling**: Risk analysis and algorithmic trading
|
||||||
|
- **Cryptocurrency**: Mining and blockchain processing
|
||||||
|
- **Protein Folding**: Drug discovery and molecular modeling
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration with Mycelium Cloud
|
||||||
|
|
||||||
|
Mycelium GPU works seamlessly with Mycelium Cloud infrastructure:
|
||||||
|
|
||||||
|
- **Unified Networking**: GPU nodes accessible via Mycelium network
|
||||||
|
- **Shared Security**: Zero-trust security model applies to GPU operations
|
||||||
|
- **Storage Integration**: Access quantum-safe storage from GPU workloads
|
||||||
|
- **Kubernetes Support**: GPU workloads can be deployed as Kubernetes resources
|
||||||
|
|
||||||
|
### Deployment Example
|
||||||
|
```yaml
|
||||||
|
# GPU workload specification for Kubernetes
|
||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: gpu-workload
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: gpu-compute
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app: gpu-compute
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: gpu-compute
|
||||||
|
image: tensorflow/tensorflow:latest-gpu
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
nvidia.com/gpu: 1
|
||||||
|
env:
|
||||||
|
- name: MYCELIUM_GPU_REGION
|
||||||
|
value: "auto"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
### Access GPU Resources
|
||||||
|
1. **Account Setup**: Create Mycelium account with GPU access
|
||||||
|
2. **Resource Request**: Use Mycelium GPU APIs to request GPU resources
|
||||||
|
3. **Workload Deployment**: Deploy your AI/ML or compute workload
|
||||||
|
4. **Monitor Usage**: Track GPU utilization and costs through dashboard
|
||||||
|
|
||||||
|
### Basic Workflow
|
||||||
|
```
|
||||||
|
Application → Mycelium GPU API → GPU Resource Allocation → Workload Execution
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Benefits
|
||||||
|
- **Deterministic Performance**: Predictable GPU allocation and performance
|
||||||
|
- **Global Distribution**: Access GPU resources worldwide
|
||||||
|
- **Transparent Costs**: Clear pricing without hidden fees
|
||||||
|
- **Sovereign Control**: Full control over GPU workloads and data
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Technical Architecture
|
||||||
|
|
||||||
|
### Distributed GPU Mesh
|
||||||
|
Mycelium GPU creates a peer-to-peer network of GPU resources accessible through the Mycelium Network.
|
||||||
|
|
||||||
|
**Components:**
|
||||||
|
- **GPU Nodes**: Physical GPU hardware distributed globally
|
||||||
|
- **Mycelium Network**: Encrypted peer-to-peer communication layer
|
||||||
|
- **Orchestration Layer**: API and smart contract-based resource management
|
||||||
|
- **Monitoring**: Real-time GPU utilization and health monitoring
|
||||||
|
|
||||||
|
### Performance Characteristics
|
||||||
|
- **Edge-to-Core Deployment**: Run workloads from edge devices to data centers
|
||||||
|
- **Adaptive Intelligence Layer**: Optimizes GPU resource allocation
|
||||||
|
- **Deterministic Performance**: Guaranteed resource availability and performance
|
||||||
|
- **Transparent Cost**: All GPU usage tracked and billed transparently
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Differentiators
|
||||||
|
|
||||||
|
### Unified Fabric
|
||||||
|
Transforms fragmented GPU resources into a single, unified acceleration fabric accessible through standard APIs.
|
||||||
|
|
||||||
|
### Sovereign Control
|
||||||
|
Complete control over GPU workloads with no vendor lock-in or geographical restrictions.
|
||||||
|
|
||||||
|
### Code-Driven Orchestration
|
||||||
|
GPU resources managed through APIs and smart contracts, enabling automated and verifiable resource allocation.
|
||||||
|
|
||||||
|
### Deterministic Performance
|
||||||
|
Guaranteed GPU allocation with consistent performance characteristics across all workloads.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Cost Efficiency
|
||||||
|
|
||||||
|
Mycelium GPU provides cost-effective access to GPU resources through:
|
||||||
|
|
||||||
|
- **Transparent Pricing**: No hidden fees or surprise charges
|
||||||
|
- **Pay-per-Usage**: Pay only for actual GPU consumption
|
||||||
|
- **Global Optimization**: Access GPUs where they're most cost-effective
|
||||||
|
- **No Vendor Lock-in**: Avoid premium pricing from single providers
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Mycelium GPU - Unifying distributed acceleration into a sovereign fabric.*
|
||||||
Reference in New Issue
Block a user