docs: add developer docs for gpu and cloud
This commit is contained in:
mik-tf
181
docs/mycelium_cloud_for_devs.md
Normal file
181
docs/mycelium_cloud_for_devs.md
Normal file
@@ -0,0 +1,181 @@
|
||||
# Mycelium Cloud for Developers
|
||||
|
||||
*Developer Guide to Decentralized Cloud Computing*
|
||||
|
||||
## Overview
|
||||
Mycelium Cloud provides a comprehensive platform for deploying and managing Kubernetes clusters on the decentralized ThreeFold Grid infrastructure. This guide covers the core features and capabilities available to developers.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Decentralized Infrastructure
|
||||
Mycelium Cloud deploys on ThreeFold Grid's distributed network, providing:
|
||||
- **Direct Node Access**: Each node gets a unique Mycelium IP address
|
||||
- **Cross-Node Communication**: Services communicate across nodes using Mycelium networking
|
||||
- **Secure Communication**: All traffic is encrypted through the Mycelium network
|
||||
- **No Public IPs Required**: Services accessible via Mycelium IPs
|
||||
|
||||
### Network Flow
|
||||
User Machine → Mycelium Network → Cluster Node → Service
|
||||
|
||||
### Kubernetes Management
|
||||
- **K3s Support**: Full K3s cluster deployment and management
|
||||
- **IPv6 Networking**: Mycelium peer-to-peer networking
|
||||
- **High Availability**: Multi-master cluster support
|
||||
|
||||
---
|
||||
|
||||
## Core Features
|
||||
|
||||
### Mycelium Networking
|
||||
Ultra-fast, decentralized networking inspired by nature. Mycelium Networking forms a resilient, adaptive mesh that routes around failures and optimizes for speed and security.
|
||||
|
||||
**Features:**
|
||||
- End-to-end encrypted
|
||||
- Nature-inspired
|
||||
- Direct node communication
|
||||
- Secure peer-to-peer connections
|
||||
|
||||
### Zero-Image Technology
|
||||
Metadata-driven zero-images cut artifacts up to 100x, slashing bandwidth and deployment overhead.
|
||||
|
||||
**Benefits:**
|
||||
- **Deterministic Deployments**: Cryptographic verification ensures every workload deploys exactly as intended—no tampering, no drift
|
||||
- **Multi-Workload Compatibility with Secure Boot**: Run containers, VMs, and Linux workloads anywhere with stateless secure boot and continuous verification
|
||||
- **Smart Contract-Based Deployment**: Cryptographically signed contracts orchestrate every workload with transparent, tamper-proof execution
|
||||
|
||||
### Quantum-Safe Storage (QSS)
|
||||
Quantum-resistant encryption secures data beyond the app layer so ownership and control stay yours.
|
||||
|
||||
**Features:**
|
||||
- **Self-Healing Storage System**: Autonomous recovery heals failures or corruption instantly, preserving integrity without human intervention
|
||||
- **Multi-Protocol Data Access**: Serve the same data via IPFS, S3, WebDAV, HTTP, and native file systems for seamless integration
|
||||
- **Geo-Aware Data Placement & Replication**: Define residency, redundancy, and distribution per workload while zone-to-zone replication hardens resilience
|
||||
|
||||
### Multi-Master Clusters
|
||||
High-availability Kubernetes clusters with multiple control plane nodes. Automatic failover, leader election, and zero-downtime upgrades built-in.
|
||||
|
||||
**Capabilities:**
|
||||
- **HA Control Plane**
|
||||
- **Automatic Failover**
|
||||
- **Zero-downtime Upgrades**
|
||||
|
||||
### Effortless Load Balancing & Scaling
|
||||
Mycelium Cloud automatically balances traffic and scales your services up or down based on demand. Enjoy high availability and optimal performance with zero manual intervention.
|
||||
|
||||
**Features:**
|
||||
- **Auto-scaling**
|
||||
- **Built-in load balancing**
|
||||
- **High availability**
|
||||
|
||||
### Simple Web Gateway Access
|
||||
Expose any service to the public web with a simple Kubernetes resource. No complex Ingress controllers. Domain and prefix-based routing is built-in.
|
||||
|
||||
**Benefits:**
|
||||
- **Simple configuration**
|
||||
- **Built-in routing**
|
||||
- **No ingress controllers**
|
||||
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
|
||||
### 1. Account Setup
|
||||
- **Sign Up**: Create your account from signup page
|
||||
- **Verify Email**: Check your email and verify your account
|
||||
- **Add Funds**: Navigate to your dashboard and add credits to your account
|
||||
- **Add SSH Key**: Navigate to Add SSH card and upload your public SSH key
|
||||
|
||||
### 2. Deploy Your First Cluster
|
||||
- **Access Deploy**: Click "Deploy Cluster" from your dashboard
|
||||
- **Configure VMs**: Define your virtual machines:
|
||||
- Choose CPU, memory, and storage requirements
|
||||
- Select the number of master and worker nodes
|
||||
- **Select Nodes**: Choose ThreeFold Grid nodes for deployment
|
||||
- **Review & Deploy**: Confirm your configuration and deploy
|
||||
|
||||
### 3. Access Your Cluster
|
||||
|
||||
#### Download Kubeconfig
|
||||
- Go to dashboard → Clusters → Click download icon (⬇️)
|
||||
- Set kubeconfig: `export KUBECONFIG=/path/to/config`
|
||||
- Test: `kubectl get nodes`
|
||||
|
||||
#### SSH Access
|
||||
- **Find Mycelium IPs**: Check cluster details page for node IPs
|
||||
- **Download Mycelium Binary**:
|
||||
```bash
|
||||
wget https://github.com/threefoldtech/mycelium/releases/latest/download/mycelium-private-x86_64-unknown-linux-musl.tar.gz
|
||||
tar -xzf mycelium-private-x86_64-unknown-linux-musl.tar.gz
|
||||
sudo chmod +x mycelium-private
|
||||
sudo mv mycelium-private /usr/local/bin/mycelium
|
||||
```
|
||||
- **Start Mycelium**:
|
||||
```bash
|
||||
sudo mycelium --peers tcp://188.40.132.242:9651 tcp://136.243.47.186:9651 tcp://185.69.166.7:9651 tcp://185.69.166.8:9651 tcp://65.21.231.58:9651 tcp://65.109.18.113:9651 tcp://209.159.146.190:9651 tcp://5.78.122.16:9651 tcp://5.223.43.251:9651 tcp://142.93.217.194:9651
|
||||
```
|
||||
- **SSH to nodes**: `ssh root@<mycelium-ip>`
|
||||
|
||||
---
|
||||
|
||||
## Use Cases
|
||||
|
||||
### AI/ML Training
|
||||
Run GPU-accelerated workloads for deep learning and data science on demand.
|
||||
- **GPU Acceleration**
|
||||
- **Scalable Compute**
|
||||
- **Cost Optimization**
|
||||
|
||||
### Enterprise Kubernetes
|
||||
Deploy production-grade clusters with full control and no vendor lock-in.
|
||||
- **High Availability**
|
||||
- **Security**
|
||||
- **Compliance**
|
||||
|
||||
### Edge & IoT
|
||||
Leverage global nodes for low-latency edge computing and IoT applications.
|
||||
- **Low Latency**
|
||||
- **Global Distribution**
|
||||
- **Real-time Processing**
|
||||
|
||||
### DigitalMe Example
|
||||
Example of what can be done with Mycelium Cloud. With a very simple execution, the following can be deployed on k8s cluster and served on Mycelium IPs:
|
||||
|
||||
- **Cryptpad**: Encrypted document collaboration
|
||||
- **Elements**: Matrix chat (the Rust one)
|
||||
- **Stallwart**: Mail/calendar/contacts
|
||||
- **Gitea**: Git hosting and code collaboration
|
||||
- **Nextcloud**: File storage and synchronization
|
||||
- **LiveKit or Jitsi**: Video conferencing integrated with elements/nextcloud
|
||||
- **SSO**: Single Sign-On (can this be gitea as backend - later)
|
||||
|
||||
This demonstrates the platform's capability to host complex, integrated applications with minimal configuration.
|
||||
|
||||
---
|
||||
|
||||
## Key Differentiators
|
||||
|
||||
### Sovereign by Design
|
||||
Control jurisdiction, residency, and governance for every workload with transparent, verifiable operations.
|
||||
|
||||
### Secure by Default
|
||||
Cryptographic verification, secure boot, and zero-image delivery protect the entire lifecycle automatically.
|
||||
|
||||
### Ready to Scale
|
||||
Autonomous orchestration keeps the platform elastic, cost-efficient, and always available across the globe.
|
||||
|
||||
---
|
||||
|
||||
## Security Architecture
|
||||
|
||||
### Provable Sovereignty
|
||||
Assign workloads to trusted zones, verify state with cryptographic proofs, and maintain full lineage for every byte.
|
||||
|
||||
### Autonomous Zero-Trust
|
||||
Identity, policy, and attestation are enforced continuously—no manual keys, no hidden backdoors, no shared control.
|
||||
|
||||
### Planetary-Scale Resilience
|
||||
Mesh-connected infrastructure routes around failure, keeping applications responsive even when regions go dark.
|
||||
|
||||
---
|
||||
|
||||
*Mycelium Cloud - Comprehensive platform for decentralized Kubernetes deployment.*
|
||||
157
docs/mycelium_gpu_for_devs.md
Normal file
157
docs/mycelium_gpu_for_devs.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# Mycelium GPU for Developers
|
||||
|
||||
*The Energy Behind Intelligence*
|
||||
|
||||
## Overview
|
||||
|
||||
Mycelium GPU provides unified access to distributed GPU acceleration across the ThreeFold Grid. It transforms fragmented GPU resources into a single sovereign fabric for running AI, ML, and rendering workloads.
|
||||
|
||||
## Core Concept
|
||||
|
||||
Mycelium GPU unifies distributed acceleration into a single sovereign fabric — turning fragmented hardware into one adaptive intelligence layer. Run AI, ML, and rendering workloads anywhere, from edge to core, with deterministic performance and transparent cost.
|
||||
|
||||
### Key Principles
|
||||
- **No Silos**: All GPU resources accessible through single interface
|
||||
- **No Intermediaries**: Direct access to GPU resources
|
||||
- **Raw, Verifiable Power**: Every GPU cycle cryptographically verified
|
||||
- **Orchestrated Through Code**: GPU resources managed through APIs and smart contracts
|
||||
|
||||
---
|
||||
|
||||
## Use Cases
|
||||
|
||||
### AI/ML Training
|
||||
Run GPU-accelerated workloads for deep learning and data science on demand.
|
||||
|
||||
**Features:**
|
||||
- **GPU Acceleration**: High-performance computing for machine learning
|
||||
- **Scalable Compute**: Scale training across multiple GPU resources
|
||||
- **Cost Optimization**: Pay only for actual GPU usage
|
||||
|
||||
### Rendering & Visualization
|
||||
Run high-performance graphics processing workloads.
|
||||
|
||||
**Applications:**
|
||||
- **3D Rendering**: Distributed rendering for film, games, and architecture
|
||||
- **Scientific Visualization**: Complex data visualization and analysis
|
||||
- **Virtual Reality**: Real-time VR/AR processing
|
||||
- **Digital Twins**: Real-time simulation and modeling
|
||||
|
||||
### General GPU Computing
|
||||
High-performance computing for various computational workloads.
|
||||
|
||||
**Applications:**
|
||||
- **Scientific Simulations**: Physics, chemistry, climate modeling
|
||||
- **Financial Modeling**: Risk analysis and algorithmic trading
|
||||
- **Cryptocurrency**: Mining and blockchain processing
|
||||
- **Protein Folding**: Drug discovery and molecular modeling
|
||||
|
||||
---
|
||||
|
||||
## Integration with Mycelium Cloud
|
||||
|
||||
Mycelium GPU works seamlessly with Mycelium Cloud infrastructure:
|
||||
|
||||
- **Unified Networking**: GPU nodes accessible via Mycelium network
|
||||
- **Shared Security**: Zero-trust security model applies to GPU operations
|
||||
- **Storage Integration**: Access quantum-safe storage from GPU workloads
|
||||
- **Kubernetes Support**: GPU workloads can be deployed as Kubernetes resources
|
||||
|
||||
### Deployment Example
|
||||
```yaml
|
||||
# GPU workload specification for Kubernetes
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: gpu-workload
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: gpu-compute
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: gpu-compute
|
||||
spec:
|
||||
containers:
|
||||
- name: gpu-compute
|
||||
image: tensorflow/tensorflow:latest-gpu
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
env:
|
||||
- name: MYCELIUM_GPU_REGION
|
||||
value: "auto"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Access GPU Resources
|
||||
1. **Account Setup**: Create Mycelium account with GPU access
|
||||
2. **Resource Request**: Use Mycelium GPU APIs to request GPU resources
|
||||
3. **Workload Deployment**: Deploy your AI/ML or compute workload
|
||||
4. **Monitor Usage**: Track GPU utilization and costs through dashboard
|
||||
|
||||
### Basic Workflow
|
||||
```
|
||||
Application → Mycelium GPU API → GPU Resource Allocation → Workload Execution
|
||||
```
|
||||
|
||||
### Key Benefits
|
||||
- **Deterministic Performance**: Predictable GPU allocation and performance
|
||||
- **Global Distribution**: Access GPU resources worldwide
|
||||
- **Transparent Costs**: Clear pricing without hidden fees
|
||||
- **Sovereign Control**: Full control over GPU workloads and data
|
||||
|
||||
---
|
||||
|
||||
## Technical Architecture
|
||||
|
||||
### Distributed GPU Mesh
|
||||
Mycelium GPU creates a peer-to-peer network of GPU resources accessible through the Mycelium Network.
|
||||
|
||||
**Components:**
|
||||
- **GPU Nodes**: Physical GPU hardware distributed globally
|
||||
- **Mycelium Network**: Encrypted peer-to-peer communication layer
|
||||
- **Orchestration Layer**: API and smart contract-based resource management
|
||||
- **Monitoring**: Real-time GPU utilization and health monitoring
|
||||
|
||||
### Performance Characteristics
|
||||
- **Edge-to-Core Deployment**: Run workloads from edge devices to data centers
|
||||
- **Adaptive Intelligence Layer**: Optimizes GPU resource allocation
|
||||
- **Deterministic Performance**: Guaranteed resource availability and performance
|
||||
- **Transparent Cost**: All GPU usage tracked and billed transparently
|
||||
|
||||
---
|
||||
|
||||
## Key Differentiators
|
||||
|
||||
### Unified Fabric
|
||||
Transforms fragmented GPU resources into a single, unified acceleration fabric accessible through standard APIs.
|
||||
|
||||
### Sovereign Control
|
||||
Complete control over GPU workloads with no vendor lock-in or geographical restrictions.
|
||||
|
||||
### Code-Driven Orchestration
|
||||
GPU resources managed through APIs and smart contracts, enabling automated and verifiable resource allocation.
|
||||
|
||||
### Deterministic Performance
|
||||
Guaranteed GPU allocation with consistent performance characteristics across all workloads.
|
||||
|
||||
---
|
||||
|
||||
## Cost Efficiency
|
||||
|
||||
Mycelium GPU provides cost-effective access to GPU resources through:
|
||||
|
||||
- **Transparent Pricing**: No hidden fees or surprise charges
|
||||
- **Pay-per-Usage**: Pay only for actual GPU consumption
|
||||
- **Global Optimization**: Access GPUs where they're most cost-effective
|
||||
- **No Vendor Lock-in**: Avoid premium pricing from single providers
|
||||
|
||||
---
|
||||
|
||||
*Mycelium GPU - Unifying distributed acceleration into a sovereign fabric.*
|
||||
Reference in New Issue
Block a user