myceliumcloud-examples/examples/nginx-nodeport/nginx-nodeport.md

# Mycelium Cloud - Nginx NodePort Secure Website Hosting

A complete, production-ready example for deploying a secure, globally accessible website on Mycelium Cloud using Kubernetes **NodePort** services with enhanced network isolation. This demonstrates **security-first IPv6 web hosting** with standard Kubernetes patterns.

## 🚨 CRITICAL: IPv6 Configuration Required

**IMPORTANT:** This example requires **dual-stack service configuration** to work with Mycelium IPv6 addresses. Without it, you'll be able to ping IPv6 addresses but not access the service.

**The Fix:** The `nginx-nodeport-service.yaml` includes:
```yaml
ipFamilies:
  - IPv4
  - IPv6
ipFamilyPolicy: RequireDualStack
```

**Why:** Kubernetes services default to IPv4-only. Mycelium uses IPv6, so we must explicitly enable dual-stack.

**Status:** ✅ Included in provided YAML files. See [IPV6_FIX.md](IPV6_FIX.md) for details.

## 🔑 Key Concept: NodePort Access Pattern

**Important:** With NodePort, you access the service via:
- **Worker Node's Mycelium IPv6 address** + **NodePort 30091**
- Example: `http://[worker-node-mycelium-ipv6]:30091/`

**This is different from hostNetwork** where pods get direct Mycelium IPs:
- **hostNetwork**: Pod has Mycelium IP → Access: `http://[pod-mycelium-ip]:8080`
- **NodePort**: Pod isolated network → Access: `http://[node-mycelium-ip]:30091`

The traffic flow is: `User → Node:30091 → Service:8080 → Pod:8080`

## 📁 What This Contains

This directory contains everything you need to deploy a secure website with global IPv6 accessibility:

- **nginx-nodeport.md** - This comprehensive guide
- **nginx-nodeport-deployment.yaml** - Secure deployment configuration
- **nginx-nodeport-service.yaml** - NodePort service configuration
- **nginx-nodeport-configmaps.yaml** - Website content and nginx configuration
- **test-nodeport-ipv6.sh** - IPv6 testing and verification script
- **update-content.sh** - Dynamic content update with IPv6 discovery
- **compare-approaches.md** - Security and architecture comparison

## 🚀 Quick Start (3 minutes)

```bash
# 1. Deploy the ConfigMaps (content and nginx config)
kubectl apply -f nginx-nodeport-configmaps.yaml

# 2. Deploy the nginx application
kubectl apply -f nginx-nodeport-deployment.yaml

# 3. Create the NodePort service
kubectl apply -f nginx-nodeport-service.yaml

# 4. Wait for deployment to be ready
kubectl wait --for=condition=ready pod -l app=nginx-nodeport --timeout=60s

# 5. Get IPv6 address of the ACTUAL node where your pod is running
POD_NODE=$(kubectl get pods -l app=nginx-nodeport -o jsonpath='{.items[0].spec.nodeName}')
NODE_IPV6=$(kubectl get node $POD_NODE -o jsonpath='{range .status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}' | grep -E '^[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+$' | head -1)
echo "Your website is accessible at: http://[$NODE_IPV6]:30091"
echo "NOTE: Pod is running on node: $POD_NODE"

# 6. Test the website
curl -6 "http://[$NODE_IPV6]:30091/"
```

**Alternative: Use the automated script for IPv6 discovery:**
```bash
# Automatically discover IPv6 addresses and update content
./update-content.sh
```

**⚠️ Important: Worker Node vs Master Node Access**
Unlike hostNetwork (which can be accessed from any node), NodePort services are **limited to the nodes where your pods are actually running**. Always check:
```bash
# Check which node your pod is running on
kubectl get pods -l app=nginx-nodeport -o wide

# Get the IPv6 of the correct WORKER node (not master)
NODE_IPV6=$(kubectl get nodes -l kubernetes.io/role!=master -o jsonpath='{range .items[0].status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}' | grep -E '^[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+$' | head -1)
```

**Expected Result:** You'll see a professional website with NodePort security branding and IPv6 address detection.

## 📋 What You'll Learn

- ✅ **Security-First IPv6 Web Hosting** - Enhanced network isolation
- ✅ **Standard Kubernetes Service Patterns** - Industry best practices
- ✅ **NodePort with External Traffic Policy** - Preserves IPv6 source IP
- ✅ **ConfigMap-based Content Management** - Dynamic updates
- ✅ **Production nginx Configuration** - Dual-stack (IPv4/IPv6) support
- ✅ **Resource Management** - CPU/memory limits and health checks
- ✅ **Security Comparison** - Understanding hostNetwork vs standard networking

## 🏗️ Architecture

This example uses **separate configuration files** for better organization and security:

### Component Overview
```
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Mycelium      │    │   NodePort       │    │   nginx Pod     │
│   IPv6 Network  │───▶│   Service        │───▶│   (Isolated)    │
│   :30090        │    │   :8080          │    │   :8080         │
└─────────────────┘    └──────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌──────────────────┐
                       │   ConfigMaps     │
                       │   • HTML Content │
                       │   • nginx Config │
                       └──────────────────┘
```

### Network Flow (NodePort Pattern)
```
User with Mycelium
        ↓
http://[worker-node-mycelium-ipv6]:30091
        ↓
Worker Node (kube-proxy forwards)
        ↓
NodePort 30091 → Service port 8080
        ↓
Kubernetes Service (load balances)
        ↓
Pod container port 8080
        ↓
nginx → HTML Content
```

### 🚨 Critical: NodePort Access Pattern

**With NodePort:**
- Pods run in **isolated network** (no `hostNetwork`)
- Pods do **NOT** have direct Mycelium IPv6 addresses
- Access via **worker node's Mycelium IPv6** + NodePort
- Service is accessible on **all worker nodes** at port 30091
- kube-proxy forwards traffic: `node:30091 → service:8080 → pod:8080`

**Comparison:**
- **hostNetwork (nginx-mycelium)**: Pod gets host's Mycelium IP → Access: `http://[pod-ip]:8080`
- **NodePort (nginx-nodeport)**: Pod isolated → Access: `http://[node-ip]:30091`

**Getting the right IPv6:**
```bash
# Get worker node Mycelium IPv6 addresses
kubectl get nodes -o wide

# OR use the fetch-ip.sh script
../../scripts/fetch-ip.sh
```

### Key Security Improvements
- **Pod Isolation**: No `hostNetwork` access, pods run in isolated network namespace
- **Standard Service Patterns**: Uses typical Kubernetes service discovery
- **External Traffic Policy**: `Local` preserves IPv6 source IP for logging
- **Resource Limits**: Prevents resource exhaustion attacks
- **Health Checks**: Liveness and readiness probes for reliability

## 📄 Configuration Files

### 1. nginx-nodeport-deployment.yaml
**Secure pod deployment without hostNetwork:**

```yaml
spec:
  hostNetwork: false  # Key security improvement
  containers:
  - name: nginx
    image: nginx:alpine
    ports:
    - containerPort: 8080  # No hostPort needed
    resources:
      requests:
        memory: "64Mi"
        cpu: "100m"
      limits:
        memory: "128Mi"
        cpu: "200m"
    livenessProbe:
      httpGet:
        path: /health
        port: 8080
    readinessProbe:
      httpGet:
        path: /health
        port: 8080
```

**What it does:**
- Creates 1 pod with nginx and custom website content
- Uses **standard pod networking** (no direct host access)
- Includes **resource limits** and **health checks**
- Uses ConfigMaps for **dynamic content management**
- **Dual-stack nginx** (IPv4 + IPv6) configuration

### 2. nginx-nodeport-service.yaml
**NodePort service with IPv6 source IP preservation:**

```yaml
spec:
  type: NodePort  # ← Service type is NodePort
  externalTrafficPolicy: Local  # Preserves IPv6 source IP
  ports:
  - port: 8080
    targetPort: 8080
    nodePort: 30091  # ← Explicitly set to 30091 (avoiding conflict with nginx-mycelium's 30090)
    protocol: TCP
```

**What it does:**
- Exposes nginx on **NodePort 30091** on all worker nodes
- **Preserves IPv6 source IP** for logging and security
- Uses **standard Kubernetes service** patterns (not LoadBalancer)
- Provides **load balancing** across pod replicas (when scaled)
- Access via: `http://[worker-node-mycelium-ipv6]:30091/`

**NodePort Range:** Kubernetes NodePorts use range 30000-32767 by default

### 3. nginx-nodeport-configmaps.yaml
**Content and nginx configuration:**

**HTML Content:** Professional website with NodePort security branding
**nginx Configuration:** Dual-stack (IPv4/IPv6) with health endpoint

```nginx
server {
    listen 8080;
    listen [::]:8080 ipv6only=on;  # IPv6 support
    server_name _;
    
    location / {
        root /usr/share/nginx/html;
        index index.html;
    }
    
    location /health {
        access_log off;
        return 200 "healthy\n";
    }
}
```

## 🔍 Verification and Testing

### Check pod status
```bash
kubectl get pods -l app=nginx-nodeport
kubectl describe pod -l app=nginx-nodeport
```

### Check service configuration
```bash
kubectl get svc nginx-nodeport-service -o yaml
kubectl get endpoints nginx-nodeport-service
```

### Test nginx configuration
```bash
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
kubectl exec $POD_NAME -- nginx -t
kubectl exec $POD_NAME -- nginx -s reload
```

### Test health endpoint
```bash
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
kubectl exec $POD_NAME -- curl -s http://localhost:8080/health
```

### Test website content
```bash
# Get node IPv6 address
NODE_IPV6=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')

# Test from outside the cluster
curl -6 "http://[$NODE_IPV6]:30091/"
curl -6 "http://[$NODE_IPV6]:30091/health"
```

### Verify ConfigMaps are mounted
```bash
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
kubectl exec $POD_NAME -- ls -la /usr/share/nginx/html/
kubectl exec $POD_NAME -- cat /usr/share/nginx/html/index.html | head -10
```

## 🔄 Updating Content

### Update website content
```bash
# Create new HTML file
cat > new-index.html << 'EOF'
<!DOCTYPE html>
<html>
<head><title>Updated Site</title></head>
<body>
    <h1>Content Updated!</h1>
    <p>New content deployed at $(date)</p>
</body>
</html>
EOF

# Update ConfigMap
kubectl create configmap nginx-nodeport-content \
  --from-file=index.html=new-index.html \
  --dry-run=client -o yaml | kubectl apply -f -

# Reload nginx to pick up changes
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
kubectl exec $POD_NAME -- nginx -s reload

echo "✅ Content updated! Access at: http://[$NODE_IPV6]:30091"
```

### Update nginx configuration
```bash
# Create custom nginx config
cat > custom-nginx.conf << 'EOF'
server {
    listen 8080;
    listen [::]:8080 ipv6only=on;
    server_name my-site.com;
    
    # Custom logging
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    
    location / {
        root /usr/share/nginx/html;
        index index.html;
        try_files $uri $uri/ =404;
    }
    
    location /health {
        access_log off;
        return 200 "healthy\n";
    }
}
EOF

# Update ConfigMap
kubectl create configmap nginx-nodeport-nginx-config \
  --from-file=default.conf=custom-nginx.conf \
  --dry-run=client -o yaml | kubectl apply -f -

# Reload nginx
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
kubectl exec $POD_NAME -- nginx -s reload
```

## 🔍 Troubleshooting

### Pod won't start
```bash
# Check pod status and events
kubectl describe pod -l app=nginx-nodeport

# Check pod logs
kubectl logs -f deployment/nginx-nodeport

# Check resource availability
kubectl top nodes
kubectl describe nodes
```

### Service not accessible
```bash
# Verify service exists and has endpoints
kubectl get svc nginx-nodeport-service
kubectl get endpoints nginx-nodeport-service

# Check if pod is ready
kubectl get pods -l app=nginx-nodeport -o wide

# Verify NodePort is open
netstat -tulpn | grep 30091
```

### IPv6 connectivity issues
```bash
# Test IPv6 on the node
ping6 -c 3 $NODE_IPV6

# Check nginx is listening on IPv6
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
kubectl exec $POD_NAME -- netstat -tuln | grep 8080

# Verify nginx configuration
kubectl exec $POD_NAME -- cat /etc/nginx/conf.d/default.conf
```

### Performance issues
```bash
# Check resource usage
kubectl top pods -l app=nginx-nodeport

# Check nginx status and connections
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
kubectl exec $POD_NAME -- ps aux | grep nginx
kubectl exec $POD_NAME -- netstat -an | grep :8080 | wc -l
```

## 🗑️ Cleanup

```bash
# Remove all resources
kubectl delete -f nginx-nodeport-deployment.yaml -f nginx-nodeport-service.yaml

# Remove ConfigMaps
kubectl delete configmap nginx-nodeport-content nginx-nodeport-nginx-config

# Verify cleanup
kubectl get all -l app=nginx-nodeport  # Should return nothing
```

## 📊 Nginx Deployment Variants Comparison

### Complete Overview: 4 Ways to Deploy Nginx on Mycelium Cloud

| Feature | hostNetwork | NodePort | LoadBalancer | Ingress |
|---------|-------------|----------|--------------|---------|
| **Example** | nginx-mycelium | nginx-nodeport | (Future) | (Future) |
| **Pod Network** | ❌ Host | ✅ Isolated | ✅ Isolated | ✅ Isolated |
| **Security** | ⚠️ Low | ✅ Medium | ✅ Medium | ✅ High |
| **Access Pattern** | `[pod-ip]:8080` | `[node-ip]:30091` | `[lb-ip]:80` | `domain.com` |
| **Port Range** | Any | 30000-32767 | 80, 443 | 80, 443 |
| **Mycelium Access** | Direct pod | Via node | Via LB | Via ingress |
| **Use Case** | Demo/POC | Testing/Dev | Production | Web apps |
| **Scalability** | ❌ Limited | ✅ Good | ✅ Excellent | ✅ Excellent |
| **Load Balancing** | Manual | K8s Service | Cloud LB | Ingress controller |
| **SSL/TLS** | Manual | Manual | Possible | Native |
| **DNS Support** | No | No | Possible | Yes |
| **Production Ready** | ⚠️ No | ✅ Yes | ✅ Yes | ✅ Yes |

### Key Differences: hostNetwork vs NodePort

| Feature | hostNetwork (nginx-mycelium) | NodePort (nginx-nodeport) |
|---------|------------------------------|---------------------------|
| **Security** | ⚠️ Direct host access | ✅ Isolated pod network |
| **Network Isolation** | ❌ Uses host interfaces | ✅ Pod namespace isolation |
| **Port Conflicts** | ❌ Limited by host ports | ✅ No port conflicts |
| **Debugging** | 🔄 Host-level tools | ✅ Standard K8s patterns |
| **Monitoring** | 🔄 Host monitoring | ✅ Pod-level monitoring |
| **Scalability** | ❌ Single instance per node | ✅ Multiple replicas |
| **Production Ready** | ⚠️ Demo/POC only | ✅ Production patterns |
| **Access URL** | `http://[pod-mycelium-ip]:8080` | `http://[node-mycelium-ip]:30091` |
| **Port Used** | 8080 (+ NodePort 30090) | NodePort 30091 |

## 📈 Scaling and High Availability

### Scale to multiple replicas
```bash
# Scale deployment
kubectl scale deployment nginx-nodeport --replicas=3

# Verify scaling
kubectl get pods -l app=nginx-nodeport

# Check load balancing
NODE_IPV6=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
for i in {1..5}; do
  curl -6 "http://[$NODE_IPV6]:30091/" | grep -o "nginx-nodeport-[a-z0-9]*-[a-z0-9]*" | head -1
done
```

### Add resource limits for better performance
```yaml
# Update deployment with enhanced resources
resources:
  requests:
    memory: "128Mi"
    cpu: "200m"
  limits:
    memory: "256Mi"
    cpu: "500m"
```

## 🎯 Best Practices

1. **Security First**: Always prefer standard pod networking over hostNetwork
2. **Resource Management**: Set appropriate requests and limits
3. **Health Checks**: Use liveness and readiness probes
4. **Monitoring**: Implement proper logging and metrics collection
5. **Updates**: Use rolling updates for zero-downtime deployments
6. **Configuration**: Store configuration in ConfigMaps for flexibility

## 🔄 Complete NodePort Flow Explained

### Understanding the Full Request Path

```
┌─────────────────────────────────────────────────────────────────┐
│ Step 1: User with Mycelium Network                             │
│ • Has Mycelium client running                                   │
│ • Can reach Mycelium IPv6 addresses globally                    │
│ • Accesses: http://[worker-node-mycelium-ipv6]:30091          │
└─────────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────────┐
│ Step 2: Mycelium Network Routes to Worker Node                 │
│ • Peer-to-peer IPv6 routing                                     │
│ • No traditional DNS needed                                     │
│ • Direct encrypted connection                                   │
└─────────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────────┐
│ Step 3: Worker Node Receives Request on Port 30091             │
│ • kube-proxy listens on NodePort 30091                          │
│ • Forwards to Service cluster IP                                │
│ • Preserves source IPv6 (externalTrafficPolicy: Local)          │
└─────────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────────┐
│ Step 4: Kubernetes Service Load Balances                       │
│ • Service receives on port 8080                                 │
│ • Selects backend pod (app=nginx-nodeport)                      │
│ • Routes to targetPort 8080                                     │
└─────────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────────┐
│ Step 5: nginx Pod Processes Request                            │
│ • Pod in isolated network namespace                             │
│ • nginx listens on container port 8080                          │
│ • Serves HTML from ConfigMap                                    │
│ • Returns response                                              │
└─────────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────────┐
│ Step 6: Response Returns to User                               │
│ • Same path in reverse                                          │
│ • User sees website in browser                                  │
│ • Access logs show original IPv6 (source IP preserved)          │
└─────────────────────────────────────────────────────────────────┘
```

### Why NodePort is Better for Production

**Security Benefits:**
1. Pods cannot access host network interfaces
2. Network policies can restrict pod-to-pod traffic
3. Resource limits prevent resource exhaustion
4. Health checks ensure reliability

**Operational Benefits:**
1. Standard Kubernetes debugging tools work
2. Can scale horizontally (multiple replicas)
3. Service provides automatic load balancing
4. Compatible with monitoring stacks (Prometheus, etc.)
5. Can evolve to LoadBalancer or Ingress later

**Mycelium Integration:**
- Worker nodes have Mycelium IPv6 addresses
- NodePort makes service accessible via these IPs
- No need for traditional cloud load balancers
- Perfect for decentralized web hosting

## <20> Learning Outcomes

After completing this example, you understand:
- **NodePort Services** - Kubernetes service exposure patterns
- **Network Security** - Pod isolation vs hostNetwork trade-offs  
- **Production Patterns** - Resource management and health checks
- **IPv6 Networking** - Dual-stack nginx configuration
- **ConfigMap Management** - Dynamic content and configuration updates
- **Service Discovery** - Kubernetes service networking
- **Load Balancing** - Service-level load distribution

## 🚀 Next Steps

- **Multi-Replica Deployment**: Scale to 3+ replicas for high availability
- **LoadBalancer Service**: Move to cloud LoadBalancer for production
- **Ingress Controller**: Implement advanced routing and SSL termination
- **Monitoring**: Add Prometheus metrics and Grafana dashboards
- **SSL/TLS**: Implement HTTPS with Let's Encrypt certificates

---

**Success Criteria**: You'll know everything is working when:
- ✅ `kubectl get pods` shows nginx-nodeport pod in "Running" status
- ✅ `kubectl get svc` shows nginx-nodeport-service with NodePort 30091
- ✅ `curl -6 "http://[$NODE_IPV6]:30091"` returns your secure website
- ✅ Website displays "NODEPORT SECURE" and "ENHANCED SECURITY" badges
- ✅ `kubectl logs deployment/nginx-nodeport` shows nginx access logs

**Access URL**: `http://[NODE-IPV6]:30091` (replace NODE-IPV6 with your node's IPv6 address)