feat: Simplify nginx-nodeport example with worker-only deployment and improved learning path
This commit is contained in:
mik-tf
@@ -1,590 +1,263 @@
|
||||
# Mycelium Cloud - Nginx NodePort Secure Website Hosting
|
||||
# nginx-nodeport - Mycelium Cloud NodePort Website Example
|
||||
|
||||
A complete, production-ready example for deploying a secure, globally accessible website on Mycelium Cloud using Kubernetes **NodePort** services with enhanced network isolation. This demonstrates **security-first IPv6 web hosting** with standard Kubernetes patterns.
|
||||
Complete, production-ready example for deploying a secure, globally accessible website on Mycelium Cloud using **NodePort** services with **worker node preference**.
|
||||
|
||||
## 🚨 CRITICAL: IPv6 Configuration Required
|
||||
## 🚀 Learn by Doing (Recommended!)
|
||||
|
||||
**IMPORTANT:** This example requires **dual-stack service configuration** to work with Mycelium IPv6 addresses. Without it, you'll be able to ping IPv6 addresses but not access the service.
|
||||
|
||||
**The Fix:** The `nginx-nodeport-service.yaml` includes:
|
||||
```yaml
|
||||
ipFamilies:
|
||||
- IPv4
|
||||
- IPv6
|
||||
ipFamilyPolicy: RequireDualStack
|
||||
```
|
||||
|
||||
**Why:** Kubernetes services default to IPv4-only. Mycelium uses IPv6, so we must explicitly enable dual-stack.
|
||||
|
||||
**Status:** ✅ Included in provided YAML files. See [IPV6_FIX.md](IPV6_FIX.md) for details.
|
||||
|
||||
## 🔑 Key Concept: NodePort Access Pattern
|
||||
|
||||
**Important:** With NodePort, you access the service via:
|
||||
- **Worker Node's Mycelium IPv6 address** + **NodePort 30091**
|
||||
- Example: `http://[worker-node-mycelium-ipv6]:30091/`
|
||||
|
||||
**This is different from hostNetwork** where pods get direct Mycelium IPs:
|
||||
- **hostNetwork**: Pod has Mycelium IP → Access: `http://[pod-mycelium-ip]:8080`
|
||||
- **NodePort**: Pod isolated network → Access: `http://[node-mycelium-ip]:30091`
|
||||
|
||||
The traffic flow is: `User → Node:30091 → Service:8080 → Pod:8080`
|
||||
|
||||
## 📁 What This Contains
|
||||
|
||||
This directory contains everything you need to deploy a secure website with global IPv6 accessibility:
|
||||
|
||||
- **nginx-nodeport.md** - This comprehensive guide
|
||||
- **nginx-nodeport-deployment.yaml** - Secure deployment configuration
|
||||
- **nginx-nodeport-service.yaml** - NodePort service configuration
|
||||
- **nginx-nodeport-configmaps.yaml** - Website content and nginx configuration
|
||||
- **test-nodeport-ipv6.sh** - IPv6 testing and verification script
|
||||
- **update-content.sh** - Dynamic content update with IPv6 discovery
|
||||
- **compare-approaches.md** - Security and architecture comparison
|
||||
|
||||
## 🚀 Quick Start (3 minutes)
|
||||
**This example is designed to teach you NodePort scaling step-by-step:**
|
||||
|
||||
### Step 1: Start with 1 Replica (Worker Node Only)
|
||||
```bash
|
||||
# 1. Deploy the ConfigMaps (content and nginx config)
|
||||
kubectl apply -f nginx-nodeport-configmaps.yaml
|
||||
|
||||
# 2. Deploy the nginx application
|
||||
kubectl apply -f nginx-nodeport-deployment.yaml
|
||||
|
||||
# 3. Create the NodePort service
|
||||
kubectl apply -f nginx-nodeport-service.yaml
|
||||
|
||||
# 4. Wait for deployment to be ready
|
||||
kubectl wait --for=condition=ready pod -l app=nginx-nodeport --timeout=60s
|
||||
|
||||
# 5. Get IPv6 address of the ACTUAL node where your pod is running
|
||||
POD_NODE=$(kubectl get pods -l app=nginx-nodeport -o jsonpath='{.items[0].spec.nodeName}')
|
||||
NODE_IPV6=$(kubectl get node $POD_NODE -o jsonpath='{range .status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}' | grep -E '^[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+$' | head -1)
|
||||
echo "Your website is accessible at: http://[$NODE_IPV6]:30091"
|
||||
echo "NOTE: Pod is running on node: $POD_NODE"
|
||||
|
||||
# 6. Test the website
|
||||
curl -6 "http://[$NODE_IPV6]:30091/"
|
||||
cd myceliumcloud-examples/examples/nginx-nodeport
|
||||
./deploy-and-test.sh
|
||||
```
|
||||
|
||||
**Alternative: Use the automated script for IPv6 discovery:**
|
||||
**What happens:**
|
||||
- ✅ Deploys 1 pod replica on a **worker node** (not master)
|
||||
- ✅ Service accessible on 1 worker node only
|
||||
- ✅ Easy to understand and test
|
||||
- ✅ Shows you the single access URL
|
||||
|
||||
### Step 2: Scale to 3 Worker Replicas (Production Ready!)
|
||||
```bash
|
||||
# Automatically discover IPv6 addresses and update content
|
||||
./update-content.sh
|
||||
```
|
||||
|
||||
**⚠️ Important: Worker Node vs Master Node Access**
|
||||
Unlike hostNetwork (which can be accessed from any node), NodePort services are **limited to the nodes where your pods are actually running**. Always check:
|
||||
```bash
|
||||
# Check which node your pod is running on
|
||||
kubectl get pods -l app=nginx-nodeport -o wide
|
||||
|
||||
# Get the IPv6 of the correct WORKER node (not master)
|
||||
NODE_IPV6=$(kubectl get nodes -l kubernetes.io/role!=master -o jsonpath='{range .items[0].status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}' | grep -E '^[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+:[0-9a-f]+$' | head -1)
|
||||
```
|
||||
|
||||
**Expected Result:** You'll see a professional website with NodePort security branding and IPv6 address detection.
|
||||
|
||||
## 📋 What You'll Learn
|
||||
|
||||
- ✅ **Security-First IPv6 Web Hosting** - Enhanced network isolation
|
||||
- ✅ **Standard Kubernetes Service Patterns** - Industry best practices
|
||||
- ✅ **NodePort with External Traffic Policy** - Preserves IPv6 source IP
|
||||
- ✅ **ConfigMap-based Content Management** - Dynamic updates
|
||||
- ✅ **Production nginx Configuration** - Dual-stack (IPv4/IPv6) support
|
||||
- ✅ **Resource Management** - CPU/memory limits and health checks
|
||||
- ✅ **Security Comparison** - Understanding hostNetwork vs standard networking
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
This example uses **separate configuration files** for better organization and security:
|
||||
|
||||
### Component Overview
|
||||
```
|
||||
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
||||
│ Mycelium │ │ NodePort │ │ nginx Pod │
|
||||
│ IPv6 Network │───▶│ Service │───▶│ (Isolated) │
|
||||
│ :30090 │ │ :8080 │ │ :8080 │
|
||||
└─────────────────┘ └──────────────────┘ └─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ ConfigMaps │
|
||||
│ • HTML Content │
|
||||
│ • nginx Config │
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
### Network Flow (NodePort Pattern)
|
||||
```
|
||||
User with Mycelium
|
||||
↓
|
||||
http://[worker-node-mycelium-ipv6]:30091
|
||||
↓
|
||||
Worker Node (kube-proxy forwards)
|
||||
↓
|
||||
NodePort 30091 → Service port 8080
|
||||
↓
|
||||
Kubernetes Service (load balances)
|
||||
↓
|
||||
Pod container port 8080
|
||||
↓
|
||||
nginx → HTML Content
|
||||
```
|
||||
|
||||
### 🚨 Critical: NodePort Access Pattern
|
||||
|
||||
**With NodePort:**
|
||||
- Pods run in **isolated network** (no `hostNetwork`)
|
||||
- Pods do **NOT** have direct Mycelium IPv6 addresses
|
||||
- Access via **worker node's Mycelium IPv6** + NodePort
|
||||
- Service is accessible on **all worker nodes** at port 30091
|
||||
- kube-proxy forwards traffic: `node:30091 → service:8080 → pod:8080`
|
||||
|
||||
**Comparison:**
|
||||
- **hostNetwork (nginx-mycelium)**: Pod gets host's Mycelium IP → Access: `http://[pod-ip]:8080`
|
||||
- **NodePort (nginx-nodeport)**: Pod isolated → Access: `http://[node-ip]:30091`
|
||||
|
||||
**Getting the right IPv6:**
|
||||
```bash
|
||||
# Get worker node Mycelium IPv6 addresses
|
||||
kubectl get nodes -o wide
|
||||
|
||||
# OR use the fetch-ip.sh script
|
||||
../../scripts/fetch-ip.sh
|
||||
```
|
||||
|
||||
### Key Security Improvements
|
||||
- **Pod Isolation**: No `hostNetwork` access, pods run in isolated network namespace
|
||||
- **Standard Service Patterns**: Uses typical Kubernetes service discovery
|
||||
- **External Traffic Policy**: `Local` preserves IPv6 source IP for logging
|
||||
- **Resource Limits**: Prevents resource exhaustion attacks
|
||||
- **Health Checks**: Liveness and readiness probes for reliability
|
||||
|
||||
## 📄 Configuration Files
|
||||
|
||||
### 1. nginx-nodeport-deployment.yaml
|
||||
**Secure pod deployment without hostNetwork:**
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
hostNetwork: false # Key security improvement
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:alpine
|
||||
ports:
|
||||
- containerPort: 8080 # No hostPort needed
|
||||
resources:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "128Mi"
|
||||
cpu: "200m"
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8080
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8080
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Creates 1 pod with nginx and custom website content
|
||||
- Uses **standard pod networking** (no direct host access)
|
||||
- Includes **resource limits** and **health checks**
|
||||
- Uses ConfigMaps for **dynamic content management**
|
||||
- **Dual-stack nginx** (IPv4 + IPv6) configuration
|
||||
|
||||
### 2. nginx-nodeport-service.yaml
|
||||
**NodePort service with IPv6 source IP preservation:**
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
type: NodePort # ← Service type is NodePort
|
||||
externalTrafficPolicy: Local # Preserves IPv6 source IP
|
||||
ports:
|
||||
- port: 8080
|
||||
targetPort: 8080
|
||||
nodePort: 30091 # ← Explicitly set to 30091 (avoiding conflict with nginx-mycelium's 30090)
|
||||
protocol: TCP
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Exposes nginx on **NodePort 30091** on all worker nodes
|
||||
- **Preserves IPv6 source IP** for logging and security
|
||||
- Uses **standard Kubernetes service** patterns (not LoadBalancer)
|
||||
- Provides **load balancing** across pod replicas (when scaled)
|
||||
- Access via: `http://[worker-node-mycelium-ipv6]:30091/`
|
||||
|
||||
**NodePort Range:** Kubernetes NodePorts use range 30000-32767 by default
|
||||
|
||||
### 3. nginx-nodeport-configmaps.yaml
|
||||
**Content and nginx configuration:**
|
||||
|
||||
**HTML Content:** Professional website with NodePort security branding
|
||||
**nginx Configuration:** Dual-stack (IPv4/IPv6) with health endpoint
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 8080;
|
||||
listen [::]:8080 ipv6only=on; # IPv6 support
|
||||
server_name _;
|
||||
|
||||
location / {
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
}
|
||||
|
||||
location /health {
|
||||
access_log off;
|
||||
return 200 "healthy\n";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🔍 Verification and Testing
|
||||
|
||||
### Check pod status
|
||||
```bash
|
||||
kubectl get pods -l app=nginx-nodeport
|
||||
kubectl describe pod -l app=nginx-nodeport
|
||||
```
|
||||
|
||||
### Check service configuration
|
||||
```bash
|
||||
kubectl get svc nginx-nodeport-service -o yaml
|
||||
kubectl get endpoints nginx-nodeport-service
|
||||
```
|
||||
|
||||
### Test nginx configuration
|
||||
```bash
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
kubectl exec $POD_NAME -- nginx -t
|
||||
kubectl exec $POD_NAME -- nginx -s reload
|
||||
```
|
||||
|
||||
### Test health endpoint
|
||||
```bash
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
kubectl exec $POD_NAME -- curl -s http://localhost:8080/health
|
||||
```
|
||||
|
||||
### Test website content
|
||||
```bash
|
||||
# Get node IPv6 address
|
||||
NODE_IPV6=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
|
||||
|
||||
# Test from outside the cluster
|
||||
curl -6 "http://[$NODE_IPV6]:30091/"
|
||||
curl -6 "http://[$NODE_IPV6]:30091/health"
|
||||
```
|
||||
|
||||
### Verify ConfigMaps are mounted
|
||||
```bash
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
kubectl exec $POD_NAME -- ls -la /usr/share/nginx/html/
|
||||
kubectl exec $POD_NAME -- cat /usr/share/nginx/html/index.html | head -10
|
||||
```
|
||||
|
||||
## 🔄 Updating Content
|
||||
|
||||
### Update website content
|
||||
```bash
|
||||
# Create new HTML file
|
||||
cat > new-index.html << 'EOF'
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Updated Site</title></head>
|
||||
<body>
|
||||
<h1>Content Updated!</h1>
|
||||
<p>New content deployed at $(date)</p>
|
||||
</body>
|
||||
</html>
|
||||
EOF
|
||||
|
||||
# Update ConfigMap
|
||||
kubectl create configmap nginx-nodeport-content \
|
||||
--from-file=index.html=new-index.html \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
|
||||
# Reload nginx to pick up changes
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
kubectl exec $POD_NAME -- nginx -s reload
|
||||
|
||||
echo "✅ Content updated! Access at: http://[$NODE_IPV6]:30091"
|
||||
```
|
||||
|
||||
### Update nginx configuration
|
||||
```bash
|
||||
# Create custom nginx config
|
||||
cat > custom-nginx.conf << 'EOF'
|
||||
server {
|
||||
listen 8080;
|
||||
listen [::]:8080 ipv6only=on;
|
||||
server_name my-site.com;
|
||||
|
||||
# Custom logging
|
||||
access_log /var/log/nginx/access.log;
|
||||
error_log /var/log/nginx/error.log;
|
||||
|
||||
location / {
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
try_files $uri $uri/ =404;
|
||||
}
|
||||
|
||||
location /health {
|
||||
access_log off;
|
||||
return 200 "healthy\n";
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Update ConfigMap
|
||||
kubectl create configmap nginx-nodeport-nginx-config \
|
||||
--from-file=default.conf=custom-nginx.conf \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
|
||||
# Reload nginx
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
kubectl exec $POD_NAME -- nginx -s reload
|
||||
```
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### Pod won't start
|
||||
```bash
|
||||
# Check pod status and events
|
||||
kubectl describe pod -l app=nginx-nodeport
|
||||
|
||||
# Check pod logs
|
||||
kubectl logs -f deployment/nginx-nodeport
|
||||
|
||||
# Check resource availability
|
||||
kubectl top nodes
|
||||
kubectl describe nodes
|
||||
```
|
||||
|
||||
### Service not accessible
|
||||
```bash
|
||||
# Verify service exists and has endpoints
|
||||
kubectl get svc nginx-nodeport-service
|
||||
kubectl get endpoints nginx-nodeport-service
|
||||
|
||||
# Check if pod is ready
|
||||
kubectl get pods -l app=nginx-nodeport -o wide
|
||||
|
||||
# Verify NodePort is open
|
||||
netstat -tulpn | grep 30091
|
||||
```
|
||||
|
||||
### IPv6 connectivity issues
|
||||
```bash
|
||||
# Test IPv6 on the node
|
||||
ping6 -c 3 $NODE_IPV6
|
||||
|
||||
# Check nginx is listening on IPv6
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
kubectl exec $POD_NAME -- netstat -tuln | grep 8080
|
||||
|
||||
# Verify nginx configuration
|
||||
kubectl exec $POD_NAME -- cat /etc/nginx/conf.d/default.conf
|
||||
```
|
||||
|
||||
### Performance issues
|
||||
```bash
|
||||
# Check resource usage
|
||||
kubectl top pods -l app=nginx-nodeport
|
||||
|
||||
# Check nginx status and connections
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
kubectl exec $POD_NAME -- ps aux | grep nginx
|
||||
kubectl exec $POD_NAME -- netstat -an | grep :8080 | wc -l
|
||||
```
|
||||
|
||||
## 🗑️ Cleanup
|
||||
|
||||
```bash
|
||||
# Remove all resources
|
||||
kubectl delete -f nginx-nodeport-deployment.yaml -f nginx-nodeport-service.yaml
|
||||
|
||||
# Remove ConfigMaps
|
||||
kubectl delete configmap nginx-nodeport-content nginx-nodeport-nginx-config
|
||||
|
||||
# Verify cleanup
|
||||
kubectl get all -l app=nginx-nodeport # Should return nothing
|
||||
```
|
||||
|
||||
## 📊 Nginx Deployment Variants Comparison
|
||||
|
||||
### Complete Overview: 4 Ways to Deploy Nginx on Mycelium Cloud
|
||||
|
||||
| Feature | hostNetwork | NodePort | LoadBalancer | Ingress |
|
||||
|---------|-------------|----------|--------------|---------|
|
||||
| **Example** | nginx-mycelium | nginx-nodeport | (Future) | (Future) |
|
||||
| **Pod Network** | ❌ Host | ✅ Isolated | ✅ Isolated | ✅ Isolated |
|
||||
| **Security** | ⚠️ Low | ✅ Medium | ✅ Medium | ✅ High |
|
||||
| **Access Pattern** | `[pod-ip]:8080` | `[node-ip]:30091` | `[lb-ip]:80` | `domain.com` |
|
||||
| **Port Range** | Any | 30000-32767 | 80, 443 | 80, 443 |
|
||||
| **Mycelium Access** | Direct pod | Via node | Via LB | Via ingress |
|
||||
| **Use Case** | Demo/POC | Testing/Dev | Production | Web apps |
|
||||
| **Scalability** | ❌ Limited | ✅ Good | ✅ Excellent | ✅ Excellent |
|
||||
| **Load Balancing** | Manual | K8s Service | Cloud LB | Ingress controller |
|
||||
| **SSL/TLS** | Manual | Manual | Possible | Native |
|
||||
| **DNS Support** | No | No | Possible | Yes |
|
||||
| **Production Ready** | ⚠️ No | ✅ Yes | ✅ Yes | ✅ Yes |
|
||||
|
||||
### Key Differences: hostNetwork vs NodePort
|
||||
|
||||
| Feature | hostNetwork (nginx-mycelium) | NodePort (nginx-nodeport) |
|
||||
|---------|------------------------------|---------------------------|
|
||||
| **Security** | ⚠️ Direct host access | ✅ Isolated pod network |
|
||||
| **Network Isolation** | ❌ Uses host interfaces | ✅ Pod namespace isolation |
|
||||
| **Port Conflicts** | ❌ Limited by host ports | ✅ No port conflicts |
|
||||
| **Debugging** | 🔄 Host-level tools | ✅ Standard K8s patterns |
|
||||
| **Monitoring** | 🔄 Host monitoring | ✅ Pod-level monitoring |
|
||||
| **Scalability** | ❌ Single instance per node | ✅ Multiple replicas |
|
||||
| **Production Ready** | ⚠️ Demo/POC only | ✅ Production patterns |
|
||||
| **Access URL** | `http://[pod-mycelium-ip]:8080` | `http://[node-mycelium-ip]:30091` |
|
||||
| **Port Used** | 8080 (+ NodePort 30090) | NodePort 30091 |
|
||||
|
||||
## 📈 Scaling and High Availability
|
||||
|
||||
### Scale to multiple replicas
|
||||
```bash
|
||||
# Scale deployment
|
||||
# Scale to 3 replicas (all on worker nodes)
|
||||
kubectl scale deployment nginx-nodeport --replicas=3
|
||||
|
||||
# Verify scaling
|
||||
kubectl get pods -l app=nginx-nodeport
|
||||
# Update content to show ALL accessible worker nodes
|
||||
./update-content-many.sh
|
||||
|
||||
# Check load balancing
|
||||
NODE_IPV6=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
|
||||
for i in {1..5}; do
|
||||
curl -6 "http://[$NODE_IPV6]:30091/" | grep -o "nginx-nodeport-[a-z0-9]*-[a-z0-9]*" | head -1
|
||||
done
|
||||
# Apply the changes
|
||||
kubectl rollout restart deployment/nginx-nodeport
|
||||
```
|
||||
|
||||
### Add resource limits for better performance
|
||||
```yaml
|
||||
# Update deployment with enhanced resources
|
||||
resources:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "200m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "500m"
|
||||
**What happens:**
|
||||
- ✅ Service now accessible on 3 worker nodes
|
||||
- ✅ Automatic load distribution across workers
|
||||
- ✅ Shows you all 3 working URLs
|
||||
- ✅ Perfect for production use
|
||||
|
||||
**Example 3-worker output:**
|
||||
```
|
||||
🌐 Access Information:
|
||||
• URL 1: http://[437:9faf:1f1a:e2b1:ff0f:1fd9:7fd5:1095]:30091 (Worker Node)
|
||||
• URL 2: http://[552:5984:2d97:72dc:ff0f:39ef:6ec:a48c]:30091 (Worker Node)
|
||||
• URL 3: http://[5c3:a162:45ab:6c53:ff0f:8c55:36b0:24af]:30091 (Worker Node)
|
||||
|
||||
## 🎯 Best Practices
|
||||
|
||||
1. **Security First**: Always prefer standard pod networking over hostNetwork
|
||||
2. **Resource Management**: Set appropriate requests and limits
|
||||
3. **Health Checks**: Use liveness and readiness probes
|
||||
4. **Monitoring**: Implement proper logging and metrics collection
|
||||
5. **Updates**: Use rolling updates for zero-downtime deployments
|
||||
6. **Configuration**: Store configuration in ConfigMaps for flexibility
|
||||
|
||||
## 🔄 Complete NodePort Flow Explained
|
||||
|
||||
### Understanding the Full Request Path
|
||||
|
||||
✅ All 3 worker replicas working! Anyone can access from anywhere with Mycelium!
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Step 1: User with Mycelium Network │
|
||||
│ • Has Mycelium client running │
|
||||
│ • Can reach Mycelium IPv6 addresses globally │
|
||||
│ • Accesses: http://[worker-node-mycelium-ipv6]:30091 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Step 2: Mycelium Network Routes to Worker Node │
|
||||
│ • Peer-to-peer IPv6 routing │
|
||||
│ • No traditional DNS needed │
|
||||
│ • Direct encrypted connection │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Step 3: Worker Node Receives Request on Port 30091 │
|
||||
│ • kube-proxy listens on NodePort 30091 │
|
||||
│ • Forwards to Service cluster IP │
|
||||
│ • Preserves source IPv6 (externalTrafficPolicy: Local) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Step 4: Kubernetes Service Load Balances │
|
||||
│ • Service receives on port 8080 │
|
||||
│ • Selects backend pod (app=nginx-nodeport) │
|
||||
│ • Routes to targetPort 8080 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Step 5: nginx Pod Processes Request │
|
||||
│ • Pod in isolated network namespace │
|
||||
│ • nginx listens on container port 8080 │
|
||||
│ • Serves HTML from ConfigMap │
|
||||
│ • Returns response │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Step 6: Response Returns to User │
|
||||
│ • Same path in reverse │
|
||||
│ • User sees website in browser │
|
||||
│ • Access logs show original IPv6 (source IP preserved) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Why NodePort is Better for Production
|
||||
|
||||
**Security Benefits:**
|
||||
1. Pods cannot access host network interfaces
|
||||
2. Network policies can restrict pod-to-pod traffic
|
||||
3. Resource limits prevent resource exhaustion
|
||||
4. Health checks ensure reliability
|
||||
|
||||
**Operational Benefits:**
|
||||
1. Standard Kubernetes debugging tools work
|
||||
2. Can scale horizontally (multiple replicas)
|
||||
3. Service provides automatic load balancing
|
||||
4. Compatible with monitoring stacks (Prometheus, etc.)
|
||||
5. Can evolve to LoadBalancer or Ingress later
|
||||
|
||||
**Mycelium Integration:**
|
||||
- Worker nodes have Mycelium IPv6 addresses
|
||||
- NodePort makes service accessible via these IPs
|
||||
- No need for traditional cloud load balancers
|
||||
- Perfect for decentralized web hosting
|
||||
|
||||
## <20> Learning Outcomes
|
||||
|
||||
After completing this example, you understand:
|
||||
- **NodePort Services** - Kubernetes service exposure patterns
|
||||
- **Network Security** - Pod isolation vs hostNetwork trade-offs
|
||||
- **Production Patterns** - Resource management and health checks
|
||||
- **IPv6 Networking** - Dual-stack nginx configuration
|
||||
- **ConfigMap Management** - Dynamic content and configuration updates
|
||||
- **Service Discovery** - Kubernetes service networking
|
||||
- **Load Balancing** - Service-level load distribution
|
||||
|
||||
## 🚀 Next Steps
|
||||
|
||||
- **Multi-Replica Deployment**: Scale to 3+ replicas for high availability
|
||||
- **LoadBalancer Service**: Move to cloud LoadBalancer for production
|
||||
- **Ingress Controller**: Implement advanced routing and SSL termination
|
||||
- **Monitoring**: Add Prometheus metrics and Grafana dashboards
|
||||
- **SSL/TLS**: Implement HTTPS with Let's Encrypt certificates
|
||||
|
||||
---
|
||||
|
||||
**Success Criteria**: You'll know everything is working when:
|
||||
- ✅ `kubectl get pods` shows nginx-nodeport pod in "Running" status
|
||||
- ✅ `kubectl get svc` shows nginx-nodeport-service with NodePort 30091
|
||||
- ✅ `curl -6 "http://[$NODE_IPV6]:30091"` returns your secure website
|
||||
- ✅ Website displays "NODEPORT SECURE" and "ENHANCED SECURITY" badges
|
||||
- ✅ `kubectl logs deployment/nginx-nodeport` shows nginx access logs
|
||||
## 🎯 Learning Journey
|
||||
|
||||
**Access URL**: `http://[NODE-IPV6]:30091` (replace NODE-IPV6 with your node's IPv6 address)
|
||||
1. **Start Simple**: Deploy 1 worker replica to understand the basics
|
||||
2. **Scale Up**: Add 2 more worker replicas to see load distribution
|
||||
3. **Production Ready**: Learn how externalTrafficPolicy: Local works on workers
|
||||
4. **Global Access**: Understand Mycelium IPv6 networking from worker nodes
|
||||
|
||||
---
|
||||
|
||||
## 📚 Available Scripts
|
||||
|
||||
### 1. `deploy-and-test.sh` - **Start Here** ⭐
|
||||
**Deploys 1 replica and shows you the basics**
|
||||
|
||||
```bash
|
||||
./deploy-and-test.sh
|
||||
```
|
||||
|
||||
Does everything: deploy 1 worker replica, test, configure, and show results.
|
||||
|
||||
### 2. `update-content-many.sh` - **For Multiple Worker Replicas** 🚀
|
||||
**Updates content to show all worker replica URLs**
|
||||
|
||||
```bash
|
||||
# After scaling to 3 worker replicas
|
||||
./update-content-many.sh
|
||||
kubectl rollout restart deployment/nginx-nodeport
|
||||
```
|
||||
|
||||
This script properly handles multiple worker replicas and shows all accessible URLs.
|
||||
|
||||
### 3. `update-content-single.sh` - **For Single Worker Replica** 📝
|
||||
**For single worker replica deployments** - updates content for 1 worker pod
|
||||
|
||||
### 4. `debug-networking.sh` - **Troubleshooting** 🔧
|
||||
**Diagnose networking issues and verify all URLs are working**
|
||||
|
||||
```bash
|
||||
./debug-networking.sh
|
||||
```
|
||||
|
||||
Tests connectivity to all nodes and shows which URLs are working.
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Manual Step-by-Step (Alternative)
|
||||
|
||||
If you want to do everything manually to learn the details:
|
||||
|
||||
```bash
|
||||
# Step 1: Deploy resources
|
||||
kubectl apply -f nginx-nodeport-configmaps.yaml
|
||||
kubectl apply -f nginx-nodeport-deployment.yaml
|
||||
kubectl apply -f nginx-nodeport-service.yaml
|
||||
|
||||
# Step 2: Wait for ready
|
||||
kubectl wait --for=condition=ready pod -l app=nginx-nodeport --timeout=60s
|
||||
|
||||
# Step 3: Test 1 worker replica
|
||||
curl -6 "http://[your-worker-node-ipv6]:30091/"
|
||||
|
||||
# Step 4: Scale to 3 worker replicas
|
||||
kubectl scale deployment nginx-nodeport --replicas=3
|
||||
|
||||
# Step 5: Update content for all worker replicas
|
||||
./update-content-many.sh
|
||||
kubectl rollout restart deployment/nginx-nodeport
|
||||
|
||||
# Step 6: Verify all URLs work
|
||||
./debug-networking.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Architecture
|
||||
|
||||
```
|
||||
User (with Mycelium)
|
||||
↓
|
||||
http://[worker-node-mycelium-ipv6]:30091
|
||||
↓
|
||||
kube-proxy (dual-stack enabled)
|
||||
↓
|
||||
Kubernetes Service (IPv4 + IPv6)
|
||||
↓
|
||||
Pod (container port 8080) ← 1-3 pods on worker nodes only
|
||||
↓
|
||||
nginx → HTML
|
||||
```
|
||||
|
||||
**Key Points:**
|
||||
- Service type: **NodePort** (not LoadBalancer)
|
||||
- IP families: **Dual-stack (IPv4 + IPv6)** ⭐ Critical
|
||||
- Pod network: **Isolated** (no hostNetwork)
|
||||
- Traffic policy: **Local** (preserves source IP)
|
||||
- Port: **30091** (NodePort)
|
||||
- Node preference: **Worker nodes only** (not master nodes)
|
||||
- Replicas: **Start with 1, scale to 3 for production**
|
||||
|
||||
---
|
||||
|
||||
## 🌐 Understanding the Worker-Only Scaling
|
||||
|
||||
### 1 Worker Replica (Learning Phase)
|
||||
- **Service accessible on**: 1 worker node (where pod runs)
|
||||
- **URLs shown**: 1 URL
|
||||
- **Use case**: Learning, testing, development
|
||||
|
||||
### 3 Worker Replicas (Production Phase)
|
||||
- **Service accessible on**: 3 worker nodes (where pods run)
|
||||
- **URLs shown**: 3 URLs
|
||||
- **Use case**: Production, high availability, global access
|
||||
- **Benefit**: Anyone can access from any of the 3 worker URLs
|
||||
- **Architecture**: Keeps master nodes free for control plane operations
|
||||
|
||||
---
|
||||
|
||||
## 📁 Files in This Directory
|
||||
|
||||
### Configuration
|
||||
- `nginx-nodeport-deployment.yaml` - Pod configuration (starts with 1 replica, worker-friendly)
|
||||
- `nginx-nodeport-deployment-worker-only.yaml` - Worker-only deployment with node affinity
|
||||
- `nginx-nodeport-service.yaml` - NodePort service (dual-stack)
|
||||
- `nginx-nodeport-configmaps.yaml` - HTML content + nginx config
|
||||
|
||||
### Scripts
|
||||
- `deploy-and-test.sh` - ⭐ **Start here** (deploy 1 worker replica + test)
|
||||
- `update-content-many.sh` - 🚀 **For scaling** (shows all worker replica URLs)
|
||||
- `update-content-single.sh` - For single worker replica deployments
|
||||
- `debug-networking.sh` - 🔧 **Troubleshooting** (tests all URLs)
|
||||
- `test-nodeport-ipv6.sh` - Testing and validation
|
||||
|
||||
### Documentation
|
||||
- `README.md` - This guide
|
||||
- `nginx-nodeport.md` - Complete technical guide
|
||||
- `TROUBLESHOOTING.md` - Common issues & solutions
|
||||
|
||||
---
|
||||
|
||||
## ✅ Success Indicators
|
||||
|
||||
**1 Worker Replica (Working):**
|
||||
- ✅ Service type: `NodePort`
|
||||
- ✅ NodePort: `30091`
|
||||
- ✅ IP families: `["IPv4","IPv6"]`
|
||||
- ✅ Pod status: `Running`
|
||||
- ✅ Node type: `Worker` (not Master)
|
||||
- ✅ Test output: `✅ External Mycelium IPv6 connectivity is working!`
|
||||
- ✅ Shows 1 accessible URL
|
||||
|
||||
**3 Worker Replicas (Production Ready):**
|
||||
- ✅ All above indicators
|
||||
- ✅ 3 pods running across 3 **worker nodes**
|
||||
- ✅ Shows 3 accessible URLs
|
||||
- ✅ All 3 worker URLs respond successfully
|
||||
- ✅ Debug script confirms: `✅ HTTP connectivity: WORKING` for all
|
||||
|
||||
---
|
||||
|
||||
## 🚨 Prerequisites
|
||||
|
||||
1. **kubectl** installed and configured
|
||||
2. **Mycelium Cloud cluster** with at least 1 worker node
|
||||
3. **Mycelium installed** on your local machine (for testing)
|
||||
4. **IPv6 connectivity** (Mycelium provides this)
|
||||
|
||||
---
|
||||
|
||||
## 🆘 Common Questions
|
||||
|
||||
**Q: Why start with 1 worker replica?**
|
||||
A: It's easier to understand and debug. You can see exactly which worker node is serving the content.
|
||||
|
||||
**Q: Why only worker nodes?**
|
||||
A: Production best practice - keep master nodes free for Kubernetes control plane operations.
|
||||
|
||||
**Q: Why scale to 3 worker replicas?**
|
||||
A: This makes the service accessible from all 3 worker nodes, providing global availability and load distribution.
|
||||
|
||||
**Q: What does "externalTrafficPolicy: Local" do on workers?**
|
||||
A: It ensures the service is only accessible on worker nodes where pods are running, preserving the source IP and improving performance.
|
||||
|
||||
**Q: How do I know if scaling worked?**
|
||||
A: Use `update-content-many.sh` - it will show you all 3 accessible worker URLs. Then run `./debug-networking.sh` to confirm all are working.
|
||||
|
||||
---
|
||||
|
||||
## 🆘 Getting Help
|
||||
|
||||
**If only 1 of 3 URLs works:**
|
||||
- Run: `./debug-networking.sh` to identify which URLs work
|
||||
- Check: Are all URLs actually on worker nodes (not masters)?
|
||||
- Fix: Use `nginx-nodeport-deployment-worker-only.yaml` for proper worker placement
|
||||
- Then: `./update-content-many.sh` and restart deployment
|
||||
|
||||
**Troubleshooting:**
|
||||
- Check worker pod locations: `kubectl get pods -l app=nginx-nodeport -o wide`
|
||||
- Test all URLs: `./debug-networking.sh`
|
||||
- Verify node types: `kubectl get nodes -o wide`
|
||||
- Check Mycelium status on affected worker nodes
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Success!
|
||||
|
||||
Once you complete both steps, you'll have:
|
||||
- ✅ **Learned** how NodePort scaling works on worker nodes
|
||||
- ✅ **Deployed** a production-ready, multi-worker replica service
|
||||
- ✅ **Understood** Mycelium IPv6 networking from workers
|
||||
- ✅ **Tested** global accessibility from all worker nodes
|
||||
- ✅ **Followed** production best practices (workers vs masters)
|
||||
|
||||
**You're now ready for production workloads on Mycelium Cloud!** 🚀
|
||||
Reference in New Issue
Block a user