docs: Add IPv6 dual-stack documentation and debug script for nginx-nodeport
This commit is contained in:
mik-tf
242
examples/nginx-nodeport/IPV6_FIX.md
Normal file
242
examples/nginx-nodeport/IPV6_FIX.md
Normal file
@@ -0,0 +1,242 @@
|
||||
# 🚨 CRITICAL: IPv6 Configuration Fix Required
|
||||
|
||||
## The Problem
|
||||
|
||||
When initially deploying NodePort services on Mycelium Cloud, you may encounter a situation where:
|
||||
- ✅ `ping -6` works (ICMP reaches the node)
|
||||
- ❌ `curl -6 http://[ipv6]:30091/` fails (HTTP service unreachable)
|
||||
|
||||
## Root Cause
|
||||
|
||||
**IPv4-only service configuration!**
|
||||
|
||||
By default, Kubernetes services created without explicit IP family configuration are often IPv4-only, even when nodes have IPv6 addresses.
|
||||
|
||||
### Symptom
|
||||
```bash
|
||||
# Service shows IPv4-only
|
||||
$ kubectl get svc nginx-nodeport-service -o json | grep ipFamily
|
||||
"ipFamilyPolicy": "SingleStack",
|
||||
"ipFamilies": ["IPv4"],
|
||||
```
|
||||
|
||||
### Result
|
||||
- Services only listen on IPv4 addresses
|
||||
- IPv6 Mycelium addresses are "reachable" (ping works) but HTTP fails
|
||||
- nginx itself is fine (listens on dual-stack)
|
||||
- **kube-proxy won't forward IPv6 traffic to the service**
|
||||
|
||||
---
|
||||
|
||||
## The Fix ⭐
|
||||
|
||||
Update the Service YAML to explicitly support dual-stack:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: nginx-nodeport-service
|
||||
spec:
|
||||
type: NodePort
|
||||
externalTrafficPolicy: Local
|
||||
ipFamilies: # ← ADD THESE 3 LINES
|
||||
- IPv4
|
||||
- IPv6
|
||||
ipFamilyPolicy: RequireDualStack # ← ADD THIS
|
||||
selector:
|
||||
app: nginx-nodeport
|
||||
ports:
|
||||
- name: http
|
||||
port: 8080
|
||||
targetPort: 8080
|
||||
nodePort: 30091
|
||||
protocol: TCP
|
||||
```
|
||||
|
||||
### Apply the fix:
|
||||
```bash
|
||||
kubectl apply -f nginx-nodeport-service.yaml
|
||||
```
|
||||
|
||||
### Verify the fix:
|
||||
```bash
|
||||
# Should now show dual-stack
|
||||
$ kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilies}'
|
||||
["IPv4","IPv6"]
|
||||
|
||||
$ kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilyPolicy}'
|
||||
RequireDualStack
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Why This Happens
|
||||
|
||||
1. **Legacy default**: Many Kubernetes setups default to IPv4-only
|
||||
2. **Service doesn't inherit node IPs**: Service has its own ClusterIP
|
||||
3. **kube-proxy needs explicit instruction**: Must know to create IPv6 rules
|
||||
4. **IP family policy**: Controls whether service listens on IPv4, IPv6, or both
|
||||
|
||||
### Without dual-stack:
|
||||
```
|
||||
User → Mycelium IPv6 → Node port 30091
|
||||
↓
|
||||
kube-proxy
|
||||
↓
|
||||
IPv4 service ← BOTTLENECK
|
||||
↓
|
||||
Pod responds
|
||||
```
|
||||
|
||||
### With dual-stack:
|
||||
```
|
||||
User → Mycelium IPv6 → Node port 30091
|
||||
↓
|
||||
kube-proxy
|
||||
↓
|
||||
IPv6 + IPv4 service ← WORKS!
|
||||
↓
|
||||
Pod responds
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing the Fix
|
||||
|
||||
### Before fix:
|
||||
```bash
|
||||
$ curl -6 "http://[552:5984:2d97:72dc:ff0f:39ef:6ec:a48c]:30091/"
|
||||
curl: (7) Failed to connect to ... Connection refused
|
||||
```
|
||||
|
||||
### After fix:
|
||||
```bash
|
||||
$ curl -6 "http://[552:5984:2d97:72dc:ff0f:39ef:6ec:a48c]:30091/"
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<title>Mycelium Cloud - Nginx NodePort Website</title>
|
||||
...
|
||||
```
|
||||
|
||||
### Automated test:
|
||||
```bash
|
||||
cd myceliumcloud-examples/examples/nginx-nodeport
|
||||
./test-nodeport-ipv6.sh
|
||||
|
||||
# Should now show:
|
||||
# ✅ External Mycelium IPv6 connectivity is working!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Other Affected Variants
|
||||
|
||||
This IP family configuration is critical for:
|
||||
|
||||
- ✅ **NodePort** - This guide
|
||||
- ✅ **LoadBalancer** - Will need same configuration (future example)
|
||||
- ✅ **Ingress** - Will need same configuration (future example)
|
||||
- ❌ **hostNetwork** - NOT affected (directly uses host network stack)
|
||||
|
||||
---
|
||||
|
||||
## Kubernetes Version Compatibility
|
||||
|
||||
### Kubernetes 1.20+ (Recommended)
|
||||
- Full dual-stack support
|
||||
- `RequireDualStack` policy available
|
||||
- Complete IPv4/IPv6 feature set
|
||||
|
||||
### Kubernetes 1.19 and earlier
|
||||
- Limited dual-stack support
|
||||
- May need `ipFamily: IPv6` instead of `RequireDualStack`
|
||||
- Some features may not work
|
||||
|
||||
**Check your version:**
|
||||
```bash
|
||||
kubectl version --short
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Service configuration checklist:
|
||||
- [ ] `type: NodePort` ✓
|
||||
- [ ] `nodePort: 30091` ✓
|
||||
- [ ] `externalTrafficPolicy: Local` ✓
|
||||
- [ ] `ipFamilies: [IPv4, IPv6]` ⭐ Critical for Mycelium
|
||||
- [ ] `ipFamilyPolicy: RequireDualStack` ⭐ Critical for Mycelium
|
||||
|
||||
### Test sequence:
|
||||
```bash
|
||||
# 1. Deploy
|
||||
kubectl apply -f nginx-nodeport-configmaps.yaml
|
||||
kubectl apply -f nginx-nodeport-deployment.yaml
|
||||
kubectl apply -f nginx-nodeport-service.yaml
|
||||
|
||||
# 2. Wait
|
||||
kubectl wait --for=condition=ready pod -l app=nginx-nodeport --timeout=60s
|
||||
|
||||
# 3. Check IP families
|
||||
kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilies}'
|
||||
|
||||
# 4. Test
|
||||
NODE_IPV6=$(kubectl get nodes -o jsonpath='{range .items[*]}{range .status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}{end}' | grep ':' | head -1)
|
||||
curl -6 "http://[$NODE_IPV6]:30091/"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Still not working after dual-stack config?
|
||||
|
||||
1. **Check kube-proxy logs:**
|
||||
```bash
|
||||
kubectl logs -n kube-system -l k8s-app=kube-proxy | grep -i error
|
||||
```
|
||||
|
||||
2. **Verify nginx listens on IPv6:**
|
||||
```bash
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
kubectl exec $POD_NAME -- netstat -tuln | grep 8080
|
||||
# Should show:
|
||||
# tcp 0 0 0.0.0.0:8080 :::* LISTEN
|
||||
# tcp 0 0 :::8080 :::* LISTEN ← IPv6
|
||||
```
|
||||
|
||||
3. **Check service endpoints:**
|
||||
```bash
|
||||
kubectl get endpoints nginx-nodeport-service
|
||||
```
|
||||
|
||||
4. **Test from within cluster:**
|
||||
```bash
|
||||
kubectl run test --rm -it --image=curlimages/curl -- sh -c "curl http://$SERVICE_IP:8080"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**For Mycelium Cloud NodePort deployments, ALWAYS include:**
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
ipFamilies:
|
||||
- IPv4
|
||||
- IPv6
|
||||
ipFamilyPolicy: RequireDualStack
|
||||
```
|
||||
|
||||
Without this, services won't be accessible via Mycelium IPv6 addresses, even though the nodes are reachable via ping.
|
||||
|
||||
---
|
||||
|
||||
**Impact:** This is a **deployment-blocking issue** for anyone trying to use NodePort with Mycelium IPv6.
|
||||
|
||||
**Status:** ✅ Fixed in updated `nginx-nodeport-service.yaml`
|
||||
**Related:** [TROUBLESHOOTING.md](TROUBLESHOOTING.md)
|
||||
157
examples/nginx-nodeport/debug-nodeport.sh
Executable file
157
examples/nginx-nodeport/debug-nodeport.sh
Executable file
@@ -0,0 +1,157 @@
|
||||
#!/bin/bash
|
||||
|
||||
# NodePort Connectivity Debug Script
|
||||
# Diagnoses why NodePort 30091 is not accessible despite successful ping
|
||||
|
||||
set -e
|
||||
|
||||
echo "🔍 NodePort Connectivity Diagnosis"
|
||||
echo "===================================="
|
||||
echo ""
|
||||
|
||||
# Colors
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
RED='\033[0;31m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Get pod info
|
||||
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
||||
POD_NODE=$(kubectl get pods -l app=nginx-nodeport -o jsonpath='{.items[0].spec.nodeName}')
|
||||
NODE_IPV6=$(kubectl get node "$POD_NODE" -o jsonpath='{range .status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}' | grep -E '^[0-9a-f]+:[0-9a-f]+:' | head -1)
|
||||
|
||||
echo "📊 Pod Information:"
|
||||
echo " Pod: $POD_NAME"
|
||||
echo " Node: $POD_NODE"
|
||||
echo " Node IPv6: $NODE_IPV6"
|
||||
echo ""
|
||||
|
||||
# Check service
|
||||
echo "🔍 Service Configuration:"
|
||||
kubectl get svc nginx-nodeport-service -o yaml | grep -E "type:|nodePort:|externalTrafficPolicy:|ipFamilies:" | sed 's/^/ /'
|
||||
echo ""
|
||||
|
||||
# Check endpoints
|
||||
echo "🔍 Service Endpoints:"
|
||||
kubectl get endpoints nginx-nodeport-service -o wide
|
||||
echo ""
|
||||
|
||||
# Test from within pod
|
||||
echo "🧪 Test 1: Internal pod access (localhost)"
|
||||
if kubectl exec $POD_NAME -- curl -s -m 5 http://localhost:8080/health > /dev/null 2>&1; then
|
||||
echo -e " ${GREEN}✅ SUCCESS${NC} - nginx responding on localhost:8080"
|
||||
else
|
||||
echo -e " ${RED}❌ FAILED${NC} - nginx not responding internally"
|
||||
exit 1
|
||||
fi
|
||||
echo ""
|
||||
|
||||
# Test service ClusterIP
|
||||
echo "🧪 Test 2: Service ClusterIP access"
|
||||
SERVICE_IP=$(kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.clusterIP}')
|
||||
echo " Service ClusterIP: $SERVICE_IP"
|
||||
|
||||
# Create a test pod to check from
|
||||
echo " Creating test pod..."
|
||||
kubectl run test-curl --image=curlimages/curl:latest --rm -i --restart=Never --command -- sh -c "curl -s -m 5 http://$SERVICE_IP:8080/health" > /tmp/test-result.txt 2>&1 || true
|
||||
|
||||
if grep -q "healthy" /tmp/test-result.txt 2>/dev/null; then
|
||||
echo -e " ${GREEN}✅ SUCCESS${NC} - Service ClusterIP accessible"
|
||||
else
|
||||
echo -e " ${YELLOW}⚠️ FAILED${NC} - Service ClusterIP not accessible"
|
||||
cat /tmp/test-result.txt 2>/dev/null || true
|
||||
fi
|
||||
rm -f /tmp/test-result.txt
|
||||
echo ""
|
||||
|
||||
# Check if kube-proxy is running
|
||||
echo "🔍 kube-proxy Status:"
|
||||
KUBE_PROXY_PODS=$(kubectl get pods -n kube-system -l k8s-app=kube-proxy --no-headers 2>/dev/null | wc -l)
|
||||
if [ "$KUBE_PROXY_PODS" -gt 0 ]; then
|
||||
echo -e " ${GREEN}✅ kube-proxy running${NC} ($KUBE_PROXY_PODS pods)"
|
||||
else
|
||||
echo -e " ${RED}❌ kube-proxy NOT running${NC}"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
# Check node ports
|
||||
echo "🔍 NodePort Listener Check:"
|
||||
echo " Checking if port 30091 is being listened on..."
|
||||
echo ""
|
||||
echo " Note: On the worker node ($POD_NODE), check with:"
|
||||
echo " ssh root@$POD_NODE 'netstat -tuln | grep 30091'"
|
||||
echo " ssh root@$POD_NODE 'ss -tuln | grep 30091'"
|
||||
echo ""
|
||||
|
||||
# Try to access NodePort from inside cluster
|
||||
echo "🧪 Test 3: NodePort access from test pod"
|
||||
echo " Testing: http://[$NODE_IPV6]:30091/"
|
||||
|
||||
kubectl run test-nodeport --image=curlimages/curl:latest --rm -i --restart=Never --command -- sh -c "curl -6 -s -m 10 http://[$NODE_IPV6]:30091/health" > /tmp/nodeport-test.txt 2>&1 || true
|
||||
|
||||
if grep -q "healthy" /tmp/nodeport-test.txt 2>/dev/null; then
|
||||
echo -e " ${GREEN}✅ SUCCESS${NC} - NodePort accessible from within cluster"
|
||||
else
|
||||
echo -e " ${RED}❌ FAILED${NC} - NodePort not accessible from within cluster"
|
||||
echo " Output:"
|
||||
cat /tmp/nodeport-test.txt 2>/dev/null | sed 's/^/ /' || true
|
||||
fi
|
||||
rm -f /tmp/nodeport-test.txt
|
||||
echo ""
|
||||
|
||||
# Check iptables rules (if we can access the node)
|
||||
echo "🔍 Potential Issues:"
|
||||
echo ""
|
||||
echo " 1. IPv6 Support in kube-proxy:"
|
||||
echo " kube-proxy might not be configured for IPv6"
|
||||
echo " Check with: kubectl logs -n kube-system -l k8s-app=kube-proxy | grep -i ipv6"
|
||||
echo ""
|
||||
echo " 2. Firewall on worker node:"
|
||||
echo " Port 30091 might be blocked by firewall"
|
||||
echo " On node: sudo iptables -L -n | grep 30091"
|
||||
echo " On node: sudo ip6tables -L -n | grep 30091"
|
||||
echo ""
|
||||
echo " 3. externalTrafficPolicy: Local:"
|
||||
echo " Service only accessible on the node where pod is running"
|
||||
echo " Pod is on: $POD_NODE"
|
||||
echo " You must use that node's IPv6: $NODE_IPV6"
|
||||
echo ""
|
||||
echo " 4. IPv6 iptables rules:"
|
||||
echo " kube-proxy might not have created IPv6 rules"
|
||||
echo " On node: sudo ip6tables -t nat -L -n | grep 30091"
|
||||
echo ""
|
||||
|
||||
# Get all node IPs to show which one to use
|
||||
echo "📋 Summary:"
|
||||
echo ""
|
||||
echo " Pod running on node: $POD_NODE"
|
||||
echo " MUST use this IPv6: $NODE_IPV6"
|
||||
echo ""
|
||||
echo " Try accessing:"
|
||||
echo " curl -6 \"http://[$NODE_IPV6]:30091/\""
|
||||
echo ""
|
||||
echo " All worker node IPs (for reference):"
|
||||
kubectl get nodes -o jsonpath='{range .items[*]}{range .status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}{end}' | grep -E '^[0-9a-f]+:[0-9a-f]+:' | nl -v 1 -w 5 -s '. ' | sed 's/^/ /'
|
||||
echo ""
|
||||
|
||||
# Suggest checking k3s/k8s configuration
|
||||
echo "🔧 Recommended Checks:"
|
||||
echo ""
|
||||
echo "1. Check if this is a K3s cluster:"
|
||||
echo " kubectl version --short"
|
||||
echo ""
|
||||
echo "2. Check kube-proxy mode:"
|
||||
echo " kubectl logs -n kube-system -l k8s-app=kube-proxy | grep -i mode"
|
||||
echo ""
|
||||
echo "3. Check if IPv6 is enabled in cluster:"
|
||||
echo " kubectl get nodes -o jsonpath='{.items[0].status.addresses}' | grep ':'"
|
||||
echo ""
|
||||
echo "4. Check service IP families:"
|
||||
echo " kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilies}'"
|
||||
echo ""
|
||||
echo "5. Try changing externalTrafficPolicy to Cluster:"
|
||||
echo " kubectl patch svc nginx-nodeport-service -p '{\"spec\":{\"externalTrafficPolicy\":\"Cluster\"}}'"
|
||||
echo ""
|
||||
|
||||
echo "======================================"
|
||||
echo "Diagnosis complete!"
|
||||
@@ -9,6 +9,10 @@ metadata:
|
||||
spec:
|
||||
type: NodePort
|
||||
externalTrafficPolicy: Local
|
||||
ipFamilies:
|
||||
- IPv4
|
||||
- IPv6
|
||||
ipFamilyPolicy: RequireDualStack
|
||||
selector:
|
||||
app: nginx-nodeport
|
||||
ports:
|
||||
|
||||
@@ -2,6 +2,22 @@
|
||||
|
||||
A complete, production-ready example for deploying a secure, globally accessible website on Mycelium Cloud using Kubernetes **NodePort** services with enhanced network isolation. This demonstrates **security-first IPv6 web hosting** with standard Kubernetes patterns.
|
||||
|
||||
## 🚨 CRITICAL: IPv6 Configuration Required
|
||||
|
||||
**IMPORTANT:** This example requires **dual-stack service configuration** to work with Mycelium IPv6 addresses. Without it, you'll be able to ping IPv6 addresses but not access the service.
|
||||
|
||||
**The Fix:** The `nginx-nodeport-service.yaml` includes:
|
||||
```yaml
|
||||
ipFamilies:
|
||||
- IPv4
|
||||
- IPv6
|
||||
ipFamilyPolicy: RequireDualStack
|
||||
```
|
||||
|
||||
**Why:** Kubernetes services default to IPv4-only. Mycelium uses IPv6, so we must explicitly enable dual-stack.
|
||||
|
||||
**Status:** ✅ Included in provided YAML files. See [IPV6_FIX.md](IPV6_FIX.md) for details.
|
||||
|
||||
## 🔑 Key Concept: NodePort Access Pattern
|
||||
|
||||
**Important:** With NodePort, you access the service via:
|
||||
|
||||
Reference in New Issue
Block a user