# 🚨 CRITICAL: IPv6 Configuration Fix Required ## The Problem When initially deploying NodePort services on Mycelium Cloud, you may encounter a situation where: - ✅ `ping -6` works (ICMP reaches the node) - ❌ `curl -6 http://[ipv6]:30091/` fails (HTTP service unreachable) ## Root Cause **IPv4-only service configuration!** By default, Kubernetes services created without explicit IP family configuration are often IPv4-only, even when nodes have IPv6 addresses. ### Symptom ```bash # Service shows IPv4-only $ kubectl get svc nginx-nodeport-service -o json | grep ipFamily "ipFamilyPolicy": "SingleStack", "ipFamilies": ["IPv4"], ``` ### Result - Services only listen on IPv4 addresses - IPv6 Mycelium addresses are "reachable" (ping works) but HTTP fails - nginx itself is fine (listens on dual-stack) - **kube-proxy won't forward IPv6 traffic to the service** --- ## The Fix ⭐ Update the Service YAML to explicitly support dual-stack: ```yaml apiVersion: v1 kind: Service metadata: name: nginx-nodeport-service spec: type: NodePort externalTrafficPolicy: Local ipFamilies: # ← ADD THESE 3 LINES - IPv4 - IPv6 ipFamilyPolicy: RequireDualStack # ← ADD THIS selector: app: nginx-nodeport ports: - name: http port: 8080 targetPort: 8080 nodePort: 30091 protocol: TCP ``` ### Apply the fix: ```bash kubectl apply -f nginx-nodeport-service.yaml ``` ### Verify the fix: ```bash # Should now show dual-stack $ kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilies}' ["IPv4","IPv6"] $ kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilyPolicy}' RequireDualStack ``` --- ## Why This Happens 1. **Legacy default**: Many Kubernetes setups default to IPv4-only 2. **Service doesn't inherit node IPs**: Service has its own ClusterIP 3. **kube-proxy needs explicit instruction**: Must know to create IPv6 rules 4. **IP family policy**: Controls whether service listens on IPv4, IPv6, or both ### Without dual-stack: ``` User → Mycelium IPv6 → Node port 30091 ↓ kube-proxy ↓ IPv4 service ← BOTTLENECK ↓ Pod responds ``` ### With dual-stack: ``` User → Mycelium IPv6 → Node port 30091 ↓ kube-proxy ↓ IPv6 + IPv4 service ← WORKS! ↓ Pod responds ``` --- ## Testing the Fix ### Before fix: ```bash $ curl -6 "http://[552:5984:2d97:72dc:ff0f:39ef:6ec:a48c]:30091/" curl: (7) Failed to connect to ... Connection refused ``` ### After fix: ```bash $ curl -6 "http://[552:5984:2d97:72dc:ff0f:39ef:6ec:a48c]:30091/" Mycelium Cloud - Nginx NodePort Website ... ``` ### Automated test: ```bash cd myceliumcloud-examples/examples/nginx-nodeport ./test-nodeport-ipv6.sh # Should now show: # ✅ External Mycelium IPv6 connectivity is working! ``` --- ## Other Affected Variants This IP family configuration is critical for: - ✅ **NodePort** - This guide - ✅ **LoadBalancer** - Will need same configuration (future example) - ✅ **Ingress** - Will need same configuration (future example) - ❌ **hostNetwork** - NOT affected (directly uses host network stack) --- ## Kubernetes Version Compatibility ### Kubernetes 1.20+ (Recommended) - Full dual-stack support - `RequireDualStack` policy available - Complete IPv4/IPv6 feature set ### Kubernetes 1.19 and earlier - Limited dual-stack support - May need `ipFamily: IPv6` instead of `RequireDualStack` - Some features may not work **Check your version:** ```bash kubectl version --short ``` --- ## Quick Reference ### Service configuration checklist: - [ ] `type: NodePort` ✓ - [ ] `nodePort: 30091` ✓ - [ ] `externalTrafficPolicy: Local` ✓ - [ ] `ipFamilies: [IPv4, IPv6]` ⭐ Critical for Mycelium - [ ] `ipFamilyPolicy: RequireDualStack` ⭐ Critical for Mycelium ### Test sequence: ```bash # 1. Deploy kubectl apply -f nginx-nodeport-configmaps.yaml kubectl apply -f nginx-nodeport-deployment.yaml kubectl apply -f nginx-nodeport-service.yaml # 2. Wait kubectl wait --for=condition=ready pod -l app=nginx-nodeport --timeout=60s # 3. Check IP families kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilies}' # 4. Test NODE_IPV6=$(kubectl get nodes -o jsonpath='{range .items[*]}{range .status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}{end}' | grep ':' | head -1) curl -6 "http://[$NODE_IPV6]:30091/" ``` --- ## Troubleshooting ### Still not working after dual-stack config? 1. **Check kube-proxy logs:** ```bash kubectl logs -n kube-system -l k8s-app=kube-proxy | grep -i error ``` 2. **Verify nginx listens on IPv6:** ```bash POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1) kubectl exec $POD_NAME -- netstat -tuln | grep 8080 # Should show: # tcp 0 0 0.0.0.0:8080 :::* LISTEN # tcp 0 0 :::8080 :::* LISTEN ← IPv6 ``` 3. **Check service endpoints:** ```bash kubectl get endpoints nginx-nodeport-service ``` 4. **Test from within cluster:** ```bash kubectl run test --rm -it --image=curlimages/curl -- sh -c "curl http://$SERVICE_IP:8080" ``` --- ## Summary **For Mycelium Cloud NodePort deployments, ALWAYS include:** ```yaml spec: ipFamilies: - IPv4 - IPv6 ipFamilyPolicy: RequireDualStack ``` Without this, services won't be accessible via Mycelium IPv6 addresses, even though the nodes are reachable via ping. --- **Impact:** This is a **deployment-blocking issue** for anyone trying to use NodePort with Mycelium IPv6. **Status:** ✅ Fixed in updated `nginx-nodeport-service.yaml` **Related:** [TROUBLESHOOTING.md](TROUBLESHOOTING.md)