242 lines
5.8 KiB
Markdown
242 lines
5.8 KiB
Markdown
# 🚨 CRITICAL: IPv6 Configuration Fix Required
|
|
|
|
## The Problem
|
|
|
|
When initially deploying NodePort services on Mycelium Cloud, you may encounter a situation where:
|
|
- ✅ `ping -6` works (ICMP reaches the node)
|
|
- ❌ `curl -6 http://[ipv6]:30091/` fails (HTTP service unreachable)
|
|
|
|
## Root Cause
|
|
|
|
**IPv4-only service configuration!**
|
|
|
|
By default, Kubernetes services created without explicit IP family configuration are often IPv4-only, even when nodes have IPv6 addresses.
|
|
|
|
### Symptom
|
|
```bash
|
|
# Service shows IPv4-only
|
|
$ kubectl get svc nginx-nodeport-service -o json | grep ipFamily
|
|
"ipFamilyPolicy": "SingleStack",
|
|
"ipFamilies": ["IPv4"],
|
|
```
|
|
|
|
### Result
|
|
- Services only listen on IPv4 addresses
|
|
- IPv6 Mycelium addresses are "reachable" (ping works) but HTTP fails
|
|
- nginx itself is fine (listens on dual-stack)
|
|
- **kube-proxy won't forward IPv6 traffic to the service**
|
|
|
|
---
|
|
|
|
## The Fix ⭐
|
|
|
|
Update the Service YAML to explicitly support dual-stack:
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: nginx-nodeport-service
|
|
spec:
|
|
type: NodePort
|
|
externalTrafficPolicy: Local
|
|
ipFamilies: # ← ADD THESE 3 LINES
|
|
- IPv4
|
|
- IPv6
|
|
ipFamilyPolicy: RequireDualStack # ← ADD THIS
|
|
selector:
|
|
app: nginx-nodeport
|
|
ports:
|
|
- name: http
|
|
port: 8080
|
|
targetPort: 8080
|
|
nodePort: 30091
|
|
protocol: TCP
|
|
```
|
|
|
|
### Apply the fix:
|
|
```bash
|
|
kubectl apply -f nginx-nodeport-service.yaml
|
|
```
|
|
|
|
### Verify the fix:
|
|
```bash
|
|
# Should now show dual-stack
|
|
$ kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilies}'
|
|
["IPv4","IPv6"]
|
|
|
|
$ kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilyPolicy}'
|
|
RequireDualStack
|
|
```
|
|
|
|
---
|
|
|
|
## Why This Happens
|
|
|
|
1. **Legacy default**: Many Kubernetes setups default to IPv4-only
|
|
2. **Service doesn't inherit node IPs**: Service has its own ClusterIP
|
|
3. **kube-proxy needs explicit instruction**: Must know to create IPv6 rules
|
|
4. **IP family policy**: Controls whether service listens on IPv4, IPv6, or both
|
|
|
|
### Without dual-stack:
|
|
```
|
|
User → Mycelium IPv6 → Node port 30091
|
|
↓
|
|
kube-proxy
|
|
↓
|
|
IPv4 service ← BOTTLENECK
|
|
↓
|
|
Pod responds
|
|
```
|
|
|
|
### With dual-stack:
|
|
```
|
|
User → Mycelium IPv6 → Node port 30091
|
|
↓
|
|
kube-proxy
|
|
↓
|
|
IPv6 + IPv4 service ← WORKS!
|
|
↓
|
|
Pod responds
|
|
```
|
|
|
|
---
|
|
|
|
## Testing the Fix
|
|
|
|
### Before fix:
|
|
```bash
|
|
$ curl -6 "http://[552:5984:2d97:72dc:ff0f:39ef:6ec:a48c]:30091/"
|
|
curl: (7) Failed to connect to ... Connection refused
|
|
```
|
|
|
|
### After fix:
|
|
```bash
|
|
$ curl -6 "http://[552:5984:2d97:72dc:ff0f:39ef:6ec:a48c]:30091/"
|
|
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<title>Mycelium Cloud - Nginx NodePort Website</title>
|
|
...
|
|
```
|
|
|
|
### Automated test:
|
|
```bash
|
|
cd myceliumcloud-examples/examples/nginx-nodeport
|
|
./test-nodeport-ipv6.sh
|
|
|
|
# Should now show:
|
|
# ✅ External Mycelium IPv6 connectivity is working!
|
|
```
|
|
|
|
---
|
|
|
|
## Other Affected Variants
|
|
|
|
This IP family configuration is critical for:
|
|
|
|
- ✅ **NodePort** - This guide
|
|
- ✅ **LoadBalancer** - Will need same configuration (future example)
|
|
- ✅ **Ingress** - Will need same configuration (future example)
|
|
- ❌ **hostNetwork** - NOT affected (directly uses host network stack)
|
|
|
|
---
|
|
|
|
## Kubernetes Version Compatibility
|
|
|
|
### Kubernetes 1.20+ (Recommended)
|
|
- Full dual-stack support
|
|
- `RequireDualStack` policy available
|
|
- Complete IPv4/IPv6 feature set
|
|
|
|
### Kubernetes 1.19 and earlier
|
|
- Limited dual-stack support
|
|
- May need `ipFamily: IPv6` instead of `RequireDualStack`
|
|
- Some features may not work
|
|
|
|
**Check your version:**
|
|
```bash
|
|
kubectl version --short
|
|
```
|
|
|
|
---
|
|
|
|
## Quick Reference
|
|
|
|
### Service configuration checklist:
|
|
- [ ] `type: NodePort` ✓
|
|
- [ ] `nodePort: 30091` ✓
|
|
- [ ] `externalTrafficPolicy: Local` ✓
|
|
- [ ] `ipFamilies: [IPv4, IPv6]` ⭐ Critical for Mycelium
|
|
- [ ] `ipFamilyPolicy: RequireDualStack` ⭐ Critical for Mycelium
|
|
|
|
### Test sequence:
|
|
```bash
|
|
# 1. Deploy
|
|
kubectl apply -f nginx-nodeport-configmaps.yaml
|
|
kubectl apply -f nginx-nodeport-deployment.yaml
|
|
kubectl apply -f nginx-nodeport-service.yaml
|
|
|
|
# 2. Wait
|
|
kubectl wait --for=condition=ready pod -l app=nginx-nodeport --timeout=60s
|
|
|
|
# 3. Check IP families
|
|
kubectl get svc nginx-nodeport-service -o jsonpath='{.spec.ipFamilies}'
|
|
|
|
# 4. Test
|
|
NODE_IPV6=$(kubectl get nodes -o jsonpath='{range .items[*]}{range .status.addresses[?(@.type=="InternalIP")]}{.address}{"\n"}{end}{end}' | grep ':' | head -1)
|
|
curl -6 "http://[$NODE_IPV6]:30091/"
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Still not working after dual-stack config?
|
|
|
|
1. **Check kube-proxy logs:**
|
|
```bash
|
|
kubectl logs -n kube-system -l k8s-app=kube-proxy | grep -i error
|
|
```
|
|
|
|
2. **Verify nginx listens on IPv6:**
|
|
```bash
|
|
POD_NAME=$(kubectl get pods -l app=nginx-nodeport -o name | head -1)
|
|
kubectl exec $POD_NAME -- netstat -tuln | grep 8080
|
|
# Should show:
|
|
# tcp 0 0 0.0.0.0:8080 :::* LISTEN
|
|
# tcp 0 0 :::8080 :::* LISTEN ← IPv6
|
|
```
|
|
|
|
3. **Check service endpoints:**
|
|
```bash
|
|
kubectl get endpoints nginx-nodeport-service
|
|
```
|
|
|
|
4. **Test from within cluster:**
|
|
```bash
|
|
kubectl run test --rm -it --image=curlimages/curl -- sh -c "curl http://$SERVICE_IP:8080"
|
|
```
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
**For Mycelium Cloud NodePort deployments, ALWAYS include:**
|
|
|
|
```yaml
|
|
spec:
|
|
ipFamilies:
|
|
- IPv4
|
|
- IPv6
|
|
ipFamilyPolicy: RequireDualStack
|
|
```
|
|
|
|
Without this, services won't be accessible via Mycelium IPv6 addresses, even though the nodes are reachable via ping.
|
|
|
|
---
|
|
|
|
**Impact:** This is a **deployment-blocking issue** for anyone trying to use NodePort with Mycelium IPv6.
|
|
|
|
**Status:** ✅ Fixed in updated `nginx-nodeport-service.yaml`
|
|
**Related:** [TROUBLESHOOTING.md](TROUBLESHOOTING.md) |