Search code examples
kubernetesflannel

Failed to install rook on k8s cluster

I am trying to create a rook cluster inside k8s cluster.

Set up - 1 master node, 1 worker node

These are the steps I have followed

Master node: 
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
sudo sysctl net.bridge.bridge-nf-call-iptables=1
sudo sysctl net.bridge.bridge-nf-call-ip6tables=1
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/32a765fd19ba45b387fdc5e3812c41fff47cfd55/Documentation/kube-flannel.yml
kubeadm token create --print-join-command

Worker node: 
kubeadm join {master_ip_address}:6443 --token {token} --discovery-token-ca-cert-hash {hash} --apiserver-advertise-address={worker_private_ip}

Master node - Install rook - (reference - https://rook.github.io/docs/rook/master/ceph-quickstart.html):
kubectl create -f ceph/common.yaml
kubectl create -f ceph/operator.yaml
kubectl create -f ceph/cluster-test.yaml

Error while creating rook-ceph-operator pod:

(combined from similar events): Failed create pod sandbox: rpc error: code = 
Unknown desc = failed to set up sandbox container "4a901f12e5af5340f2cc48a976e10e5c310c01a05a4a47371f766a1a166c304f" 
network for pod "rook-ceph-operator-fdfbcc5c5-jccc9": networkPlugin cni failed to 
set up pod "rook-ceph-operator-fdfbcc5c5-jccc9_rook-ceph" network: failed to set bridge addr: 
"cni0" already has an IP address different from 10.244.1.1/24

enter image description here

Can anybody help me with this issue?

Solution

  • This issue start if you did kubeadm reset and after that kubeadm init reinitialize Kubernetes.

    kubeadm reset
systemctl stop kubelet
systemctl stop docker
rm -rf /var/lib/cni/
rm -rf /var/lib/kubelet/*
rm -rf /etc/cni/
ifconfig cni0 down
ifconfig flannel.1 down
ifconfig docker0 down

    After this start docker and kubelet and kubeadm again.

    Work around

    You can also try this way as simple easy solution 

    ip link delete cni0
ip link delete flannel.1

    that depends on which network you are using inside k8s.