10 Apr 2020 8716字 30分次 k8s
BY-SA 4.0（除特别声明或转载文章外）

操作系统Centos7.6 node最低配置要求 CPU 2核内存 2G

每一个节点分别修改hostname 比如master1这个节点： hostnamectl set-hostname master1

配置host绑定 vi /etc/hosts

168.33.11 master1
168.33.12 master2
168.33.13 master3
168.33.14 worker1
168.33.15 worker2

配置DNS vi /etc/resolv.conf

环境部署（所有节点） yum clean all yum -y update yum install -y conntrack ipvsadm ipset jq sysstat curl iptables libseccomp ntpdate setenforce 0 systemctl stop firewalld systemctl disable firewalld 禁用虚拟内存，禁止swap并禁用开机启动: swapoff -a sed -i ‘/swap/s/^(.*)$/#\1/g’ /etc/fstab

停止并禁用dnsmasq。dnsmasq是小型网络用于配置DNS和DHCP的工具 service dnsmasq stop systemctl disable dnsmasq

时间同步 timedatectl set-timezone Asia/Shanghai ntpdate cn.pool.ntp.org

cat > /etc/sysctl.d/kubernetes.conf «EOF net.bridge.bridge-nf-call-iptables=1 net.bridge.bridge-nf-call-ip6tables=1 net.ipv4.ip_forward=1 vm.swappiness=0 vm.overcommit_memory=1 vm.panic_on_oom=0 fs.inotify.max_user_watches=89100 EOF

sysctl -p /etc/sysctl.d/kubernetes.conf

如果报下面的报错： sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: 没有那个文件或目录 sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-ip6tables: 没有那个文件或目录可以执行一下： modprobe br_netfilter 然后再： sysctl -p /etc/sysctl.d/kubernetes.conf

安装docker（所有节点）根据kubernetes对docker版本的兼容测试情况，我们选择17.03.1版本

由于近期docker官网速度极慢甚至无法访问，使用yum安装很难成功。我们直接使用rpm方式安装

手动下载rpm包

mkdir -p /opt/kubernetes/docker && cd /opt/kubernetes/docker #wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-selinux-17.03.1.ce-1.el7.centos.noarch.rpm #wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-17.03.1.ce-1.el7.centos.x86_64.rpm

1.18.2-00

#wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm #wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-19.03.8-3.el7.x86_64.rpm #wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-cli-19.03.8-3.el7.x86_64.rpm

wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.2-3.el7.x86_64.rpm wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-18.09.9-3.el7.x86_64.rpm wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-cli-18.09.9-3.el7.x86_64.rpm

清理原有版本 yum remove -y docker* container-selinux

安装rpm包 yum localinstall -y *.rpm

开机启动 systemctl enable docker

#mkdir -p /etc/docker #vi /etc/docker/daemon.json #```

“graph”: “/docker/data/path”,

“exec-opts”: [“native.cgroupdriver=systemd”]

#``` service docker restart

安装kubernetes工具配置yum源（有条件科学上网的，可以把”mirrors.aliyun.com”替换为”packages.cloud.google.com”） cat «EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF

yum list kubeadm –showduplicates

sort -r

#yum install -y kubeadm-1.18.2-0 kubelet-1.18.2-0 kubectl-1.18.2-0 –disableexcludes=kubernetes

#yum install -y kubeadm-1.14.0-0 kubelet-1.14.0-0 kubectl-1.14.0-0 –disableexcludes=kubernetes

yum install -y kubeadm-1.16.4-0 kubelet-1.16.4-0 kubectl-1.16.4-0 –disableexcludes=kubernetes

kubelet版本是1.16.4，该版本支持的docker版本为1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09

开机启动kubelet

systemctl enable kubelet

配置master节点（仅master节点）：这里是单master：生成配置文件 kubeadm config print init-defaults ClusterConfiguration > kubeadm.conf

修改配置文件,注意三个地方

kubernetesVersion: v1.18.2

localAPIEndpoint:
  advertiseAddress: 10.0.2.51
  bindPort: 6443

imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers

networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}

拉取镜像 kubeadm config images pull –config ./kubeadm.conf

检查镜像 docker images

初始化 kubeadm init –config ./kubeadm.conf

初始化成功的提示：

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.33.12:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:366892d67eedc0a66f5e40535661cac8df59ab321e15bb3b0da727b1a4f64de5

这里面有这一段：

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

把这三行命令照着执行一下。加载环境变量。你也可以用其他办法比如：

echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source .bash_profile

然后最后面还有一段

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.33.12:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:366892d67eedc0a66f5e40535661cac8df59ab321e15bb3b0da727b1a4f64de5

这个要记下来，后面加worker节点会用到。

如果需要重新初始化，可以用下面这个命令重置 kubeadm reset rm -rf $HOME/.kube/config

删除之前绑定的IP ip addr del 10.244.0.1/24 dev cni0 #ip addr del 10.245.1.0/32 dev flannel.1

重置命令要慎用。如果要重置，要把所有节点都重置，然后重启服务器。再在master上执行kubeadm init –config ./kubeadm.conf

初始化完成了，我们启动kubelet systemctl enable kubelet && systemctl start kubelet

systemctl status kubelet

journalctl -xefu kubelet 查看详细信息

[root@k8snode1 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8snode1 NotReady master 10m v1.14.0

[root@k8snode1 ~]# kubectl get cs NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-0 Healthy {“health”:”true”}

获取并上传kube-flannel.yml

kubectl apply -f kube-flannel.yml

稍等一会，就可以检查一下master的网络状态，可以看到已经状态已经变成Ready [root@node2 opt]# kubectl get node NAME STATUS ROLES AGE VERSION node2 Ready master 62m v1.14.0

如果是master集群，配置文件就不一样了。先在master1节点上执行： [root@localhost ~]# cat kubeadm.yaml apiVersion: kubeadm.k8s.io/v1beta1 imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers kind: ClusterConfiguration kubernetesVersion: v1.18.2 apiServer: certSANs:

master1
master2
master3
worker1
worker2
192.168.33.11
192.168.33.12
192.168.33.13
192.168.33.14
192.168.33.15 controlPlaneEndpoint: “192.168.33.11:6443” networking: dnsDomain: cluster.local podSubnet: “10.244.0.0/16” serviceSubnet: 10.96.0.0/12 scheduler: {}

master集群初始化命令： kubeadm init –config=kubeadm.yaml

初始化成功以后会类似如下的提示：

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join 192.168.33.11:6443 --token 0a0mf9.fc10msqpsk1lll8h \
    --discovery-token-ca-cert-hash sha256:c913cd2c5fa9c47c177c3016e0addea2d9d78e64cb28b3c75b764ad7d1c52676 \
    --control-plane

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.33.11:6443 --token 0a0mf9.fc10msqpsk1lll8h \
    --discovery-token-ca-cert-hash sha256:c913cd2c5fa9c47c177c3016e0addea2d9d78e64cb28b3c75b764ad7d1c52676

环境变量： mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config

systemctl restart kubelet

获取 kube-flannel.yml

kubectl apply -f kube-flannel.yml

稍等一会，就可以检查一下master的网络状态，可以看到已经状态已经变成Ready [root@node2 opt]# kubectl get node NAME STATUS ROLES AGE VERSION node2 Ready master 62m v1.14.0

其他节点和master之间做好密钥登录 ssh-keygen -t rsa

配置第二个master节点，我们需要从第一个master节点复制一些配置文件到第二个master节点上远程登录第二个master节点执行命令： scp -r root@master1:/etc/kubernetes/pki . scp -r root@master1:/etc/kubernetes/admin.conf .

删除多余文件： rm -f pki/apiserver* rm -f pki/front-proxy-client.* rm -f pki/etcd/healthcheck-client.* pki/etcd/peer.* pki/etcd/server.* 复制到本地目录： cp -rf pki /etc/kubernetes/ cp -rf admin.conf /etc/kubernetes/

#scp -r root@master1:~/kubeadm.yaml .

环境变量： mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config

加入集群： kubeadm join 192.168.33.11:6443 –token 0a0mf9.fc10msqpsk1lll8h \ –discovery-token-ca-cert-hash sha256:c913cd2c5fa9c47c177c3016e0addea2d9d78e64cb28b3c75b764ad7d1c52676 \ –control-plane

好了，下面是部署worker节点，node3：

scp /etc/kubernetes/admin.conf root@node3:~

远程登录node3，执行：

mkdir -p $HOME/.kube
sudo cp -i $HOME/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

node3 加入master集群： kubeadm join 192.168.33.12:6443 –token abcdef.0123456789abcdef \ –discovery-token-ca-cert-hash sha256:366892d67eedc0a66f5e40535661cac8df59ab321e15bb3b0da727b1a4f64de5

上master把文件传到node3 scp kube-flannel.yml root@node3:/opt

上node3，启动flannel网络： kubectl apply -f kube-flannel.yml

过一会，检查node3是否加入集群： kubectl get nodes

集群节点状态查看： kubectl get po -o wide -n kube-system

https://www.kubernetes.org.cn/6632.html https://blog.csdn.net/qq_36160277/article/details/97989131 https://kuboard.cn/install/install-k8s.html#%E5%88%9D%E5%A7%8B%E5%8C%96-master-%E8%8A%82%E7%82%B9

k8s安装笔记 fangfu的技术博客

手动下载rpm包

“graph”: “/docker/data/path”,

“exec-opts”: [“native.cgroupdriver=systemd”]

开机启动kubelet