K8S集群相关(v1.20.1)

K8S集群安装(v1.20.1)

虚拟机配置: centos7(节点CPU核数必须是 :>= 2核 ,否则k8s无法启动)
IP:
master:10.211.55.25
node1:10.211.55.26
node2:10.211.55.27

1. 机器环境配置

  • 1.1 给每一台机器设置主机名
hostnamectl set-hostname k8s-01
hostnamectl set-hostname k8s-02
hostnamectl set-hostname k8s-03
  • 1.2 配置IP host映射关系
vi /etc/hosts
#######
10.211.55.25 k8s-01
10.211.55.26 k8s-02
10.211.55.27 k8s-03
  • 1.3 安装依赖环境,注意:每一台机器都需要安装此依赖环境
yum install-y conntrack ntpdate ntp ipvsadm ipset jq iptablescurl sysstat libseccompwget vim net-toolsgit iproute lrzsz bash-completion tree bridge- utils unzip bind-utilsgcc

1.4 防火墙相关

安装iptables,启动iptables,设置开机自启,清空iptables规则,保存当前规则到默认规则

  • 关闭防火墙
systemctlstop firewalld && systemctl disable firewalld
  • 置空规则
iptablesyum-y install iptables-services && systemctlstart iptables && systemctl enable iptables && iptables-F &&service iptables save
  • 关闭swap分区【虚拟内存】并且永久关闭虚拟内存
swapoff -a && sed -i '/ swap / s/^\(.*\)$/#\1/g'/etc/fstab
  • 关闭selinux(Linux安全内核模块)
setenforce 0 && sed -i 's/^SELINUX=.*/SELINUX=disabled/'/etc/selinux/config

1.5 升级Linux内核为4.44版本

  • 获取升级包
rpm-Uvh http://www.elrepo.org/elrepo-release-7.0-4.el7.elrepo.noarch.rpm
  • 可选:添加rpm源
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
  • 查看可用版本( 长期支持版本lt 稳定主线版本ml ) 找到4.4版本
yum --enablerepo="elrepo-kernel" list --showduplicates | sort -r | grep kernel-lt.x86_64
yum --enablerepo="elrepo-kernel" list --showduplicates | sort -r | grep kernel-ml.x86_64
image
  • 安装指定版本内核
yum --enablerepo="elrepo-kernel" install kernel-lt-4.4.249-1.el7.elrepo.x86_64 -y
  • 查看安装完后所有内核
grep 'menuentry' /etc/grub2.cfg
  • 设置开机从新内核启动
grub2-set-default 'CentOS Linux (4.4.249-1.el7.elrepo.x86_64) 7 (Core)'

** 注意 设置完内核后,需要重启服务器才会生效 **

  • 查询当前内核
uname-r
4.4.249-1.el7.elrepo.x86_64

1.6 修改内核参数

cat > kubernetes.conf <<EOF 
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF
  • 将优化内核文件拷贝到/etc/sysctl.d/文件夹下,这样优化文件开机的时候能够被调用
cp kubernetes.conf /etc/sysctl.d/kubernetes.conf 
  • 手动刷新,让优化文件立即生效
sysctl-p /etc/sysctl.d/kubernetes.conf
  • 调整系统临时区(如果已经设置时区,可略过 )
  • 设置系统时区为中国/上海
timedatectl set-timezone Asia/Shanghai
  • 将当前的 UTC 时间写入硬件时钟
timedatectl set-local-rtc0 
  • 重启依赖于系统时间的服务
systemctlrestart rsyslog systemctlrestart crond
  • 关闭系统不需要的服务
systemctlstop postfix && systemctl disable postfix
  • 设置日志保存方式
  • 修改打开文件数调整
echo "* soft nofile 65536" >> /etc/security/limits.conf 
echo "* hard nofile 65536" >> /etc/security/limits.conf

1.7 设置日志保存方式

  • 创建保存日志的目录
mkdir /var/log/journal 
  • 创建配置文件存放目录
mkdir /etc/systemd/journald.conf.d 
  • 创建配置文件
cat > /etc/systemd/journald.conf.d/99-prophet.conf <<EOF 
[Journal]
Storage=persistent
Compress=yes
SyncIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000
SystemMaxUse=10G
SystemMaxFileSize=200M
MaxRetentionSec=2week
ForwardToSyslog=no
EOF
  • 重启systemd journald的配置
systemctl restart systemd-journald

1.8 kube-proxy 开启 ipvs 前置条件

modprobe br_netfilter
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
  • 使用lsmod命令查看这些文件是否被引导
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

2. docker安装

2.1安装docker

  • yum安装
yum install -y yum-utils device-mapper-persistent-data lvm2
  • 紧接着配置一个稳定的仓库、仓库配置会保存到/etc/yum.repos.d/docker-ce.repo文件中
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
  • 更新Yum安装的相关Docker软件包&安装Docker CE
yum update -y && yum install docker-ce
  • 设置docker daemon文件 创建/etc/docker目录
mkdir /etc/docker 
  • 更新daemon.json文件
cat > /etc/docker/daemon.json <<EOF
{"exec-opts":["native.cgroupdriver=systemd"],"log-driver":"json-file","log-opts":{"max-size":"100m"}}
EOF
  • 创建,存储docker配置文件
mkdir -p /etc/systemd/system/docker.service.d
  • 重启docker服务
systemctl daemon-reload && systemctl restart docker && systemctl enable docker

4. 集群安装

4.1在线安装 kubelet kubeadm kubectl

  • yum阿里源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
       http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
  • 安装kubeadm、kubelet、kubectl
yum install -y kubeadm-1.15.1 kubelet-1.15.1 kubectl-1.15.1
  • 启动 kubelet
systemctl enable kubelet && systemctl start kubelet

4.2准备k8s镜像

  • 初始化配置文件
kubeadm config print init-defaults > kubeadm.conf

由于国内无法访问k8s.gcr.io, 采用从阿里云中下载后,重新修改tag的方式

  • 查看kubeadm所需的镜像列表
kubeadm config images list
image.png
  • 从国内阿里云下载对应版本号的镜像
docker image pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.20.1
docker image pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.20.1
docker image pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.20.1
docker image pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.20.1
docker image pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
docker image pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.13-0
docker image pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0
  • 镜像更名
docker image tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.20.1 k8s.gcr.io/kube-apiserver:v1.20.1
docker image tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.20.1 k8s.gcr.io/kube-controller-manager:v1.20.1
docker image tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.20.1 k8s.gcr.io/kube-scheduler:v1.20.1
docker image tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.20.1 k8s.gcr.io/kube-proxy:v1.20.1
docker image tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2 k8s.gcr.io/pause:3.2
docker image tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.13-0 k8s.gcr.io/etcd:3.4.13-0
docker image tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0 k8s.gcr.io/coredns:1.7.0
  • 删除之前的下载的镜像
docker image rm -f registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.20.1
docker image rm -f registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.20.1
docker image rm -f registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.20.1
docker image rm -f registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.20.1 
docker image rm -f registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0
docker image rm -f registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.13-0
docker image rm -f registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2

4.3 初始化集群

4.3.1 初始化主节点 --- 只需要在主节点执行
  • 获取yaml资源配置文件
kubeadm config print init-defaults > kubeadm-config.yaml
  • 修改yaml资源文件
localAPIEndpoint:
    advertiseAddress: 10.211.55.25 # 注意:修改配置IP为master节点
kubernetesVersion: v1.20.1 #修改版本号,必须和kubectl版本保持一致

#添加以下配置
#指定使用ipvs网络进行通信
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1 
kind: kubeProxyConfiguration
featureGates:
      SupportIPVSProxyMode: true
mode: ipvs
  • 初始化主节点,开始部署

执行此命令,CPU核心数量必须大于1核,否则无法执行成功

kubeadm init --config=kubeadm-config.yaml --experimental-upload-certs | tee kubeadm-init.log

执行完成后,继续执行日志输出中的后续配置命令

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
  • 查看node节点,

此时应该只有master,且使用ipvs+flannel的方式进行网络通信,但是flannel网络插件还没有部署,因此节点状态 此时为NotReady

kubectl get node
4.3 flannel插件 (只需要在主节点执行)
#1 下载flannel网络插件
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
#2 部署flannel
kubectl create -f kube-flannel.yml
#也可进行部署网络
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
  • 查看pod
 kubectl get pod -n kube-system
4.4 将两个node节点加入集群
  • 从主节点的安装日志中,获取节点添加命令
cat kubeadm-init.log
-----
kubeadm join 10.211.55.25:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:2395b257958a5d583b9d8059df71b751f0929f5a91eb6247649193a81c0af841
  • 在两个node节点上分别执行一次
4.5 验证最终状态
[root@k8s-01 ~]#  kubectl get node
NAME     STATUS   ROLES                  AGE   VERSION
k8s-01   Ready    control-plane,master   23h   v1.20.1
k8s-02   Ready    <none>                 22h   v1.20.1
k8s-03   Ready    <none>                 22h   v1.20.1
[root@k8s-01 ~]# kubectl get pod -n kube-system
NAME                             READY   STATUS              RESTARTS   AGE
coredns-74ff55c5b-mbh7t          0/1     ContainerCreating   0          23h
coredns-74ff55c5b-qzfqq          0/1     ContainerCreating   0          23h
etcd-k8s-01                      1/1     Running             1          23h
kube-apiserver-k8s-01            1/1     Running             1          23h
kube-controller-manager-k8s-01   1/1     Running             1          23h
kube-flannel-ds-amd64-bzjvn      0/1     CrashLoopBackOff    19         22h
kube-flannel-ds-amd64-fl5qt      0/1     CrashLoopBackOff    19         22h
kube-flannel-ds-amd64-qvkw6      0/1     CrashLoopBackOff    26         22h
kube-proxy-2vljc                 1/1     Running             1          23h
kube-proxy-ptwd9                 1/1     Running             0          22h
kube-proxy-rjwps                 1/1     Running             0          22h
kube-scheduler-k8s-01            1/1     Running             1          23h

推荐阅读更多精彩内容