使用rke2安装高可用k8s集群

• 服务器rke集群节点角色规划

• 安装一些个人常用的基础安装包

   yum  -y install  epel-release.noarch
   yum  -y install   psmisc gcc gcc-c++ texinfo wget unzip zip gcc libticonv-devel  libcurl-devel curl nmap iotop dstat tree mlocate ntpdate openssh-clients  net-tools vim  ntsysv nmap curl lrzsz  sysstat  libselinux-python   pcre pcre-devel zlib zlib-devel openssl openssl-devel readline-devel bzip2  httpd-devel python-devel python-pip python-setuptools   lsof  sqlite-devel  nscd bind-utils telnet rsync tcpdump expect  nc    ntp  lftp   bash-completion ipset ipvsadm 
  
  

• 关闭防火墙

  systemctl stop firewalld
  systemctl stop iptables
  systemctl disable firewalld
  systemctl disable iptables
  
  

• 关闭selinux

  sed -i 's/SELINUX=enforcing/SELINUX=disabled/'  /etc/sysconfig/selinux
  sed -i 's/SELINUX=enforcing/SELINUX=disabled/'   /etc/selinux/config
  setenforce 0
  

• 三台时间同步要一致

  systemctl start ntpd
  systemctl enable ntpd
  
  

• 关闭swap分区

  swapoff -a
  sed  -i  '/swap/d' /etc/fstab
  mount  -a
  

  • 然后修改/etc/fstab，把swap分区相关的配置注释掉

• 内核参数调整

  cat >> /etc/sysctl.conf <<EOF
  fs.file-max = 2442652
  net.ipv4.ip_local_port_range = 1024     65535
  vm.swappiness=0
  net.ipv4.ip_forward=1
  net.bridge.bridge-nf-call-iptables=1
  net.bridge.bridge-nf-call-ip6tables = 1
  EOF
  
  modprobe br_netfilter  
  sysctl  -p 
  

• 配置资源限制

     sh -c " cat  >>/etc/security/limits.conf <<EOF
     *       soft    nofile          1048576
     *       hard    nofile          1048576
     *       soft     core            unlimited
     *       hard    core            unlimited
     *       soft	    nproc            unlimited
     *       hard	    nproc           unlimited
     EOF"
     sh -c  "cat >> /etc/security/limits.d/20-nproc.conf << EOF
     *  soft    nproc     unlimited
     *  hard    nproc     unlimited
     EOF"
  

• NetworkManager 网络配置

 systemctl status NetworkManager
 cat >> /etc/NetworkManager/conf.d/rke2-canal.conf << EOF
 [keyfile]
 unmanaged-devices=interface-name:cali*;interface-name:flannel*
 EOF
 systemctl daemon-reload
 systemctl restart NetworkManager

• 加载ipvs相关模块
  由于ipvs已经加入到了内核的主干，所以为kube-proxy开启ipvs的前提需要加载以下的内核模块：

  cat > /etc/sysconfig/modules/ipvs.modules <<EOF
   #!/bin/bash
   modprobe -- ip_vs
   modprobe -- ip_vs_rr
   modprobe -- ip_vs_wrr
   modprobe -- ip_vs_sh
   modprobe -- nf_conntrack_ipv4
   EOF
   chmod 755 /etc/sysconfig/modules/ipvs.modules 
   bash /etc/sysconfig/modules/ipvs.modules
   lsmod | grep -e ip_vs -e nf_conntrack_ipv4
  

• 在线三个节点安装rke-server(controlplane、worker、etcd角色)

  curl -sfL https://get.rke2.io | sh -
  
  

• 在rke-sersver-01上启动rke-server

  systemctl enable rke2-server.service
  systemctl start rke2-server.service
  
  

  • 安装完成后 /var/lib/rancher/rke2/bin/ 目录 生成有ctr、crictl、kubectl 等二进制文件
  • A kubeconfig file will be written to /etc/rancher/rke2/rke2.yaml
  • A token that can be used to register other server or agent nodes will be created at /var/lib/rancher/rke2/server/node-token

• 三个节点均创建rke-server高可用集群需要的/etc/rancher/rke2/config.yaml 配置文件，

  server: https://192.168.2.131:9345
  token: my-shared-secret  #/var/lib/rancher/rke2/server/node-token中值
  tls-san:
     - my-kubernetes-domain.com
     - another-kubernetes-domain.com
  node-label:
     - "host=k8s-master"
  #node-taint:  ####(打污点)
  #   - "host=k8s-master:NoExecute"   
  

• 验证集群

   /var/lib/rancher/rke2/bin/kubectl \
       --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes
  

• 集群添加agent(woker角色)节点

  curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" sh -
  systemctl enable rke2-agent.service
  mkdir -p /etc/rancher/rke2/
  cat > /etc/rancher/rke2/config.yaml <<EOF
  server: https://192.168.2.131:9345
  token: my-shared-secret #/var/lib/rancher/rke2/server/node-token中值
  EOF
  systemctl start rke2-agent.service
  

• 停止某个节点上的服务

  rke2-killall.sh
  

• 清理某个节点上的rke 服务

  rke2-uninstall.sh
  

  rke2官方文档连接