一、先安装一台负载均衡器。这里是安装HAPROXY

apt install haproxy -y

执行以下命令编辑 HAProxy 的配置文件：

vi /etc/haproxy/haproxy.cfg
在 HAProxy 的配置文件中添加以下信息并保存文件（将 <IP address> 替换为 KubeSphere 企业版集群各控制平面节点的私网 IP 地址）：

global
    log /dev/log  local0 warning
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

   stats socket /var/lib/haproxy/stats

defaults
  log global
  option  httplog
  option  dontlognull
        timeout connect 5000
        timeout client 50000
        timeout server 50000

frontend kube-apiserver
  bind *:6443
  mode tcp
  option tcplog
  default_backend kube-apiserver

backend kube-apiserver
    mode tcp
    option tcplog
    option tcp-check
    balance roundrobin
    default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
    server kube-apiserver-1 <IP address>:6443 check
    server kube-apiserver-2 <IP address>:6443 check
    server kube-apiserver-3 <IP address>:6443 check
执行以下命令重启 HAProxy：

systemctl restart haproxy
执行以下命令设置 HAProxy 开机后自动运行：

systemctl enable haproxy

二、在所有控制节点和工作节点上配置内核，执行

加载内核模块
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter

配置内核参数
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
sudo sysctl --system


cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system

关闭swap
swapoff -a
sed -i '/ swap / s/^/#/' /etc/fstab

三、安装依赖，并添加源，并安装K8S

apt install -y curl gnupg2 software-properties-common apt-transport-https ca-certificates

添加阿里云源
curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.asc
echo   "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu \
    $(. /etc/os-release && echo "$VERSION_CODENAME") stable" |   sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
apt update
curl -fsSL https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-aliyun-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-aliyun-keyring.gpg] https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
apt update
curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
echo   "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] http://mirrors.aliyun.com/docker-ce/linux/ubuntu \
   $(. /etc/os-release && echo "$VERSION_CODENAME") stable" |   sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

apt update
apt install -y kubelet kubeadm kubectl

四、安装容器运行时containerd，并配置代理

apt install -y containerd.io

生成containerd默认配置文件
containerd config default | sudo tee /etc/containerd/config.toml
修改默认配置文件，启用阿里源，并指定镜像代理地址
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml
[plugins]
  [plugins.'io.containerd.cri.v1.images']
    [plugins.'io.containerd.cri.v1.images'.pinned_images]
      sandbox = 'registry.aliyuncs.com/google_containers/pause:3.9'
    [plugins.'io.containerd.cri.v1.images'.registry]
      config_path = '/etc/containerd/certs.d'
mkdir -p /etc/containerd/certs.d/docker.io/
配置代理：
more /etc/containerd/certs.d/docker.io/hosts.toml
server = "https://registry-1.docker.io"

[host."https://docker.1panel.live"]
  capabilities = ["pull", "resolve"]

[host."https://hub.rat.dev"]
  capabilities = ["pull", "resolve"]

[host."https://docker.1ms.run"]
  capabilities = ["pull", "resolve"]

五、在控制节点初始化集群

kubeadm init --control-plane-endpoint "192.168.23.48:6443" --upload-certs --image-repository=registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16
正常情况下，应该运行成功，显示如下内容：
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 192.168.23.48:6443 --token 4xjy2y.xp2afw7bpmdw6btx --discovery-token-ca-cert-hash sha256:bf0e07f7826630927669f18bb2444881b1a827835d0447850dbf0cc6edbfe4a6 --control-plane --certificate-key 1850d50ffa927c9ad09aefcb51d01e5156bcd63d3fa80a2a7556540577b1c448

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

新增控制节点和工作节点
工作节点
kubeadm join 192.168.23.48:6443 --token 4xjy2y.xp2afw7bpmdw6btx \
        --discovery-token-ca-cert-hash sha256:bf0e07f7826630927669f18bb2444881b1a827835d0447850dbf0cc6edbfe4a6

控制节点
 kubeadm join 192.168.23.48:6443 --token 4xjy2y.xp2afw7bpmdw6btx --discovery-token-ca-cert-hash sha256:bf0e07f7826630927669f18bb2444881b1a827835d0447850dbf0cc6edbfe4a6 --control-plane --certificate-key 1850d50ffa927c9ad09aefcb51d01e5156bcd63d3fa80a2a7556540577b1c448

安装网络：
 export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

查看安装结果
时间可能比较久
kubectl get nodes
kubectl get pods -n kube-flannel

六、

使用KK安装k8s
准备三台节点，一台主控，两台工作。不要安装docker和containerd。操作系统版本为：ubuntu20.04
主控节点执行：
curl -sfL https://get-kk.kubesphere.io | sh -
./kk create config --with-kubernetes v1.30.0
修改配置文件，主要是把节点IP和密码配置上去
./kk create cluster -f config-sample.yaml
安装完成后，执行以下命令查看集群状态：
kubectl get nodes
kubectl get pods -A

安装kubesphere
chart=oci://hub.kubesphere.com.cn/kse/ks-core
version=1.1.1
helm upgrade --install -n kubesphere-system --create-namespace ks-core $chart --version $version --debug --wait