使用docker engine 运行时 集群部署 kubernetes v1.23
VMware 安装 K8S集群kubernetes v1.23
硬件环境:
- 使用 VMware16 模拟3台服务器,建立1个control plane(master),2个work 节点的kubernetes集群
- 使用 Ubuntu server 20.04 TLS 作为Linux系统环境
系统安装
- 使用典型配置即可。操作系统选择 Ubuntu 64位
-
系统安装过程中,在网络连接的时候,选择DHCPv4,自动获取IP地址即可。
- 如果DHCPv4 无法自动获取IP,打开虚拟网络编辑器还原默认设置。
-
设置镜像来源的时候,将默认的更改为
https://mirrors.tuna.tsinghua.edu.cn/ubuntu
-
中途无需选择其他组件安装,只需要打开open SSH即可。
初始环境设置
-
安装net-tools,查看IP地址,以便在外部使用SSH连接(因为内部终端不好用,没办法复制粘贴)
sudo apt update sudo apt install net-tools
查看IP
ifconfig
=====> 看到 IP 192.168.145.140
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.145.140 netmask 255.255.255.0 broadcast 192.168.145.255 inet6 fe80::250:56ff:fe3e:d6c7 prefixlen 64 scopeid 0x20<link> ether 00:50:56:3e:d6:c7 txqueuelen 1000 (Ethernet) RX packets 67260 bytes 82597310 (82.5 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 21345 bytes 4844931 (4.8 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
-
外部SSH客户端连接
例如:
ssh master@192.168.145.140
-
关闭swap分区(root 权限)
swapoff -a
永久关闭(不然重启过后,kubelet就无法运行)
vim /etc/fstab
注释掉最后一行
#/swap.img none swap sw 0 0
-
安装docker
sudo apt install docker.io
验证
docker version
-
修改docker驱动
sudo vim /etc/docker/daemon.json
{ "exec-opts":["native.cgroupdriver=systemd"] }
重启
systemctl restart docker
设置开启自启
systemctl enable docker.service
-
安装kubernetes 前置
sudo apt-get update && sudo apt-get install -y apt-transport-https curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - cat <<EOF >/etc/apt/sources.list.d/kubernetes.list deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main EOF
apt-get update
-
安装 kubelet kubectl kubeadm
尽量选择一个固定的版本
apt install kubelet=1.23.6-00 kubectl=1.23.6-00 kubeadm=1.23.6-00
- 不同组件的版本支持情况 https://kubernetes.feisky.xyz/setup/upgrade
- 最好就是保持版本相同
- kubernetes自1.20,放弃了docker支持,在 Kubernetes 1.24 版本中移除了dockershim, 所以如果 kubelet版本高于1.23,后面的安装可能就有问题。除非你使用其他运行时。
- 不同组件的版本支持情况 https://kubernetes.feisky.xyz/setup/upgrade
-
验证
kubectl version、 kubelet --version、 kubeadm version
root@work1:/home/master# kubectl version Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.6", GitCommit:"ad3338546da947756e8a88aa6822e9c11e7eac22", GitTreeState:"clean", BuildDate:"2022-04-14T08:49:13Z", GoVersion:"go1.17.9", Compiler:"gc", Platform:"linux/amd64"} The connection to the server localhost:8080 was refused - did you specify the right host or port? root@work1:/home/master# kubeadm version kubeadm version: &version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.6", GitCommit:"ad3338546da947756e8a88aa6822e9c11e7eac22", GitTreeState:"clean", BuildDate:"2022-04-14T08:48:05Z", GoVersion:"go1.17.9", Compiler:"gc", Platform:"linux/amd64"} root@work1:/home/master# kubelet --version Kubernetes v1.23.6
设置开启自启
systemctl enable kubelet
创建其他节点
- 以此虚拟机作为母版,创建 控制平面节点、工作节点
- 建议每几个关键步骤都生成一个快照。方便操作错误,快速回滚。
control plan
-
创建连接克隆
- 选择该虚拟机==>右键==>管理==>克隆==>链接克隆
-
修改 mac 地址
因为克隆出来的 mac 地址也是一样的,而mac需要唯一,不然两个同时起来,会有冲突
- 编辑虚拟机设置-网络适配器-高级-mac地址生成
-
开机
-
修改主机名字,方便后面清楚在操作谁
hostnamectl set-hostname master
-
-
固定虚拟机的IP地址(非常重要,不然IP一直变化,集群直接失效)
sudo vim /etc/netplan/00-installer-config.yaml
写如下信息(按ifconfig中的信息,适当改写)
# This is the network config written by 'subiquity' network: ethernets: ens33: #配置的网卡的名称 addresses: [192.168.145.140/24] #配置的静态ip地址和掩码 dhcp4: no #关闭DHCP,如果需要打开DHCP则写yes optional: true gateway4: 192.168.145.2 nameservers: addresses: [8.8.8.8,8.8.4.4] version: 2 renderer: networkd
使生效
sudo netplan apply
work node
操作如上。注意IP地址的分配。
control plane 初始化
-
只在 control plane 中运行以下命令,work 节点不需要
-
若某一步骤失败,重新运行之前,先还原设置 运行
kubeadm reset
-
初始化命令
- 可以指定 kubernetes 版本 --kubernetes-version=v1.23.14,若不指定,则系统默认。
- pod的网络,可以按需填写。可直接使用下面的地址。
kubeadm init --image-repository=registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16
成功后
Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.145.132:6443 --token zzo4m1.zbwbhl223km7kv6h --discovery-token-ca-cert-hash sha256:478678ef11485c6a131f9778bdfbb19cb7f1aefddcf249618be1ffd76891461d
- 若失败,请考虑是否是 kebuctl kubelet kubeadm 版本问题
-
运行提示中的指令
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
-
记住work 节点加入集群的指令
kubeadm join 192.168.145.132:6443 --token zzo4m1.zbwbhl223km7kv6h --discovery-token-ca-cert-hash sha256:478678ef11485c6a131f9778bdfbb19cb7f1aefddcf249618be1ffd76891461d
-
输入指令查看节点(root)
kubectl get nodes
- not ready 是因为还没有部署pod
-
部署一个 flannel
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
-
再次查看节点,应该ready了
kubectl get nodes
work 节点部署
-
直接运行上面提到的加入指令
kubeadm join 192.168.145.132:6443 --token zzo4m1.zbwbhl223km7kv6h --discovery-token-ca-cert-hash sha256:478678ef11485c6a131f9778bdfbb19cb7f1aefddcf249618be1ffd76891461d
-
如果不成功,先运行
kubeadm reset
再排查原因,再次运行指令
-
-
两个节点都运行结束后,control plane 中应该可以看到信息
-
回到 control plane,运行
kubectl get nodes
应该可以看到
root@master:/home/master# kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready control-plane,master 3h37m v1.23.6 work1 Ready <none> 10m v1.23.6 work2 Ready <none> 111s v1.23.6
创建 Pod 验证集群
- 在control plane 中运行指令
-
使用 kubectl 创建一个 nginx
kubectl create deployment nginx --image=nginx
-
创建一个服务,并且暴露端口让外部可以访问(后面可以跟–target-port=80指定暴露的外部访问的端口,我这里让它随机选择)
kubectl expose deployment nginx --port=80 --type=NodePort
-
查看pod和service的状态
kubectl get pods,svc
root@master:/home/master# kubectl get pods,svc NAME READY STATUS RESTARTS AGE pod/nginx-85b98978db-mn4cc 1/1 Running 0 3h36m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h41m service/nginx NodePort 10.109.113.11 <none> 80:30432/TCP 3h36m
使用此指令可以看pod详细信息
kubectl describe pod nginx
-
访问一下 nginx
-
集群中随意一个IP + 端口
http://192.168.145.132:30432/
-
可视化界面 dashboard
-
dashboard 作为一个普通的work节点,部署在k8s上。
-
去官网找到对应的兼容版本 https://github.com/kubernetes/dashboard/releases/
-
安装指令
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.1/aio/deploy/recommended.yaml
-
验证是否成功
master@master:~$ kubectl get pods --namespace=kubernetes-dashboard NAME READY STATUS RESTARTS AGE dashboard-metrics-scraper-799d786dbf-98d6p 1/1 Running 0 38s kubernetes-dashboard-fb8648fd9-z85h8 1/1 Running 0 38s
-
暴露端口,让外面访问
kubectl patch svc kubernetes-dashboard -p '{"spec":{"type":"NodePort"}}' -n kubernetes-dashboard
可以看到已有端口
master@master:~$ kubectl get svc -n kubernetes-dashboard NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE dashboard-metrics-scraper ClusterIP 10.100.43.199 <none> 8000/TCP 14m kubernetes-dashboard NodePort 10.108.211.17 <none> 443:32699/TCP 14m
-
集群中任意一台主机IP+端口访问,注意加 HTTPS
-
创建用户,进入dashboard
cat <<EOF > account.yml # Create Service Account apiVersion: v1 kind: ServiceAccount metadata: name: admin-user namespace: kube-system --- # Create ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: admin-user namespace: kube-system EOF
-
应用 yml
kubectl apply -f account.yml
-
查看token
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
可以得到
Name: admin-user-token-lt7q6 Namespace: kube-system Labels: <none> Annotations: kubernetes.io/service-account.name: admin-user kubernetes.io/service-account.uid: 7aee1787-fc20-4c9a-84b6-645ddfd08158 Type: kubernetes.io/service-account-token Data ==== ca.crt: 1099 bytes namespace: 11 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6IkR2RWxIbkkwYW1Fb0Nnb1I1YWU1TUdaVDYwN1d2S1lRWk5XOVB5QTZmQjAifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLWx0N3E2Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI3YWVlMTc4Ny1mYzIwLTRjOWEtODRiNi02NDVkZGZkMDgxNTgiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.ziGadAjGozAPE4uLljn0Y1Fl-nAsJSti3gSwDglAssBBVI7I2MyX1XAMtFi7UYpj_uUQj5QRoXMC34PGztq1xHLDrJZgOz_LmOAnt5NJUC0k5GrV0pg5syob8NAaJmx4piEhbIitnPwczxS1FD4Et-IEQyyRN8kaJX2aAoL89DCWXDIpbmUqNbRfZ1R2Ldug1VPTNziIXtpquyTnpKHgCIZETBNijoPtduxfEVqwodVAAUkbuVYyE0eUWgQGSxwU2BcFpAzhMzJ3rkyNkAgJLJ8izu3wuQMDlbXxx5yf3DhvJXvphEVPT3PUyWnNj8_guIghHlAepiTq9nBSaVJXaA
-
进入dashboard
得到token后,就可以进入dashboard
2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.ziGadAjGozAPE4uLljn0Y1Fl-nAsJSti3gSwDglAssBBVI7I2MyX1XAMtFi7UYpj_uUQj5QRoXMC34PGztq1xHLDrJZgOz_LmOAnt5NJUC0k5GrV0pg5syob8NAaJmx4piEhbIitnPwczxS1FD4Et-IEQyyRN8kaJX2aAoL89DCWXDIpbmUqNbRfZ1R2Ldug1VPTNziIXtpquyTnpKHgCIZETBNijoPtduxfEVqwodVAAUkbuVYyE0eUWgQGSxwU2BcFpAzhMzJ3rkyNkAgJLJ8izu3wuQMDlbXxx5yf3DhvJXvphEVPT3PUyWnNj8_guIghHlAepiTq9nBSaVJXaA
- 进入dashboard
得到token后,就可以进入dashboard
更多推荐
所有评论(0)