K8s-day1-kubeadmin安装K8s+部署
文章目录kubeadm安装k8s一、准备环境1、部署软件、系统要求2、节点规划二、开始部署k8s1、系统优化(所有节点)1)关闭swap分区2)关闭selinux、firewalld2、修改主机名并且做域名解析1)修改主机名(所有节点)2)追加以下内容到hosts文件(主节点)3、配置免密登录、分发公钥(主节点)4、更新并优化系统(所有节点)1)配置国内yum源2)升级内核版本5、安装 IPVS
·
文章目录
- KubeAdmin安装k8s
- 一、准备环境
- 二、准备虚拟机网络环境
- 三、开始部署k8s
- 1、系统优化(所有节点)
- 2、修改主机名并且做域名解析
- 3、配置免密登录、分发公钥(所有主节点)
- 4、更新并优化系统(所有节点)
- 5、安装 IPVS 、加载 IPVS 模块 (所有节点)
- 6、授权(所有节点)
- 7、内核参数优化(所有节点)
- 8、更新系统内核(所有节点)
- 9、安装基础软件(所有节点)
- 10、安装Docker (所有节点)
- 11、创建docker目录、启动服务(所有节点)
- 12、 同步集群时间(所有节点)
- 12、安装kubernetes(所有节点)
- 14、主节点初始化配置
- 15、安装集群网络插件(主节点)
- 16、将其他节点加入集群
- 17、测试集群DNS(主节点)
- 18、新增Master节点
- 19、让其他节点可用kubectl获取状态
KubeAdmin安装k8s
一、准备环境
1、部署软件、系统要求
软件 | 版本 |
---|---|
Centos | CentOS Linux release 7.5及以上 |
Docker | 19.03.12 |
Kubernetes | V1.19.1 |
Flannel | V0.13.0 |
Kernel-lm | kernel-lt-4.4.245-1.el7.elrepo.x86_64.rpm |
Kernel-lm-deve | kernel-lt-devel-4.4.245-1.el7.elrepo.x86_64.rpm |
2、节点规划
- IP建议采用192网段,避免与kubernetes内网冲突
准备机器 | IP | 配置 | 系统内核版本 |
---|---|---|---|
k8s-master1 | 192.168.12.11 | 2核2G | 4.4+ |
k8s-node1 | 192.168.12.12 | 2核2G | 4.4+ |
k8s-node2 | 192.168.12.13 | 2核2G | 4.4+ |
二、准备虚拟机网络环境
- 配置IP网段为192.168.12.0网段
- 指定网关,后续的eth0/eth1且与此网关地址对应
三、开始部署k8s
1、系统优化(所有节点)
1)关闭swap分区
- 一旦触发 swap,会导致系统性能急剧下降,所以一般情况下,K8S 要求关闭 swap
swapoff -a
echo 'KUBELET_EXTRA_ARGS="--fail-swap-on=false"' > /etc/sysconfig/kubelet
2)关闭selinux、firewalld
sed -i 's#enforcing#disabled#g' /etc/selinux/config
setenforce 0
systemctl disable --now firewalld
--now 可以立即生效,避免再stop一次~
2、修改主机名并且做域名解析
1)修改主机名(所有节点)
hostnamectl set-hostname k8s-master1
hostnamectl set-hostname k8s-node1
hostnamectl set-hostname k8s-node2
2)追加以下内容到hosts文件(所有节点)
cat >> /etc/hosts <<EOF
192.168.12.11 k8s-master1 m1
192.168.12.12 k8s-node1 n1
192.168.12.13 k8s-node2 n2
EOF
3、配置免密登录、分发公钥(所有主节点)
ssh-keygen -t rsa
for i in m1 n1 n2;do ssh-copy-id -i ~/.ssh/id_rsa.pub root@$i;done
4、更新并优化系统(所有节点)
1)配置国内yum源
- 默认情况下,CentOS 使用的是官方 yum 源,所以一般情况下在国内使用是非常慢,所以我们可以替换成 国内的一些比较成熟的 yum 源,例如:清华大学镜像源,网易云镜像源等等
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
curl -o /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
#查看内核版本,若内核高于4.0,可不加--exclud选项
[root@k8s-master1 ~]# uname -r
5.4.106-1.el7.elrepo.x86_64
# 刷新缓存 && 更新系统
yum makecache && yum update -y
2)升级内核版本
- 由于 Docker 运行需要较新的系统内核功能,例如 ipvs 等等,所以一般情况下,我们需要使用 4.0+以上版 本的系统内核
- 内核要求是 4.18+,如果是 CentOS 8 则不需要升级内核
yum update --exclud=kernel* -y # 不更新内核升级系统
wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-4.4.245-1.el7.elrepo.x86_64.rpm
wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-devel-4.4.245-1.el7.elrepo.x86_64.rpm
yum localinstall -y kernel-lt*
grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg
grubby --default-kernel
reboot
5、安装 IPVS 、加载 IPVS 模块 (所有节点)
- ipvs 是系统内核中的一个模块,其网络转发性能很高。一般情况下,我们首选 ipvs
yum install -y conntrack-tools ipvsadm ipset conntrack libseccomp
cat > /etc/sysconfig/modules/ipvs.modules <EOF
#!/bin/bash
ipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_fo ip_vs_nq ip_vs_sed ip_vs_ftp nf_conntrack"
for kernel_module in \${ipvs_modules};
do /sbin/modinfo -F filename \${kernel_module} > /dev/null 2>&1
if [ $? -eq 0 ]; then
/sbin/modprobe \${kernel_module}
fi
done
EOF
6、授权(所有节点)
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep ip_vs
7、内核参数优化(所有节点)
-
加载IPVS 模块、生效配置
-
内核参数优化的主要目的是使其更适合 kubernetes 的正常运行
cat > /etc/sysctl.d/k8s.conf <<EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
fs.may_detach_mounts = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
fs.file-max=52706963
fs.nr_open=52706963
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp.keepaliv.probes = 3
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp.max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp.max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.ip_conntrack_max = 65536
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.top_timestamps = 0
net.core.somaxconn = 16384
EOF
# 立即生效
sysctl --system
8、更新系统内核(所有节点)
1)下载完成后,推送至所有节点
wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-5.4.106-1.el7.elrepo.x86_64.rpm
wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-devel-5.4.106-1.el7.elrepo.x86_64.rpm
for i in n1 n2 m1 ; do scp kernel-lt-* $i:/root; done
2)执行安装
yum localinstall -y kernel-lt*
grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg
grubby --default-kernel
3)重启
reboot
9、安装基础软件(所有节点)
- 安装一些基础软件,是为了方便我们的日常使用
yum install wget expect vim net-tools ntp bash-completion ipvsadm ipset jq iptables conntrack sysstat libseccomp -y
10、安装Docker (所有节点)
- Docker 主要是作为 k8s 管理的常用的容器工具之一
1)CentOS 7版
# 安装扩展
yum install -y yum-utils device-mapper-persistent-data lvm2
# 配置源
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# 安装docker
yum install docker-ce -y
安装Docker报错解决
yum install docker-ce -y
# 此步骤若出现报错:
GPG key retrieval failed: [Errno 14] curl#6 - "Could not resolve host: download.docker.com; Unknown error"
# 原因:无法解析主机
# 解决方法1:临时添加114域名解析到配置文件(临时,且不可重启网卡)
[root@k8s-master1 ~] cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 114.114.114.114
nameserver 223.5.5.5
# 解决方法2:添加到eth0网卡配置文件内(永久,需重启网卡)
dns1=114.114.114.114
2)CentOS 8版
# 下载rpm包
wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io1.2.13-3.2.el7.x86_64.rpm
# 安装
yum install containerd.io-1.2.13-3.2.el7.x86_64.rpm -y
# 安装扩展
yum install -y yum-utils device-mapper-persistent-data lvm2
# 配置源
yum-config-manager --add-repo
https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# 安装docker
yum install docker-ce -y
11、创建docker目录、启动服务(所有节点)
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://8mh75mhz.mirror.aliyuncs.com"]
}
# 启动+重启+自启
sudo systemctl daemon-reload;systemctl restart docker;systemctl enable --now docker.service
# 现在启动,相当于systemctl start docker.service
--now docker.service
12、 同步集群时间(所有节点)
- 在集群当中,时间是一个很重要的概念,一旦集群当中某台机器时间跟集群时间不一致,可能会导致集群面 临很多问题。所以,在部署集群之前,需要同步集群当中的所有机器的时间
1)CentOS7 版
yum install ntp -y
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' > /etc/timezone
ntpdate time2.aliyun.com
# 写入定时任务
*/1 * * * * ntpdate time2.aliyun.com > /dev/null 2>&1
2)CentOS8 版
rpm -ivh http://mirrors.wlnmp.com/centos/wlnmp-release-centos.noarch.rpm
yum install wntp -y
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' > /etc/timezone
ntpdate time2.aliyun.com
# 写入定时任务
*/1 * * * * ntpdate time2.aliyun.com >
12、安装kubernetes(所有节点)
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
#此处指定版本下载为了与下边初始化节点版本对应
yum install kubectl-1.20.2 kubeadm-1.20.2 kubelet-1.20.2 -y
# 此时只需开机自启,无需启动,因为还未初始化
systemctl enable kubelet
14、主节点初始化配置
1)集群初始化
# 关闭swap
sed -i 's/KUBELET_EXTRA_ARGS=/KUBELET_EXTRA_ARGS="--fail-swap-on=false"/' /etc/sysconfig/kubelet
# 初始化master节点
kubeadm init \
--image-repository=registry.cn-hangzhou.aliyuncs.com/k8sos \
--kubernetes-version=v1.20.2 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--ignore-preflight-errors=Swap
# 过程中可监控初始化日志,出现successfully即为成功!
# tailf /var/log/messages
# [root@k8s-master1 ~] cat /var/log/messages | grep successfully
# Mar 24 21:02:07 k8s-master1 containerd: time="2021-03-24T21:02:07.063840628+08:00" level=info msg="containerd successfully booted in 0.181480s"
# 初始化完成后,会自动启动,无需启动
systemctl status kubelet
2) 配置 kubernetes 用户信息
- master节点执行
# 建立用户集群权限
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 添加至环境变量
export KUBECONFIG=/etc/kubernetes/admin.conf
3)增加命令提示
- 所有节点执行
yum install -y bash-completion
source /usr/share/bash-completion/bash_completion
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc
15、安装集群网络插件(主节点)
- kubernetes 需要使用第三方的网络插件来实现 kubernetes 的网络功能,这样一来,安装网络插件成为必要前提
- 第三方网络插件有多种,常用的有 flanneld、calico 和 cannel(flanneld+calico),不同的网络组件,都提供基本的网络功能,为各个 Node 节点提供 IP 网络等
# 此方式可能会失败,建议复制下方flannel.yaml直接部署~
docker pull registry.cn-hangzhou.aliyuncs.com/k8sos/flannel:v0.12.0-amd64 ;\
docker tag registry.cn-hangzhou.aliyuncs.com/k8sos/flannel:v0.12.0-amd64 quay.io/coreos/flannel:v0.12.0-amd64
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
1)编写flannel.yaml
cat >> /root/flannel.yaml <<EOF
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp.flannel.unprivileged
annotations:
seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
privileged: false
volumes:
- configMap
- secret
- emptyDir
- hostPath
allowedHostPaths:
- pathPrefix: "/etc/cni/net.d"
- pathPrefix: "/etc/kube-flannel"
- pathPrefix: "/run/flannel"
readOnlyRootFilesystem: false
# Users and groups
runAsUser:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
fsGroup:
rule: RunAsAny
# Privilege Escalation
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
# Capabilities
allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
defaultAddCapabilities: []
requiredDropCapabilities: []
# Host namespaces
hostPID: false
hostIPC: false
hostNetwork: true
hostPorts:
- min: 0
max: 65535
# SELinux
seLinux:
# SELinux is unused in CaaSP
rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
rules:
- apiGroups: ['extensions']
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- ""
resources:
- nodes
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:
- kind: ServiceAccount
name: flannel
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- linux
hostNetwork: true
priorityClassName: system-node-critical
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: registry.cn-hangzhou.aliyuncs.com/alvinos/flanned:v0.13.1-rc1
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: registry.cn-hangzhou.aliyuncs.com/alvinos/flanned:v0.13.1-rc1
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN", "NET_RAW"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
EOF
2)安装网络插件
[root@k8s-master1 ~] cat flannel.yaml | grep image
image: registry.cn-hangzhou.aliyuncs.com/alvinos/flanned:v0.13.1-rc1
image: registry.cn-hangzhou.aliyuncs.com/alvinos/flanned:v0.13.1-rc1
# 安装网络插件
kubectl apply -f flannel.yaml
3)检查集群初始化状态
- READY状态为1/1即为成功!
[root@k8s-master1 ~] kubectl get pods -o wide -n kube-system
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-f68b4c98f-j8b2z 1/1 Running 0 83m 10.244.0.2 k8s-master1 <none> <none>
coredns-f68b4c98f-psg8n 1/1 Running 0 83m 10.244.0.3 k8s-master1 <none> <none>
etcd-k8s-master1 1/1 Running 0 83m 192.168.12.11 k8s-master1 <none> <none>
kube-apiserver-k8s-master1 1/1 Running 0 83m 192.168.12.11 k8s-master1 <none> <none>
kube-controller-manager-k8s-master1 1/1 Running 0 83m 192.168.12.11 k8s-master1 <none> <none>
kube-flannel-ds-nswg6 1/1 Running 0 41m 192.168.12.11 k8s-master1 <none> <none>
kube-proxy-lv9t7 1/1 Running 0 83m 192.168.12.11 k8s-master1 <none> <none>
kube-scheduler-k8s-master1 1/1 Running 0 83m 192.168.12.11 k8s-master1 <none> <none>
4)检查集群状态
- 状态为Ready即为成功!
[root@k8s-master1 ~] kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready control-plane,master 87m v1.20.5
16、将其他节点加入集群
1)主节点创建集群
- node节点要查看token,主节点生成token可重复执行查看,不会改变~
[root@k8s-master1 ~] kubeadm token create --print-join-command
kubeadm join 192.168.12.11:6443 --token iypm65.p5nmdzzw1zifxy6c --discovery-token-ca-cert-hash sha256:8bdbe324980e3350aaa3b9cea58edf576dc0a6d937da6b7bff6dbe6a01e0b525
2)从节点加入集群
[root@k8s-node1 ~] kubeadm join 192.168.12.11:6443 --token iypm65.p5nmdzzw1zifxy6c --discovery-token-ca-cert-hash sha256:8bdbe324980e3350aaa3b9cea58edf576dc0a6d937da6b7bff6dbe6a01e0b525 --ignore-preflight-errors=Swap
[root@k8s-node2 ~] kubeadm join 192.168.12.11:6443 --token iypm65.p5nmdzzw1zifxy6c --discovery-token-ca-cert-hash sha256:8bdbe324980e3350aaa3b9cea58edf576dc0a6d937da6b7bff6dbe6a01e0b525 --ignore-preflight-errors=Swap
3)加入集群报错解决
1> 报错原因1
从节点加入集群可能会出现如下报错:
[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
PS:前提安装Docker+启动,再次尝试加入节点!
# 报错原因:
swap没关,一旦触发 swap,会导致系统性能急剧下降,所以一般情况下,所以K8S 要求关闭 swap
# 解决方法:
1> 执行以下三条命令后再次执行添加到集群命令:
modprobe br_netfilter
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward
2> 追加 --ignore-preflight-errors=Swap 参数重新执行即可!
[root@k8s-node1 ~] kubeadm join 192.168.12.11:6443 --token iypm65.p5nmdzzw1zifxy6c --discovery-token-ca-cert-hash sha256:8bdbe324980e3350aaa3b9cea58edf576dc0a6d937da6b7bff6dbe6a01e0b525 --ignore-preflight-errors=Swap
2> 报错原因2
# 报错原因:
可能是内核参数忘记优化所有节点都需优化
# 解决方法:
回到上面第七小节,复制粘贴优化参数即可: cat > /etc/sysctl.d/k8s.conf <<EOF
3> 报错原因3
- ode节点加入集群失败,状态为NotReady
情况1:软件版本不一致
[root@gdx1 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
gdx1 Ready control-plane,master 73m v1.20.2
gdx2 NotReady <none> 10m v1.21.0
gdx3 NotReady <none> 26s v1.21.0
# 原因分析:
默认下载是最新版本,难免出现版本不一致的问题,所以下载时指定同一版本才行
# 解决方法:
yum install kubectl-1.20.2 kubeadm-1.20.2 kubelet-1.20.2 -y # 主从节点安装指定版本格式
kubectl reset # 重新初始化
kubeadm join 192.168.12.11:6443 --token fm0387.iqixomz5jmsukwsi --discovery-token-ca-cert-hash sha256:d8ff83ffed5967000034d07b3da738ae4f1f0254e8417bb30c21f3ed15ac5d18 # 重新将node节点加入集群
# PS:查看token值命令
[root@gdx1 ~]# kubeadm token list
kubeadm join 192.168.12.11:6443 --token fm0387.iqixomz5jmsukwsi --discovery-token-ca-cert-hash sha256:d8ff83ffed5967000034d07b3da738ae4f1f0254e8417bb30c21f3ed15ac5d18
8ff83ffed5967000034d07b3da738ae4f1f0254e8417bb30c21f3ed15ac5d18
情况2:软件版本一致,touken值可能不对
[root@gdx1 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
gdx1 Ready control-plane,master 73m v1.20.2
gdx2 NotReady <none> 10m v1.20.2
gdx3 NotReady <none> 26s v1.20.2
# node节点为notready状态,加入从节点时报错:
[ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists
[ERROR Port-10250]: Port 10250 is in use
[ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
# 原因分析:因为多次生成主节点的token值,导致token值加入不一致或输入错误
# 解决方法:
# 从集群移除状态为notready的node节点
[root@gdx1 ~]# kubectl delete node gdx3
# node节点重置touken值以及证书端口号等信息
[root@gdx3 ~]# kubeadm reset
# 主节点重新创建token
[root@gdx1 ~]# kubeadm token create --print-join-command
kubeadm join 192.168.12.11:6443 --token fm0387.iqixomz5jmsukwsi --discovery-token-ca-cert-hash sha256:d8ff83ffed5967000034d07b3da738ae4f1f0254e8417bb30c21f3ed15ac5d18
# 将node节点重新加入集群
kubeadm join 192.168.12.11:6443 --token fm0387.iqixomz5jmsukwsi --discovery-token-ca-cert-hash sha256:d8ff83ffed5967000034d07b3da738ae4f1f0254e8417bb30c21f3ed15ac5d18
# 再次查看状态为ready
[root@gdx1 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
gdx1 Ready control-plane,master 93m v1.20.2
gdx2 Ready <none> 30m v1.20.2
gdx3 Ready <none> 16m v1.20.2
3> 报错原因4
# 加入集群是种失败,一直未NotReady状态,没什么报错,只是显示部分提醒
# 解决方法:
查看node节点是否缺失bin目录即里面的文件,若缺失,将其他节点或master节点的bin推送过来即可
建议直接scp覆盖,node节点再重新执行:
kubeadm reset
kubeadm join ··· 重新加入集群即可
[root@k8s-m1 code] ls -l /opt/cni/bin/
总用量 56484
-rwxr-xr-x 1 root root 3254624 9月 10 2020 bandwidth
-rwxr-xr-x 1 root root 3581192 9月 10 2020 bridge
-rwxr-xr-x 1 root root 9837552 9月 10 2020 dhcp
-rwxr-xr-x 1 root root 4699824 9月 10 2020 firewall
-rwxr-xr-x 1 root root 2650368 9月 10 2020 flannel
-rwxr-xr-x 1 root root 3274160 9月 10 2020 host-device
-rwxr-xr-x 1 root root 2847152 9月 10 2020 host-local
-rwxr-xr-x 1 root root 3377272 9月 10 2020 ipvlan
-rwxr-xr-x 1 root root 2715600 9月 10 2020 loopback
-rwxr-xr-x 1 root root 3440168 9月 10 2020 macvlan
-rwxr-xr-x 1 root root 3048528 9月 10 2020 portmap
-rwxr-xr-x 1 root root 3528800 9月 10 2020 ptp
-rwxr-xr-x 1 root root 2849328 9月 10 2020 sbr
-rwxr-xr-x 1 root root 2503512 9月 10 2020 static
-rwxr-xr-x 1 root root 2820128 9月 10 2020 tuning
-rwxr-xr-x 1 root root 3377120 9月 10 2020 vlan
[root@k8s-m1 code] scp -r /opt/cni/bin n2:/opt/cni/
4)查看从节点是否加入集群(主节点)
- status若为Ready即为加入成功!
[root@k8s-master1 ~] kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready control-plane,master 12h v1.20.5
k8s-node1 Ready <none> 3m2s v1.20.5
k8s-node2 Ready <none> 2m21s v1.20.5
17、测试集群DNS(主节点)
[root@k8s-master1 ~] kubectl run test -it --rm --image=busybox:1.28.3
# 出现如下内容表示集群正常可用!
If you don't see a command prompt, try pressing enter.
/ #
/ #
/ # nslookup kubernetes
Server: 10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
Name: kubernetes
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local
/ #
/ #
18、新增Master节点
- 新节点准备目录
rm -rf /etc/kubernetes
mkdir -p /etc/kubernetes/pki/etcd
- 推送原Master节点的配置文件到新节点
scp /etc/kubernetes/pki/{ca.crt,ca.key,sa.key,sa.pub,front-proxy-ca.crt,front-proxy-ca.key} m2:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/etcd/{ca.crt,ca.key} m2:/etc/kubernetes/pki/etcd/{ca.crt,ca.key} m2:/etc/kubernetes/pki/etcd
scp /etc/kubernetes/pki/etcd/{ca.crt,ca.key} m2:/etc/kubernetes/pki/etcd/
scp /etc/kubernetes/admin.conf m2:/etc/kubernetes/
- 原Master节点查看token值,并复制到新节点执行
kubeadm token create --print-join-command
# 复制到新节点执行
kubeadm join 172.23.0.241:6443 --token lnvj7t.c7mc3254dnz3kp0u --discovery-token-ca-cert-hash sha256:2c0cdaae024d5668cece036cca6e2696eee92da5a92188b89da74c8364bb5251 --ignore-preflight-errors=Swap
加入报错解决:
# 报错1:
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
# 解决:关闭Swap配置
增加 --ignore-preflight-errors=Swap 选项即可
# 报错2
[ERROR Port-10250]: Port 10250 is in use
可能还有报错1的提示,添加选项后可无视
# 解决:提示端口占用,可能是因为之前加入过集群,加入失败残留端口占用,重置后重新加入即刻
kubeadmin reset
kubeadm join 172.23.0.241:6443 --token lnvj7t.c7mc3254dnz3kp0u --discovery-token-ca-cert-hash sha256:2c0cdaae024d5668cece036cca6e2696eee92da5a92188b89da74c8364bb5251 --ignore-preflight-errors=Swap
此时m2新master节点已成功加入集群!
[root@k8s-m1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-m1 Ready control-plane,master 15d v1.20.2
k8s-m2 Ready <none> 15m v1.20.2
k8s-n1 Ready <none> 15d v1.20.2
k8s-n2 Ready <none> 15d v1.20.2
19、让其他节点可用kubectl获取状态
- 让其他节点可用kubectl获取nodes、cs等状态信息
# 将master1节点的admin.conf推送至其他节点对应目录
for i in m1 m2 n1 n2 ;do scp /etc/kubernetes/admin.conf $i:/etc/kubernetes/
# 其他节点加入环境变量,即可使用
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source ~/.bash_profile
# 查看
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-m1 Ready control-plane,master 18d v1.20.2
k8s-m2 Ready <none> 3d4h v1.20.2
k8s-n1 Ready <none> 18d v1.20.2
k8s-n2 Ready <none> 18d v1.20.2
更多推荐
已为社区贡献8条内容
所有评论(0)