K8S部署(kubeadm方式)
系统环境软件版本操作系统CentOS7.8_x64 (mini)Docker19-ceKubernetes1.18节点组件角色IP组件k8s-master192.168.104.200k8s-node1192.168.104.201k8s-node2192.168.104.202基础环境配置–所有节点hosts配置cat >> /etc/hosts << EOF192.16
系统环境
软件 | 版本 |
---|---|
操作系统 | CentOS7.8_x64 (mini) |
Docker | 19-ce |
Kubernetes | 1.18 |
节点组件
角色 | IP | 组件 |
---|---|---|
k8s-master | 192.168.104.200 | |
k8s-node1 | 192.168.104.201 | |
k8s-node2 | 192.168.104.202 |
基础环境配置–所有节点
hosts配置
cat >> /etc/hosts << EOF
192.168.104.200 k8s-master
192.168.104.201 k8s-node1
192.168.104.202 k8s-node2
EOF
## 主机名修改
例如master
cat > /etc/hostname << EOF
k8s-master
EOF
系统配置文件sysctl设置
## 将桥接的IPv4流量传递到iptables的链
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
## 生效
sysctl --system
## 关闭缓存,配置/etc/fstab,永久关闭
swapoff -a
## 时间同步
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
ntpdate time.windows.com
## 创建目录结构
mkdir -pv /opt/k8s/{bin,cfg,ssl,logs}
##配置环境变量(根据节点情况)
echo 'export PATH=$PATH:/opt/k8s/bin/' >> /etc/profile
source /etc/profile
节点安装Docker
卸载本机docker
yum remove docker
安装依赖软件
yum install yum-utils device-mapper-persistent-data lvm2
添加docker yum源
阿里镜像源
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
Docker官方镜像源
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
安装docker
yum list docker-ce --showduplicates | sort -r
yum install docker-ce docker-ce-cli containerd.io
配置阿里云镜像加速
vim /etc/docker/daemon.json
{
"registry-mirrors": ["https://gsm39obv.mirror.aliyuncs.com"]
}
systemctl restart docker
## 使用docker info查看生效情况
启动docker
## 查看docker版本
docker -v
## 启动docker
systemctl start docker
# 配置docker开机启动
systemctl enable docker
kubeadm安装集群
组件安装(所有节点)
配置yum源
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
安装组件
yum install kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0 -y
systemctl enable kubelet
注意版本号,要与后面指定K8S版本号对应
创建集群(Master节点)
kubeadm init \
--apiserver-advertise-address=192.168.104.200 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.18.0 \
--service-cidr=10.10.0.0/16 \
--pod-network-cidr=10.244.0.0/16
由于默认拉取镜像地址k8s.gcr.io国内无法访问,这里指定阿里云镜像仓库地址。
使用kubectl工具:
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get nodes
安装flannel网络
kubectl create -f kube-flannel.yaml
## Node节点加入集群
kubeadm join 192.168.104.200:6443 --token 12n909.ptwiysemj7834ybh \
--discovery-token-ca-cert-hash sha256:12ce7522eaac8956410f31aa1c3e489fe0dc2ba978cbfd0a4db7f3a706f1d207
安装flannel网络
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
确保能够访问到quay.io这个registery。
如果Pod镜像下载失败,可以改成这个镜像地址:lizhenliang/flannel:v0.11.0-amd64
安装flannel后,节点依旧为NotReady状态,排查是节点无法加载cni插件,报错如下
no networks found in /etc/cni/net.d
master执行:
scp /etc/cni/net.d/10-flannel.conflist root@k8s-node1:/etc/cni/net.d/10-flannel.conflist
scp /etc/cni/net.d/10-flannel.conflist root@k8s-node2:/etc/cni/net.d/10-flannel.conflist
正常情况下不会有上面的报错,办证所有安装组件版本和k8s版本一直,node先加入集群,再安装flannel,基本不会出错
默认token有效期为24小时,当过期之后,该token就不可用了。这时就需要重新创建token,操作如下:
# kubeadm token create
# kubeadm token list
# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
63bca849e0e01691ae14eab449570284f0c3ddeea590f8da988c07fe2729e924
# kubeadm join 192.168.31.61:6443 --token nuja6n.o3jrhsffiqs9swnu --discovery-token-ca-cert-hash sha256:63bca849e0e01691ae14eab449570284f0c3ddeea590f8da988c07fe2729e924
部署Dashboard
下载地址
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml
文件内容
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: v1
kind: Namespace
metadata:
name: kubernetes-dashboard
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
ports:
- port: 443
targetPort: 8443
selector:
k8s-app: kubernetes-dashboard
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-certs
namespace: kubernetes-dashboard
type: Opaque
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-csrf
namespace: kubernetes-dashboard
type: Opaque
data:
csrf: ""
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-key-holder
namespace: kubernetes-dashboard
type: Opaque
---
kind: ConfigMap
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-settings
namespace: kubernetes-dashboard
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
rules:
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
verbs: ["get", "update", "delete"]
# Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kubernetes-dashboard-settings"]
verbs: ["get", "update"]
# Allow Dashboard to get metrics.
- apiGroups: [""]
resources: ["services"]
resourceNames: ["heapster", "dashboard-metrics-scraper"]
verbs: ["proxy"]
- apiGroups: [""]
resources: ["services/proxy"]
resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
verbs: ["get"]
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
rules:
# Allow Metrics Scraper to get metrics from the Metrics server
- apiGroups: ["metrics.k8s.io"]
resources: ["pods", "nodes"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubernetes-dashboard
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
containers:
- name: kubernetes-dashboard
image: kubernetesui/dashboard:v2.0.4
imagePullPolicy: Always
ports:
- containerPort: 8443
protocol: TCP
args:
- --auto-generate-certificates
- --namespace=kubernetes-dashboard
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
volumeMounts:
- name: kubernetes-dashboard-certs
mountPath: /certs
# Create on-disk volume to store exec logs
- mountPath: /tmp
name: tmp-volume
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
volumes:
- name: kubernetes-dashboard-certs
secret:
secretName: kubernetes-dashboard-certs
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-metrics-scraper
name: dashboard-metrics-scraper
namespace: kubernetes-dashboard
spec:
ports:
- port: 8000
targetPort: 8000
selector:
k8s-app: kubernetes-metrics-scraper
---
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: kubernetes-metrics-scraper
name: kubernetes-metrics-scraper
namespace: kubernetes-dashboard
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-metrics-scraper
template:
metadata:
labels:
k8s-app: kubernetes-metrics-scraper
spec:
containers:
- name: kubernetes-metrics-scraper
image: kubernetesui/metrics-scraper:v1.0.0
ports:
- containerPort: 8000
protocol: TCP
livenessProbe:
httpGet:
scheme: HTTP
path: /
port: 8000
initialDelaySeconds: 30
timeoutSeconds: 30
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
版本可以去dockerhub上去查
https://hub.docker.com/r/kubernetesui/dashboard/tags
默认Dashboard只能集群内部访问,修改Service为NodePort类型,暴露到外部(kubernetes-dashboard部分), 如下:
vi recommended.yaml
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
ports:
- port: 443
targetPort: 8443
nodePort: 30001
type: NodePort
selector:
k8s-app: kubernetes-dashboard
创建dashboard-admin帐号
cat >> recommended.yaml << EOF
---
# ------------------- dashboard-admin ------------------- #
apiVersion: v1
kind: ServiceAccount
metadata:
name: dashboard-admin
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: dashboard-admin
subjects:
- kind: ServiceAccount
name: dashboard-admin
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
EOF
如果不使用上面的方法授权帐号,也可以使用命令
// 先创建一个帐号
kubectl create serviceaccount dashboard-admin-01 -n kubernetes-dashboard
// 给账号授权角色
kubectl create clusterrolebinding dashboard-admin-01 --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin-01
// 获取角色帐号TOKEN令牌
kubectl describe secrets -n kubernetes-dashboard $(kubectl -n kubernetes-dashboard get secret | awk ‘/dashboard-admin-01/{print $1}’)
相关查询命令:
kubectl -n kubernetes-dashboard get/describe serviceaccount/clusterrolebinding/secret dashboard-admin-01
serviceaccount: 创建帐号
clusterrolebinding: 绑定角色
secret: token相关
部署kubernetes-dashboard
## 部署
kubectl apply -f recommended.yaml
## 查看pod分配节点信息
kubectl get all -n kubernetes-dashboard -o wide
## 查看svc信息
kubectl get -n kubernetes-dashboard svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.0.0.96 <none> 8000/TCP 2m16s
kubernetes-dashboard NodePort 10.0.0.142 <none> 443:30001/TCP 2m16s
获取TOKEN令牌
kubectl describe secrets -n kubernetes-dashboard dashboard-admin
Name: dashboard-admin-token-wn89j
Namespace: kubernetes-dashboard
Labels: <none>
Annotations: kubernetes.io/service-account.name: dashboard-admin
kubernetes.io/service-account.uid: d8bca1b6-7c35-424c-baa0-e7a70ffb2ffc
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1025 bytes
namespace: 20 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6InoyUmRYcjZQSTEwdUJiQV9sUW83VEZ0UVFlbWxzN3JxRXFFUmhhdElpTncifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4td244OWoiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZDhiY2ExYjYtN2MzNS00MjRjLWJhYTAtZTdhNzBmZmIyZmZjIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmVybmV0ZXMtZGFzaGJvYXJkOmRhc2hib2FyZC1hZG1pbiJ9.gKomAYQjXNz1is6RpeNUhtKBUgNUuotHI87O6YHYvp4xxeCuQimUG-9PY1MKqX-Myc1QOn_h1lTCj5lnOd9nzJsHsbC8DZKlif41lS2qEFtegukBegBkhAtBXMso1F6YwHH4LvWoOXpZMjZx_nbNFaoy38iTH8yJ6ErXCdtKPD3yoptmrTd3tpzjflrQkCNSMryAxgO_4E4DuUCbVYY8qMw_aP5v1hxL-rEX_lB9vTiO3nt1_WIekWTNT57clpxxhIosjAroZSmG9knON0PkqdsL_0JriBgFPmuYKZdc5wOXa-2wKVRuPgIBImo7wXvJ0euMiHjX6hsiufE2dFTtDw
浏览器访问节点https://IP:30001,使用上面生成的token,即可登录kubernetes-dashboard
创建一个示例
kubectl create deployment nginx --image=nginx
kubectl expose deployment/nginx --port=80 --type=NodePort --name=service-nginx
常见问题处理
浏览器不能访问
-
二进制 部署
注意你部署Dashboard的命名空间(之前部署默认是kube-system,新版是kubernetes-dashboard)
# 1、 删除默认的secret,用自签证书创建新的secret
kubectl delete secret kubernetes-dashboard-certs -n kubernetes-dashboard
kubectl create secret generic kubernetes-dashboard-certs \
--from-file=/opt/k8s/ssl/server-key.pem --from-file=/opt/k8s/ssl/server.pem -n kubernetes-dashboard
# 2、修改 dashboard.yaml 文件,在args下面增加证书两行
args:
# PLATFORM-SPECIFIC ARGS HERE
- --auto-generate-certificates
- --tls-key-file=server-key.pem
- --tls-cert-file=server.pem
kubectl apply -f kubernetes-dashboard.yaml
-
kubeadm 部署
注意你部署Dashboard的命名空间(之前部署默认是kube-system,新版是kubernetes-dashboard)
# 1、删除默认的secret,用自签证书创建新的secret
kubectl delete secret kubernetes-dashboard-certs -n kubernetes-dashboard
kubectl create secret generic kubernetes-dashboard-certs \
--from-file=/etc/kubernetes/pki/apiserver.key --from-file=/etc/kubernetes/pki/apiserver.crt -n kube-system
# 2、修改 dashboard的yaml 文件,在args下面增加证书两行
args:
# PLATFORM-SPECIFIC ARGS HERE
- --auto-generate-certificates
- --tls-key-file=apiserver.key
- --tls-cert-file=apiserver.crt
kubectl apply -f kubernetes-dashboard.yaml
Token过期
## 创建token
kubeadm token create
## 查看token
kubeadm token list
## 获取token
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt |openssl rsa -pubin -outform der 2>/dev/null |openssl dgst -sha256 -hex |sed 's/^.* //'
12ce7522eaac8956410f31aa1c3e489fe0dc2ba978cbfd0a4db7f3a706f1d207
kubeadm join 192.168.104.200:6443 --token 12n909.ptwiysemj7834ybh \
--discovery-token-ca-cert-hash sha256:12ce7522eaac8956410f31aa1c3e489fe0dc2ba978cbfd0a4db7f3a706f1d207
环境清理
kubeadm reset
在master上执行就是清除master上的,在node上执行就是清除node的
更多推荐
所有评论(0)