4、docker+k8s+kubesphere:master安装

检查docker、kubectl、kubadm状态

#docker 
systemctl status docker  查看状态
systemctl start docker   重启
systemctl stop docker    停止
systemctl restart docker 重启

#kubectl
systemctl status kubelet  查看状态
systemctl start kubelet   重启
systemctl stop kubelet    停止
systemctl restart kubelet 重启

第一种，yaml方式安装

cat <<EOF > ./kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.16.9
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers

#master地址
controlPlaneEndpoint: "192.168.5.151:6443"	
networking:
  serviceSubnet: "10.96.0.0/16"	

  #k8s容器组所在的网段
  podSubnet: "10.20.0.1/16"	
  dnsDomain: "cluster.local"
EOF

PS:注意这里的网段后面会用到

此时会生成一个kubeadm-config.yaml的yaml文件

安装master的yaml文件

kubeadm init --config=kubeadm-config.yaml --upload-certs

第二种，命令安装

 kubeadm init \
--apiserver-advertise-address=192.168.5.151   \
--image-repository registry.cn-hangzhou.aliyuncs.com/google_containers  \
--kubernetes-version= v1.16.9  \
--service-cidr=10.10.0.0/16  \
--pod-network-cidr=10.244.0.0/16 \

PS:注意这里的网段后面会用到

不管采用哪一种方式安装
1、检查docker是否启动
2、版本必须要与kubeadm、kubelet、kubectl保持一致
3、查看版本kubectl version
4、安装过程大概3分钟，根据机器，网络情况而定

设置k8s命令自动补全
yum -y install bash-completion
source /usr/share/bash-completion/bash_completion
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc

查看镜像

此时会有7个镜像产生
[root@node151 ~]# docker images
REPOSITORY                                                                    TAG                 IMAGE ID            CREATED             SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver            v1.16.9             dd3b6beaa554        3 months ago        160MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager   v1.16.9             b6f6512bb3ba        3 months ago        152MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy                v1.16.9             a197b1cf22e3        3 months ago        82.8MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler            v1.16.9             476ac3ab84e5        3 months ago        83.6MB
registry.cn-hangzhou.aliyuncs.com/google_containers/etcd                      3.3.15-0            b2756210eeab        11 months ago       247MB
registry.cn-hangzhou.aliyuncs.com/google_containers/coredns                   1.6.2               bf261d157914        11 months ago       44.1MB
registry.cn-hangzhou.aliyuncs.com/google_containers/pause                     3.1                 da86e6ba6ca1        2 years ago         742kB

安装成功结果如下

[root@node151 ~]# kubeadm init --config=kubeadm-config.yaml --upload-certs
W0724 14:34:58.312794    2137 validation.go:28] Cannot validate kubelet config - no validator is available
W0724 14:34:58.312851    2137 validation.go:28] Cannot validate kube-proxy config - no validator is available
[init] Using Kubernetes version: v1.17.5
[preflight] Running pre-flight checks
	[WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [node151 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.5.151 192.168.5.151]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [node151 localhost] and IPs [192.168.5.151 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [node151 localhost] and IPs [192.168.5.151 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
W0724 14:35:43.150495    2137 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[control-plane] Creating static Pod manifest for "kube-scheduler"
W0724 14:35:43.152092    2137 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 35.004800 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
85bfab9efd786219f55e164208471f58c63ab740d9c77c23677fb4ba82c17987
[mark-control-plane] Marking the node node151 as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node node151 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: mcudha.jbcama8vtpsxwexy
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:
 
 要执行这段代码
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.   集群创建成功
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 192.168.5.151:6443 --token mcudha.jbcama8vtpsxwexy \
    --discovery-token-ca-cert-hash sha256:9cd6057cfb7372fbe19a056d372d439c0a2688487d0b140a184b91cf2aa3ccdf \
    --control-plane --certificate-key 85bfab9efd786219f55e164208471f58c63ab740d9c77c23677fb4ba82c17987

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:


节点执行指端代码，加入集群，但是这里的token只有24小时就要过期，需要创建永不过期的token
kubeadm join 192.168.5.151:6443 --token mcudha.jbcama8vtpsxwexy \
    --discovery-token-ca-cert-hash sha256:9cd6057cfb7372fbe19a056d372d439c0a2688487d0b140a184b91cf2aa3ccdf 

执行kubeconfig 文件

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

查看服务

kubectl get pod --all-namespaces

结果如下

[root@node151 ~]# kubectl get pod --all-namespaces
NAMESPACE     NAME                              READY   STATUS    RESTARTS   AGE
kube-system   coredns-7f9c544f75-gmclr          0/1     Pending   0          16m
kube-system   coredns-7f9c544f75-t7jh6          0/1     Pending   0          16m
kube-system   etcd-node151                      1/1     Running   0          16m
kube-system   kube-apiserver-node151            1/1     Running   0          16m
kube-system   kube-controller-manager-node151   1/1     Running   0          16m
kube-system   kube-proxy-5t7jg                  1/1     Running   0          16m
kube-system   kube-scheduler-node151            1/1     Running   0          16m

查看node

 kubectl get node

结果如下

[root@node151 ~]#  kubectl get node
NAME      STATUS     ROLES    AGE   VERSION
node151   NotReady   master   20m   v1.17.5

这里是正常的，因为还没有在node上执行token，查看下面token执行方式
服务还没有启动

token管理(过期、删除、重新生成等)

查看token
[root@node151 ~]# kubeadm token list
TOKEN                     TTL         EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
9e3xwd.nuladzqb5rjbb427   1h          2020-07-24T16:36:18+08:00   <none>                   Proxy for managing TTL for the kubeadm-certs secret        <none>
mcudha.jbcama8vtpsxwexy   23h         2020-07-25T14:36:19+08:00   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token

删除token（可以不管只是为了演示）
kubeadm token delete mcudha.jbcama8vtpsxwexy

创建临时token
kubeadm token create --print-join-command

创建永久token（推荐）
kubeadm token create --ttl 0 --print-join-command

创建永久token结果如下
[root@node151 ~]# kubeadm token create --ttl 0 --print-join-command
W0724 14:44:33.276404    3652 validation.go:28] Cannot validate kubelet config - no validator is available
W0724 14:44:33.276496    3652 validation.go:28] Cannot validate kube-proxy config - no validator is available
kubeadm join 192.168.5.151:6443 --token uwt7uq.w932sasaow3jd8sg     --discovery-token-ca-cert-hash sha256:9cd6057cfb7372fbe19a056d372d439c0a2688487d0b140a184b91cf2aa3ccdf 
[root@node151 ~]# kubeadm token list
TOKEN                     TTL         EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
9e3xwd.nuladzqb5rjbb427   1h          2020-07-24T16:36:18+08:00   <none>                   Proxy for managing TTL for the kubeadm-certs secret        <none>
mcudha.jbcama8vtpsxwexy   23h         2020-07-25T14:36:19+08:00   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token
uwt7uq.w932sasaow3jd8sg   <forever>   <never>   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token

[root@node151 ~]# 

master安装失败怎么办

[root@node151 ~]# kubeadm token list
TOKEN                     TTL         EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
9e3xwd.nuladzqb5rjbb427   1h          2020-07-24T16:36:18+08:00   <none>                   Proxy for managing TTL for the kubeadm-certs secret        <none>
mcudha.jbcama8vtpsxwexy   23h         2020-07-25T14:36:19+08:00   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token
uwt7uq.w932sasaow3jd8sg   <forever>   <never>   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token
[root@node151 ~]# kubeadm config images list
I0724 14:46:39.575724    3683 version.go:251] remote version is much newer: v1.18.6; falling back to: stable-1.17
W0724 14:46:40.454686    3683 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0724 14:46:40.454722    3683 validation.go:28] Cannot validate kubelet config - no validator is available
k8s.gcr.io/kube-apiserver:v1.17.9
k8s.gcr.io/kube-controller-manager:v1.17.9
k8s.gcr.io/kube-scheduler:v1.17.9
k8s.gcr.io/kube-proxy:v1.17.9
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.4.3-0
k8s.gcr.io/coredns:1.6.5


新建文件master_images.sh，内容如下（注意版本）
#!/bin/bash
images=(
	kube-apiserver:v1.17.9
	kube-controller-manager:v1.17.9
	kube-scheduler:v1.17.9
	kube-proxy:v1.17.5
	pause:3.1
	etcd:3.4.3-0
	coredns:1.6.5
)
for imageName in ${images[@]} ; do
#    docker pull registry.aliyuncs.com/google_containers/$imageName
    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
#   docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName  k8s.gcr.io/$imageName
done

执行
sh master_images.sh
或者
chmod 700 master_images.sh
./master_images.sh

下载完后执行，查看下载了那些镜像
docker images

命令集合

watch kubectl get pod -n kube-system -o wide 监控kube-system服务
kubeadm token list 查看
kubeadm token delete xxxx.xxxxxxx 删除
kubeadm token create --print-join-command 创建临时
kubeadm token create --ttl 0 --print-join-command 创建永久token（推荐）
kubectl get node 查看所有node
kubectl get pod --all-namespaces 查看所有服务