前言

本文搜集大量关于Kubernetes的文章，取其精华，去其糟粕，以小白的视角进行学习。

Kubernetes咱也是第一次接触，甚至对Docker都没怎么玩过，写的有错请及时指正。

kubernetes是什么？

• 名称
  • Kubernetes源于希腊语，意为「舵手」或「飞行员」，简称是k8s，用8代替[ubernete]。
• 起源
  • 在容器技术之前，是在硬件层面实现虚拟化的虚拟机。虚拟机技术的代表是VMware和OpenStack。虚拟机虽然可以隔离出很多“子电脑”，但占用空间大，启动慢。而容器技术在操作系统层面实现虚拟化，只需要虚拟一个小规模的环境（类似“沙箱”），启动时间快，占的空间小，对资源的利用率很高（一台主机可以同时运行几千个Docker容器）。
  • 目前Docker容器技术非常火热，但是将Docker用于实际业务，面对成千上万的容器时，存在编排、管理、调度各方面的困难，手里的Docker突然感觉不香了，因此迫切需要一套管理系统，对Docker及容器灵活高效的管理。这个时候，Google家的Kubernetes闪亮登场了。

kubernetes架构

每个Kubernetes集群都需要一个Master(Control plane 控制平面)，它决定资源分配，运行状态检查和调度、协调通信。其他的服务器为Slave(Compute machines 计算设备|节点)，主要接受和运行工作负载，创建或销毁容器等。

• Master
  • etcd：保存了整个集群的状态
  • API Server：提供了资源操作的唯一入口，并提供认证、授权、访问控制、API注册和发现等机制
  • Controller Manager：负责维护集群的状态，比如故障检测、自动扩展、滚动更新等
  • Scheduler：负责资源的调度，按照预定的调度策略将Pod调度到相应的机器上
• Slave(Node)
  • Kubelet：负责维护容器的生命周期，同时也负责Volume（CSI）和网络（CNI）的管理
  • Kube-proxy：负责为Service提供cluster内部的服务发现和负载均衡
  • Pod：Kubernetes管理的基本单元（最小单元），Pod 内部是容器，Kubernetes不直接管理容器，而是管理 Pod
  • Fluentd：获取容器日志文件、过滤和转换日志数据
  • Flannel：为 Kubernetes 提供 overlay network 的网络插件，它基于 Linux TUN/TAP，使用 UDP 封装 IP 包来创建 overlay 网络，并借助 etcd维护网络的分配情况。

构建一个简单的Kubernetes集群

• 概述
  • 建议至少3台服务器，每台服务器4G内存，2个CPU核心以上，基本架构为1台master节点，2台slave节点。没有服务器的可以通过在线的测试平台https://www.katacoda.com/courses/kubernetes/playground或者https://labs.play-with-k8s.com/（本文所用），可以节点信息如下:

角色	主机名	IP地址
Master	Node1	192.168.0.13
Slave	Node2	192.168.0.12
Slave	Node3	192.168.0.11

• 安装准备
  • 手工搭建 Kubernetes 集群是一件很繁琐的事情，为了简化这些操作，就产生了很多安装配置工具，如 Kubeadm ，Kubespray，RKE 等组件。本文使用官方提供的Kubeadm，Kubeadm安装完以后，就可以使用它来快速安装部署Kubernetes集群了。先安装 Master 节点，然后将 Slave 节点一个个加入到集群中去。
  • 使用在线测试环境时，可以省去安装准备工作；完整的环境准备工作参考：https://www.qikqiak.com/k8strain/k8s-basic/install/，感觉把修改主机名称放到添加hosts信息前比较合适。
  • 下面只是安装下kubelet kubeadm kubectl

Ubuntu使用
apt-get install -y kubelet kubeadm kubectl --allow-unauthenticated
CentOS使用
yum -y install kubelet kubeadm kubectl

• Master节点初始化

  • 通过kubeadm init初始化Master节点

[node1 ~]$ kubeadm init --apiserver-advertise-address $(hostname -i) --pod-network-cidr 10.5.0.0/16
Initializing machine ID from random generator.
I0927 01:56:10.530784     645 version.go:252] remote version is much newer: v1.19.2; falling back to: stable-1.18
W0927 01:56:10.843108     645 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.18.9
[preflight] Running pre-flight checks
        [WARNING Service-Docker]: docker service is not active, please run 'systemctl start docker.service'
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.4.0-179-generic
DOCKER_VERSION: 19.03.11
DOCKER_GRAPH_DRIVER: vfs
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
        [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "", err: exit status 1
        [WARNING SystemVerification]: unsupported graph driver: vfs
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [node1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.13]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [node1 localhost] and IPs [192.168.0.13 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [node1 localhost] and IPs [192.168.0.13 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
W0927 01:56:40.015648     645 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[control-plane] Creating static Pod manifest for "kube-scheduler"
W0927 01:56:40.016844     645 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 24.502899 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node node1 as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node node1 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: 9cxumz.aplwrc79m3v5cftk
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.0.13:6443 --token 9cxumz.aplwrc79m3v5cftk \
    --discovery-token-ca-cert-hash sha256:016cdbc248a3a47b93d0f04e08c0fe19a6891adc7e2c5886098715503d50b716
Waiting for api server to startup
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
daemonset.apps/kube-proxy configured
No resources found

• 执行上面打印中的以下内容配置Kubectl。（中间那条执行可能会报错，因为在线的环境已经提前做好了链接）

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

• 这样Master的节点就配置好了，并且可以使用Kubectl来进行各种操作了，根据上面的提示接着往下做，将Slave节点加入到集群。

 

• Slave节点加入集群
  • 在所有Slave节点执行如下的命令（刚刚Master init打印中的内容），将Slave节点加入集群，正常的返回信息如下：

[node2 ~]$ kubeadm join 192.168.0.13:6443 --token 9cxumz.aplwrc79m3v5cftk \
>     --discovery-token-ca-cert-hash sha256:016cdbc248a3a47b93d0f04e08c0fe19a6891adc7e2c5886098715503d50b716
Initializing machine ID from random generator.
W0927 02:06:18.571095    2665 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
        [WARNING Service-Docker]: docker service is not active, please run 'systemctl start docker.service'
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.4.0-179-generic
DOCKER_VERSION: 19.03.11
DOCKER_GRAPH_DRIVER: vfs
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
        [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "", err: exit status 1
        [WARNING SystemVerification]: unsupported graph driver: vfs
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

• 在Master节点查看节点状态。coredns将在网络组建安装完成后Running起来。

[node1 ~]$ kubectl get node
NAME    STATUS     ROLES    AGE   VERSION
node1   NotReady   master   10m   v1.18.4
node2   NotReady   <none>   64s   v1.18.4
node3   NotReady   <none>   61s   v1.18.4

[node1 ~]$ kubectl get pod -n kube-system -o wide
NAME                            READY   STATUS    RESTARTS   AGE     IP             NODE     NOMINATED NODE   READINESS GATES
coredns-66bff467f8-rgzhs        0/1     Pending   0          12m     <none>         <none>   <none>           <none>
coredns-66bff467f8-xgctv        0/1     Pending   0          12m     <none>         <none>   <none>           <none>
etcd-node1                      1/1     Running   0          11m     192.168.0.13   node1    <none>           <none>
kube-apiserver-node1            1/1     Running   0          12m     192.168.0.13   node1    <none>           <none>
kube-controller-manager-node1   1/1     Running   0          11m     192.168.0.13   node1    <none>           <none>
kube-proxy-5lbv4                1/1     Running   0          12m     192.168.0.13   node1    <none>           <none>
kube-proxy-6m8xh                1/1     Running   0          3m34s   192.168.0.11   node3    <none>           <none>
kube-proxy-r5qvp                1/1     Running   0          3m37s   192.168.0.12   node2    <none>           <none>
kube-scheduler-node1            1/1     Running   0          11m     192.168.0.13   node1    <none>           <none>

• 安装网络插件
  • 安装

[node1 ~]$ kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml
configmap/kube-router-cfg created
daemonset.apps/kube-router created
serviceaccount/kube-router created
clusterrole.rbac.authorization.k8s.io/kube-router created
clusterrolebinding.rbac.authorization.k8s.io/kube-router created

• 再次查看节点状态

[node1 ~]$ kubectl get nodesNAME    STATUS   ROLES    AGE   VERSION
node1   Ready    master   36m   v1.18.4
node2   Ready    <none>   26m   v1.18.4
node3   Ready    <none>   26m   v1.18.4

[node1 ~]$ kubectl get pod -n kube-system -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP             NODE    NOMINATED NODE   READINESS GATES
coredns-66bff467f8-rgzhs        1/1     Running   0          36m   10.5.1.2       node2   <none>           <none>
coredns-66bff467f8-xgctv        1/1     Running   0          36m   10.5.2.2       node3   <none>           <none>
etcd-node1                      1/1     Running   0          35m   192.168.0.13   node1   <none>           <none>
kube-apiserver-node1            1/1     Running   0          35m   192.168.0.13   node1   <none>           <none>
kube-controller-manager-node1   1/1     Running   0          35m   192.168.0.13   node1   <none>           <none>
kube-proxy-5lbv4                1/1     Running   0          36m   192.168.0.13   node1   <none>           <none>
kube-proxy-6m8xh                1/1     Running   0          27m   192.168.0.11   node3   <none>           <none>
kube-proxy-r5qvp                1/1     Running   0          27m   192.168.0.12   node2   <none>           <none>
kube-router-tphnv               1/1     Running   0          67s   192.168.0.13   node1   <none>           <none>
kube-router-v2btr               1/1     Running   0          67s   192.168.0.12   node2   <none>           <none>
kube-router-w85lf               1/1     Running   0          67s   192.168.0.11   node3   <none>           <none>
kube-scheduler-node1            1/1     Running   0          35m   192.168.0.13   node1   <none>           <none>

参考：https://www.cnblogs.com/xiao987334176/p/12696740.html