单机k8s实验环境安装 安装记录

2020-08

kubernetes v1.18.5
CentOS Linux release 7.6.1810 (Core)

1. 关掉 selinux

   $ setenforce 0
   $ sed -i “s/^SELINUX=enforcing/SELINUX=disabled/g” /etc/sysconfig/selinux

2. 关掉防火墙

   $ systemctl stop firewalld
   $ systemctl disable firewalld

3.关闭 swap

$ swapoff -a 
$ sed -i 's/.*swap.*/#&/' /etc/fstab

1. 配置转发参数

   $ cat < /etc/sysctl.d/k8s.conf
   net.bridge.bridge-nf-call-ip6tables = 1
   net.bridge.bridge-nf-call-iptables = 1
   EOF
   $ sysctl --system

2. 安装docker

   $ yum -y install docker

   $ vim /usr/lib/systemd/system/docker.service

   修改 --exec-opt native.cgroupdriver=systemd 中 systemd 为 cgroupfs

   $ systemctl start docker.service
   $ systemctl enable docker.service

3. 配置阿里云源

   $ cat < /etc/yum.repos.d/kubernetes.repo
   [kubernetes]
   name=Kubernetes
   baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
   enabled=1
   gpgcheck=0
   repo_gpgcheck=0
   gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
   http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
   EOF

4. 安装基础包（所有节点）

   $ yum install -y kubelet kubeadm kubectl

5. 镜像准备

   参考 国内拉去 k8s 镜像
   注意，k8s镜像版本 v1.13.2

6. 初始化

7. #kubelet开机自启
   systemctl enable kubelet
   
   #生成安装k8s的配置文件kubeadm.conf
   kubeadm config print init-defaults ClusterConfiguration > kubeadm.conf
   
   #查看k8s安装过程都需要安装哪些镜像
   kubeadm config images list --config ./kubeadm.conf
   
   #提前拉取k8s相关的镜像
   kubeadm config images pull --config ./kubeadm.conf

8. v1.13.2 是准备 k8s 镜像的版本，10.1.6.168 是本机器ip
   cpay-cloud-registry.default.svc.cluster.local
   ,"–spring.profiles.active=test"

   [root@op ~]# kubeadm init --kubernetes-version=v1.13.2 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=10.1.6.168
   [init] Using Kubernetes version: v1.13.2
   [preflight] Running pre-flight checks
   [WARNING Service-Kubelet]: kubelet service is not enabled, please run ‘systemctl enable kubelet.service’
   [preflight] Pulling images required for setting up a Kubernetes cluster
   [preflight] This might take a minute or two, depending on the speed of your internet connection
   [preflight] You can also perform this action in beforehand using ‘kubeadm config images pull’
   [kubelet-start] Writing kubelet environment file with flags to file “/var/lib/kubelet/kubeadm-flags.env”
   [kubelet-start] Writing kubelet configuration to file “/var/lib/kubelet/config.yaml”
   [kubelet-start] Activating the kubelet service
   [certs] Using certificateDir folder “/etc/kubernetes/pki”
   [certs] Generating “front-proxy-ca” certificate and key
   [certs] Generating “front-proxy-client” certificate and key
   [certs] Generating “etcd/ca” certificate and key
   [certs] Generating “apiserver-etcd-client” certificate and key
   [certs] Generating “etcd/server” certificate and key
   [certs] etcd/server serving cert is signed for DNS names [op localhost] and IPs [10.1.6.168 127.0.0.1 ::1]
   [certs] Generating “etcd/peer” certificate and key
   [certs] etcd/peer serving cert is signed for DNS names [op localhost] and IPs [10.1.6.168 127.0.0.1 ::1]
   [certs] Generating “etcd/healthcheck-client” certificate and key
   [certs] Generating “ca” certificate and key
   [certs] Generating “apiserver-kubelet-client” certificate and key
   [certs] Generating “apiserver” certificate and key
   [certs] apiserver serving cert is signed for DNS names [op kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.1.6.168]
   [certs] Generating “sa” key and public key
   [kubeconfig] Using kubeconfig folder “/etc/kubernetes”
   [kubeconfig] Writing “admin.conf” kubeconfig file
   [kubeconfig] Writing “kubelet.conf” kubeconfig file
   [kubeconfig] Writing “controller-manager.conf” kubeconfig file
   [kubeconfig] Writing “scheduler.conf” kubeconfig file
   [control-plane] Using manifest folder “/etc/kubernetes/manifests”
   [control-plane] Creating static Pod manifest for “kube-apiserver”
   [control-plane] Creating static Pod manifest for “kube-controller-manager”
   [control-plane] Creating static Pod manifest for “kube-scheduler”
   [etcd] Creating static Pod manifest for local etcd in “/etc/kubernetes/manifests”
   [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory “/etc/kubernetes/manifests”. This can take up to 4m0s
   [apiclient] All control plane components are healthy after 24.003887 seconds
   [uploadconfig] storing the configuration used in ConfigMap “kubeadm-config” in the “kube-system” Namespace
   [kubelet] Creating a ConfigMap “kubelet-config-1.13” in namespace kube-system with the configuration for the kubelets in the cluster
   [patchnode] Uploading the CRI Socket information “/var/run/dockershim.sock” to the Node API object “op” as an annotation
   [mark-control-plane] Marking the node op as control-plane by adding the label “node-role.kubernetes.io/master=’’”
   [mark-control-plane] Marking the node op as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
   [bootstrap-token] Using token: 9uh8ic.4zlffjmxukja9idr
   [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
   [bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
   [bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
   [bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
   [bootstraptoken] creating the “cluster-info” ConfigMap in the “kube-public” namespace
   [addons] Applied essential addon: CoreDNS
   [addons] Applied essential addon: kube-proxy

   Your Kubernetes master has initialized successfully!

   To start using your cluster, you need to run the following as a regular user:

   mkdir -p $HOME/.kube
   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
   sudo chown (id−u):(id -u):(id−u):(id -g) $HOME/.kube/config

   You should now deploy a pod network to the cluster.
   Run “kubectl apply -f [podnetwork].yaml” with one of the options listed at:
   Installing Addons | Kubernetes

   You can now join any number of machines by running the following on each node
   as root:

   kubeadm join 192.168.206.128:6443 --token um7jc2.044efo248u26sg3t --discovery-token-ca-cert-hash sha256:dcb899cbfe03b9543f9270b5e1c1f0eeca37715d3e6402cd0a1cd28181f8f985
   [root@op ~]#

9. 配置 kubectl

   mkdir -p $HOME/.kube
   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
   sudo chown (id−u):(id -u):(id−u):(id -g) $HOME/.kube/config

   #如果用root
   export KUBECONFIG=/etc/kubernetes/admin.conf

10. 配置网络

    sysctl net.bridge.bridge-nf-call-iptables=1
    kubectl apply  -  f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml

11. 去掉 master 污点

k8s是通过污点的机制来禁止 pod 部署在master上的，去除后pod会部署到master上，就成了单机版 k8s了

$ kubectl describe node op
可以看到 
Taints:             node-role.kubernetes.io/master:NoSchedule

# 去除污点 nodes --all 这个应该也可以制定单个节点的，待实验
$ kubectl taint nodes --all node-role.kubernetes.io/master-

• 注

其实和就是和k8s小集群部署一样，少了个节点加入，多了个去除 master 节点的污点。

 安装问题总结：

1. kubelet 没加cgroup导致无法初始化节点
   1. /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

   Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --cgroup-driver=cgroupfs"

七、部署Dashboard

注：在master节点上进行如下操作

1.创建Dashboard的yaml文件

wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml

使用如下命令或直接手动编辑kubernetes-dashboard.yaml文件

sed -i 's/k8s.gcr.io/loveone/g' kubernetes-dashboard.yaml
sed -i '/targetPort:/a\ \ \ \ \ \ nodePort: 30001\n\ \ type: NodePort' kubernetes-dashboard.yaml

手动编辑kubernetes-dashboard.yaml文件时，需要修改两处内容，首先在Dashboard Deployment部分修改Dashboard镜像下载链接，由于默认从官方社区下载，而不“科学上网”是无法下载的，因此修改为：image: loveone/kubernetes-dashboard-amd64:v1.10.1 修改后内容如图：

此外，需要在Dashboard Service内容加入nodePort： 30001和type: NodePort两项内容，将Dashboard访问端口映射为节点端口，以供外部访问，编辑完成后，状态如图

2.部署Dashboard

kubectl create -f kubernetes-dashboard.yaml

3.创建完成后，检查相关服务运行状态

kubectl get deployment kubernetes-dashboard -n kube-system

kubectl get pods -n kube-system -o wide

kubectl get services -n kube-system

netstat -ntlp|grep 30001

4.在Firefox浏览器输入Dashboard访问地址：https://10.10.10.10:30001

5.查看访问Dashboard的认证令牌

kubectl create serviceaccount  dashboard-admin -n kube-system
kubectl create clusterrolebinding  dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')

6.使用输出的token登录Dashboard。

认证通过后，登录Dashboard首页如图

1. 参考
   kubeadm安装Kubernetes 1.14最佳实践_Kubernetes中文社区