集群信息

192.168.11.156 node1
192.168.11.158 node2  
192.168.11.159 node3

1-7 步骤 所有节点执行，8-9 主节点执行，10 子节点执行，11 主节点执行

1、安装基本服务

yum install -y net-tools epel-release 
yum install -y vim  yum-utils device-mapper-persistent-data lvm2

2、配置docker-ce 和 k8s yum 源

yum-config-manager \
    --add-repo \
    http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

3、防火墙和Selinux

sudo systemctl stop firewalld.service   #停止firewall
sudo systemctl disable firewalld.service #禁止firewall开机启动
sudo firewall-cmd --state             #查看防火墙状态

sudo setenforce 0
sudo vi /etc/selinux/config
#SELINUX修改为disabled
SELINUX=disabled 

4、安装docker-ce 和k8s

yum install docker-ce-18.06.0.ce    
# 直接 yum install docker-ce，docker-ce的版本太高，可能存在兼容问题，未测试
systemctl enable docker
systemctl start docker

yum install kubectl kubelet kubernetes-cni kubeadm 
systemctl enable kubelet 
# 默认安装需要禁用swap，这里配置/etc/sysconfig/kubelet  忽略禁用swap
vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--fail-swap-on=false"

5、桥接网络设置

modprobe br_netfilter
cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl -p /etc/sysctl.d/k8s.conf
ls /proc/sys/net/bridge

6、配置 /etc/hosts

添加
vim /etc/hosts
192.168.11.156 node1
192.168.11.158 node2
192.168.11.159 node3

7、准备k8s 需要的docker 镜像（因为外网原因，不能下载镜像）

docker 镜像下载
docker pull mirrorgooglecontainers/kube-apiserver-amd64:v1.13.1
docker pull mirrorgooglecontainers/kube-controller-manager-amd64:v1.13.1
docker pull mirrorgooglecontainers/kube-scheduler-amd64:v1.13.1
docker pull mirrorgooglecontainers/kube-proxy-amd64:v1.13.1
docker pull mirrorgooglecontainers/pause:3.1
docker pull mirrorgooglecontainers/etcd-amd64:3.2.24
docker pull coredns/coredns:1.2.6


docker tag mirrorgooglecontainers/kube-controller-manager-amd64:v1.13.1 k8s.gcr.io/kube-controller-manager:v1.13.1
docker tag mirrorgooglecontainers/kube-scheduler-amd64:v1.13.1 k8s.gcr.io/kube-scheduler:v1.13.1
docker tag mirrorgooglecontainers/kube-apiserver-amd64:v1.13.1 k8s.gcr.io/kube-apiserver:v1.13.1
docker tag mirrorgooglecontainers/kube-proxy-amd64:v1.13.1 k8s.gcr.io/kube-proxy:v1.13.1
docker tag mirrorgooglecontainers/pause:3.1 k8s.gcr.io/pause:3.1
docker tag mirrorgooglecontainers/etcd-amd64:3.2.24 k8s.gcr.io/etcd:3.2.24
docker tag coredns/coredns:1.2.6 k8s.gcr.io/coredns:1.2.6

#删除不需要的镜像
docker rmi mirrorgooglecontainers/kube-apiserver-amd64:v1.13.1
docker rmi mirrorgooglecontainers/kube-controller-manager-amd64:v1.13.1
docker rmi mirrorgooglecontainers/kube-scheduler-amd64:v1.13.1
docker rmi mirrorgooglecontainers/kube-proxy-amd64:v1.13.1
docker rmi mirrorgooglecontainers/pause:3.1
docker rmi mirrorgooglecontainers/etcd-amd64:3.2.24
# 查看
docker images

8、（主节点执行）集群初始化（请记录初始化最后打印出的kubeadm join 信息）

# --ignore-preflight-errors=Swap 忽略禁用swap ，必须要加上
kubeadm init \
   --kubernetes-version=v1.13.0 \
   --pod-network-cidr=10.244.0.0/16 \
   --apiserver-advertise-address=192.168.11.158 \
   --ignore-preflight-errors=Swap

#### 请记录初始化最后打印出的kubeadm join 信息
#### 日志里出现如下 join 信息说明初始化成功
#### You can now join any number of machines by running the following on each node
#### as root:
#### 
####   kubeadm join 192.168.61.11:6443 --token 702gz5.49zhotgsiyqimwqw --discovery-token-ca-cert-hash sha256:2bc50229343849e8021d2aa19d9d314539b40ec7a311b5bb6ca1d3cd10957c2f

#复制配置
mkdir ~/.kube
cp -i /etc/kubernetes/admin.conf ~/.kube/config
chown $(id -u):$(id -g) ~/.kube/config

   9、（主节点执行）配置flannel 网络（使用的镜像是quay.io/coreos/flannel:v0.10.0-amd64）

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl  apply -f kube-flannel.yml

### kube-flannel.yml 有可能下载不下来，多试几次，

10、（子节点加入集群）子节点执行命令

# 初始化log 中的join信息拼接上 --ignore-preflight-errors=Swap，否则会报错

kubeadm join 192.168.11.158:6443 --token yw5jli.0h16u7ta3iegkv58 --discovery-token-ca-cert-hash sha256:5c744e46595d5b486b683e266487dddf20e200701ab9e272972a421044996bf8  --ignore-preflight-errors=Swap
#确认日志无报错

#主节点查询 Ready 说明节点加入集群正常
kubectl get nodes
# NAME    STATUS   ROLES    AGE   VERSION
# node1   Ready    <none>   17h   v1.13.1
# node2   Ready    master   20h   v1.13.1
# node3   Ready    <none>   17h   v1.13.1

11、dashboard 安装

#为所有节点添加dashboard 镜像,因为不确定dashboard 在哪台机器上启动

# 所有节点执行
docker pull mirrorgooglecontainers/kubernetes-dashboard-amd64:v1.10.1
docker tag mirrorgooglecontainers/kubernetes-dashboard-amd64:v1.10.1 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1
docker rmi  mirrorgooglecontainers/kubernetes-dashboard-amd64:v1.10.1

主节点执行

wget https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
kubectl apply -f kubernetes-dashboard.yaml

# 查看dashboard 信息，Running 说明正常
kubectl get pods -n kube-system
  ### kubernetes-dashboard-79ff88449c-fpxrh   1/1     Running   0          17h

#创建 serviceaccount 用于登录 dashboard
kubectl create serviceaccount dashboard-admin -n kube-system

#创建clusterrolebinding
kubectl create clusterrolebinding cluster-dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin

#将dashboard的service 的类型改成NodePort  ，也可以在部署dashboard的时候修改下yaml文件
kubectl patch svc kubernetes-dashboard -p '{"spec":{"type":"NodePort"}}' -n kube-system

#查看secret 的具体名字 dashboard-admin-token-xxxxx
kubectl get secret -n kube-system
### 找到下面这一行 dashboard-admin-token-xxxxx
### dashboard-admin-token-9rj88                      kubernetes.io/service-account-token   3      17h


# 查看 token 信息
kubectl describe secret dashboard-admin-token-9rj88 -n kube-system
#结果如下图所示

# 复制如下图中的token 值，登录dashboard 会用到

# 查看dashboard 具体端口

# 通过如下命令查到dashboard 映射的端口是 31465
kubectl get svc -n kube-system
NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
kube-dns               ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP   19h
kubernetes-dashboard   NodePort    10.99.138.89   <none>        443:31465/TCP   18h

# 火狐浏览器访问 https://ip:31465，这个31465端口是上个命令查询到的，（我的谷歌访问不了，原因未知）

# 给网页添加信任，选择令牌，粘贴之前查询到的token 值，登录

 

12、问题参考

***** 主节点负载问题，使用kubeadm初始化的集群，出于安全考虑Pod不会被调度到Master Node上，也就是说Master Node不参与工作负载，这是因为当前的master节点被打上了node-role.kubernetes.io/master:NoSchedule的污点：如果想要让master节点参与工作负载

#kubectl describe node node2 | grep Taint
Taints:             node-role.kubernetes.io/master:NoSchedule
#kubectl taint nodes node2 node-role.kubernetes.io/master-
node "node2" untainted

***** kubernetes-dashboard.yaml 和 kube-flannel.yml 下载不下来，参考https://blog.csdn.net/qq_42006894/article/details/86219510

***** k8s 命令参考https://blog.csdn.net/qq_42006894/article/details/85987089