服务器环境

• centos7
• mac装的pd虚拟机

作用	IP	部署服务	配置
master	10.211.55.10	etcd、kube-apiserver、kube-controller-manager、kube-scheduler	2C、2G
node1	10.211.55.11	docker 、kubelet、kube-proxy	2C、2G
node2	10.211.55.12	docker 、kubelet、kube-proxy	2C、2G

- 计划采用二进制包进行部署：

所需二进制包下载地址：
1.https://dl.k8s.io/v1.10.4/kubernetes-server-linux-amd64.tar.gz
2.https://dl.k8s.io/v1.10.4/kubernetes-node-linux-amd64.tar.gz
3.https://github.com/coreos/etcd/releases/download/v3.2.22/etcd-v3.2.22-linux-amd64.tar.gz
注意所有服务器都需要关闭防火墙

Master部署

二进制安装基本都是以下几个步骤：
1、复制对应的二进制文件到/usr/bin目录下
2、创建systemd service启动服务文件
3、创建service中对应的配置参数文件
4、将该应用加入到开机自启
5、启动服务并查看服务状态

etcd部署

• 下载二进制安装包并安装：

wget https://github.com/coreos/etcd/releases/download/v3.2.22/etcd-v3.2.22-linux-amd64.tar.gz
cd etcd-v3.2.22-linux-amd64/
cp etcd /usr/bin/
cp etcdctl /usr/bin/
mkdir /var/lib/etcd
mkdir /etc/etcd

• 编辑systemd管理文件

vim /usr/lib/systemd/system/etcd.service

[Unit]
Description=Etcd Server
After=network.target

[Service]
Type=simple
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd

[Install]
WantedBy=multi-user.target

• 启动服务,并设置开机启动

systemctl daemon-reload
systemctl start etcd
systemctl enable etcd

• 查看服务状态的三种命令

systemctl status etcd.service

curl -L http://127.0.0.1:2379/version

etcdctl cluster-health

这个安装的还挺顺利，很快就ok了。继续。。。。

kube-apiserver

• 下载并安装

wget https://dl.k8s.io/v1.10.4/kubernetes-server-linux-amd64.tar.gz 
tar -xzvf kubernetes-server-linux-amd64.tar.gz  
cd kubernetes/server/bin
cp kube-apiserver /usr/bin/

# 一起拷贝吧，后面就直接配置了
cp kube-controller-manager /usr/bin/
cp kube-scheduler /usr/bin/

• 编辑systemd的启动文件

vim /usr/lib/systemd/system/kube-apiserver.service

[Unit]
Description=Kubernetes API Server
Documentation=https://kubernetes.io/docs/concepts/overview
After=network.target
After=etcd.service

[Service]
EnvironmentFile=/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver $KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

• 配置参数文件

mkdir /etc/kubernetes/
vim /etc/kubernetes/apiserver 

KUBE_API_ARGS="--storage-backend=etcd3 \
               --etcd-servers=http://127.0.0.1:2379 \
               --bind-address=0.0.0.0 \
               --secure-port=6443  \
               --service-cluster-ip-range=192.168.2.0/16  \
               --service-node-port-range=1-65535 \
               --client-ca-file=/etc/kubernetes/ssl/ca.crt \
               --tls-private-key-file=/etc/kubernetes/ssl/server.key  \
               --tls-cert-file=/etc/kubernetes/ssl/server.crt  \
               --enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,DefaultStorageClass,ResourceQuota \
               --logtostderr=false \
               --log-dir=/var/log/kubernetes \
               --v=2"

service-cluster-ip-range是servcies的虚拟IP的IP范围，这里可以自己定义，不能当前的宿主机网段重叠。
bind-addres 指定的apiserver监听地址，对应的监听端口是6443，使用的https的方式。(0.0.0.0 表示绑定所有地址)
client-ca-file 这是认证的相关文件，这预先定义，后面会创建证书文件，并放置到对应的路径。

• 创建日志目录和证书目录

mkdir -p /etc/kubernetes/ssl
mkdir -p /var/log/kubernete

kube-controller-manager

kube-controller-manager 依赖 kube-apiserver服务

• 编辑systemd启动文件

vim /usr/lib/systemd/system/kube-controller-manager.service 

[Unit]
Description=Kubernetes Controller Manager
Documentation=https://kubernetes.io/docs/setup
After=kube-apiserver.service
Requires=kube-apiserver.service

[Service]
EnvironmentFile=/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

• 配置启动参数

vim /etc/kubernetes/controller-manager 

KUBE_CONTROLLER_MANAGER_ARGS="--master=https://10.211.55.10:6443   \
               --service-account-private-key-file=/etc/kubernetes/ssl/server.key  \
               --root-ca-file=/etc/kubernetes/ssl/ca.crt \
               --kubeconfig=/etc/kubernetes/kubeconfig \
               --logtostderr=false \
               --log-dir=/var/log/kubernetes \
               --v=2"

kube-scheduler

kube-scheduler也依赖kubu-apiserver
- 编辑systemd启动文件

vim /usr/lib/systemd/system/kube-scheduler.service 

[Unit]
Description=Kubernetes Controller Manager 
Documentation=https://kubernetes.io/docs/setup
After=kube-apiserver.service
Requires=kube-apiserver.service

[Service]
EnvironmentFile=/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

• 配置参数文件

vim /etc/kubernetes/scheduler 

KUBE_SCHEDULER_ARGS="--master=https://10.211.55.10:6443 --kubeconfig=/etc/kubernetes/kubeconfig \ 
               --logtostderr=false \
               --log-dir=/var/log/kubernetes \
               --v=2"

创建CA证书

注意生成证书前先同步一下服务器时间：ntpdate s2m.time.edu.cn

• 创建kube-apiserver的CA证书和私钥文件

cd  /etc/kubernetes/ssl/
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=10.211.55.10" -days 5000 -out ca.crt
openssl genrsa -out server.key 2048

• 创建master_ssl.cnf文件

vim master_ssl.cnf

[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = k8s_master
IP.1 = 192.168.2.1     # ClusterIP 地址
IP.2 = 10.211.55.10    # master IP地址

• 生成apiserver证书

openssl req -new -key server.key -subj "/CN=10.211.55.10" -config master_ssl.cnf -out server.csr

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 5000 -extensions v3_req -extfile master_ssl.cnf -out server.crt

• 设置kube-controller-manager相关证书

openssl genrsa -out cs_client.key 2048
openssl req -new -key cs_client.key -subj "/CN=10.211.55.10" -out cs_client.csr
openssl x509 -req -in cs_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cs_client.crt -days 5000

• 创建kubeconfig文件,kube-controller-manager和kube-scheduler公用的配置文件

vim /etc/kubernetes/kubeconfig 

apiVersion: v1
kind: Config
users:
- name: controllermanager
  user:
    client-certificate: /etc/kubernetes/ssl/cs_client.crt
    client-key: /etc/kubernetes/ssl/cs_client.key
clusters:
- name: local
  cluster:
    certificate-authority: /etc/kubernetes/ssl/ca.crt
contexts:
- context:
    cluster: local
    user: controllermanager
  name: my-context
current-context: my-context

启动服务

• 启动kube-apiserver

systemctl daemon-reload
systemctl enable kube-apiserver
systemctl start kube-apiserver

• 启动kube-controller-manager

systemctl enable kube-controller-manager
systemctl start kube-controller-manager

• 启动kube-scheduler

systemctl enable kube-scheduler
systemctl start kube-scheduler

Node

安装docker

• 使用aliyun的yum源

curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
curl -o /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache

• yum安装docker工具

yum install docker-ce
systemctl start docker
systemctl enable docker

docker -v

安装kubelet服务

• 安装包下载，整理

wget https://dl.k8s.io/v1.10.4/kubernetes-node-linux-amd64.tar.gz
tar -xzvf kubernetes-node-linux-amd64.tar.gz
cd kubernetes/node/bin
cp * /usr/bin

• 添加systemctl启动配置

vim /usr/lib/systemd/system/kubelet.service
mkdir -p /var/lib/kubelet
mkdir -p /etc/kubernetes/
mkdir -p /var/log/kubernetes

[Unit]
Description=Kubelet Service
After=docker.service
Requires=docker.service
[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet $KUBELET_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

• kuberlet运行参数配置

安装kube-proxy服务

• 添加systemctl启动配置

vim /usr/lib/systemd/system/kube-proxy.service

[Unit]
Description=K8s kube-proxy Service
After=network.target
After=docker.service
After=network.target
After=network.service

[Service]
EnvironmentFile=/etc/kubernetes/kube-proxy
ExecStart=/usr/bin/kube-proxy $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

生成CA证书

• 将master节点上的kube-apiserver证书ca.crt和ca.key拷贝到Node上
• 使用ca.crt和ca.key生成node证书

openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=10.211.55.11" -out kubelet_client.csr
openssl x509 -req -in kubelet_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000

mkdir /etc/kubernetes/ssl
mv kubelet_client.* /etc/kubernetes/ssl/
mv ca.crt /etc/kubernetes/ssl/

• 配置kubeconfig

vim /etc/kubernetes/kubeconfig

apiVersion: v1
kind: Config
users:
- name: kubelet
  user:
      client-certificate: /etc/kubernetes/ssl/kubelet_client.crt
      client-key: /etc/kubernetes/ssl/kubelet_client.key
clusters:
- name: local
  cluster:
      certificate-authority: /etc/kubernetes/ssl/ca.crt
      server: https://10.211.55.10:6443
contexts:
- context:
      cluster: local
      user: kubelet
  name: my-context
current-context: my-context

• kubelet启动参数配置

vim /etc/kubernetes/kubelet

KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=10.211.55.11 --logtostderr=false --log-dir=/var/log/kubernetes --v=2 --fail-swap-on=false"

这里要注意–fail-swap-on=false或者禁用swap，我这里选择配置–fail-swap-on=false

• 设置kube-proxy启动参数

vim /etc/kubernetes/kube-proxy

KUBE_PROXY_ARGS="--master=https://10.211.55.10:6443 --kubeconfig=/etc/kubernetes/kubeconfig --logtostderr=false --log-dir=/var/log/kubernetes --v=2"

启动服务

 systemctl daemon-reload
 systemctl start kubelet.service
 systemctl status kubelet.service

 systemctl start kube-proxy
 systemctl status kube-proxy

node 2就按照上面的步骤进行安装即可