文章目录

K8S多节点部署

一、实验过程

1、实验准备

先具备单master节点部署环境

Master01：192.168.150.128/24          kube-apiserver kube-controller-manager kube-scheduler etcd 

Node01：   192.168.150.179/24          kubelet kube-proxy docker flannel etcd

Node02：   192.168.150.163/24          kubelet kube-proxy docker flannel etcd

Master02：192.168.150.130/24          安装软件同Master01

lb01：         192.168.150.131/24          nginx  keepalived

lb02：         192.168.150.132/24          nginx  keepalived

2、上传文件（master01）

systemctl stop firewalld.service

setenforce 0

scp -r /opt/kubernetes/ root@192.168.150.130:/opt               ##复制kubernetes目录到master02

yes

Abc123                             ##输入master02的密码

scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.150.130:/usr/lib/systemd/system/

##复制master中的三个组件启动脚本kube-apiserver.service          kube-controller-manager.service     kube-scheduler.service

Abc123

scp -r /opt/etcd/ root@192.168.150.130:/opt/                     ##上传etch证书，证书一定要有

Abc123

3、修改配置文件（Master02）

cd /opt/kubernetes/cfg/

vim kube-apiserver ##配置文件修改内容如下所示

KUBE_APISERVER_OPTS="–logtostderr=true \

–v=4 \

–etcd-servers=https://192.168.150.128:2379,https://192.168.150.179:2379,https://192.168.150.163:2379 \

–bind-address=192.168.150.130 \

–secure-port=6443 \

–advertise-address=192.168.150.130 \

–allow-privileged=true \

–service-cluster-ip-range=10.0.0.0/24 \

–enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \

–authorization-mode=RBAC,Node \

–kubelet-https=true \

–enable-bootstrap-token-auth \

–token-auth-file=/opt/kubernetes/cfg/token.csv \

–service-node-port-range=30000-50000 \

–tls-cert-file=/opt/kubernetes/ssl/server.pem \

–tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \

–client-ca-file=/opt/kubernetes/ssl/ca.pem \

–service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \

–etcd-cafile=/opt/etcd/ssl/ca.pem \

–etcd-certfile=/opt/etcd/ssl/server.pem \

–etcd-keyfile=/opt/etcd/ssl/server-key.pem"

4、启动三个组件服务（master02）

systemctl start kube-apiserver.service

systemctl start kube-controller-manager.service

systemctl start kube-scheduler.service

vim /etc/profile

#末尾添加一行

export PATH=$PATH:/opt/kubernetes/bin

source /etc/profile

kubectl get node

5、安装nginx服务（lb01和lb02）

systemctl stop firewalld.service

setenforce 0

vim /etc/yum.repos.d/nginx.repo       ##文件中新增内容如下所示

[nginx]

name=nginx repo

baseurl=http://nginx.org/packages/centos/7/$basearch/

gpgcheck=0

yum install nginx -y

vim /etc/nginx/nginx.conf ##添加四层转发

events {

worker_connections 1024;

}

stream {

log_format main ‘$remote_addr u p s t r e a m a d d r − [ upstream_addr - [ upstreama​ddr−[time_local] $status $upstream_bytes_sent’;

access_log /var/log/nginx/k8s-access.log main;

upstream k8s-apiserver {

​ server 192.168.150.128:6443;

​ server 192.168.150.130:6443;

}

server {

​ listen 6443;

​ proxy_pass k8s-apiserver;

}

}

http {

systemctl start nginx

yum install keepalived -y                                                             ##部署keepalived服务

cp keepalived.conf /etc/keepalived/keepalived.conf              ##修改配置文件

yes

vim /etc/keepalived/keepalived.conf

//注意：lb01是Mster配置如下：

! Configuration File for keepalived

global_defs {

# 接收邮件地址

notification_email {

acassen@firewall.loc

failover@firewall.loc

sysadmin@firewall.loc

}

# 邮件发送地址

notification_email_from Alexandre.Cassen@firewall.loc

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id NGINX_MASTER

}

vrrp_script check_nginx {

script “/etc/nginx/check_nginx.sh”

}

vrrp_instance VI_1 {

state MASTER

interface ens33

virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的

priority 100 # 优先级，备服务器设置 90

advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒

authentication {

​ auth_type PASS

​ auth_pass 1111

}

virtual_ipaddress {

​ 192.168.150.100/24

}

track_script {

​ check_nginx

}

}

//注意：lb02是Backup配置如下：

! Configuration File for keepalived

global_defs {

# 接收邮件地址

notification_email {

acassen@firewall.loc

failover@firewall.loc

sysadmin@firewall.loc

}

# 邮件发送地址

notification_email_from Alexandre.Cassen@firewall.loc

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id NGINX_MASTER

}

vrrp_script check_nginx {

script “/etc/nginx/check_nginx.sh”

}

vrrp_instance VI_1 {

state BACKUP

interface ens33

virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的

priority 90 # 优先级，备服务器设置 90

advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒

authentication {

​ auth_type PASS

​ auth_pass 1111

}

virtual_ipaddress {

​ 192.168.195.100/24

}

track_script {

​ check_nginx

}

}

vim /etc/nginx/check_nginx.sh ##新增内容如下所示

count=$(ps -ef |grep nginx |egrep -cv "grep|$$")

if [ "$count" -eq 0 ];then

  systemctl stop keepalived

fi

chmod +x /etc/nginx/check_nginx.sh

systemctl start keepalived

ip a

//验证地址漂移（lb01中使用pkill nginx，再在lb02中使用ip a 查看）

//恢复操作（在lb01中先启动nginx服务，再启动keepalived服务）

//nginx站点/usr/share/nginx/html

6、修改node节点配置文件统一为VIP地址

vim /opt/kubernetes/cfg/bootstrap.kubeconfig

vim /opt/kubernetes/cfg/kubelet.kubeconfig

vim /opt/kubernetes/cfg/kube-proxy.kubeconfig

//三个配置文件中统统修改为VIP

server: https://192.168.150.100:6443

systemctl restart kubelet.service

systemctl restart kube-proxy.service 

cd /opt/kubernetes/cfg

grep 100 *                                            ##替换完成直接自检

7、查看k8s日志（lb01）

tail /var/log/nginx/k8s-access.log

8、测试（master01）

kubectl run nginx --image=nginx               ##测试创建pod

[root@localhost ~]# kubectl get pods                                           ##查看状态

NAME           READY  STATUS        RESTARTS  AGE

nginx-dbddb74b8-gcf9h   0/1   ContainerCreating  0      33s  //正在创建中

[root@localhost ~]# kubectl get pods

NAME           READY  STATUS   RESTARTS  AGE

nginx-dbddb74b8-gcf9h   1/1   Running  0      80s  //创建完成，运行中

kubectl logs nginx-dbddb74b8-gcf9h                      //查看日志

Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-nf9sk)

kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

kubectl get pods -o wide             ##查看pod网络

nginx-dbddb74b8-gcf9h   1/1     Running   0          6m    172.17.56.2   192.168.150.163   <none>

9、在节点上操作可以直接访问（node02）

curl 172.17.56.2                   ##直接访问可以看到nginx界面信息

10、在k8s结点上查看日志（master01）

kubectl logs nginx-dbddb74b8-gcf9h
# docker # nginx # linux # kubernetes # 云计算
Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes