K8S环境部署

一、master部署

1、配置hosts文件

vim /etc/hosts

10.200.124.177      ECFDP-DW-AW1-01

10.200.124.178      ECFDP-DW-AW2-01

10.200.124.179      ECFDP-DW-AW3-01

10.200.124.180      ECFDP-DW-AW4-01

10.200.124.181      ECFDP-DW-AW1-02-01

10.200.124.182      ECFDP-DW-AW2-02-01

10.200.124.183      ECFDP-DW-AW3-02-01

10.200.124.184      ECFDP-DW-AW4-02-01

备注:

User:root
Pass:  XXXXXX 

2、设置对应hostname

hostnamectl set-hostname --static ECFDP-DW-AW1-01

hostnamectl set-hostname --transient  ECFDP-DW-AW1-01

3、关闭firewalld,并设置开机不启动。

systemctl stop firewalld

systemctl disable firewalld

4、关闭swap

swapoff -a

sed -i '/ swap /  s/^\(.*\)$/#\1/g' /etc/fstab

5、关闭selinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

6、修改支持k8s的内核参数

cat > /etc/sysctl.d/k8s.conf <<EOF

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_forward = 1

EOF

7、开启br_netfilter

modprobe br_netfilter

sysctl -p /etc/sysctl.d/k8s.conf

8、安装依赖工具包:

cd /ecfdpuser/k8s_c75-sign_install

tar xvf rpms/util_rpm.tar.gz

rpm -ivh --replacefiles --replacepkgs util_rpm/*.rpm

9、安装docker

cd /ecfdpuser/k8s_c75-sign_install

tar xvf rpms/docker_rpm.tar.gz

rpm -ivh --replacefiles --replacepkgs docker_rpm/*.rpm

10、设置dockercgroupdriver

cat > /etc/docker/daemon.json <<EOF

{

"exec-opts": ["native.cgroupdriver=systemd"]

}

EOF

11、启动docker

systemctl enable docker.service

systemctl start docker.service

12、安装k8s

tar xvf rpms/k8s_rpm.tar.gz

rpm -ivh --replacefiles --replacepkgs k8s_rpm/*.rpm

13、导入docker image

docker load -i imgs/k8s_18.tar

docker load -i imgs/dashboard2.tar

14、初始化集群

kubeadm init \

    --kubernetes-version=v1.18.2 \

    --apiserver-advertise-address 172.20.10.6 \

    --pod-network-cidr=10.244.0.0/16

sha256 编码获取

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

        

15、配置kubectl配置文件

mkdir -p $HOME/.kube

cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

chown $(id -u):$(id -g) $HOME/.kube/config

systemctl enable kubelet.service

systemctl start kubelet.service

16、安装flannel网络组件

mkdir -p /run/flannel/

cat >> /run/flannel/subnet.env << EOF

FLANNEL_NETWORK=10.244.0.0/16

FLANNEL_SUBNET=10.244.0.1/24

FLANNEL_MTU=1450

FLANNEL_IPMASQ=true

EOF

kubectl apply -f yamls/kube-flannel.yml   (这yml文件里要注意实际网卡名称,否则报错)

kubectl get pod --all-namespaces

二、node节点部署

1、修改hosts

vim /etc/hosts

10.200.124.177      ECFDP-DW-AW1-01

10.200.124.178      ECFDP-DW-AW2-01

10.200.124.179      ECFDP-DW-AW3-01

10.200.124.180      ECFDP-DW-AW4-01

10.200.124.181      ECFDP-DW-AW1-02-01

10.200.124.182      ECFDP-DW-AW2-02-01

10.200.124.183      ECFDP-DW-AW3-02-01

10.200.124.184      ECFDP-DW-AW4-02-01

2、关闭防火墙

systemctl stop firewalld

systemctl disable firewalld

 3、关闭swap

swapoff -a

sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

4、关闭selinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

 5、修改支持k8s的内核参数

cat > /etc/sysctl.d/k8s.conf <<EOF

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_forward = 1

EOF

6、开启br_netfilter

modprobe br_netfilter

sysctl -p /etc/sysctl.d/k8s.conf

7、安装依赖工具包:

cd /ecfdpuser/k8s_c75-sign_install

tar xvf rpms/util_rpm.tar.gz

rpm -ivh --replacefiles --replacepkgs util_rpm/*.rpm

8、安装docker

cd /ecfdpuser/k8s_c75-sign_install

tar xvf rpms/docker_rpm.tar.gz

rpm -ivh --replacefiles --replacepkgs docker_rpm/*.rpm

9、设置dockercgroupdriver

cat > /etc/docker/daemon.json <<EOF

{

"exec-opts": ["native.cgroupdriver=systemd"]

}

EOF

10、启动docker

systemctl enable docker.service

systemctl start docker.service

  1. 安装k8s

cd /ecfdpuser/k8s_c75-sign_install

tar xvf rpms/k8s_rpm.tar.gz

rpm -ivh --replacefiles --replacepkgs k8s_rpm/*.rpm

12、导入docker image

docker load -i imgs/k8s_18.tar

docker load -i imgs/dashboard2.tar

13、安装flannel网络组件

mkdir -p /run/flannel/

cat >> /run/flannel/subnet.env << EOF

FLANNEL_NETWORK=10.244.0.0/16

FLANNEL_SUBNET=10.244.0.1/24

FLANNEL_MTU=1450

FLANNEL_IPMASQ=true

EOF

14、加入k8s-master

kubeadm token list  查看token

3v38rf.6gyx26pc40r91a6f

kubeadm join 172.20.10.6:6443 --token 3v38rf.6gyx26pc40r91a6f

15、重启k8s服务

systemctl enable kubelet.service

systemctl start kubelet.service

三、dashboard部署

1、安装

cd /ecfdpuser/k8s_c75-sign_install

kubectl apply -f yamls/dashboard2_recommended.yaml

kubectl apply -f yamls/dashboard2_admin.yaml

查看登录oken:

kubectl describe secret dashboard -n kube-system

记录token

浏览器访问https://10.200.124.177:30000

2、设置账号密码登录

(1)生成密码文件

echo 'admin,admin,1' > /etc/kubernetes/pki/basic_auth_file

(2)编辑/etc/kubernetes/manifests/kube-apiserver.yaml,在- command:下面增加- --basic-auth-file=/etc/kubernetes/pki/basic_auth_file参数:

vim /etc/kubernetes/manifests/kube-apiserver.yaml

...

spec:

  containers:

  - command:

    - kube-apiserver

    - --advertise-address=10.1.0.160

......

    - --basic-auth-file=/etc/kubernetes/pki/basic_auth_file

......

(3)重启api-server

cd /etc/kubernetes/manifests

mv ./kube-apiserver.yaml ../

mv ../kube-apiserver.yaml ./

(4)更新api-server配置

kubectl apply -f /etc/kubernetes/manifests/kube-apiserver.yaml

(5)将用户与权限绑定

kubectl create clusterrolebinding  login-on-dashboard-with-cluster-admin  --clusterrole=cluster-admin --user=admin

查看绑定:

kubectl get clusterrolebinding login-on-dashboard-with-cluster-admin

6、修改dashboard2_recommended.yaml

开启authentication-mode=basic配置:

vim /ecfdpuser/k8s_c75_sign_instal/yamls/dashboard2_recommended.yaml

增加:

--token-ttl=43200

--authentication-mode=basic

args:

  - --auto-generate-certificates

  - --namespace=kubernetes-dashboard

  - --token-ttl=43200

  - --authentication-mode=basic

7、更新dashboard2_recommended.yaml

kubectl apply -f /data/k8s_c75_sign_instal/yamls/kubernetes-dashboard.yaml

注意:

生成密码文件的时候,basic验证方式要求用户名和密码要保持一致,id不能重复。

/etc/kubernetes/pki/basic_auth_file文件不会热更新,每次添加新用户之后都需要手动重启一下api-server。

要通过火狐浏览器访问dashboard。

四、Jenkins配置

登录10.200.124.184服务器操作:

cd /data/k8s_c75_sign_install/rpms

安装JDK支持

rpm -ivh jdk-8u251-linux-i586.rpm

cp jenkins.war apache-tomcat-8.5.55/webapps/

cd apache-tomcat-8.5.55/bin/

./startup.sh

通过访问 http://10.200.124.184:8080/jenkins 访问Jenkins

首次访问需要根据屏幕提示设置密码

cat /root/.jenkins/secrets/initialAdminPassword

五、Nexus 配置方法

登录服务器10.200.124.184服务器操作:

cd /data/k8s_c75_sign_install/rpms

tar xvf nexus-3.23.0-03-unix.tar.gz

sed -i 's/run_as_root=true/run_as_root=false/' nexus-3.23.0-03/bin/nexus

./nexus-3.23.0-03/bin/nexus start

* 通过访问 http://10.200.124.184:8081访问Nexus

* 首次访问:

登录名:admin  

密码服务器中查看:# cat sonatype-work/nexus3/admin.password

六、服务部署:

在master(10.200.124.177)主节点操作:

cd /ecfdpuser/seaboxdata/local-path_install

kubectl apply -f local-path-storage.yaml

执行:

kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

=========================================================================

部署mongodb

cd /ecfdpuser/seaboxdata/5_base-services/03_mongodb

执行以下命令:

kubectl apply -f mongodb-pv.yaml

kubectl apply -f deployment.yaml

验证是否部署成功:

kubectl get pod -n default

===============================================================================

部署redis

执行以下命令:

cd /ecfdpuser/seaboxdata/5_base-services/04_redis

kubectl apply -f deployment.yaml

===============================================================================

部署elasticsearch

cd /ecfdpuser/seaboxdata/5_base-services/05_elasticsearch

kubectl create  configmap es-config --from-file=./elasticsearch.yml

kubectl apply -f elasticsearch-pv.yaml

kubectl apply -f deployment.yaml

验证部署是否成功:

kubectl get pod -n default

安装orientdb

cd /ecfdpuser/seaboxdata/5_base-services/06_orientdb

kubectl apply -f orient-pv.yaml

kubectl apply -f deployment.yaml

验证:

kubectl get pod -n default

部署rockermq

cd /ecfdpuser/seaboxdata/5_base-services/07_rocketmq/console

kubectl apply -f deployment.yaml

cd /data/seaboxdata/5_base-services/07_rocketmq/broker

kubectl apply -f broker/broker-pv.yaml

kubectl apply -f deployment.yaml

cd /ecfdpuser/seaboxdata/5_base-services/07_rocketmq/name-server

kubectl apply -f ns-pv.yaml

kubectl apply -f deployment.yaml

检查是否发布成功:

kubectl get pod -n default

部署ambry

cd /ecfdpuser/seaboxdata/5_base-services/08_ambry

kubectl apply -f ambry-server-pv.yaml

kubectl apply -f server-deployment.yaml

kubectl apply -f ambry-client-pv.yaml

kubectl apply -f client-deployment.yaml

部署mysql

cd /ecfdpuser/seaboxdata/5_base-services/02_mysql

kubectl apply -f mysql-pv.yaml

kubectl apply -f deployment.yaml

kubectl get pod -n default

部署数据管控系统,权限认证中心

kubectl create namespace deploy

然后执行:

kubectl label node ecfdp-dw-aw1-01 ecfdp-dw-aw1-02-01 ecfdp-dw-aw2-01 ecfdp-dw-aw2-02-01 ecfdp-dw-aw3-01 ecfdp-dw-aw3-02-01 ecfdp-dw-aw4-01 env=deploy

部署eureka

cd /ecfdpuser/seaboxdata/7_eureka-service

kubectl apply -f eureka-service.yaml

查看是否部署成功:

kubectl get pod -n deploy

部署jvm及timezone

cd /ecfdpuser/seaboxdata/jvm/jvm

kubectl apply -f jvm.yaml

cd /ecfdpuser/seaboxdata/jvm

kubectl create configmap time-config --from-file=./timezone -n deploy

验证是否创建成功:

kubectl get configmap -n deploy

部署数据管控,权限认证中心

cd /ecfdpuser/seaboxdata/6_ej-service/server/

sh install-configmap.sh auth-frontier-service auth-service dqs-service ds-service mds-service tag-service usercenter-service kc-service pmc-service workflow-service mds-all-frontier-service

使用以下命令验证configmap是否部署成功

kubectl get configmap -n deploy

部署前端应用

创建前端configmap:

cd /ecfdpuser/seaboxdata/6_ej-service/webapp/runtimes/dev-ceb/

kubectl create -n deploy configmap ac-runtime-args --from-file=./ac-webapp

kubectl create -n deploy configmap ds-runtime-args --from-file=./ds-webapp

kubectl create -n deploy configmap mds-runtime-args --from-file=./mds-webapp

kubectl create -n deploy configmap dqs-runtime-args --from-file=./dqs-webapp

kubectl create -n deploy configmap kc-runtime-args --from-file=./kc-webapp

kubectl create -n deploy configmap login-runtime-args --from-file=./login

cd /ecfdpuser/seaboxdata/6_ej-service/webapp/deploy/

kubectl apply -f ac-deployment.yaml

kubectl apply -f login-deployment.yaml

kubectl apply -f mds-deployment.yaml

kubectl apply -f ds-deployment.yaml

kubectl apply -f dqs-deployment.yaml

kubectl apply -f kc-deployment.yaml

可通过以下命令查看是否部署成功:

 kubectl get pod -n deploy

初始化操作:

初始化orientdb数据库

进入orientdb数据库管理页面 地址为:http://10.200.124.177:30007

点击 NEW DB ->

ame项填mds_server_deploy

user项填 root

ServerPassword项填 root

->点击CREATE DATABASE 创建数据库

在主服务器上进入MDS_SERVER的pod中执行:

kubectl exec -it mds-service-749855b994-l7g5w -n deploy bas

curl -X GET http://10.96.41.153:8080/admin/graph/initSchema?token=ewekii1ecoiShae7Xuephoruh

curl -X GET http://10.96.41.153:8080/admin/tmpfile/init?token=ewekii1ecoiShae7Xuephoruh

curl -X GET http://10.96.41.153:8080/admin/es/init?token=ewekii1ecoiShae7Xuephoruh

dqs初始化:

kubectl exec -it dqs-service-8778bf57d-kjbwl  -n deploy bash

curl -X GET http://10.96.218.157:8080/admin/es/initExecution?token=DqsText

curl -X GET http://10.96.218.157:8080/admin/es/initSchedule?token=DqsText

Logo

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐