构建和交付dubbo

5. 构建和交付dubbo5.1. 交付Provider到K8S5.1.1. CI流水线构建1.1 参数构建要点jenkins流水线配置的java项目的十个常用参数:参数名作用举例或说明app_name项目名dubbo_demo_serviceimage_namedocker镜像名app/dubbo-demo-servicegit_repo项目...

包泽旭

418人浏览 · 2020-09-07 17:59:00

包泽旭 · 2020-09-07 17:59:00 发布

5. 构建和交付dubbo

5.1. 交付Provider到K8S

5.1.1. CI流水线构建

1.1 参数构建要点

jenkins流水线配置的java项目的十个常用参数:

参数名	作用	举例或说明
app_name	项目名	dubbo_demo_service
image_name	docker镜像名	app/dubbo-demo-service
git_repo	项目的git地址	https://x.com/x/x.git
git_ver	项目的git分支或版本号	master
add_tag	镜像标签,常用时间戳	191203_1830
mvn_dir	执行mvn编译的目录	./
target_dir	编译产生包的目录	./target
mvn_cmd	编译maven项目的命令	mvc clean package -Dmaven.
base_image	项目的docker底包	不同的项目底包不一样,下拉选择
maven	maven软件版本	不同的项目可能maven环境不一样

除了base_image和maven是choice parameter，其他都是string parameter

1.2 创建流水线

1.2.1 创建流水线

创建名为dubbo-demo的流水线(pipeline),并设置Discard old builds 为如下

Discard old builds选项保留多少次老的构建	值
Days to keep builds	3
Max # of builds to keep	30

1.2.2 添加10个构建参数

This project is parameterized点击Add Parameter,分别添加如下10个参数

#第1个参数
参数类型 : String Parameter
Name : app_name
Description : 项目名 eg:dubbo-demo-service

#第2个参数
参数类型 : String Parameter
Name : image_name
Description : docker镜像名 eg: app/dubbo-demo-service

#第3个参数
参数类型 : String Parameter
Name : git_repo
Description : 仓库地址 eg: https://gitee.com/xxx/xxx.git

#第4个参数
参数类型 : String Parameter
Name : git_ver
Description : 项目的git分支或版本号

#第5个参数
参数类型 : String Parameter
Name : add_tag
Description : 
给docker镜像添加标签组合的一部分,如
$git_ver_$add_tag=master_191203_1830

#第6个参数
参数类型 : String Parameter
Name : mvn_dir
Default Value : ./
Description : 执行mvn编译的目录,默认是项目根目录, eg: ./

#第7个参数
参数类型 : String Parameter
Name : target_dir
Default Value : ./target
Description : 编译产生的war/jar包目录 eg: ./dubbo-server/target

#第8个参数
参数类型 : String Parameter
Name : mvn_cmd
Default Value : mvn clean package -Dmaven.test.skip=true
Description : 编译命令,常加上-e -q参数只输出错误

#第9个参数
参数类型 : Choice Parameter
Name : base_image
Choices :
base/jre7:7u80
base/jre8:8u112
Description : 项目的docker底包

#第10个参数
参数类型 : Choice Parameter
Name : maven
Choices :
3.6.1-8u232
2.2.1-xxxx
Description : 执行编译使用maven软件版本

1.2.3 添加完成效果如下:

mark

1.2.4 添加pipiline代码

流水线构建所用的pipiline代码语法比较有专门的生成工具
以下语句的作用大致是分为四步:拉代码->构建包->移动包-打docker镜像并推送

pipeline {
  agent any 
    stages {
      stage('pull') { //get project code from repo 
        steps {
          sh "git clone ${params.git_repo} ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.app_name}/${env.BUILD_NUMBER} && git checkout ${params.git_ver}"
        }
      }
      stage('build') { //exec mvn cmd
        steps {
          sh "cd ${params.app_name}/${env.BUILD_NUMBER}  && /var/jenkins_home/maven-${params.maven}/bin/${params.mvn_cmd}"
        }
      }
      stage('package') { //move jar file into project_dir
        steps {
          sh "cd ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.target_dir} && mkdir project_dir && mv *.jar ./project_dir"
        }
      }
      stage('image') { //build image and push to registry
        steps {
          writeFile file: "${params.app_name}/${env.BUILD_NUMBER}/Dockerfile", text: """FROM harbor.od.com/${params.base_image}
ADD ${params.target_dir}/project_dir /opt/project_dir"""
          sh "cd  ${params.app_name}/${env.BUILD_NUMBER} && docker build -t harbor.od.com/${params.image_name}:${params.git_ver}_${params.add_tag} . && docker push harbor.od.com/${params.image_name}:${params.git_ver}_${params.add_tag}"
        }
      }
    }
}

1.3 用流水线完成dubbo-service的构建

记得先在harbor中创建私有仓库app

1.3.1 选择参数化构建

进入dubbo-demo后,选择的参数化构建build with parameters ,填写10个构建的参数

参数名	参数值
app_name	dubbo-demo-service
image_name	app/dubbo-demo-service
git_repo	https://gitee.com/noah-luo/dubbo-demo-service.git
git_ver	master
add_tag	200509_0800
mvn_dir	./
target_dir	./dubbo-server/target
mvn_cmd	mvn clean package -Dmaven.test.skip=true
base_image	base/jre8:8u112
maven	3.6.1-8u332

1.3.2 填写完成效果如下

mark

1.3.3 执行构建并检查

填写完以后执行bulid
第一次构建需要下载很多依赖包，时间很长，抽根烟，喝杯茶
经过漫长的等待后，已经构建完成了点击打开 Blue Ocean查看构建历史及过程：

检查harbor是否已经有这版镜像：

mark

# 编译内容存放路径，数字目录表示某一个app编译的序号，每个目录下有自己的 Dockerfile
[root@hdss7-21 ~]# kubectl exec jenkins-54b8469cf9-v8g28 -n infra -- ls -l /var/jenkins_home/workspace/dubbo-demo/dubbo-demo-service
drwxr-xr-x 6 root root 101 Feb  2 11:51 1
drwxr-xr-x 6 root root 119 Feb  2 12:33 2
drwxr-xr-x 6 root root 101 Feb  2 12:35 3
drwxr-xr-x 6 root root 119 Feb  2 12:35 4
drwxr-xr-x 6 root root 119 Feb  2 14:16 5
drwxr-xr-x 6 root root 119 Feb  2 14:19 6
drwxr-xr-x 6 root root 119 Feb  2 14:28 7
# 第一次编译时会下载很多的第三方库文件，速度较慢，可以将下载后第三方库持久化，避免重启pod后速度变慢
# 第三方库的缓存目录在: /root/.m2/repository
[root@hdss7-21 ~]# kubectl exec jenkins-54b8469cf9-v8g28 -n infra -- ls -a /root/.m2/repository

5.1.2. 交付provider到k8s

# 准备工作
[root@hdss7-21 ~]# kubectl create namespace app
[root@hdss7-22 ~]# kubectl create secret docker-registry harbor --docker-username='app_view' --docker-password='Harbor12345' --docker-server='http://harbor.od.com/' -n app

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: dubbo-demo-service
  namespace: app
  labels: 
    name: dubbo-demo-service
    tier: provider
spec:
  replicas: 1
  selector:
    matchLabels: 
      name: dubbo-demo-service
  template:
    metadata:
      labels: 
        app: dubbo-demo-service
        name: dubbo-demo-service
        version: master_20200202_1233
    spec:
      containers:
      - name: dubbo-demo-service
        image: harbor.od.com/app/dubbo-demo-service:master_20200202_1233
        env:
        - name: JAR_BALL
          value: dubbo-server.jar
      imagePullSecrets:
      - name: harbor
      terminationGracePeriodSeconds: 30
  strategy:
    type: RollingUpdate
    rollingUpdate: 
      maxUnavailable: 0
      maxSurge: 1
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/app/dubbo-demo-service/deployment.yaml
[root@hdss7-21 ~]# kubectl get pod -n app -o wide
NAME                                  READY   STATUS    RESTARTS   AGE   IP           NODE                NOMINATED NODE   READINESS GATES
dubbo-demo-service-6949888ff6-b26t6   1/1     Running   0          10s   172.7.21.4   hdss7-21.host.com   <none>           <none>
[root@hdss7-21 ~]# kubectl logs dubbo-demo-service-6949888ff6-b26t6 -n app | tail -n 2
Dubbo server started
Dubbo 服务端已经启动

# 查看dubbo-demo-service是否连接到了ZK，通过以下方式查看
1. 通过 zkCli.sh 查看
[root@hdss7-11 ~]# /opt/apps/zookeeper/bin/zkCli.sh -server localhost:2181
[zk: localhost:2181(CONNECTED) 0] ls /  # 确认存在 dubbo 即可
[dubbo, zookeeper]
[zk: localhost:2181(CONNECTED) 1] exit
2. 通过5.2中Monitor页面查看: Applications --> dubbo-demo-service
----
补充：dubbo-demo-service 连接zk的地址是写在配置文件中:
dubbo-server/src/main/java/config.properties:dubbo.registry=zookeeper://zk1.od.com:2181?backup=zk2.od.com:2181,zk3.od.com:2181

5.2. 交付Monitor到K8S

当前项目的Monitor非常丑陋，有很多不完善的地方，不太建议使用，以下只供参考：

5.2.1. 修改dubbo-monitor配置

[root@hdss7-200 src]# git clone https://github.com/Jeromefromcn/dubbo-monitor.git
[root@hdss7-200 src]# cd dubbo-minitor
# 修改配置文件
[root@hdss7-200 dubbo-minitor]# vim dubbo-monitor-simple/conf/dubbo_origin.properties
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=dubbo-monitor
dubbo.application.owner=duduniao
dubbo.registry.address=zookeeper://zk1.od.com:2181?backup=zk2.od.com:2181,zk3.od.com:2181
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.statistics.directory=/dubbo-monitor-simple/statistics
dubbo.log4j.file=logs/dubbo-monitor-simple.log
dubbo.log4j.level=WARN
# 修改启动脚本
[root@hdss7-200 dubbo-minitor]# tail dubbo-monitor-simple/bin/start.sh
# 修改 启动参数，并且让 java 进程在前台运行，删除java之后的所有行
if [ -n "$BITS" ]; then
    JAVA_MEM_OPTS=" -server -Xmx256m -Xms256m -Xmn32m -XX:PermSize=16m -Xss256k -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:LargePageSizeInBytes=128m -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 "
else
    JAVA_MEM_OPTS=" -server -Xms256m -Xmx256m -XX:PermSize=32m -XX:SurvivorRatio=2 -XX:+UseParallelGC "
fi

echo -e "Starting the $SERVER_NAME ...\c"
exec java $JAVA_OPTS $JAVA_MEM_OPTS $JAVA_DEBUG_OPTS $JAVA_JMX_OPTS -classpath $CONF_DIR:$LIB_JARS com.alibaba.dubbo.container.Main > $STDOUT_FILE 2>&1
[root@hdss7-200 dubbo-minitor]# docker build . -t harbor.od.com/infra/dubbo-monitor:latest
[root@hdss7-200 dubbo-minitor]# docker push harbor.od.com/infra/dubbo-monitor:latest

5.2.2. monitor的资源配置清单

[root@hdss7-200 ~]# vim /data/k8s-yaml/devops/dubbo-monitor/deployment.yaml 
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: dubbo-monitor
  namespace: infra
  labels: 
    name: dubbo-monitor
spec:
  replicas: 1
  selector:
    matchLabels: 
      name: dubbo-monitor
  template:
    metadata:
      labels: 
        app: dubbo-monitor
        name: dubbo-monitor
    spec:
      containers:
      - name: dubbo-monitor
        image: harbor.od.com/infra/dubbo-monitor:latest
      imagePullSecrets:
      - name: harbor
      terminationGracePeriodSeconds: 30
  strategy:
    type: RollingUpdate
    rollingUpdate: 
      maxUnavailable: 1
      maxSurge: 1
  revisionHistoryLimit: 7
  progressDeadlineSeconds: 600

kind: Service
apiVersion: v1
metadata: 
  name: dubbo-monitor
  namespace: infra
spec:
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  selector: 
    app: dubbo-monitor

kind: Ingress
apiVersion: extensions/v1beta1
metadata: 
  name: dubbo-monitor
  namespace: infra
spec:
  rules:
  - host: dubbo-monitor.od.com
    http:
      paths:
      - path: /
        backend: 
          serviceName: dubbo-monitor
          servicePort: 80

5.2.3. 交付monitor

[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/devops/dubbo-monitor/deployment.yaml
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/devops/dubbo-monitor/service.yaml
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/devops/dubbo-monitor/ingress.yaml

[root@hdss7-11 ~]# vim /var/named/od.com.zone  # 配置DNS解析
......
dubbo-monitor      A    10.4.7.10
[root@hdss7-11 ~]# systemctl restart named

5.3. 交付Consumer到K8S

5.3.1. CI流水线构建

5.3.2. 资源配置清单

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: dubbo-demo-consumer
  namespace: app
  labels: 
    name: dubbo-demo-consumer
spec:
  replicas: 1
  selector:
    matchLabels: 
      name: dubbo-demo-consumer
  template:
    metadata:
      labels: 
        app: dubbo-demo-consumer
        name: dubbo-demo-consumer
    spec:
      containers:
      - name: dubbo-demo-consumer
        image: harbor.od.com/app/dubbo-demo-consumer:master_20200202_1630
        env:
        - name: JAR_BALL
          value: dubbo-client.jar
        imagePullPolicy: IfNotPresent
      imagePullSecrets:
      - name: harbor
      terminationGracePeriodSeconds: 30
  strategy:
    type: RollingUpdate
    rollingUpdate: 
      maxUnavailable: 1
      maxSurge: 1
  revisionHistoryLimit: 7
  progressDeadlineSeconds: 600

kind: Service
apiVersion: v1
metadata: 
  name: dubbo-demo-consumer
  namespace: app
spec:
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  selector: 
    app: dubbo-demo-consumer

kind: Ingress
apiVersion: extensions/v1beta1
metadata: 
  name: dubbo-demo-consumer
  namespace: app
spec:
  rules:
  - host: demo.od.com
    http:
      paths:
      - path: /
        backend: 
          serviceName: dubbo-demo-consumer
          servicePort: 80

5.3.3. 交付Consumer

[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/app/dubbo-demo-consumer/deployment.yaml
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/app/dubbo-demo-consumer/service.yaml
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/app/dubbo-demo-consumer/ingress.yaml
[root@hdss7-21 ~]# kubectl get pod -n app
NAME                                   READY   STATUS    RESTARTS   AGE
dubbo-demo-consumer-694b5645f9-dc4dg   1/1     Running   0          65s
dubbo-demo-service-6949888ff6-b26t6    1/1     Running   0          144m
[root@hdss7-21 ~]# kubectl logs dubbo-demo-consumer-694b5645f9-dc4dg -n app | tail -n 2
Dubbo client started
Dubbo 消费者端启动

[root@hdss7-11 ~]# vim /var/named/od.com.zone
......
demo               A    10.4.7.10
[root@hdss7-11 ~]# systemctl restart named

4.4 模拟版本升级

接下来我们模拟升级发版，之前已经用同一个镜像打了不同的tag并推送到从库
当然正常发版的顺序是:

提交修改过的代码的代码块
使用jenkins构建新镜像
上传到私有harbor仓库中
更新de文件并apply

4.4.1 修改dp.yaml资源配置清单

修改harbor镜像仓库中对应的tag版本：

sed -i 's#20200202_1630#master_200510_1430#g' dp.yaml

4.4.2 应用修改后的资源配置清单

当然也可以在dashboard中进行在线修改：

kubectl apply -f http://k8s-yaml.od.com/dubbo-consumer/dp.yaml
~]# kubectl -n app  get pod
NAME                                   READY   STATUS    RESTARTS   AGE
dubbo-demo-consumer-84f75b679c-kdwd7   1/1     Running   0          54s
dubbo-demo-service-79574b6879-cxkls    1/1     Running   0          4h58m

4.4.3 使用浏览器验证

使用浏览器验证：http://dubbo-demo.od.com/hello?name=lg
在短暂的超时后,即可正常访问
至此，我们一套完成的dubbo服务就已经交付到k8s集群当中了，并且也演示了如何发版。

K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 | 韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。长夜有穷，真水无香。领略过了 K8s“身在江

K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub