一. 持续集成/部署/交付概述

持续集成(Continuous Integration，CI): 代码合并、构建、部署、测试都在一起，不断地执行这个过程，并对结果反馈。
持续部署(Continuous Deployment，CD): 部署到测试环境、预生产环境、生产环境。
持续交付(Continuous Delivery，CD): 将最终产品发布到生产环境，给用户使用。

环境配置

角色	IP	备注	推荐配置
K8S	10.40.6.201 10.40.6.210 10.40.6.213	自行准备	CPU:2C+ 内存:4G+
Harbor	10.40.6.165
Git	10.40.6.165
Jenkins	部署在K8S平台

二. 准备工作

1. 对项目的理解

• 单体架构、微服务
• 怎么部署
• 启动是否有依赖

2. 部署到k8s平台流程

(1). 制作镜像
(2). 容器放到Pod
(3). 控制器管理Pod
(4). 暴露应用
(5). 对外发布应用
(6). 日志管理/监控

3. 不同环境区分配置文件

• configmap
• entrypoint.sh
• 统一配置中心，例如 Apollo，Nacos

4. Harbor镜像仓库

Harbor 部署参考: https://blog.csdn.net/agonie201218/article/details/115691894

kubernetes从harbor拉取镜像没有权限解决方法 unauthorized https://andyoung.blog.csdn.net/article/details/120313644

5. GitLab代码版本仓库

GitLab 详细安装及使用教程: https://blog.csdn.net/agonie201218/article/details/120859712

三. Kubernetes中部署Jenkins

参考 github jenkinsci/kubernetes-plugin 插件 :https://github.com/jenkinsci/kubernetes-plugin/tree/master/src/main/kubernetes

NFS服务部署 PV、 PVC 参考：https://andyoung.blog.csdn.net/article/details/120147604

https://andyoung.blog.csdn.net/article/details/120843117

1. 部署有状态的jenkins Pod

# jenkins

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: jenkins
  labels:
    name: jenkins
spec:
  selector:
    matchLabels:
      name: jenkins
  serviceName: jenkins
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      name: jenkins
      labels:
        name: jenkins
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: jenkins
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts-alpine
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
            - containerPort: 50000
          resources:
            limits:
              cpu: 1
              memory: 1Gi
            requests:
              cpu: 0.5
              memory: 500Mi
          env:
            - name: LIMITS_MEMORY
              valueFrom:
                resourceFieldRef:
                  resource: limits.memory
                  divisor: 1Mi
            - name: JAVA_OPTS
              # value: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
              value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
          livenessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12 # ~2 minutes
          readinessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12 # ~2 minutes
      securityContext:
        fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
      name: jenkins-home
      # annotations:
      #   volume.beta.kubernetes.io/storage-class: anything
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

---
## 创建jenkins service
apiVersion: v1
kind: Service
metadata:
  name: jenkins
spec:
  # type: LoadBalancer
  selector:
    name: jenkins
  # ensure the client ip is propagated to avoid the invalid crumb issue when using LoadBalancer (k8s >=1.7)
  #externalTrafficPolicy: Local
  ports:
    -
      name: http
      port: 80
      targetPort: 8080
      protocol: TCP
    -
      name: agent
      port: 50000
      protocol: TCP

---
## 创建jenkins ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: jenkins
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/tls-acme: "true"
    # "413 Request Entity Too Large" uploading plugins, increase client_max_body_size
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    # For nginx-ingress controller < 0.9.0.beta-18
    ingress.kubernetes.io/ssl-redirect: "true"
    # "413 Request Entity Too Large" uploading plugins, increase client_max_body_size
    ingress.kubernetes.io/proxy-body-size: 50m
    ingress.kubernetes.io/proxy-request-buffering: "off"
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: jenkins
            port:
              number: 80
    host: jenkins.example.com
  tls:
  - hosts:
    - jenkins.example.com
    secretName: tls-jenkins

  securityContext:
    fsGroup: 1000  这个配置可以去掉，但得先给后端存储切换组，如果用这个配置，后面jenkins 使用久了，jenkins 重启后会需要很久参能启动成功，这样就会影响健康检测失败，pod一直启动失败

2. jenkins认证授权

# In GKE need to get RBAC permissions first with
# kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin [--user=<user-name>|--group=<group-name>]

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins

---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: jenkins
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins

3. 获取jenkins密码并登录

# kubectl create -f .
登录容器获取jenkins登录密码：
# kubectl exec -it jenkins-0 bash
bash-4.4$ cat /var/jenkins_home/secrets/initialAdminPassword
0f8c730f4b7a426098283cf94aa57231

jenkins地址：http://jenkins.example.com:30080

4. jenkins 安装插件

1. pipeline

2. git

3. kubernetes

   

4. Role-based Authorization Strategy(推荐)

   我们可以利用Role-based Authorization Strategy 插件来管理Jenkins用户权限

   https://blog.csdn.net/agonie201218/article/details/120168454

5. Chinese (中文汉化插件)

6. Extended Choice Parameter(参数化构建过程 多选框)

   

   6.1 多选框：项目名称。设置4个参数，用 , 隔开

6.2 设置多选框值值

6.3 最终效果