环境介绍

本次 jenkins 部署在本地服务器上，下面我们开始动态slave配置。

k8s创建RBAC

## 首先需要创建命名空间 pipeline
[root@master1 ~]# cat pipeline-acount.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-slave
  namespace: pipeline
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: pipeline
  name: jenkins-slave
rules:
  - apiGroups: [""]
    resources: ["pods", "configmaps", "namespaces"]
    verbs: ["get", "watch", "list", "create", "update", "delete", "patch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["get", "watch", "list", "create", "update", "delete", "patch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get", "watch", "list"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: jenkins-slave-binding
  namespace: pipeline
subjects:
- kind: ServiceAccount
  name: jenkins-slave
  namespace: pipeline
roleRef:
  kind: Role
  name: jenkins-slave
  apiGroup: rbac.authorization.k8s.io
 
## kubectl apply -f pipeline-acount.yaml
## 创建好后会自动生成一个 token

Jenkins 配置

1、配置全局凭据

image-20231102100814838

2、配置 kuberetes cloud

首先回到系统设置

2.1证书key获取
## 找到  k8s 配置文件，一般在 /root/.kube/  这里用我自己的做实例
[root@master1 ~]# cd .kube/
[root@master1 .kube]# cat config
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeU1UQXhNakE0TURZeU5Gb1hEVE15TVRBd09UQTRNRFl5TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTm5NCjcrY1N4TXFIMUw2dzUxNWpEU09Bc0ttTEIvV2Q4c0NvMUt3cml1dlo0aHRCOTQ5RkROSGdLZ1BOdy9RV1NnUHQKRi9abmRua1RlR2loVHJ1VzhoQzJkMHVFNVdFKzN5Qmp3a3dMUmo0ZVBEMjRhODREc3cxNC8weFVNUzRKZ0hCMgp5TitZUlp5OXpBRmhFSlBWSURQUEtCRGplbmRrN01GMGMxSTl6Q2t3cjlmTFlrM2xVL2hwcWdzM01JekpycUhGCmZHUkhibVVNMGVPdmFZOHVaVlR1cStSQ3h1OTlFeFl0MzRKYWtnSVVFa2RLSTh4TEYrczFNeVBsNjk0enFFZFcKdVBXdGZ3aGU1TS9nVk96Vk1RN3o2eCs1R2FQV2NkVjFNendUbGhWZFdDZEJla1p0Y2ZQOWF3OVBVOWZWTFEwWgoyY3Q4NGRTREJsaDVycHV2UUEwQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZFQjdTeDk5RGw2eFF2a3V0Tk0vS3pCU1pJUW5NQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBanM0TG1icDR1SWtBNGRlcmhsTnNxb3VVNWx0Q250M3l0aVkxTjdCUW9XV0p4YU9PbgpycCsyOW8wcVU2d0lNU1Mvd2J1dlRMWEtSVHI2T2lqQTNkbllmanRjcFNEQ1JCZkdXdTU4dVFEUkQ4dHU3d052CnZlL0hUVTd0b2taZFlnSnVCenVBUjJLNm5mZWtvRVArSFdFazRvM29CVFgzMFB0NjZ5dUhiZWRBWmV1b3QwMGEKSnJWdG80VTJxZWwwOGl3Y1ZUdmxxVGVZa2ZodlFYcWk1TDQ3MWlJNFVIMGF4WGpqWUhaNENxTjRmYTJYcGdadwp0K1FTVG5zZFNrcEM1bGpvelM0QkZIeWVyZ09rQldrcDR0K09uM0I1cExnSW1oVHlRa2UxVmV5MW1iT01wQlFOCkx5TUNEcTFNNEtQYU9FdkhEZVNYMjQrRVN2RWEvT3BFU0xTaAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://lb.kubesphere.local:6443
  name: cluster.local
contexts:
- context:
    cluster: cluster.local
    user: kubernetes-admin
  name: kubernetes-admin@cluster.local
current-context: kubernetes-admin@cluster.local
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFekNDQWZ1Z0F3SUJBZ0lJYzBGSzhoZDBhK293RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TWpFd01USXdPREEyTWpSYUZ3MHlOREE1TXpBeE9UQXdNREphTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXhSMy9tVzNvb25uWEEvaHcKT0ZiVXN5cVE5R2t0K05PNFZKMGJ1b1dxaEJVZ3E0dlVMU203N0dYYktpUkVuOFJqcHNNOURSZFpSb3VaWVFyegp3VHRkS2pvektpbVowK0IyWlFKMWxqUXl0SkExanVyRm4weEltT29YZGFhL3dMVmp5bFR5QVQreTFmOEd4R0J0CnRlT1hSd3RXVmF3d0hSLzlJbitRempQNlpKQW5NRGZlYTBTbDZLVWFlNlF3S0l3QnpKdFFFRVk5aG5wZFRLVXgKRFdJVDRFSFV6V2hDakNiQWtRaEYxd3VlNnNyVmhSejBESmxzeXVnME5VQlgxc3l2d1VBTW5vUXJyVmVCNDBRUApXU0s3WjZnVDhYV3l4NS9aeDBRNWxyUkdxMFVmdExiU0tnQUpjTGExZ1g2dks3enBNUEMwWmZwRjgzbjRmZ0FyCnZOY3Fxd0lEQVFBQm8wZ3dSakFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0h3WURWUjBqQkJnd0ZvQVVRSHRMSDMwT1hyRkMrUzYwMHo4ck1GSmtoQ2N3RFFZSktvWklodmNOQVFFTApCUUFEZ2dFQkFMdnJpb3BoSE9odFRzYlNWOG5yNEMvWmxodlRQS3Y4cmg2RWh4K0pOdThtRjlpTmtDR01yZGV2CnNkdjMzRzlBWUxDSlJhZ2UxREd3RGJ1RjRYTldwblZ1NjN2dnJVOXE4SHhaQmxqYmMrOTh0SFlhOG8rMlVjNHAKQlpNRG5aYXBiSFoycWJrdWJqYzNzeHlicEFveGRmY1FzaUZwODVWU2dLQUVMNDZUN0pKaHk0NXJ1b3FpR1luRgpKNXlwc3YxQ3lWaFp1Z0dqSTErQzVxdUNRQWJ2T3NvRzFqN2t6cU5NbE1iMElaNTFmanNUSG5leUtqYUoyRS9HCjlURWtZVU0zT2tMZmQyZnYrZ1pnZzNQSis4cVErRm1zUGJEa3d6MVVaVUxBVGFxNjZPTzJuMW9uZTBXWXdGbDMKeThtYTFIb1ZzdFZKQWNzSmJER2pQVzBaWmpES1UwST0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBeFIzL21XM29vbm5YQS9od09GYlVzeXFROUdrdCtOTzRWSjBidW9XcWhCVWdxNHZVCkxTbTc3R1hiS2lSRW44Umpwc005RFJkWlJvdVpZUXJ6d1R0ZEtqb3pLaW1aMCtCMlpRSjFsalF5dEpBMWp1ckYKbjB4SW1Pb1hkYWEvd0xWanlsVHlBVCt5MWY4R3hHQnR0ZU9YUnd0V1Zhd3dIUi85SW4rUXpqUDZaSkFuTURmZQphMFNsNktVYWU2UXdLSXdCekp0UUVFWTlobnBkVEtVeERXSVQ0RUhVeldoQ2pDYkFrUWhGMXd1ZTZzclZoUnowCkRKbHN5dWcwTlVCWDFzeXZ3VUFNbm9RcnJWZUI0MFFQV1NLN1o2Z1Q4WFd5eDUvWngwUTVsclJHcTBVZnRMYlMKS2dBSmNMYTFnWDZ2Szd6cE1QQzBaZnBGODNuNGZnQXJ2TmNxcXdJREFRQUJBb0lCQUdIcW0reHdYMjVPVGRQSQpGcDYxNjJTdzBWN0gzTEZOSXJUTmNsZStURXplWGVNNVM0Vy94SDBTdjhMNFR5cDZHUEplNjdYVUVtSHJ1SjR2CkhaU3dVNEJGZHVNTDFVRWRzRkpPM1hCbXI0Vm9XV0tNRnUwaHJMSmhIcFF2NS9MbmFCRzdEcGcyTnpUOFFUVk4KblZJTUl6cUVWRGVzbUIzdnBRUDlFRWFPSldUTUlrUUZpVk9ZZVpXTkpUMXd4eVZyUWtJUDEwb2dIZ2VRWW96YwpWbDhxQlMrWUlzeFBtd0ZVdUlROWZJTG5rbGNIS0U5L1BFNVVOc1RGMmFTRCs5WklBV2duLy9NOFkvckt0dGR3CnNrMkJOOFVicGN1TUJKa3F6Ly9OTy91Q1Y3VHdTRk81a0srdG13MC9LamJ3dEJ5MFZWelhHdTFoZDI2T2Ixa0gKeER2dVZWa0NnWUVBKzJNTTdmd0lkQmc3dGJYUUJOd3dYU0RBb1hOajg5MDJnRysyTEVoYUlQMTF2NXdqNnNKTwpnZ0ptNVVlNTJuQWFTVHlqQzVWMXNhZllMRWNuWkJlV0xlTUhneU5SaHpVQXkxc29LcFExdzdwSG8zeForazZpCktkR0ZXWWhQQ1pnMEdUK3B1ODhyVmNiS1Y3QldpaGd0cXRqZEF2SlpIREt6WFVHOXlmc2VBSTBDZ1lFQXlMd0EKeFpHajErN0tmTUtTdHpUSG1QVzVhSXBZb0VZSDV0c1NKTUZBNE91c05WM3AxSW1QMGxpamhxSGtnaTBvaVVZbwo5aWpNVTM2ZkowT0IwNDhleXdla3B5NGx2cE5xWWY4ZzM1SGU5OVQ3Z3JNVE5nV0t3RjNSbDZKazNnWlYzSGtzCktUM0Z3MWhRSU9lVm9TazZhUXJJSWk0KzZxTzBmMHJVWmpXOUZoY0NnWUJlMERiNk00ckVycmNtaTlKUFl2VGkKeXcyY1Z6Y2xyUk4zVWFyMS9MdnhvV2NkdzdoUVBNVDdpQWhqQmJCMVVMNjVUS042SlA3azZKZEI2L3hSWmd3QQpkcFpJd2JOb09YZDVPNEprdk0yaWFzbkdRWXAyTzk0eHc1SjErRkZISHg3WFB3bTNpaVdnRG9BV25xMmxIQVZ0CllLbkxDTGpQUjlnYW5rY0V2Uy9OcVFLQmdRQ1lqRytWdFptclJ6aXcyWG1BSzJjb2NrMW1DZ1ZTUUFaUFJkc2kKL1k2ZG45eXViZUYrK00vSXpqM3YyZVo3bTIvNzZzckVUN3NBRlBGRWVJUVlUK0xaN3NRTm5QOW9Oa1dabGJiWQowTlYreUdnZktwSUY4dTVBUk13OGpWaFVkc0lYbkNxWWFPNTdCd2xXQ0VqcFFoaTJyVS9zMGZKVWhQWnQ2bU5DCkRjTFdId0tCZ0hobmJiM1I0ZFdNcnRjUU9aVS9qKzl4MjFGS3pGZUZuSXpQZ0JlQUtvVXJNNk52RU1JLzU0QXEKaG1JY053MFUyQWZmUERTemxSSCt1UHBlaC9xQW41aE56bnhYTjBxcHc4aFRUQUJTR0lPV0tSa0tLWFMvTlovdwpFRVhGeDN3Skx5NEhkM2lYTEltd2VCZkpwcGcwcERLOFZzNDZHQmJTUE5QaHVMekI3L0VLCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
​
## 将下方单引号内的值，替换为 配置文件 中对应的加密编码即可
echo 'certificate-authority-data-value' | base64 -d > ./ca.crt （在kube_config中能找到对应的值）
echo 'client-certificate-data' | base64 -d > ./client.crt
echo 'client-key-data' | base64 -d > ./client.key
openssl pkcs12 -export -out cert.pfx -inkey client.key -in client.crt -certfile ca.crt(必须输入密码不然会jenkins验证报错)
​
## 这时 在你的目录中会生成几个文件
[root@master1 .kube]# ls
cache  ca.crt  cert.pfx  client.crt  client.key  config  config-hw  http-cache
[root@master1 .kube]# cat client.key 
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAxR3/mW3oonnXA/hwOFbUsyqQ9Gkt+NO4VJ0buoWqhBUgq4vU
LSm77GXbKiREn8RjpsM9DRdZRouZYQrzwTtdKjozKimZ0+B2ZQJ1ljQytJA1jurF
n0xImOoXdaa/wLVjylTyAT+y1f8GxGBtteOXRwtWVawwHR/9In+QzjP6ZJAnMDfe
a0Sl6KUae6QwKIwBzJtQEEY9hnpdTKUxDWIT4EHUzWhCjCbAkQhF1wue6srVhRz0
DJlsyug0NUBX1syvwUAMnoQrrVeB40QPWSK7Z6gT8XWyx5/Zx0Q5lrRGq0UftLbS
KgAJcLa1gX6vK7zpMPC0ZfpF83n4fgArvNcqqwIDAQABAoIBAGHqm+xwX25OTdPI
Fp6162Sw0V7H3LFNIrTNcle+TEzeXeM5S4W/xH0Sv8L4Typ6GPJe67XUEmHruJ4v
HZSwU4BFduML1UEdsFJO3XBmr4VoWWKMFu0hrLJhHpQv5/LnaBG7Dpg2NzT8QTVN
nVIMIzqEVDesmB3vpQP9EEaOJWTMIkQFiVOYeZWNJT1wxyVrQkIP10ogHgeQYozc
Vl8qBS+YIsxPmwFUuIQ9fILnklcHKE9/PE5UNsTF2aSD+9ZIAWgn//M8Y/rKttdw
sk2BN8UbpcuMBJkqz//NO/uCV7TwSFO5kK+tmw0/KjbwtBy0VVzXGu1hd26Ob1kH
xDvuVVkCgYEA+2MM7fwIdBg7tbXQBNwwXSDAoXNj8902gG+2LEhaIP11v5wj6sJO
ggJm5Ue52nAaSTyjC5V1safYLEcnZBeWLeMHgyNRhzUAy1soKpQ1w7pHo3xZ+k6i
KdGFWYhPCZg0GT+pu88rVcbKV7BWihgtqtjdAvJZHDKzXUG9yfseAI0CgYEAyLwA
xZGj1+7KfMKStzTHmPW5aIpYoEYH5tsSJMFA4OusNV3p1ImP0lijhqHkgi0oiUYo
9ijMU36fJ0OB048eywekpy4lvpNqYf8g35He99T7grMTNgWKwF3Rl6Jk3gZV3Hks
KT3Fw1hQIOeVoSk6aQrIIi4+6qO0f0rUZjW9FhcCgYBe0Db6M4rErrcmi9JPYvTi
yw2cVzclrRN3Uar1/LvxoWcdw7hQPMT7iAhjBbB1UL65TKN6JP7k6JdB6/xRZgwA
dpZIwbNoOXd5O4JkvM2iasnGQYp2O94xw5J1+FFHHx7XPwm3iiWgDoAWnq2lHAVt
YKnLCLjPR9gankcEvS/NqQKBgQCYjG+VtZmrRziw2XmAK2cock1mCgVSQAZPRdsi
/Y6dn9yubeF++M/Izj3v2eZ7m2/76srET7sAFPFEeIQYT+LZ7sQNnP9oNkWZlbbY
0NV+yGgfKpIF8u5ARMw8jVhUdsIXnCqYaO57BwlWCEjpQhi2rU/s0fJUhPZt6mNC
DcLWHwKBgHhnbb3R4dWMrtcQOZU/j+9x21FKzFeFnIzPgBeAKoUrM6NvEMI/54Aq
hmIcNw0U2AffPDSzlRH+uPpeh/qAn5hNznxXN0qpw8hTTABSGIOWKRkKKXS/NZ/w
EEXFx3wJLy4Hd3iXLImweBfJppg0pDK8Vs46GBbSPNPhuLzB7/EK
-----END RSA PRIVATE KEY-----
## 在Kubernetes 服务证书 key中填入解码的client.key内容即可
## 继续下方操作

测试：pipeline调用jenkins-slave

image-20231102104428227

## 测试脚本内容
pipeline{
    //使用Kubernetes拉取Slave pod
    agent {
    kubernetes {
      cloud 'kubernetes'
  }
}
    stages{
        stage('输出主机名称'){
            steps{
                sh 'hostname'
            }
        }
 
        stage('等待一段时间'){
            steps{
                sh 'sleep 20'
            }
        }
 
    }
 
}

image-20231102104505490

查看 ns 中新创建了 pod  表示配置成功。

Jenkins配置 方法二

配置 kubernetes cloud

## 找到  k8s 配置文件，一般在 /root/.kube/  这里用我自己的做实例
[root@master1 ~]# cd .kube/
[root@master1 .kube]# cat config
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeU1UQXhNakE0TURZeU5Gb1hEVE15TVRBd09UQTRNRFl5TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTm5NCjcrY1N4TXFIMUw2dzUxNWpEU09Bc0ttTEIvV2Q4c0NvMUt3cml1dlo0aHRCOTQ5RkROSGdLZ1BOdy9RV1NnUHQKRi9abmRua1RlR2loVHJ1VzhoQzJkMHVFNVdFKzN5Qmp3a3dMUmo0ZVBEMjRhODREc3cxNC8weFVNUzRKZ0hCMgp5TitZUlp5OXpBRmhFSlBWSURQUEtCRGplbmRrN01GMGMxSTl6Q2t3cjlmTFlrM2xVL2hwcWdzM01JekpycUhGCmZHUkhibVVNMGVPdmFZOHVaVlR1cStSQ3h1OTlFeFl0MzRKYWtnSVVFa2RLSTh4TEYrczFNeVBsNjk0enFFZFcKdVBXdGZ3aGU1TS9nVk96Vk1RN3o2eCs1R2FQV2NkVjFNendUbGhWZFdDZEJla1p0Y2ZQOWF3OVBVOWZWTFEwWgoyY3Q4NGRTREJsaDVycHV2UUEwQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZFQjdTeDk5RGw2eFF2a3V0Tk0vS3pCU1pJUW5NQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBanM0TG1icDR1SWtBNGRlcmhsTnNxb3VVNWx0Q250M3l0aVkxTjdCUW9XV0p4YU9PbgpycCsyOW8wcVU2d0lNU1Mvd2J1dlRMWEtSVHI2T2lqQTNkbllmanRjcFNEQ1JCZkdXdTU4dVFEUkQ4dHU3d052CnZlL0hUVTd0b2taZFlnSnVCenVBUjJLNm5mZWtvRVArSFdFazRvM29CVFgzMFB0NjZ5dUhiZWRBWmV1b3QwMGEKSnJWdG80VTJxZWwwOGl3Y1ZUdmxxVGVZa2ZodlFYcWk1TDQ3MWlJNFVIMGF4WGpqWUhaNENxTjRmYTJYcGdadwp0K1FTVG5zZFNrcEM1bGpvelM0QkZIeWVyZ09rQldrcDR0K09uM0I1cExnSW1oVHlRa2UxVmV5MW1iT01wQlFOCkx5TUNEcTFNNEtQYU9FdkhEZVNYMjQrRVN2RWEvT3BFU0xTaAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://lb.kubesphere.local:6443
  name: cluster.local
contexts:
- context:
    cluster: cluster.local
    user: kubernetes-admin
  name: kubernetes-admin@cluster.local
current-context: kubernetes-admin@cluster.local
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFekNDQWZ1Z0F3SUJBZ0lJYzBGSzhoZDBhK293RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TWpFd01USXdPREEyTWpSYUZ3MHlOREE1TXpBeE9UQXdNREphTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXhSMy9tVzNvb25uWEEvaHcKT0ZiVXN5cVE5R2t0K05PNFZKMGJ1b1dxaEJVZ3E0dlVMU203N0dYYktpUkVuOFJqcHNNOURSZFpSb3VaWVFyegp3VHRkS2pvektpbVowK0IyWlFKMWxqUXl0SkExanVyRm4weEltT29YZGFhL3dMVmp5bFR5QVQreTFmOEd4R0J0CnRlT1hSd3RXVmF3d0hSLzlJbitRempQNlpKQW5NRGZlYTBTbDZLVWFlNlF3S0l3QnpKdFFFRVk5aG5wZFRLVXgKRFdJVDRFSFV6V2hDakNiQWtRaEYxd3VlNnNyVmhSejBESmxzeXVnME5VQlgxc3l2d1VBTW5vUXJyVmVCNDBRUApXU0s3WjZnVDhYV3l4NS9aeDBRNWxyUkdxMFVmdExiU0tnQUpjTGExZ1g2dks3enBNUEMwWmZwRjgzbjRmZ0FyCnZOY3Fxd0lEQVFBQm8wZ3dSakFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0h3WURWUjBqQkJnd0ZvQVVRSHRMSDMwT1hyRkMrUzYwMHo4ck1GSmtoQ2N3RFFZSktvWklodmNOQVFFTApCUUFEZ2dFQkFMdnJpb3BoSE9odFRzYlNWOG5yNEMvWmxodlRQS3Y4cmg2RWh4K0pOdThtRjlpTmtDR01yZGV2CnNkdjMzRzlBWUxDSlJhZ2UxREd3RGJ1RjRYTldwblZ1NjN2dnJVOXE4SHhaQmxqYmMrOTh0SFlhOG8rMlVjNHAKQlpNRG5aYXBiSFoycWJrdWJqYzNzeHlicEFveGRmY1FzaUZwODVWU2dLQUVMNDZUN0pKaHk0NXJ1b3FpR1luRgpKNXlwc3YxQ3lWaFp1Z0dqSTErQzVxdUNRQWJ2T3NvRzFqN2t6cU5NbE1iMElaNTFmanNUSG5leUtqYUoyRS9HCjlURWtZVU0zT2tMZmQyZnYrZ1pnZzNQSis4cVErRm1zUGJEa3d6MVVaVUxBVGFxNjZPTzJuMW9uZTBXWXdGbDMKeThtYTFIb1ZzdFZKQWNzSmJER2pQVzBaWmpES1UwST0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBeFIzL21XM29vbm5YQS9od09GYlVzeXFROUdrdCtOTzRWSjBidW9XcWhCVWdxNHZVCkxTbTc3R1hiS2lSRW44Umpwc005RFJkWlJvdVpZUXJ6d1R0ZEtqb3pLaW1aMCtCMlpRSjFsalF5dEpBMWp1ckYKbjB4SW1Pb1hkYWEvd0xWanlsVHlBVCt5MWY4R3hHQnR0ZU9YUnd0V1Zhd3dIUi85SW4rUXpqUDZaSkFuTURmZQphMFNsNktVYWU2UXdLSXdCekp0UUVFWTlobnBkVEtVeERXSVQ0RUhVeldoQ2pDYkFrUWhGMXd1ZTZzclZoUnowCkRKbHN5dWcwTlVCWDFzeXZ3VUFNbm9RcnJWZUI0MFFQV1NLN1o2Z1Q4WFd5eDUvWngwUTVsclJHcTBVZnRMYlMKS2dBSmNMYTFnWDZ2Szd6cE1QQzBaZnBGODNuNGZnQXJ2TmNxcXdJREFRQUJBb0lCQUdIcW0reHdYMjVPVGRQSQpGcDYxNjJTdzBWN0gzTEZOSXJUTmNsZStURXplWGVNNVM0Vy94SDBTdjhMNFR5cDZHUEplNjdYVUVtSHJ1SjR2CkhaU3dVNEJGZHVNTDFVRWRzRkpPM1hCbXI0Vm9XV0tNRnUwaHJMSmhIcFF2NS9MbmFCRzdEcGcyTnpUOFFUVk4KblZJTUl6cUVWRGVzbUIzdnBRUDlFRWFPSldUTUlrUUZpVk9ZZVpXTkpUMXd4eVZyUWtJUDEwb2dIZ2VRWW96YwpWbDhxQlMrWUlzeFBtd0ZVdUlROWZJTG5rbGNIS0U5L1BFNVVOc1RGMmFTRCs5WklBV2duLy9NOFkvckt0dGR3CnNrMkJOOFVicGN1TUJKa3F6Ly9OTy91Q1Y3VHdTRk81a0srdG13MC9LamJ3dEJ5MFZWelhHdTFoZDI2T2Ixa0gKeER2dVZWa0NnWUVBKzJNTTdmd0lkQmc3dGJYUUJOd3dYU0RBb1hOajg5MDJnRysyTEVoYUlQMTF2NXdqNnNKTwpnZ0ptNVVlNTJuQWFTVHlqQzVWMXNhZllMRWNuWkJlV0xlTUhneU5SaHpVQXkxc29LcFExdzdwSG8zeForazZpCktkR0ZXWWhQQ1pnMEdUK3B1ODhyVmNiS1Y3QldpaGd0cXRqZEF2SlpIREt6WFVHOXlmc2VBSTBDZ1lFQXlMd0EKeFpHajErN0tmTUtTdHpUSG1QVzVhSXBZb0VZSDV0c1NKTUZBNE91c05WM3AxSW1QMGxpamhxSGtnaTBvaVVZbwo5aWpNVTM2ZkowT0IwNDhleXdla3B5NGx2cE5xWWY4ZzM1SGU5OVQ3Z3JNVE5nV0t3RjNSbDZKazNnWlYzSGtzCktUM0Z3MWhRSU9lVm9TazZhUXJJSWk0KzZxTzBmMHJVWmpXOUZoY0NnWUJlMERiNk00ckVycmNtaTlKUFl2VGkKeXcyY1Z6Y2xyUk4zVWFyMS9MdnhvV2NkdzdoUVBNVDdpQWhqQmJCMVVMNjVUS042SlA3azZKZEI2L3hSWmd3QQpkcFpJd2JOb09YZDVPNEprdk0yaWFzbkdRWXAyTzk0eHc1SjErRkZISHg3WFB3bTNpaVdnRG9BV25xMmxIQVZ0CllLbkxDTGpQUjlnYW5rY0V2Uy9OcVFLQmdRQ1lqRytWdFptclJ6aXcyWG1BSzJjb2NrMW1DZ1ZTUUFaUFJkc2kKL1k2ZG45eXViZUYrK00vSXpqM3YyZVo3bTIvNzZzckVUN3NBRlBGRWVJUVlUK0xaN3NRTm5QOW9Oa1dabGJiWQowTlYreUdnZktwSUY4dTVBUk13OGpWaFVkc0lYbkNxWWFPNTdCd2xXQ0VqcFFoaTJyVS9zMGZKVWhQWnQ2bU5DCkRjTFdId0tCZ0hobmJiM1I0ZFdNcnRjUU9aVS9qKzl4MjFGS3pGZUZuSXpQZ0JlQUtvVXJNNk52RU1JLzU0QXEKaG1JY053MFUyQWZmUERTemxSSCt1UHBlaC9xQW41aE56bnhYTjBxcHc4aFRUQUJTR0lPV0tSa0tLWFMvTlovdwpFRVhGeDN3Skx5NEhkM2lYTEltd2VCZkpwcGcwcERLOFZzNDZHQmJTUE5QaHVMekI3L0VLCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
​
## 将下方单引号内的值，替换为 配置文件 中对应的加密编码即可
echo 'certificate-authority-data-value' | base64 -d > ./ca.crt （在kube_config中能找到对应的值）
echo 'client-certificate-data' | base64 -d > ./client.crt
echo 'client-key-data' | base64 -d > ./client.key
openssl pkcs12 -export -out cert.pfx -inkey client.key -in client.crt -certfile ca.crt(必须输入密码不然会jenkins验证报错)
​
## 这时 在你的目录中会生成几个文件
[root@master1 .kube]# ls
cache  ca.crt  cert.pfx  client.crt  client.key  config  config-hw  http-cache
[root@master1 .kube]# cat ca.crt 
-----BEGIN CERTIFICATE-----
MIIC5zCCAc+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
cm5ldGVzMB4XDTIyMTAxMjA4MDYyNFoXDTMyMTAwOTA4MDYyNFowFTETMBEGA1UE
AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANnM
7+cSxMqH1L6w515jDSOAsKmLB/Wd8sCo1KwriuvZ4htB949FDNHgKgPNw/QWSgPt
F/ZndnkTeGihTruW8hC2d0uE5WE+3yBjwkwLRj4ePD24a84Dsw14/0xUMS4JgHB2
yN+YRZy9zAFhEJPVIDPPKBDjendk7MF0c1I9zCkwr9fLYk3lU/hpqgs3MIzJrqHF
fGRHbmUM0eOvaY8uZVTuq+RCxu99ExYt34JakgIUEkdKI8xLF+s1MyPl694zqEdW
uPWtfwhe5M/gVOzVMQ7z6x+5GaPWcdV1MzwTlhVdWCdBekZtcfP9aw9PU9fVLQ0Z
2ct84dSDBlh5rpuvQA0CAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
/wQFMAMBAf8wHQYDVR0OBBYEFEB7Sx99Dl6xQvkutNM/KzBSZIQnMA0GCSqGSIb3
DQEBCwUAA4IBAQAjs4Lmbp4uIkA4derhlNsqouU5ltCnt3ytiY1N7BQoWWJxaOOn
rp+29o0qU6wIMSS/wbuvTLXKRTr6OijA3dnYfjtcpSDCRBfGWu58uQDRD8tu7wNv
ve/HTU7tokZdYgJuBzuAR2K6nfekoEP+HWEk4o3oBTX30Pt66yuHbedAZeuot00a
JrVto4U2qel08iwcVTvlqTeYkfhvQXqi5L471iI4UH0axXjjYHZ4CqN4fa2XpgZw
t+QSTnsdSkpC5ljozS4BFHyergOkBWkp4t+On3B5pLgImhTyQke1Vey1mbOMpBQN
LyMCDq1M4KPaOEvHDeSX24+ESvEa/OpESLSh
-----END CERTIFICATE-----
## 在Kubernetes 服务证书 key中填入解码的ca.crt内容即可
​
## 将cert.pfx 导出
## 继续下方操作
​
配置全局凭据

image-20231102122231873

image-20231102122523417

这里是我的 Pipeline 流水线

#!groovy
import groovy.json.JsonSlurperClassic
​
pipeline {
    agent {
        kubernetes {
            yaml """
                apiVersion: v1
                kind: Pod
                spec:
                  imagePullSecrets:
                    - name: docker-registry
                  containers:
                  - name: maven
                    image: 192.167.57.74:30002/library/pipelone-docker:8.3
                    command: ['cat']
                    tty: true
                    resources:
                      requests:
                        memory: "1Gi"  # 请求1 GB 内存
                        cpu: "0.5"     # 请求0.5核 CPU
                      limits:
                        memory: "2Gi"  # 限制最大使用2 GB 内存
                        cpu: "1"       # 限制最大使用1核 CPU
                    volumeMounts:
                      - mountPath: /etc/localtime
                        name: agent-time
                      - mountPath: /etc/maven/
                        name: ks-devops-agent
                      - mountPath: /root/.kube/
                        name: kubeconfig
                      - mountPath: /var/run/docker.sock
                        name: dockersock
                      - mountPath: /etc/docker/daemon.json
                        name: dockerdaemon
                      - mountPath: /root
                        name: agent-jenkins-job
                      - mountPath: /root/mvn_data/
                        name: mvn-cache
                  volumes:
                    - configMap:
                        name: kubeconfig
                      name: kubeconfig
                    - configMap:
                        name: maven
                      name: ks-devops-agent
                    - hostPath:
                        path: /etc/localtime
                      name: agent-time
                    - hostPath:
                        path: /var/run/docker.sock
                      name: dockersock
                    - hostPath:
                        path: /etc/docker/daemon.json
                      name: dockerdaemon
                    - hostPath:
                        path: /home/repository/
                      name: mvn-cache
                    - name: agent-jenkins-job
                      persistentVolumeClaim:
                        claimName: ks-devops-agent-pvc
            """
        }
    }
​
    options {
        timestamps()
        retry(1)
        buildDiscarder(logRotator(numToKeepStr: '100'))
    }
​
    stages {
        stage('拉取代码') {
            steps {
                echo "-----------------------------------------------------------------【拉取代码阶段】-----------------------------------------------------------------------"
                container('maven') {
                    git(credentialsId: "${GITLAB_ID}", url: "${gitlab_url}", branch: "${branch_name}", changelog: true, poll: false)
                }
            }
        }
​
        stage('执行构建') {
            steps {
                echo "-----------------------------------------------------------------【执行构建阶段】-----------------------------------------------------------------------"
                container('maven') {
                    script {
                        sh """
                            \${IYIQI_MVN_SED}
                            mvn -version
                            ls /etc/maven/
                            cat /etc/maven/xkhy-settings.xml
                            \${MVN_SHELL}
​
                        """
                    }
                }
            }
        }
​
        stage('生成docker镜像') {
            steps {
                echo "-----------------------------------------------------------------【生成镜像阶段】-----------------------------------------------------------------------"
                container('maven') {
                    sh """    
cat > Dockerfile << EOF
FROM \${BASE_IMAGES}
​
USER root
​
# 设置时区
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN echo "Asia/Shanghai" > /etc/timezone
​
# 设置环境变量，以支持中文
ENV LANG=zh_CN.UTF-8
ENV LANGUAGE=zh_CN.UTF-8
ENV LC_ALL=zh_CN.UTF-8
​
COPY \${DOCKER_JAR_PATH} /var/app/app.jar
​
\${APK_INSTALL}
​
WORKDIR /usr/local/
​
EXPOSE 8080
​
#运行初始化进程作为pid 1
\${PID_1}
​
CMD ["sh", "-c", "java \${JAVA_OPTS} -Dserver.port=8080 -Dfile.encoding=UTF-8  -Dsun.jnu.encoding=UTF-8 -jar /var/app/app.jar --spring.profiles.active=\${ACTIVE_ENV}"]
EOF
                    """
                    
                    script {
                        withCredentials([usernamePassword(credentialsId: "${HARBOR_ID}", passwordVariable: 'DOCKER_PASSWORD', usernameVariable: 'DOCKER_USER')]) {
                            sh """
                                cat Dockerfile
                                echo \${DOCKER_PASSWORD} | docker login --username=\${DOCKER_USER} --password-stdin \${HARBOR_URL}
                            """
                            sh "docker build --no-cache -t ${HARBOR_URL}\${DEPLOY_NAME}:${BUILD_TIMESTAMP}.${BUILD_NUMBER} ."
                            sh "docker push \${HARBOR_URL}\${DEPLOY_NAME}:${BUILD_TIMESTAMP}.${BUILD_NUMBER}"
                            sh "rm -rf *"
                        }
                    }
                }
            }
        }
​
        stage('更新编排文件') {
            steps {
                echo "-----------------------------------------------------------------【更新编排文件阶段】-----------------------------------------------------------------------"
                container('maven') {
                    script {
                        if (ENV == "prod") {
                            withCredentials([usernamePassword(credentialsId: "GIT_ARGOCD", passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
                                git(url: "http://139.9.242.148:5000/xiyadong/argocd-deploy.git", branch: "master", changelog: true, poll: false, credentialsId: "GIT_ARGOCD")
                                sh '''
                                    pwd
                                    sed -i -e 's#tag: .*#tag: \"'"${BUILD_TIMESTAMP}.${BUILD_NUMBER}"'\"#' ./${namespace}/${service}/values.yaml
                                    sed -i -e "s#namespace: .*#namespace: ${namespace}#g" -e "s#repository: .*#repository: ${HARBOR_URL}${DEPLOY_NAME}#g" -e "s#build_id: .*#build_id: \${BUILD_NUMBER}#g" ./${namespace}/${service}/values.yaml
                                    cat ./${namespace}/${service}/values.yaml
                                    cat ./${namespace}/${service}/Chart.yaml
                                    git config --global credential.helper 'store --file=.git-credentials'
                                    git config --global user.email "xiyadong@example.com"
                                    git config --global user.name "xiyadong"
                                    git commit -am "Update ${DEPLOY_NAME} service，current version ${BUILD_TIMESTAMP}.${BUILD_NUMBER}."
                                    git remote set-url origin http://${GIT_USERNAME}:${GIT_PASSWORD}@139.9.242.148:5000/xiyadong/argocd-deploy.git
                                    git push origin master
                                    rm -rf *
                                '''
                            }                
                        } else {
                            sh """
                                cd /root/jenkins/.helm/
                                sed -i -e 's#tag: .*#tag: \"'"\${BUILD_TIMESTAMP}.\${BUILD_NUMBER}"'\"#g' -e "s#  repository: .*#  repository: \${HARBOR_URL}\${DEPLOY_NAME}#g" -e "s#build_id: .*#build_id: \${BUILD_NUMBER}#g" ./${HELM_CHART_PATH}/\${service}/values.yaml
                                cat ./\${HELM_CHART_PATH}/\${service}/values.yaml
                                pwd
                            """
                            sh "helm upgrade --install ${DEPLOY_NAME} /root/jenkins/.helm/${HELM_CHART_PATH}/${service} --kubeconfig=/root/.kube/${KUBE_CONFIG} -n ${namespace}"
                        }
                    }
                }
            }
        }
​
        stage('argocd-sync') {
            steps {
                container('maven') {
                    script {
                        if (ENV == "prod") {
                            echo "-----------------------------------------------------------------【argo同步阶段】-----------------------------------------------------------------------"
                            withCredentials([usernamePassword(credentialsId: "${ARGOCD_SYNC_ID}", passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
                                def loginCmd = "argocd login --insecure \$ARGO_API --username \$GIT_USERNAME --password \$GIT_PASSWORD"
                                def loginResult = sh(script: loginCmd, returnStatus: true)
                                if (loginResult != 0) {
                                    error "ArgoCD登录失败"
                                }
​
                                def appExists = sh(
                                    script: "argocd app list | grep -q ${ARGO_PROJECT_NAME}",
                                    returnStatus: true
                                )
​
                                if (appExists == 0) {
                                    echo "-------------------- 【Application ${ARGO_PROJECT_NAME} exists. Syncing...】 ------------------------"
                                    sh """
                                        argocd app sync \${ARGO_PROJECT_NAME} --prune --force
                                        sleep 10
                                    """
​
                                    timeout(time: 5, unit: 'MINUTES') {
                                        def syncStatus = sh(
                                            script: "argocd app get ${ARGO_PROJECT_NAME} -o json",
                                            returnStdout: true
                                        ).trim()
​
                                        def jsonSlurper = new JsonSlurperClassic()
                                        def json = jsonSlurper.parseText(syncStatus)
                                        def lastSyncResult = json.status.operationState.syncResult
​
                                        if (lastSyncResult.resources.every { it.status == 'Synced' }) {
                                            echo "ArgoCD同步成功.}"
                                        } else {
                                            currentBuild.result = 'FAILURE'
                                            error "ArgoCD同步失败,请重试流水线或联系运维处理."
                                        }
                                    }
                                } else {
                                    echo "------------------ 【Application ${ARGO_PROJECT_NAME} does not exist. Creating...】 ---------------"
                                    sh """
                                        argocd app create \${ARGO_PROJECT_NAME} \
                                        --repo \$GIT_ARGOCD_URL \
                                        --path oiltax/\${service} \
                                        --dest-server \$CLUSTER_URL \
                                        --dest-namespace \$namespace
                                    """
​
                                    def appexists = sh(
                                        script: "argocd app list | grep -q ${ARGO_PROJECT_NAME}",
                                        returnStatus: true
                                    )
​
                                    if (appexists == 0) {
                                        echo "----------------- 【Application ${ARGO_PROJECT_NAME} created，syncing...】 ----------------------"
                                        sh """
                                        argocd app sync \${ARGO_PROJECT_NAME} --prune --force
                                        sleep 5
                                        """
                                    }
                                    timeout(time: 5, unit: 'MINUTES') {
                                        def syncStatus = sh(
                                            script: "argocd app get ${ARGO_PROJECT_NAME} -o json",
                                            returnStdout: true
                                        ).trim()
​
                                        def jsonSlurper = new JsonSlurperClassic()
                                        def json = jsonSlurper.parseText(syncStatus)
                                        def lastSyncResult = json.status.operationState.syncResult
​
                                        if (lastSyncResult.resources.every { it.status == 'Synced' }) {
                                            echo "ArgoCD同步成功.}"
                                        } else {
                                            currentBuild.result = 'FAILURE'
                                            error "ArgoCD同步失败,请重试流水线或联系运维处理."
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
​

# jenkins # kubernetes # 运维 # ci/cd
Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes