k8s部署jenkins

【代码】k8s部署jenkins。

被一米六支配的恐惧

1500人浏览 · 2023-11-28 19:35:42

被一米六支配的恐惧 · 2023-11-28 19:35:42 发布

1.先决条件

1.因为国内的容器镜像加速器无法实时更新docker hub上的镜像资源.所以可以自己进行jenkins的容器镜像创建,.

2.这里用到了storageClass k8s的动态制备.详情参考:

k8s-StoargClass的使用-基于nfs-CSDN博客

3.安装docker服务.(用于构建docker image)

2.构建jenkins镜像

1.创建Dockerfile用于构建jenkins容器镜像.

这里原始镜像我个人修改成了自己仓库的镜像。具体情况个人自己分析。

[root@master /zpf/jenkins]$cat Dockerfile
FROM jenkins/jenkins:latest
ADD ./apache-maven-3.9.4-bin.tar.gz /usr/local/
COPY ./sonar-scanner-4.8.0.2856-linux /usr/local


ENV MAVEN_HOME=/user/local/apache-maven-3.9.4
ENV PATH=$JAVE_HOME/bin/:$MAVEN_HOME/bin:$PATH
USER root
RUN echo "jenkins ALL=NOPASSWORD: ALL" >> /etc/sudoers

USER jenkins

maven下载地址 Index of /dist/maven/maven-3/3.9.4/binaries

sonar-scanner下载地址 SonarSource Downloads-CDN

2.构建docker镜像

这里下载了maven 和sonarqube将这两个包一起打入容器镜像中.下载地址:

容器镜像仓库地址各自自己定义。没有harbor的可以使用阿里云容器镜像仓库。(免费）

[root@master /zpf/jenkins]$ls
apache-maven-3.9.4-bin.tar.gz  Dockerfile  github-jenkins  manifests  sonar-scanner-4.8.0.2856-linux  sonar-scanner-cli-4.8.0.2856-linux.zip
[root@master /zpf/jenkins] docker build -t 192.168.75.35:8858/scorpio/jenkins-maven:v1 .

3.上传到自己搭建的harbor容器镜像仓库

[root@master /zpf/jenkins]$ docker push 192.168.75.35:8858/scorpio/jenkins-maven:v1

4.编辑jenkins-ServiceAccount 文件

[root@master /zpf/jenkins/manifests]$vim jenkins-serviceAccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-admin
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: jenkins-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: jenkins-admin
  namespace: default

5.创建sa

[root@master /zpf/jenkins/manifests]$kubectl create -f jenkins-serviceAccount.yaml

6.查看sa创建状态,以及角色权限

[root@master /zpf/jenkins/manifests]$kubectl get sa |grep jenkins
jenkins-admin            1         5d2h
[root@master /zpf/jenkins/manifests]$kubectl get ClusterRoleBinding |grep jenkins
jenkins-admin                                          ClusterRole/jenkins-admin                                                          5d2h
[root@master /zpf/jenkins/manifests]$kubectl get ClusterRole |grep jenkins
jenkins-admin                                                          2023-11-23T08:33:36Z

7.创建pvc文件(这里用到了storageClass)

[root@master /zpf/jenkins/manifests]$cat jenkins-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pvc
  namespace: default
spec:
  storageClassName: managed-nfs-storage
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1G

8. 创建pvc

[root@master /zpf/jenkins/manifests]$kubectl create -f jenkins-pvc.yaml

9.查看创建结果

[root@master /zpf/jenkins/manifests]$kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                              STORAGECLASS          REASON   AGE
pvc-b1c1b782-47d7-42ce-be60-8e5d832d21bc   1Gi        RWX            Delete           Bound    default/sonarqube-data-pvc                                         managed-nfs-storage            3h25m
pvc-b35e5912-99be-4c02-a164-5ea5e385b0a4   1G         RWX            Delete           Bound    default/jenkins-pvc                                                managed-nfs-storage            29h
pvc-b7cdc67e-f66a-4602-84ef-e59c741487e2   2Gi        RWX            Delete           Bound    default/postgres-data                                              managed-nfs-storage            3h24m
pvc-d215b650-e7cf-4216-972d-4d56d0b650a6   20Gi       RWO            Delete           Bound    kubesphere-monitoring-system/prometheus-k8s-db-prometheus-k8s-0    local                      14d
[root@master /zpf/jenkins/manifests]$kubectl get pvc
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
jenkins-pvc          Bound    pvc-b35e5912-99be-4c02-a164-5ea5e385b0a4   1G         RWX            managed-nfs-storage   29h
postgres-data        Bound    pvc-b7cdc67e-f66a-4602-84ef-e59c741487e2   2Gi        RWX            managed-nfs-storage   3h24m
sonarqube-data-pvc   Bound    pvc-b1c1b782-47d7-42ce-be60-8e5d832d21bc   1Gi        RWX            managed-nfs-storage   3h25m

10.创建sts yaml文件并绑定svc

[root@master /zpf/jenkins/manifests]$cat sts-jenkins.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: jenkins
  namespace: default
spec:
  serviceName: "jenkins"
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      serviceAccountName: jenkins-admin
      containers:
      - name: jenkins
        image: 192.168.75.35:8858/scorpio/jenkins-maven:v1
#        image: jenkins/jenkins:latest
        imagePullPolicy: IfNotPresent
        securityContext:
          privileged: true
          runAsUser: 0
        ports:
        - containerPort: 8080
        - containerPort: 50000
#        env:
#        - name: JAVA_OPTS
#          value: "-Dhudson.model.DownloadService.noSignatureCheck=true"
        volumeMounts:
        - name: jenkins-data
          mountPath: /var/jenkins_home
        - name: docker
          mountPath: /run/docker.sock
        - name: docker-home
          mountPath: /usr/bin/docker
        - name: mvn-setting
          mountPath: /usr/local/apache-maven-3.9.4/conf/settings.xml
#        - name: mvn-setting
#          mountPath: /usr/local/apache-maven-3.9.4/conf/settings.xml
        - name: daemon
          mountPath: /etc/docker
          subPath: daemon.json
        - name: kubectl
          mountPath: /usr/bin/kubectl

      volumes:
      - name: jenkins-data
        persistentVolumeClaim:
          claimName: jenkins-pvc
      - name: docker
        hostPath:
          path: /run/docker.sock
      - name: docker-home
        hostPath:
          path: /usr/bin/docker
      - name: mvn-setting
        hostPath:
          path: /zpf/jenkins/manifests/conf/settings.xml
#      - name: mvn-setting
#        configMap:
#          name: mvn-settings
#          items:
#          - key: settings.xml
#            path: settings.xml
      - name: daemon
        hostPath:
          path: /etc/docker/
      - name: kubectl
        hostPath:
          path: /usr/local/bin/kubectl

11.创建sts服务

[root@master /zpf/jenkins/manifests]$kubectl get deploy
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
nfs-client-provisioner   1/1     1            1           7d3h
postgres-sonar           1/1     1            1           3h25m
sonarqube                1/1     1            1           3h22m

12.创建svc并绑定svc

[root@master /zpf/jenkins/manifests]$cat service-jenkins.yaml
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: default
spec:
  type: NodePort
  ports:
  - name: http
    port: 8080
    targetPort: 8080
    nodePort: 31400
  - name: agent
    port: 50000
    targetPort: 50000
    nodePort: 31401
  selector:
    app: jenkins

13.创建svc

[root@master /zpf/jenkins/manifests]$kubectl get svc
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                          AGE
jenkins           NodePort    10.233.54.32    <none>        8080:31400/TCP,50000:31401/TCP   7d
jenkins-service   NodePort    10.233.57.17    <none>        8080:32000/TCP                   5d2h
kubernetes        ClusterIP   10.233.0.1      <none>        443/TCP                          14d
sonarqube         NodePort    10.233.55.245   <none>        9000:32273/TCP                   3h15m

14.查看页面

[root@master /zpf/jenkins/manifests]$kubectl get svc
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                          AGE
jenkins           NodePort    10.233.54.32    <none>        8080:31400/TCP,50000:31401/TCP   7d
jenkins-service   NodePort    10.233.57.17    <none>        8080:32000/TCP                   5d2h
kubernetes        ClusterIP   10.233.0.1      <none>        443/TCP                          14d
sonarqube         NodePort    10.233.55.245   <none>        9000:32273/TCP                   3h15m

15.页面验证

16.遇到的问题,

1.插件下载源更换,这里更换的插件源是

https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json

设置位置是 DashBoard-->系统管理--> 插件管理-->Advanced settings

或者从容器内部进行修改(自行查阅配置文件位置.)

17.遇到的问题2.

系统版本太老(安装完成的原始版本是2.3版本,插件版本比jenkins版本高,导致插件无法安装运行.)

问题解决,这里直接将原本老版本的jenkins升级为最新版本的jenkins即可.升级方式

Dashboard--> 系统管理--> 页面中间就有版本升级信息.点击升级即可,(这里说明一点,部署的时候尽量不要使用daployment,否则重启服务后版本设置又变回原来的版本了,)也可以将数据挂在出来,解决这个问题.

K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 | 韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。长夜有穷，真水无香。领略过了 K8s“身在江

K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub