Gitea+K8s-Jenkins-master-slave(webhook钩子)

节点规划

服务器规划：centos7.9.2009最小化安装

当前的k8s容器底层为 containerd

主机名称	IP地址	节点
k8s-master	10.11.121.111	Jenkins
k8s-node1	10.11.121.112	node
k8s-node2	10.11.121.113	Gitea/NFS存储

部署Jenkins和Gitea

1.部署动态供应

这里原来的博客撤销啦，所有访问404
部署NFS动态供应 👈

2.部署Gitea

1.查看当前的StorageClass的存储情况：

[root@k8s-master ~]# kubectl get sc 
NAME                    PROVISIONER       RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
storage-nfs (default)   storage.pri/nfs   Delete          Immediate           false                  5h23m

2.使用Docker启动一个Gitea的容器

[root@k8s-master ~]# docker run -d \
--name gitea  \
-p 222:22 \
-p 3000:3000 \
-v /etc/timezone:/etc/timezone:ro \
-v /etc/localtime:/etc/localtime:ro \
-v /data/gitea:/data \
gitea/gitea:latest

3.修改gieta的配置文件并且重启gitea容器

[root@k8s-master ~]# vim /data/gitea/gitea/conf/app.ini
#在app.ini配置文件最后添加下面这个配置
[webhook]
ALLOWED_HOST_LIST = *

3.访问Gitea： http://10.11.121.113:3000

注册一个账号设置用户名密码绑定邮箱

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-9lhSvg6S-1680265300281)(null)]

创建一个项目仓库，仓库的名称叫做devops。

3.部署Jenkins

最新版镜像

• jenkins/jenkins:lts

当前使用的镜像

• jenkins/jenkins:lts-jdk11

关闭Jenkins的CSRF

• Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true

部署要求：

1. 创建RBAC策略，绑定Jenkins
2. 创建Deployment作为Jenkins容器控制器
3. 创建Service的服务暴露Jenkins的端口

[root@k8s-master ~]# mkdir /opt/jenkins && cd /opt/jenkins/
[root@k8s-master jenkins]# cat > Jenkins-rbac.yaml << EOF
apiVersion: v1
kind: Namespace
metadata:
  name: jenkins
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: jenkins
rules:
- apiGroups:
  - '*'
  resources:
  - statefulsets
  - services
  - replicationcontrollers
  - replicasets
  - podtemplates
  - podsecuritypolicies
  - pods
  - pods/log
  - pods/exec
  - podpreset
  - poddisruptionbudget
  - persistentvolumes
  - persistentvolumeclaims
  - jobs
  - endpoints
  - deployments
  - deployments/scale
  - daemonsets
  - cronjobs
  - configmaps
  - namespaces
  - events
  - secrets
  verbs:
  - create
  - get
  - watch
  - delete
  - list
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
  - list
  - watch
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkins
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:jenkins
EOF

[root@k8s-master jenkins]# cat > Jenkins-Deployment.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
  namespace: jenkins
  labels:
    app: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
      - name: jenkins
        image: jenkins/jenkins:lts-jdk11
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        - containerPort: 50000
          name: agent
          protocol: TCP
        #resources:
          #limits:
            #memory: 4Gi
            #cpu: "2000m"
          #requests:
            #memory: 4Gi
            #cpu: "2000m"
        env:
        - name: LIMITS_MEMORY
          valueFrom:
            resourceFieldRef:
              resource: limits.memory
              divisor: 1Mi
        - name: JAVA_OPTS
          value: -Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true
        volumeMounts:
        - name: jenkins-home
          mountPath: /var/jenkins_home
      volumes:
      - name: jenkins-home
        persistentVolumeClaim:
          claimName: jenkins-home
---
apiVersion: v1
kind:  PersistentVolumeClaim
metadata:
  name: jenkins-home
  namespace: jenkins
spec:
  storageClassName: "storage-nfs"
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 20Gi
EOF

[root@k8s-master jenkins]# cat > Jenkins-Service.yaml << EOF
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: jenkins
  labels:
    app: jenkins
spec:
  selector:
    app: jenkins
  type: NodePort
  ports:
  - name: web
    port: 8080
    targetPort: web
    nodePort: 31000
  - name: agent
    port: 50000
    targetPort: agent
EOF

创建所有的yaml文件。

[root@k8s-master jenkins]#  kubectl apply -f Jenkins-rbac.yaml -f Jenkins-Deployment.yaml -f Jenkins-Service.yaml
[root@k8s-master jenkins]# kubectl get -n jenkins all 
NAME                           READY   STATUS    RESTARTS   AGE
pod/jenkins-746b5b5d65-tllqz   1/1     Running   0          30s

NAME              TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                          AGE
service/jenkins   NodePort   10.96.213.109   <none>        8080:31000/TCP,50000:31204/TCP   24s

NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/jenkins   1/1     1            1           30s

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/jenkins-746b5b5d65   1         1         1       30s

4.配置Jenkins

安装Jenkins插件

Localization: Chinese (Simplified)

Pipeline

Kubernetes

1、使用命令 kubectl exec -it $jenkins-pod cat /var/jenkins_home/secrets/initialAdminPassword查看密码。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-JYIz1biA-1680265300379)(null)]

2、选择插件来安装

3、这里勾选无，只需要再后面选择的时候再安装插件即可。

4、创建一个用户登录。

5、配置Jenkins使用国内的源

配置插件源 默认从国外网络下载插件，会比较慢，建议修改成国内源：

只需要到nfs上，修改PVC挂载的内容即可

[root@k8s-master nfsmount]# cd jenkins-jenkins-home-pvc-1175cde6-4c32-45b1-bed2-7c90ac173972/
[root@k8s-master jenkins-jenkins-home-pvc-1175cde6-4c32-45b1-bed2-7c90ac173972]# cd updates/
[root@k8s-master updates]# ls 
default.json  hudson.tasks.Maven.MavenInstaller
'先备份好配置文件'
[root@k8s-master updates]# cp default.json default.json.bak  
'修改插件的下载地址为国内的地址'
[root@k8s-master updates]# sed -i s#https://updates.jenkins.io/download#https://mirrors.tuna.tsinghua.edu.cn/jenkins#g default.json
'修改jenkins启动时检测的URL网址，改为国内baidu的地址'
[root@k8s-master updates]# sed -i s#http://www.google.com#https://www.baidu.com#g default.json default.json
'删除pod重建（pod名称改成你实际的）'
[root@k8s-master updates]# kubectl delete pod -n jenkins jenkins-746b5b5d65-tllqz 
pod "jenkins-746b5b5d65-tllqz" deleted

6、安装需要的插件。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-8yKCN3sv-1680265300224)(null)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-KN8mIWmx-1680265300252)(null)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-25Qr82lx-1680265300238)(null)]

7、配置jenkins连接k8s

https://kubernetes.default
http://jenkins.jenkins:8080

或者

# 可以是apiserver的地址
https://kubernetes.default.svc.cluster.local

# 这里是jenkins命名空间下的jenkins地址
http://jenkins.jenkins.svc.cluster.local:8080

选择 系统管理 > 节点管理 找到Config cloud

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-7HlLG3g8-1680265300267)(null)]

8、在全局安全配置中开启（匿名用户具有可读权限 ）

在全局安全配置中的授权策略 把登录用户可以做任何事 下的匿名用户具有可读权限 勾上

在全局安全配置中关闭跨站请求伪造保护 （在安装Jenkins关闭）

9、给jenkins用户添加新Token（点击右上角jenkins用户，然后点设置就可以跳转到配置Token页面）

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-PBeB0C0S-1680265300345)(null)]

5.Pipeline（动态pod）

创建一个pipeline流水线测试

在构建触发器 中勾上 ：触发远程构建 (例如,使用脚本)

身份验证令牌 就填上面创建的Token

pipeline {
    agent {
        kubernetes {
            yaml '''
apiVersion: v1
kind: Pod
metadata:
  name: jenkins-slave
  namespace: jenkins
spec:
  containers:
  - name: jnlp
    image: jenkins/inbound-agent:latest
    volumeMounts:
'''
        }
    }
    stages {
        stage('1、拉取代码') {
            steps {
                echo 'Hello World'
            }
        }
        stage('2、代码编译') {
            steps {
                echo 'Hello World'
            }
        }
        stage('3、单元测试') {
            steps {
                echo 'Hello World'
            }
        }
        stage('4、部署') {
            steps {
                echo 'Hello World'
			}
		}
	}
}

1、新建一个流水线项目

2、配置流水线的脚本，测试动态Pod。

3、保存退出之后，单击立即构建。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-0TiqFBgM-1680265300296)(null)]

6.创建一个仓库

在gitea创建一个名字为devops仓库，在此之前我们已经创建好了，现在只需要上传就好。

[root@k8s-master demo]# echo  add index.html  > README.md
[root@k8s-master demo]# git init 
Initialized empty Git repository in /root/demo/.git/
[root@k8s-master demo]# git add README.md
[root@k8s-master demo]# git commit -m "first commit"
[master (root-commit) 315edf5] first commit
 1 file changed, 1 insertion(+)
 create mode 100644 README.md
[root@k8s-master demo]#  git remote add origin http://10.11.121.113:3000/devops/devops.git
[root@k8s-master demo]# git push -u origin master
Username for 'http://10.11.121.113:3000': devops
Password for 'http://devops@10.11.121.113:3000': 
Counting objects: 3, done.
Writing objects: 100% (3/3), 224 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: . Processing 1 references
remote: Processed 1 references in total
To http://10.11.121.113:3000/devops/devops.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

查看当前的仓库源码。

7.配置webhook钩子

1.设置gitea的Webhook的设置，配置webhook的钩子。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-0XvfoEhr-1680265300320)(null)]

2.查看当前的Jenkins的流水线配置，设置开启触发远程构建。

测试一下推送代码。

8.推代码测试钩子

[root@k8s-master demo]# git clone http://10.11.121.113:3000/devops/devops.git   #先克隆仓库
Cloning into 'devops'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), done.
[root@k8s-master demo]# cd devops/         #进入仓库
[root@k8s-master devops]# echo  abc > index.html #创建一个新文件来模拟仓库代码改动
[root@k8s-master devops]# git add .
[root@k8s-master devops]# git commit -m "add index.html two"
[master 91cc098] add index.html two
 1 file changed, 1 insertion(+)
 create mode 100644 index.html
[root@k8s-master devops]# git push  origin master
Username for 'http://10.11.121.113:3000': devops
Password for 'http://devops@10.11.121.113:3000': 
Counting objects: 4, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 284 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: . Processing 1 references
remote: Processed 1 references in total
To http://10.11.121.113:3000/devops/devops.git
   315edf5..91cc098  master -> master
# 提交后，回到jenkins页面，查看是否成功自动触发构建代码

以下是自动触发构建。