Jenkins+Gitlab+Harbor+k8s CICD

机器环境准备

主机名	IP地址	操作系统	配置
Gitlab	192.168.3.160	debian11	2核4G
Jenkins	192.168.3.161	debian11	2核4G
Harbor	192.168.3.162	debian11	2核4G

基础依赖安装

安装Gitlab(官方教程安装)

1.安装并配置必要的依赖项

sudo apt-get update && sudo apt-get install -y curl openssh-server ca-certificates perl

2.安装 Postfix

sudo apt-get install -y postfix

选择Internet Site

填写邮件名称

3.添加GitLab包存储库并安装Gitlab

curl -s https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | bash

# 然后做了DNS解析的话就填域名，如果没有就填IP地址
#sudo EXTERNAL_URL="https://<domain/address>" apt-get install gitlab-ce

export GITLAB_URL="http://192.168.3.160"
sudo EXTERNAL_URL=${GITLAB_URL} apt-get install -y gitlab-ce

4.启动Gitlab

gitlab-ctl start

常用命令

gitlab-ctl status:  查看gitlab组件状态
gitlab-ctl restart:  重启全部服务
gitlab-ctl stop： 停止全部服务
gitlab-ctl tail <service name>  查看服务的日志
gitlab-ctl show-config:  验证配置文件

#使配置文件生效（一般修改完主配置文件/etc/gitlab/gitlab.rb，需要执行此命令）
gitlab-ctl reconfigure
gitlab-ctl restart

5.登录Gitlab

访问地址：https://192.168.3.160

默认账号：root

默认密码存储在：cat /etc/gitlab/initial_root_password #只保留24小时，及时修改密码

6.常用操作介绍

常用目录文件介绍

/etc/gitlab/gitlab.rb # 主配置文件
/var/log/gitlab/ # 日志目录
/var/opt/gitlab/ # 各个服务的主目录
/var/opt/gitlab/git-data/repositories # Git仓库数据目录

修改访问端口

nano /etc/gitlab/gitlab.rb
external_url 'http://127.0.0.1:8090'

修改备份目录

nano /etc/gitlab/gitlab.rb
gitlab_rails["backup_path"] = "/data/gitlab/backups"  #添加此行

修改仓库数据目录

nano /etc/gitlab/gitlab.rb
git_data_dirs({
   "default" => {
    "path" => "/data/gitlab/gitlab-data"
    }
 })

安装Jenkins(官方教程安装)

1.安装Java

sudo apt update && sudo apt install -y fontconfig openjdk-17-jre

#查看java版本
java -version
openjdk version "17.0.8" 2023-07-18
OpenJDK Runtime Environment (build 17.0.8+7-Debian-1deb12u1)
OpenJDK 64-Bit Server VM (build 17.0.8+7-Debian-1deb12u1, mixed mode, sharing)

2.安装Jenkins稳定版

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update && sudo apt-get -y install jenkins

3.启动jenkins

sudo systemctl enable jenkins
sudo systemctl start jenkins
sudo systemctl status jenkins

4.登录Jenkins

访问地址：http://192.168.3.161:8080

默认密码存储在：cat /var/lib/jenkins/secrets/initialAdminPassword

安装Harbor

1.安装docker compose

curl -SL https://github.com/docker/compose/releases/download/v2.24.6/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose --version

2.安装docker

 curl -fsSL https://get.docker.com -o get-docker.sh
 sudo sh get-docker.sh

3.下载安装包

export harbor_version="v2.9.2"
wget https://github.com/goharbor/harbor/releases/download/${harbor_version}/harbor-offline-installer-${harbor_version}.tgz
tar -zxvf harbor-offline-installer-${harbor_version}.tgz

4.配置harbor

cd harbor
cp harbor.yml.tmpl harbor.yml
nano harbor.yml
./install.sh

5.安装启动Harbor

./install
docker-compose up -d

6.登录Harbor

访问地址：http://192.168.3.162

默认账号：admin

默认密码存储在第四点的harbor.yml文件里

配置 CICD

1.安装相关插件

gitlab

安装docker环境

 curl -fsSL https://get.docker.com -o get-docker.sh
 sudo sh get-docker.sh

2.连接Gitlab仓库

如果证书报不安全的话需要跳过验证

Ignore SSL Certificate Errors

3.配置harbor私有仓库

nano /etc/docker/daemon.json

配置完需要重启docker：systemctl restart docker

{
  "insecure-registries": ["http://192.168.3.162"]
}

export REGISTRY_DOMAIN="192.168.3.162"
export REGISTRY_USERNAME="admin"
export REGISTRY_PASSWORD="xxc12345"

4.添加环境变量

5.添加流水线项目并配置pipeline

def appName="demo"
def label = "jenkins-${appName}-${UUID.randomUUID().toString()}"

podTemplate(label: label, serviceAccount: 'jenkins', containers: [
  containerTemplate(name: 'maven', image: 'maven:3.8.6-eclipse-temurin-17', command: 'cat', ttyEnabled: true),
  containerTemplate(name: 'docker', image: 'docker:dind', ttyEnabled: true, privileged: true)
]) {
  node(label) {

    def myRepo = checkout scm
    def gitCommit = myRepo.GIT_COMMIT
    def gitBranch = myRepo.GIT_BRANCH
    def imageTag = sh(script: "git rev-parse --short HEAD", returnStdout: true).trim()
    def dockerRegistryUrl = "${DOCKER_REGISTRY}"
    def imageEndpoint = "${dockerImageNamePrefix}/${appName}"
    def image = "${dockerRegistryUrl}/${imageEndpoint}"
    def namespace ="${k8sNamespace}"

    try {

      stage('Code compiles and packages') {
        container('maven') {
          echo "Code compiles and packages"
          sh "mvn clean install -Dmaven.test.skip=true -U"
        }
      }

      stage('Build Docker images') {
        withCredentials([[$class: 'UsernamePasswordMultiBinding',
          credentialsId: 'harbor',
          usernameVariable: 'DOCKER_HUB_USER',
          passwordVariable: 'DOCKER_HUB_PASSWORD']]) {
            container('docker') {
              echo "Build Docker images"
              sh """
                docker login ${dockerRegistryUrl} -u "${DOCKER_HUB_USER}" -p "${DOCKER_HUB_PASSWORD}"
                docker build -t ${image}:${imageTag} -f manifest/Dockerfile .
                docker push ${image}:${imageTag}
                """
            }
        }
      }

      stage('Deploy images to Kubernetes') {
        echo "Deploy images to Kubernetes:${appName}"
        sh """
          sed -i "s#<IMAGE>#${image}#g" manifest/k8s.yaml
          sed -i "s/<IMAGE_TAG>/${imageTag}/g" manifest/k8s.yaml
          sed -i "s/<NAMESPACE>/${namespace}/g" manifest/k8s.yaml
        """
        deploy_to_k8s()
      }

    } catch (exc) {
      println "Build error - ${currentBuild.fullDisplayName}"
      throw(exc)
    }
  }
}

// Generates a script block to download kubectl
def deploy_to_k8s(){
  sh """
      git clone git@192.168.3.160:root/demo.git
      chmod +x demo/curl-amd64
      demo/curl-amd64 -LO https://storage.googleapis.com/kubernetes-release/release/v1.24.1/bin/linux/amd64/kubectl
      chmod +x kubectl
      ./kubectl apply -f manifest/k8s.yaml
     """
}