安装harbor

harbor是容器镜像注册仓库

官方首页:
https://goharbor.io/

官方文档:

• 安装和配置文档(针对版本2.1)
  https://goharbor.io/docs/2.1.0/install-config/

• 下载地址:
  https://github.com/goharbor/harbor/releases

• 将docker login的凭据保存在外部存储
  https://docs.docker.com/engine/reference/commandline/login/#credentials-store

当前版本:2.1.3 (2021.2.1)

系统要求:

• docker 17.06.0-ce+
• docker-compose 1.18.0+

1.前置任务

准备如下包或软件

• SELinux : permessive
• docker : 17.06.0-ce+
• docker-compose : 1.18.0+
• harbor离线安装包 : 文件名:harbor-offline-installer-v2.1.3.tgz .目前版本是2.1.3 (2021.2.1)
• 自签名证书：
  • ca.crt
  • server.crt 和 server.key 注意:server.crt中的DNS和IP必须包含本服务器的主机名和IP
  • client.crt,client.key docker daemon为私库的客户端,因此使用客户端证书

1.1 硬件要求:

https://goharbor.io/docs/2.1.0/install-config/installation-prereqs/

Resource	Minimum	Recommended
CPU		2 CPU	4 CPU
Mem		4 GB	8 GB
Disk		40 GB	160 GB

网络端口(可以在配置文件中修改):

Port	Protocol	Description
443	HTTPS		
4443	HTTPS		
80	HTTP		

2.安装步骤:

安装分为2部分:

• 安装harbor私库
• 设置k8s的worker节点的docker,以便能访问harbor私库(k8s的master节点一般不会下载应用的镜像，因此无需设置master节点的docker)

2.1 安装harbor私库

1.下载docker-compose,并拷贝到/usr/local/bin目录

chmod +x docker-compose-Linux-x86_64

cp -p docker-compose-Linux-x86_64 /usr/local/bin/docker-compose

2.上载harbor离线安装包到~/tool目录
3.编辑.~/.bashrc,设置harbor的环境变量。并让设置生效

# 先备份
cp -p ~/.bashrc ~/.bashrc.20210202

tee -a >>~/.bashrc<<EOF

# harbor
HARBOR_ROOT=~/program

export HARBOR_ROOT
EOF

# 让设置生效
source ~/.bashrc

4.解压harbor的TAR包,得到harbor子目录

# 创建目录
mkdir -p $HARBOR_ROOT

cd $HARBOR_ROOT
tar -zvxf ~/tool/harbor-offline-installer-v2.1.3.tgz

5.配置$HARBOR_HOME/harbor.yml文件

cd $HARBOR_ROOT/harbor

# 从模板复制配置文件
cp -p harbor.yml.tmpl harbor.yml

必须配置的参数如下:

• hostname : 可以是IP地址

sed -i "s/^hostname:.*/hostname: 192.168.8.xxx(本机IP地址或域名)/" harbor.yml

• https下的certificate 和 private_key
  假设服务端证书在~/certs目录下

certificate: ~/certs/server.crt
private_key: ~/certs/server.key

• harbor_admin_password(可选) : 管理员密码.缺省用户名是admin,密码是Harbor12345。此密码只用于第一次启动harbor,之后的密码在门户中设置
• data_volume(可选) : 存放harbor数据的目录.缺省是/data

9.执行harbor的install.sh脚本

cd $HARBOR_ROOT/harbor

# 启用上述所有选项
./install.sh

输出如下:

[Step 0]: checking if docker is installed ...

Note: docker version: 20.10.2

[Step 1]: checking docker-compose is installed ...

Note: docker-compose version: 1.28.2

[Step 2]: loading Harbor images ...
[Step 3]: preparing environment ...
...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/program/harbor
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Successfully called func: create_root_cert
Successfully called func: create_cert
Copying certs for notary signer
Copying nginx configuration file for notary
Generated configuration file: /config/nginx/conf.d/notary.upstream.conf
Generated configuration file: /config/nginx/conf.d/notary.server.conf
Generated configuration file: /config/notary/server-config.postgres.json
Generated configuration file: /config/notary/server_env
Generated and saved secret to file: /data/secret/keys/defaultalias
Generated configuration file: /config/notary/signer_env
Generated configuration file: /config/notary/signer-config.postgres.json
Generated configuration file: /config/clair/postgres_env
Generated configuration file: /config/clair/config.yaml
Generated configuration file: /config/clair/clair_env
Generated configuration file: /config/clair-adapter/env
Generated configuration file: /config/chartserver/env
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir


[Step 5]: starting Harbor ...
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating network "harbor_harbor" with the default driver
Creating network "harbor_harbor-clair" with the default driver
Creating network "harbor_harbor-notary" with the default driver
Creating network "harbor_harbor-chartmuseum" with the default driver
Creating network "harbor_notary-sig" with the default driver
Creating harbor-log ... done
Creating registryctl   ... done
Creating chartmuseum   ... done
Creating harbor-portal ... done
Creating registry      ... done
Creating redis         ... done
Creating harbor-db     ... done
Creating clair         ... done
Creating notary-signer ... done
Creating harbor-core   ... done
Creating clair-adapter     ... done
Creating notary-server     ... done
Creating nginx             ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----

注意:如果执行install.sh发生报错,则多半是harobr.yml的格式错误导致

10.页面访问(假设本机IP为192.168.8.172):
https://192.168.8.172

• 用户名:admin
• 初始密码:Harbor12345

11.设置harbor为系统服务
服务文件harbor.service的内容如下:

[Unit]
Description=Start Harbor private registry
Documentation=https://goharbor.io/
After=docker.service

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/root/program/harbor

ExecStart=/usr/local/bin/docker-compose up -d
ExecStop=/usr/local/bin/docker-compose down

[Install]
WantedBy=multi-user.target

1.修改harbor.service中的WorkingDirectory的值为$HARBOR_ROOT/harbor的值,比如:/root/program/harbor
2.将harbor.service复制到/etc/systemd/system目录下
3.执行如下命令

systemctl daemon-reload
systemctl start harbor
systemctl status harbor
systemctl enable harbor

2.2 设置k8s的worker节点的docker,以便能访问harbor私库

假设harbor私库安装在192.168.8.172

1.为docker配置证书
在/etc/docker目录下创建certs.d/192.168.8.172子目录

cd /etc/docker
mkdir -p certs.d/192.168.8.172

2.将ca.crt,client.crt 和 client.key拷贝到/etc/docker/certs.d/192.168.8.172子目录下,且将文件更名为

• CA证书 ca.crt
• 服务端证书 client.crt -> client.cert **注意:**证书的扩展名必须是.cert
• 服务端私钥 client.key -> client.key

3.重启docker

systemctl restart docker

3. 管理Harbor

在$HARBOR_HOME目录下执行docker-compose命令，以便手动启动或停止harbor

3.1停止harbor

cd $HARBOR_HOME
docker-compose down

3.2以后台方式启动harbor

docker-compose up -d

3.3重新配置harbor

# 删除容器且保持镜像和数据库
docker-compose down -v
vim harbor.yml
./prepare --with-notary --with-clair --with-chartmuseum
docker-compose up -d

缺省下,registry仓库数据存放在/data目录.日志在/var/log/harbor目录

4.将镜像推到harbor

1.对原镜像重新打标签.新标签必须是IP/library/xxx:yyy格式,比如：192.168.8.172/library/pause:3.2

docker tag registry.aliyuncs.com/google_containers/pause:3.2 \
192.168.8.172/library/pause:3.2

# admin/Harbor12345
docker login 192.168.8.172

docker push 192.168.8.172/library/pause:3.2

在执行docker login 192.168.8.172后,会生成~/.docker/config.json,内容如下:

{
	"auths": {
		"192.168.8.172": {
			"auth": "YWRtaW46SGFyYm9yMTIzNDU="
		}
	}
}

5.在k8s中拉取私库镜像

前置条件:

• 私库安装完毕
• 私库服务器已信任ca(需要重启服务器)
• k8s的worker节点已设置docker为私库的客户端

5.1 拉取原版镜像

docker pull alpine

5.2 重新打标签

docker tag alpine:latest 192.168.2.28/library/alpine:latest

5.3 登录私库

docker login 192.168.2.28 -u admin -p Harbor12345

5.4 推送镜像到私库

docker push 192.168.2.28/library/alpine:latest

5.5 创建k8s的secret对象

kubectl create secret docker-registry registry-secret
–docker-username=admin
–docker-password=Harbor12345

5.6 创建k8s的service account对象,并引用上面的secret

apiVersion: v1
kind: ServiceAccount
metadata:
name: mysa
imagePullSecrets:

• name: registry-secret

5.7 创建k8s的deployment对象,并设置podspec的serviceAccountName值为上述sa

apiVersion: apps/v1
kind: Deployment
metadata:
name: myjenkins
spec:

以下为pod模板

template:
spec:
restartPolicy: Always
# 用于运行和拉取镜像的SA
serviceAccountName: mysa
containers:
- name: jenkins
image: 192.168.2.28/library/alpine:latest
imagePullPolicy: IfNotPresent