kubernetes(K8S)集群组件(二) ------ containerd(容器运行时)的安装和使用
容器技术除了docker之外,还有其它不同的容器技术,为了保证容器生态的标准和健康可持续发展,包括Linux基金会Docker、微软、红帽、谷歌、IBM和华为等公司在2015年6月共同成立了open container(OCI)组织,目的是制定开放标准的容器规范。
containerd 容器技术简介
容器技术除了docker之外,还有其它不同的容器技术,为了保证容器生态的标准和健康可持续发展,包括Linux基金会Docker、微软、红帽、谷歌、IBM和华为等公司在2015年6月共同成立了open container(OCI)组织,目的是制定开放标准的容器规范。目前OCI发布了 runtime spec(运行时规范)、image format spec(镜像格式规范)、distribution-spec(镜像分发规范),这样不同的容器公司开发的容器只要兼容以上规范,就可以保证容器的可移植性和相互可操作性
常见的运行时包括:runc(go语言开发)、crun(C语言开发)、gVisor(go语言开发)
运行时分为高级运行时和低级运行时:
容器运行时级别 | 功能 |
---|---|
High-Level | 高级别运行时提供基于API的远程管理操作,客户端可以通过高级别运行时管理容器的整个生命周期,高级别运行时不是真正直接运行容器,而是调用低级别运行时,比如dockerd和containerd都是高级别运行时 |
Low-Level | 低级别运行时接受高级别运行时的指令,按照响应的指令运行容器,因此低级别运行时是真正运行容器的地方,例如runc |
为了兼容OCI规范,Docker项目做了架构调整,Docker由一个单一组件拆分为Docker Engine(docker-daemon)、containerd、containerd-shim和RunC等4个独立项目,并将containerd捐赠给CNCF。
containerd是一个守护进程,可以实现容器运行时所需要的创建、启动 、停止、中止、信号处理和删除以及镜像管理等所有功能,同时通过gRPC向上层调用者公开其API,可以被兼容的任何上层系统调用。
containerd项目github地址:https://github.com/containerd/containerd
通过apt/yum仓库安装containerd
系统版本:ubuntu20.04.3
root@k8s-master1:~# uname -a
Linux k8s-master1 5.4.0-81-generic #91-Ubuntu SMP Thu Jul 15 19:09:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
更新apt索引
root@k8s-master1:~# apt-get update
如果update报错:(需要更新证书)
root@k8s-master1:~# apt-get update
Ign:1 https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal InRelease
Ign:2 https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-updates InRelease
Ign:3 https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-backports InRelease
Ign:4 https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-security InRelease
Err:5 https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal Release
Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 101.6.15.130 443]
Err:6 https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-updates Release
Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 101.6.15.130 443]
Err:7 https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-backports Release
Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 101.6.15.130 443]
Err:8 https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-security Release
Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 101.6.15.130 443]
Reading package lists... Done
E: The repository 'https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal Release' no longer has a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-updates Release' no longer has a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-backports Release' no longer has a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-security Release' no longer has a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
更改源文件,将所有的https改成http:
root@k8s-master1:~# cp /etc/apt/sources.list{,.bak}
root@k8s-master1:~# sed -i '/^[^#]/s/https:/http:/' /etc/apt/sources.list
更新源:
root@k8s-master1:~# apt-get update
安装/更新证书ca-certificates:
root@k8s-master1:~# apt-get install --reinstall ca-certificates
将镜像源文件改回https:
root@k8s-master1:~# \cp /etc/apt/sources.list.bak /etc/apt/sources.list
再次更新源:
root@k8s-master1:~# apt-get update
验证仓库版本:
root@k8s-master1:~# apt-cache madison containerd
root@k8s-master1:~# apt-cache madison containerd
containerd | 1.6.12-0ubuntu1~20.04.1 | https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-updates/main amd64 Packages
containerd | 1.5.9-0ubuntu1~20.04.6 | https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal-security/main amd64 Packages
containerd | 1.3.3-0ubuntu2 | https://mirrors.tuna.tsinghua.edu.cn/ubuntu focal/main amd64 Packages
安装containerd:
root@k8s-master1:~# apt-get install containerd=1.6.12-0ubuntu1~20.04.1
查看containerd的service⽂件,可以用作二进制安装containerd的服务启动文件:
root@k8s-master1:~# cat /usr/lib/systemd/system/containerd.service
# Copyright The containerd Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
验证runc环境:
root@k8s-master1:~# whereis runc
runc: /usr/sbin/runc /usr/share/man/man8/runc.8.gz
root@k8s-master1:~# whereis containerd
containerd: /usr/bin/containerd /usr/share/man/man8/containerd.8.gz
root@k8s-master1:~# runc -v
runc version 1.1.4-0ubuntu1~20.04.1
spec: 1.0.2-dev
go: go1.18.1
libseccomp: 2.5.1
containerd配置⽂件:
root@k8s-master1:~# containerd --help
root@k8s-master1:~# containerd config --help
root@k8s-master1:~# containerd config default #查看默认配置输出
root@k8s-master1:~# mkdir /etc/containerd/
root@k8s-master1:~# containerd config default > /etc/containerd/config.toml #默认没有配置文件,输出后可以对默认配置进行优化修改
root@k8s-master1:~# cp /etc/containerd/config.toml{,.default}
root@k8s-master1:~# vi /etc/containerd/config.toml
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.7"
#第61行默认进行下载镜像的地址,需要根据实际修改为公网镜像或者公司内部harbor仓库地址
SystemdCgroup = true #第125行,如果宿主机系统是用systemd引导的,需要改为true
[plugins."io.containerd.grpc.v1.cri".registry.mirrors] #153行
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"] #154行
endpoint = ["https://pmdiavi9.mirror.aliyuncs.com"] #155行
#153行开始需要添加镜像加速的地址,此处是我自己的阿里云的镜像加速地址
root@k8s-master1:~# systemctl restart containerd
root@k8s-master1:~# systemctl status containerd
⼆进制安装containerd:
通过官⽅⼆进制安装containerd、 runc及CNI,kubernetes从 v1.24.0开始默认使⽤containerd作为容器运⾏时,因
此需要提前安装好containerd之后再安装v1.24或更⾼版本的kubernetes(如果要继续使⽤docker,则需要单独安装
docker及cri-dockerd,官网: https://github.com/Mirantis/cri-dockerd ),实际生产中多以二进制安装为主。
containerd官方下载链接:https://github.com/containerd/containerd/releases
下载containerd⼆进制文件并创建service文件:
root@k8s-master01:~# cd /usr/local/src/
#下载并解压
root@k8s-master01:/usr/local/src# wget https://github.com/containerd/containerd/releases/download/v1.6.20/containerd-1.6.20-linux-amd64.tar.gz
root@k8s-master01:/usr/local/src# tar -xvf containerd-1.6.20-linux-amd64.tar.gz
#复制二进制文件至/usr/local/bin/
root@k8s-master01:/usr/local/src# cp bin/* /usr/local/bin/
# 验证二进制文件可以执行
root@k8s-master01:/usr/local/src# containerd -v
containerd github.com/containerd/containerd v1.6.20 2806fc1057397dbaeefbea0e4e17bddfbd388f38
#创建service⽂件
root@k8s-master01:/usr/local/src# cat >/usr/lib/systemd/system/containerd.service <<'EOF'
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target
[Service]
ExecStartPre=-/usr/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
EOF
#开机自启动
root@k8s-master01:/usr/local/bin# systemctl daemon-reload
root@k8s-master01:/usr/local/bin# systemctl enable containerd;systemctl start containerd
编辑containerd配置⽂件并重启服务:
root@k8s-master01:~# mkdir /etc/containerd/
root@k8s-master01:~# containerd config default > /etc/containerd/config.toml
root@k8s-master01:~# vi /etc/containerd/config.toml
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.7"
#第61行默认进行下载镜像的地址,需要根据实际修改为公网镜像或者公司内部harbor仓库地址
SystemdCgroup = true #第125行,如果宿主机系统是用systemd引导的,需要改为true
[plugins."io.containerd.grpc.v1.cri".registry.mirrors] #153行
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"] #154行
endpoint = ["https://pmdiavi9.mirror.aliyuncs.com"] #155行
#153行开始需要添加镜像加速的地址,此处是我自己的阿里云的镜像加速地址
root@k8s-master01:~# systemctl restart containerd && systemctl enable containerd
部署runc:
官方下载地址:https://github.com/opencontainers/runc/
root@k8s-master01:~# cd /usr/local/src/
root@k8s-master01:/usr/local/src# wget https://github.com/opencontainers/runc/releases/download/v1.1.5/runc.amd64
root@k8s-master01:/usr/local/src# mv runc.amd64 /usr/local/bin/runc
root@k8s-master01:/usr/local/src# chmod +x /usr/local/bin/runc
root@k8s-master01:/usr/local/src# runc --version
下载测试镜像并启动容器验证:
root@k8s-master01:/usr/local/src# ctr images pull docker.io/library/alpine:latest
root@k8s-master01:/usr/local/src# ctr images ls
root@k8s-master01:/usr/local/src# ctr -n k8s.io images ls #-n指定namespaces
root@k8s-master01:/usr/local/src# ctr run -t --net-host docker.io/library/alpine:latest testcontainer sh
/ # ls
containerd前端管理工具
containerd的命令⾏客户端有ctr、 crictl、 nerdctl等, containerd相⽐docker多了⼀个命名空间的逻辑概念,用来隔离不同命名空间的容器,⾃身的命令⾏客户端ctr命令默认是在default命名空间⾥、 nerdctl也是在default,当使⽤crictl命令的时候,是在k8s.io这个命名空间,⽽k8s的创建的pod也是在k8s.io命名空间,因此在使⽤nerdctl管理kubernetes环境的pod的时候要指定命名空间为k8s.io,否则看不到kubernetes环境中的pod。
ctr命令
ctr命令是containerd自带的前端管理工具,非常不好用,不推荐使用
测试下载镜像:
root@k8s-master1:~# ctr images pull docker.io/library/alpine:latest #注意,ctr下载需要指定镜像完整的:域名+仓库+名称+tag,均不能省略
验证镜像:
root@k8s-master1:~# ctr images ls
创建测试容器:
root@k8s-master1:~# ctr run -t --net-host docker.io/library/alpine:latest testcontainer sh
/ #
查看容器
root@k8s-master1:~# ctr container ls
crictl工具
crictl是kubernetes-sigs(Special Interest Group,特别兴趣⼩组)中的⼀个项⽬,旨在提供⼀个调⽤kubelet管理本机pod的命令⾏客户端。
项目地址:https://github.com/kubernetes-sigs
crictl安装包下载地址:https://github.com/kubernetes-sigs/cri-tools/releases
crictl⽀持以下的容器运⾏时、⽬前已经没有默认调⽤的后端运⾏时路径(Default is now deprecated and the endpoint should be set instead),因此需要单独设置使⽤的容器运⾏时:
unix:///var/run/dockershim.sock
unix:///run/containerd/containerd.sock
unix:///run/crio/crio.sock
unix:///var/run/cri-dockerd.sock
下载安装crictl:
root@k8s-master01:/usr/local/src# wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.26.1/crictl-v1.26.1-linux-amd64.tar.gz
root@k8s-master01:/usr/local/src# tar -xvf crictl-v1.26.1-linux-amd64.tar.gz -C /usr/local/bin/
root@k8s-master01:/usr/local/src# crictl -v
配置crictl运⾏时环境:
root@k8s-master01:/usr/local/src# crictl --help|grep yaml #找到默认配置文件位置
root@k8s-master01:/usr/local/src# vim /etc/crictl.yaml #默认不存在此文件,添加以下几行配置
runtime-endpoint: "unix:///run/containerd/containerd.sock"
image-endpoint: "unix:///run/containerd/containerd.sock"
timeout: 10
debug: false
下载并验证镜像
root@k8s-master01:/usr/local/src# crictl image ls
IMAGE TAG IMAGE ID SIZE
root@k8s-master01:/usr/local/src# crictl pull nginx:1.20.2
root@k8s-master01:/usr/local/src# crictl image ls
IMAGE TAG IMAGE ID SIZE
docker.io/library/nginx 1.20.2 50fe74b50e0d0 56.7MB
root@k8s-master01:/usr/local/src# crictl ps
crictl无法直接运行容器,需要编写Json或yaml文件,具体参考:使用 crictl 对 Kubernetes 节点进行调试 | Kubernetes
nerdctl工具
nerdctl工具是由docker开源的,和docker命令使用基本一致,非常推荐使用
官方地址:https://github.com/containerd/nerdctl
安装nerdctl:
root@k8s-master01:/usr/local/bin# cd /usr/local/src/
root@k8s-master01:/usr/local/src# wget https://github.com/containerd/nerdctl/releases/download/v1.3.0/nerdctl-1.3.0-linuxamd64.tar.gz
root@k8s-master01:/usr/local/src# tar -xf nerdctl-1.3.0-linux-amd64.tar.gz -C /usr/local/bin/
root@k8s-master01:/usr/local/src# nerdctl version
root@k8s-master01:~# nerdctl ps
root@k8s-master01:~# nerdctl -n k8s.io ps
nerdctl配置⽂件:
root@k8s-master01:/usr/local/src# nerdctl --help|grep toml #查找默认配置文件路径
Config file ($NERDCTL_TOML): /etc/nerdctl/nerdctl.toml
root@k8s-master01:/usr/local/src# mkdir /etc/nerdctl/
root@k8s-master01:/usr/local/src# vi /etc/nerdctl/nerdctl.toml
namespace = "k8s.io"
debug = false
debug_full = false
insecure_registry = true
安装CNI插件:
CNI(container network interface),容器网络接口,用于在创建容器时为容器提供网络配置信息
注意,如果需要像docker一样创建带网络非host容器,必须安装CNI插件,k8s也需要安装此插件
CNI插件也是开源的,官网地址:https://github.com/containernetworking/plugins/
root@k8s-master01:~# nerdctl run -it -p 80:80 nginx #没有安装CNI插件之前会报错,此报错是containerd抛出来的
FATA[0026] failed to verify networking settings: failed to create default network: needs CNI plugin "bridge" to be installed in CNI_PATH ("/opt/cni/bin"), see https://github.com/containernetworking/plugins/releases: exec: "/opt/cni/bin/bridge": stat /opt/cni/bin/bridge: no such file or directory
下载并安装CNI插件:
root@k8s-master01:~# cd /usr/local/src/
root@k8s-master01:/usr/local/src# wget https://github.com/containernetworking/plugins/releases/download/v1.2.0/cni-pluginslinux-amd64-v1.2.0.tgz
root@k8s-master01:/usr/local/src# vi /etc/containerd/config.toml #cni默认的路径是有containerd配置的,需要在containerd的配置文件找到cni相关的路径,71-73行是cni相关配置
71 [plugins."io.containerd.grpc.v1.cri".cni]
72 bin_dir = "/opt/cni/bin"
73 conf_dir = "/etc/cni/net.d"
root@k8s-master01:/usr/local/src# mkdir -pv /opt/cni/bin
root@k8s-master01:/usr/local/src# tar -xvf cni-plugins-linux-amd64-v1.2.0.tgz -C /opt/cni/bin/
使用nerdctl创建容器
创建nginx容器:
root@k8s-master01:/usr/local/src# nerdctl run -it -p 80:80 nginx
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/04/18 14:55:31 [notice] 1#1: using the "epoll" event method
2023/04/18 14:55:31 [notice] 1#1: nginx/1.23.4
2023/04/18 14:55:31 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
2023/04/18 14:55:31 [notice] 1#1: OS: Linux 5.4.0-81-generic
2023/04/18 14:55:31 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1024:1024
2023/04/18 14:55:31 [notice] 1#1: start worker processes
2023/04/18 14:55:31 [notice] 1#1: start worker process 29
2023/04/18 14:55:31 [notice] 1#1: start worker process 30
root@k8s-master01:/usr/local/src# nerdctl ps -a #查看容器
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fc315f98c71a docker.io/library/nginx:latest "/docker-entrypoint.…" 9 minutes ago Up 0.0.0.0:80->80/tcp nginx-fc315
访问宿主机(192.168.100.101)的80端口即可访问此容器的nginx服务
创建tomcat容器:
root@k8s-master01:/usr/local/src# nerdctl run -d -p 8080:8080 --name=tomcat-web1 --restart=always tomcat:7.0.88-alpine #创建容器
root@k8s-master01:/usr/local/src# nerdctl ps #查看tomcat容器
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dff069fa02e0 docker.io/library/tomcat:7.0.88-alpine "catalina.sh run" 6 seconds ago Up 0.0.0.0:8080->8080/tcp tomcat-web1
fc315f98c71a docker.io/library/nginx:latest "/docker-entrypoint.…" 14 minutes ago Up 0.0.0.0:80->80/tcp nginx-fc315
root@k8s-master01:/usr/local/src# nerdctl logs -f dff069fa02e0 #查看容器日志
访问宿主机(192.168.100.101)的8080端口即可访问此容器的tomcat服务
!(tomcat-1.JPG)
root@k8s-master01:/usr/local/src# nerdctl exec -it dff069fa02e0 bash #进入tomcat容器写入测试页面
bash-4.4# cd /usr/local/tomcat/webapps/
bash-4.4# mkdir app1
bash-4.4# echo `date +%T` >app1/index.jsp
访问宿主机(192.168.100.101)的tomcat测试页面
创建mysql容器并指定端口:
root@k8s-master01:/usr/local/src# nerdctl pull mysql:5.7.35 #拉取mysql镜像
root@k8s-master01:/usr/local/src# nerdctl image ls
root@k8s-master01:/usr/local/src# mkdir -p /data/mysql
root@k8s-master01:/usr/local/src# nerdctl run -d -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 -v /data/mysql:/var/lib/mysql mysql:5.7.35 #启动mysql容器
root@k8s-master01:/usr/local/src# nerdctl ps #查看mysql容器
root@k8s-master01:/usr/local/src# ll /data/mysql/ #查看容器关联的宿主机目录
root@k8s-master01:/usr/local/src# nerdctl logs -f 7ec2a88318fc #查看日志
root@k8s-master01:/usr/local/src# apt-get install mariadb-client #安装mysql客户端
root@k8s-master01:/usr/local/src# mysql -uroot -p -h 192.168.100.101 #测试连接
nter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.35 MySQL Community Server (GPL)
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [(none)]>
更多推荐
所有评论(0)