前言

记录一下本地搭建k8s过程

一、主机清单

主机名ip描述
master192.168.88.50k8s控制节点
harbor192.168.88.30镜像仓库
registry192.168.88.35私有镜像仓库
node1192,168.88.51节点1
node2192,168.88.52节点2
node3192,168.88.53节点3
node4192,168.88.54节点4
node5192,168.88.55节点5

二、配置本地yum仓库

1、上传软件到虚拟机

软件清单:
containerd.io-1.6.12-3.1.el8.x86_64.rpm
docker-ce-20.10.21-3.el8.x86_64.rpm
docker-ce-cli-20.10.21-3.el8.x86_64.rpm
docker-ce-rootless-extras-20.10.21-3.el8.x86_64.rpm
docker-compose-plugin-2.12.2-3.el8.x86_64.rpm
docker-distribution-2.6.2-2.git48294d9.el8.x86_64.rpm
docker-scan-plugin-0.21.0-3.el8.x86_64.rpm
上传方式:rsync或scp
上传主机:master
上传地址:/var/localrepo
创建yum软件清单:
下载createrepo
创建软件清单

2、配置nginx转发

master主机:下载nginx将/var/localrepo软连接到/usr/share/nginx/html
启动nginx并开机自起

3、配置其他机器yum的repo文件

# vim /etc/yum.repos.d/http.repo
[master]
name=master
baseurl=http://192.168.88.50/localrepo
gpgcheck=0
enabled=1

将http.repo发送给其他所有主机,可用方式:Ansible,scp,rsync

三、配置registry私有仓库(非必须)

yum install -y docker-distribution
systemctl enable docker-distribution --now
# 修改主机清单
# vim /etc/hosts
添加 
192.168.88.35 registry

#修改配置文件
# vim /etc/docker/daemon.json
{
    "registry-mirrors": ["http://registry:5000"],
    "insecure-registries":["registry:5000"]
}
# 重启服务
systemctl restart docker

# 打标签上传镜像
docker tag  nginx:latest registry:5000/img/myimg:web
docker push registry:5000/img/myimg:web
docker tag  php-fpm:latest registry:5000/img/myimg:php-fpm
docker push registry:5000/img/myimg:php-fpm
 docker tag  httpd:latest registry:5000/library/httpd:latest
 docker push registry:5000/library/httpd:latest

四、配置harbor

harbor主机

1、安装docker和https证书起服务

# 配置主机清单
# vim /etc/hosts
192.168.88.30    harbor

# 安装dockercompose组件
 dnf install -y docker-ce docker-compose-plugin
systemctl enable --now docker

# 导入harbor镜像
tar -zxf harbor-v2.7.0.tgz -C /usr/local/
cd /usr/local/harbor
docker load -i harbor.v2.7.0.tar.gz
# 创建https证书
mkdir tls
openssl genrsa -out tls/cert.key 2048
openssl req -new -x509 -days 3650 -key tls/cert.key -out tls/cert.crt 
# 会要求填写国家名,省名,城市名,公司名,组织名,用户名,邮件随便填填就行
# 修改配置文件
cp harbor.yml.tmpl harbor.yml 
# vim harbor.yml
05:    hostname: harbor
08:    # http:
10:      # port: 80
17:    certificate: /usr/local/harbor/tls/cert.crt
18:    private_key: /usr/local/harbor/tls/cert.key
34:    harbor_admin_password: admin123
# 检查预安装环境
/usr/local/harbor/prepare
# 创建并启动项目
docker compose -f docker-compose.yml up -d
# 添加开机自启动
chmod 0755 /etc/rc.d/rc.local
echo "/usr/bin/docker compose -p harbor start" >>/etc/rc.d/rc.local

# 查看项目
docker compose ls
# 查看容器状态
docker compose -p harbor ps
通过ELB发布harbor服务,通过浏览器配置管理

2、管理harbor并上传镜像

harbor主机:

# 添加主机配置
# vim /etc/hosts
# 添加
192.168.88.35 registry
# 添加私有仓库配置
# vim /etc/docker/daemon.json
{
    "registry-mirrors": ["https://harbor:443", "http://registry:5000"],
    "insecure-registries":["harbor:443", "registry:5000"]
}
systemctl restart docker

上传镜像需要设置标签和登录,未登录无法上传成功,且上传的项目需要用户有上传权限。

docker login harbor:443
docker tag rockylinux:8.5 harbor:443/myimg/rockylinux:8.5
docker push harbor:443/myimg/rockylinux:8.5

五、配置master的yum软件仓库存放k8s相关软件

通过ssh或者scp或者rsync上传以下软件到master主机上/var/localrepo/
软件清单:
cri-tools-1.25.0-0.x86_64.rpm
kubeadm-1.26.0-0.x86_64.rpm
kubectl-1.26.0-0.x86_64.rpm
kubelet-1.26.0-0.x86_64.rpm
kubernetes-cni-1.1.1-0.x86_64.rpm
libnetfilter_cthelper-1.0.0-15.el8.x86_64.rpm
libnetfilter_cttimeout-1.0.0-11.el8.x86_64.rpm
libnetfilter_queue-1.0.4-3.el8.x86_64.rpm

更新软件清单:
createrepo --update /var/localrepo/

六、配置系统环境和安装软件包

1、禁用firewall和swap

sed '/swap/d' -i /etc/fstab
swapoff -a
dnf remove -y firewalld-*

2、安装软件包

配置主机清单:

#vim /etc/hosts
192.168.1.30    harbor
192.168.1.50    master
192.168.1.51    node-0001
192.168.1.52    node-0002
192.168.1.53    node-0003
192.168.1.54    node-0004
192.168.1.55    node-0005

下载软件:

dnf install -y kubeadm kubelet kubectl containerd.io ipvsadm ipset iproute-tc
# 配置配置文件
containerd config default >/etc/containerd/config.toml
vim /etc/containerd/config.toml
61     sandbox_image = "harbor:443/k8s/pause:3.9"
125    SystemdCgroup = true
154 行新插入:
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
          endpoint = ["https://harbor:443"]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor:443"]
          endpoint = ["https://harbor:443"]
      	[plugins."io.containerd.grpc.v1.cri".registry.configs."harbor:443".tls]
          insecure_skip_verify = true
# 起服务并开机自起
systemctl enable --now kubelet containerd

3、配置内核参数

# 加载内核模块
cat >/etc/modules-load.d/containerd.conf<<EOF
overlay
br_netfilter
xt_conntrack
EOF
# 起服务
systemctl start systemd-modules-load.service
# 设置内核参数
cat >/etc/sysctl.d/99-kubernetes-cri.conf<<EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.netfilter.nf_conntrack_max = 1000000
EOF
# 加载参数文件
sysctl -p /etc/sysctl.d/99-kubernetes-cri.conf

4、导入k8s镜像包

上传资源包
安装docker-ce
创建/etc/docker目录
编写镜像配置文件:

# vim /etc/docker/daemon.json 
{
    "registry-mirrors":["https://harbor:443"],
    "insecure-registries":["harbor:443"]
}
# 起服务并开机自起
systemctl enable --now docker

登录harbor仓库,上传镜像:

docker login harbor:443
docker load -i init/v1.26.0.tar.xz
docker images|while read i t _;do

    [[ "${t}" == "TAG" ]] && continue

    [[ "${i}" =~ ^"harbor:443/".+ ]] && continue

    docker tag ${i}:${t} harbor:443/k8s/${i##*/}:${t}

    docker push harbor:443/k8s/${i##*/}:${t}

    docker rmi ${i}:${t} harbor:443/k8s/${i##*/}:${t}

done

5、设置Tab键

source <(kubeadm completion bash|tee /etc/bash_completion.d/kubeadm)
source <(kubectl completion bash|tee /etc/bash_completion.d/kubectl)

6、master安装

# 测试安装环境
kubeadm init --config=init/init.yaml --dry-run 2>error.log
cat error.log
rm -rf error.log /etc/kubernetes/tmp
# 主控节点初始化
kubeadm init --config=init/init.yaml |tee init/init.log
# 管理授权
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 验证安装结果
kubectl get nodes

7、安装网络插件

上传包和yaml文件:
calico.tar.xz
calico.yaml
上传镜像

docker load -i calico.tar.xz
docker images|while read i t _;do
    [[ "${t}" == "TAG" ]] && continue
    [[ "${i}" =~ ^"harbor:443/".+ ]] && continue
    docker tag ${i}:${t} harbor:443/plugins/${i##*/}:${t}
    docker push harbor:443/plugins/${i##*/}:${t}
    docker rmi ${i}:${t} harbor:443/plugins/${i##*/}:${t}
done

修改yaml文件

sed -ri 's,^(\s*image: )(.*/)?(.+),\1harbor:443/plugins/\3,' calico.yaml
# 根据yaml文件生成容器
kubectl apply -f calico.yaml
# 验证
kubectl get nodes

8、安装计算节点

# 查看token
kubeadm token list
# 删除token
kubeadm token delete token码
# 创建token码
kubeadm token create --ttl=0 --print-join-command
# 输出
kubeadm join 192.168.88.50:6443 --token 225c9x.vbyxxtssecxp5rdj --discovery-token-ca-cert-hash sha256:7fd324d99d5c4c165c6095cc61d9f223c498d4869c9784446cefeaf134cf3a8b 
# 获取token_hash
# 1、查看安装日志  2、在创建token时候显示  3、使用 openssl 计算得到

node按住那该同控制节点导入镜像前一致,
使用
kubeadm join 192.168.88.50:6443 --token 225c9x.vbyxxtssecxp5rdj --discovery-token-ca-cert-hash sha256:7fd324d99d5c4c165c6095cc61d9f223c498d4869c9784446cefeaf134cf3a8b
用kubectl get nodes验证

也可以使用ansible批量部署。

Logo

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐