前置条件

  • Windows 电脑注意

    关闭系统 Hyper-V 功能，否则可能导致后续 Kubernetes 启动失败：

    进入【控制面板——程序——启用或关闭 Windows 功能——取消勾选 "Hyper-V"、"Windows 沙盒"、"Windows 虚拟机监控程序平台"、"虚拟机平台" 】，点击确定后等待卸载完成，然后重启电脑。

  • 安装 VMWare

    不要使用 VirtualBox，因为 VirtualBox 要配置双网卡才能访问外网，而我自己在本地实测双网卡会导致 iptables 转发流量有问题

  • 安装虚拟机

    CentOS-7 镜像：http://mirrors.aliyun.com/centos/7/isos/x86_64/CentOS-7-x86_64-DVD-2009.iso

配置网络

首先需要在 VMWare 配置网络模式为桥接模式：

image-20240205160103695

接着，找到 VMWare 菜单栏的 编辑 - 虚拟网络编辑器 配置正确的桥接网卡：

image-20240205160117481

然后，在 CentOS 中配置下网卡：

这里我的网卡名字为 ens33，具体可以通过 ip addr 命令查看

vi /etc/sysconfig/network-scripts/ifcfg-ens33

修改为静态分配 IP，并配置 IP 地址、子网掩码、网关和 DNS：

注意：子网掩码和网关需要根据自己的实际网络环境配置，具体自行查看主机网络环境。

TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="ens33"
UUID="31185fa5-58a1-4ac5-9ccb-6029876d9347"
DEVICE="ens33"
ONBOOT="yes"
IPADDR=192.168.10.101
NETMASK=255.255.255.0
GATEWAY=192.168.10.1
DNS1=119.29.29.29

最后，重启网络服务：

systemctl restart network

关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

关闭 SeLinux

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

关闭 Swap

swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab

配置 yum 源

  • 使用阿里云镜像源

curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

  • 建立缓存

yum clean all
yum makecache

安装基础软件包

yum install ‐y yum-utils device-mapper-persistent-data lvm2 wget net-tools ntp unzip ifconfig telnet nfs-utils rpcbind git

同步系统时间

ntpdate time1.aliyun.com
systemctl enable ntpd

配置 Host

这里配置为自己实际的 IP

cat >> /etc/hosts << EOF
192.168.10.101 master
192.168.10.102 node1
192.168.10.103 node2
EOF

配置桥接流量

cat > /etc/modules-load.d/k8s.conf << EOF
br_netfilter
EOF

modprobe br_netfilter

cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --system

安装 Docker

  • 设置阿里云 Docker 的 yum 源

yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

  • 设置阿里云 K8s 的 yum 源

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

  • 安装 Docker 19.03.11

yum install -y docker-ce-19.03.11

  • 配置 Docker 镜像源

mkdir -p /etc/docker

cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "registry-mirrors": ["https://vjji27go.mirror.aliyuncs.com"],
  "insecure-registries": ["art.local:8081", "192.168.10.101:8081"]
}

EOF

  • 启动 Docker 并设置自启动

systemctl daemon-reload && systemctl enable docker --now

安装 K8s

  • 安装 1.19.3 版本

yum install -y kubelet-1.19.3 kubeadm-1.19.3 kubectl-1.19.3

  • 设置自启动 kubelet 并立即应用

systemctl enable kubelet --now

  • 添加环境变量和别名

echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
echo "alias kc=\"kubectl\"" >> ~/.bash_profile
echo "alias kcsys=\"kubectl -n kube-system\"" >> ~/.bash_profile
source ~/.bash_profile

后续可以通过 kc 简写命令执行 kubectl，kcsys 表示 kubectl -n kube-system，自己也可以定义其他别名。

创建虚拟机 Snapshot

执行完以上所有步骤后，对当前虚拟机状态生成 snapshot，后续 worker 节点可以直接从此 snapshot 克隆，无需执行之前的繁琐操作。

image-20240129161249754

初始化 Master 节点

设置主机名

hostnamectl set-hostname master

初始化主节点

注意只有主节点需要 init，worker node 只需要 join

将 192.168.10.101 修改为自己实际的 master 节点的 ip

kubeadm init --kubernetes-version=1.19.3 \
--apiserver-advertise-address=192.168.10.101 \
--image-repository registry.aliyuncs.com/google_containers \
--service-cidr=10.1.0.0/16 \
--pod-network-cidr=10.244.0.0/16

此时会等待镜像拉取，网络 5M 的情况下 1 分钟能完成。

注意：如果你用的是云环境，需要把 apiserver-advertise-address 改为云服务器的内网 IP 地址。

如果启动失败提示：kubelet 没有启动，则先启动 kubelet，接着执行 kubeadm reset，然后执行 kubeadm init。

  • 初始化成功后，记录下 worker join 命令，后续会用到

image-20240129164133439

  • 接着执行以下命令

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

安装网络组件

curl https://docs.projectcalico.org/v3.20/manifests/calico.yaml -O

kubectl apply -f calico.yaml

配置 Worker 节点

设置主机名

hostnamectl set-hostname node1

Join 主节点

  • 拷贝 master 节点的 admin.conf

在 master 节点上执行下面这行命令，然后输入 node 节点的密码

scp /etc/kubernetes/admin.conf root@node1:/etc/kubernetes/

  • join 主节点

如果之前未记录到 join 命令，则在 master 上执行这行命令：kubeadm token create --print-join-command

kubeadm join 192.168.10.101:6443 --token 0l852h.9pnn31g43sv0j3hu \
    --discovery-token-ca-cert-hash sha256:54d6ff6fb2c6784b423f50a458ac5827ec18bd4914802f4d69a6674873db0781

安装 Kubernetes Dashboard

部署

参考：https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml

cat > kubernetes-dashboard.yaml << EOF
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: Namespace
metadata:
  name: kubernetes-dashboard

---

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 31111
  selector:
    k8s-app: kubernetes-dashboard

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque

---

kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
    # Allow Dashboard to get metrics.
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  # Allow Metrics Scraper to get metrics from the Metrics server
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.3.1
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --token-ttl=86400
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
            # Uncomment the following line to manually specify Kubernetes API server Host
            # If not specified, Dashboard will attempt to auto discover the API server and connect
            # to it. Uncomment only if the default does not work.
            # - --apiserver-host=http://my-address:port
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              # Create on-disk volume to store exec logs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
      annotations:
        seccomp.security.alpha.kubernetes.io/pod: 'runtime/default'
    spec:
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.6
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
            - mountPath: /tmp
              name: tmp-volume
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}
EOF

kubectl apply -f kubernetes-dashboard.yaml

创建访问账号

cat > dashboard-account.yaml << EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: admin-user
    namespace: kubernetes-dashboard
EOF

kc apply -f dashboard-account.yaml

查看 Pod

[root@master ~]# kubectl get pod -n kubernetes-dashboard -o wide
NAME                                         READY   STATUS    RESTARTS   AGE   IP              NODE     NOMINATED NODE   READINESS GATES
dashboard-metrics-scraper-79c5968bdc-4ksm7   1/1     Running   0          28m   10.244.219.65   master   <none>           <none>
kubernetes-dashboard-658485d5c7-vzvgg        1/1     Running   0          28m   10.244.219.66   master   <none>           <none>

可以看到被分配到了哪个节点运行

令牌访问

获取访问令牌：

kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"

访问地址：https://192.168.10.101:31111（这里要修改为实际运行的节点的 ip）

如果使用谷歌浏览器，会出现如下提示，点击页面然后键盘敲击：thisisunsafe

image-20240129201314300

登录成功后的页面：

image-20240205160936668

创建第一个 Pod

VSCode 安装插件

在 VSCode 中安装 Microsoft 提供的官方 Kubernetes 插件

image-20240129204154345

编写 YAML

在 VSCode 中创建一个 my-nginx.yaml 文件，然后使用 Pod 模板：

image-20240129204621587

就能快速地帮我们生成一个模板文件：

完整配置文件如下：

apiVersion: v1
kind: Pod
metadata:
  name: my-nginx
  labels:
    name: my-nginx
spec:
  containers:
  - name: my-nginx
    image: nginx
    resources:
      limits:
        memory: "128Mi"
        cpu: "500m"
    ports:
      - containerPort: 80

运行 Pod

kubectl create -f my-nginx.yaml

查看 Pod

kubectl describe pod my-nginx

image-20240129205437912

常见问题

如何重置 K8s

  • 重置 kubeadm

kubeadm reset

  • 清除 kubeconfig 环境变量

rm -rf $HOME/.kube

  • 清理网络环境

systemctl stop kubelet && systemctl stop docker \
&& rm -rf /var/lib/cni/* && rm -rf /var/lib/kubelet/* && rm -rf /etc/cni/* \
&& ifconfig cni0 down \
&& ifconfig flannel.1 down \
&& ifconfig docker0 down \
&& ip link delete cni0 \
&& ip link delete flannel.1

  • 重启服务

systemctl start docker && systemctl start kubelet

create 和 apply 的区别

  • create 是在 Kubernetes 中创建新资源，如果 yaml 文件中的内容已经在 Kubernetes 中创建过，那么再次执行 create 会报错

  • apply 是对已有对象进行修改，如果对象不存在则创建。大部分情况下可以用 apply 替代 create，但当有些对象的字段是 immutable 不可变类型的时候，如果你修改了这些字段，则必须先删除该对象，然后再用 create 命令创建。

# kubernetes # k8s # docker # 容器 # 云原生 # 后端 # 运维
Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes