有一些业务需求要求我们在k8s pod或者docker container中用程序挂载本机的硬盘执行mount操作,下面我们以k8s pod为例,讲解一下在实现过程中遇到的各种坑。

至于docker container的实现方法和k8s原理一致,查一下相关参数就好

containers:
        - name: xxx
          terminationMessagePath: "/tmp/termination-log"  # 重要
          securityContext:
            privileged: true  # 重要
          image: xxx-image
          imagePullPolicy: Always
          command: ["/bin/sh"]
          args:
            [
              "-c",
              "sleep inf",
            ]
          volumeMounts:
            - name: mount-volume
              mountPath: /disks_mount_point
              mountPropagation: Bidirectional  # 重要
      volumes:
        - name: mount-volume
          hostPath:
            path: /disks_mount_point
            type: Directory

1. 没有权限mount

首先我们在docker image中已经指定了user=root,但是我们在pod里面尝试进行mount操作的时候,可以看到disk read-only,所以我们需要设置pod的权限:

参考:https://kubernetes.io/docs/concepts/policy/pod-security-policy/

securityContext:
	privileged: true

2. 在k8s pod中挂载的硬盘,在宿主机上看不到

经过上面的设置,我们可以成功的在pod中挂载硬盘,但是在宿主机执行lsblk,可以看到硬盘并没有被挂载

如果我们的业务场景是,需要在其他pod中进行硬盘上的数据处理,那么就行不通。

volumeMounts:
    - name: mount-volume
    mountPath: /disks_mount_point
    mountPropagation: Bidirectional

所以我们需要设置mountPropagation,这个设置等同于docker中的rshared,这样设置后,在pod中mount,我们也可以在宿主机上看到mount的硬盘了

参考:https://kuboard.cn/learning/k8s-intermediate/persistent/volume-mount-point.html#%E6%8C%82%E8%BD%BD%E4%BC%A0%E6%92%AD


3. mount /dev 目录失败

如果尝试mount /dev目录到k8s pod的话,我们可能得到报错:

Error: failed to start container "xxx": Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "/var/lib/kubelet/pods/c337b284-1aea-472e-9b10-4afecd3e56e0/containers/xxx/e983d0ba" to rootfs at "/dev/termination-log" caused: mount through procfd: possibly malicious path detected -- refusing to operate on /var/lib/docker/overlay2/d72bbf76d11b215231e7c2776a18239797c5281b21223f5746397f5bc5ed7c7b/merged/dev/termination-log (deleted): unknown

可以看到与/dev/termination-log 目录有关,所以我们需要设置:

terminationMessagePath: "/tmp/termination-log"

参考:https://kubernetes.io/zh/docs/tasks/debug-application-cluster/determine-reason-pod-failure/

Logo

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐