1.安装Kubernetes krew

注意：建议先安装代理，方便下载外网数据

参考链接：https://docs.min.io/minio/k8s/deployment/deploy-minio-operator.html

1.1 确认 git 已经安装

1.2 安装krew

mkdir minio
cd minio
OS="$(uname | tr '[:upper:]' '[:lower:]')"
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')"
KREW="krew-${OS}_${ARCH}"
curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz"
tar zxvf "${KREW}.tar.gz" &&
./"${KREW}" install krew

1.3 添加环境变量

echo export PATH="\${PATH}:\${HOME}/.krew/bin" >> ~/.bashrc
source ~/.bashrc

1.4 测试安装

kubectl krew

2.确认kube-controller-manager存在TLS证书

kubectl get pod kube-controller-manager-unode1 \
  -n kube-system -o yaml

spec:
  containers:
  - command:
    - kube-controller-manager
    - --allocate-node-cidrs=true
    - --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
    - --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
    - --bind-address=127.0.0.1
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --cluster-cidr=192.168.0.0/16
    - --cluster-name=kubernetes
    - --cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt
    - --cluster-signing-key-file=/etc/kubernetes/pki/ca.key

3.安装 MinIO Kubernetes Operator

kubectl krew update
kubectl krew install minio

验证安装

root@unode1:~/minio# kubectl minio version
v4.4.21

4.初始化MinIO Kubernetes Operator

关闭代理，再初始化，如果是export临时变量，重新开启窗口

kubectl minio init

验证Operator安装

root@unode1:~# kubectl get all --namespace minio-operator
NAME                                  READY   STATUS    RESTARTS   AGE
pod/console-6d894f47b8-kmr7f          1/1     Running   0          3m34s
pod/minio-operator-6c648d8c67-4sz5l   1/1     Running   0          3m34s
pod/minio-operator-6c648d8c67-ks6jv   1/1     Running   0          3m34s

NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/console    ClusterIP   10.108.129.225   <none>        9090/TCP,9443/TCP   3m34s
service/operator   ClusterIP   10.108.89.200    <none>        4222/TCP,4221/TCP   3m34s

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/console          1/1     1            1           3m34s
deployment.apps/minio-operator   2/2     2            2           3m34s

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/console-6d894f47b8          1         1         1       3m34s
replicaset.apps/minio-operator-6c648d8c67   2         2         2       3m34s

5.打开Operator控制台

root@unode1:~# kubectl minio proxy
Starting port forward of the Console UI.

To connect open a browser and go to http://localhost:9090

Current JWT to login: eyJhbGciOiJSUzI1NiIsImtpZCI6ImxsZDJ6Z1NldGUwV0dIVXN2NjBlOTEzRmlEMTgtdHBwNV9pSVlCckM1TWsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtaW5pby1vcGVyYXRvciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjb25zb2xlLXNhLXRva2VuLTRsOHhnIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImNvbnNvbGUtc2EiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI4Njk1YWY2YS0xNWNiLTQ3MjQtYTI5ZS03NDc5NTRmNGUxOGMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6bWluaW8tb3BlcmF0b3I6Y29uc29sZS1zYSJ9.ObK4a6kDz11fk1nd2f6KWfyz3Fu1lWQe8YlWi5jEAD4M3-GtLbJeDUlmrNk3YeGdjJqOvbnjGdy0dNw8-F1TXQsB7GW65HraOmuZz-dNSUUvJVTGgczSef0v01FuUp9KqvEuBF2QD54Sm3EOirTkMK5xDjZGrsuXdBlqdID1oPrD_o0Ud5tC3zFQwW9OBPnKTGdZy6qbVS3xF9AAA8bQcGnfMdATzJA1ERsSDdXdZHOXwPVyQm5gHpwxsKhTilN06KpY4UI6t7Vkrr4gkPh_TpFuktIF2yWMEp5FIS7YzYuAoZqHovjiy81PG9rVy3VnqFYHOSRVXcVGIMkKczlsrQ

6.登录创建租户

注意点：

Setup、Audit log、Monitoring都有Storage Class设置，选择nfs(预先安装好nfs), Number of Servers和Driver per Server设置等于服务器的数量，由于k8s集群worker节点数量为2, 所以这个地方设置为2.

7.创建用户相关信息

7.1点击进入租户

7.2 点击console

7.3用户Policy编写

在用户管理配置相应的策略即可，保证数据的隔离安全

将用户指定wangzy-p bucket

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
				"arn:aws:s3:::wangzy-p/*"
            ]
        }
    ]
}

7.4 Pod读取minio数据

7.4.1 创建ServiceAccount

7.4.2 保存ServiceAccount的access key相关信息

{"url":"http://minio.storage.svc.cluster.local","accessKey":"9GhmJRsHrY4Dsui8","secretKey":"0N1bLrFstBz1iRirYfSH7Lvs3eENqlj7","api":"s3v4","path":"auto"}

7.4.3在pod中安装minio的sdk包

参考链接: https://docs.min.io/docs/python-client-quickstart-guide.html

安装pip3 install minio

>>> from minio import Minio
>>> client = Minio("minio.storage.svc.cluster.local", "9GhmJRsHrY4Dsui8", "0N1bLrFstBz1iRirYfSH7Lvs3eENqlj7", secure=False)
>>> found = client.bucket_exists("wangzy-p")
# 通过客户端去访问
>>> response = client.get_object("wangzy-p", "test.py")
>>> response.data.decode("utf-8")

# 通过分享生成url链接，通过url去访问

>>> from urllib import request
>>> URL="http://minio.storage.svc.cluster.local/wangzy-p/test.py?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=MY7GD4K8LBPJH6ZAV2MB%2F20220612%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220612T103356Z&X-Amz-Expires=604800&X-Amz-Security-Token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJNWTdHRDRLOExCUEpINlpBVjJNQiIsImV4cCI6MTY1NTAzMjA0MiwicGFyZW50Ijoid2FuZ3p5LXAifQ.vPPNba5v6m-RY_GY9-P7ZtTDLAxIx4YUKX2jVF2UzFcdkF7uUx_VP4D-cW_qQVlf5Fzj_MTnJ8-AqFlcmfQX3w&X-Amz-SignedHeaders=host&versionId=null&X-Amz-Signature=cc346b804e5f46a465518a4b1117f17777b054ed3a244bd5d485bbeaa35af6b0"
>>> req=request.Request(URL)
>>> resp=request.urlopen(req)
>>> print(resp.read().decode('utf-8'))