部署heketi kubernetes glusterfs

环境说明：

• glusterfs node1 192.168.0.211
• glusterfs node2 192.168.0.212
• hekeit 192.168.0.210
• kubernetes 192.168.0.210

安装glusterfs略

下载heketi：

wget https://github.com/heketi/heketi/releases/download/v10.2.0/heketi-v10.2.0.linux.amd64.tar.gz

wget https://github.com/heketi/heketi/releases/download/v10.2.0/heketi-client-v10.2.0.linux.amd64.tar.gz

解压heketi-v10.2.0.linux.amd64.tar.gz后 编辑heketi.json文件

文件内容如下：

heketi.json

{

"_port_comment": "Heketi Server Port Number",

"port": "8080",

"_use_auth": "Enable JWT authorization. Please enable for deployment",

"use_auth": false,

"_jwt": "Private keys for access",

"jwt": {

"_admin": "Admin has access to all APIs",

"admin": {

"key": "hsop"

},

"_user": "User only has access to /volumes endpoint",

"user": {

"key": "hsop"

}

},

"_glusterfs_comment": "GlusterFS Configuration",

"glusterfs": {

"_executor_comment": [

"Execute plugin. Possible choices: mock, ssh",

"mock: This setting is used for testing and development.",

" It will not send commands to any node.",

"ssh: This setting will notify Heketi to ssh to the nodes.",

" It will need the values in sshexec to be configured.",

"kubernetes: Communicate with GlusterFS containers over",

" Kubernetes exec api."

],

"executor": "ssh",

"_sshexec_comment": "SSH username and private key file information",

"sshexec": {

"keyfile": "/root/.ssh/id_rsa",

"user": "root"

},

"_kubeexec_comment": "Kubernetes configuration",

"kubeexec": {

"host" :"http://192.168.0.210:8100",

"insecure": true,

"user": "kubernetes username",

"password": "password for kubernetes user",

"namespace": "OpenShift project or Kubernetes namespace",

"fstab": "Optional: Specify fstab file on node. Default is /etc/fstab"

},

"_db_comment": "Database file name",

"db": "/var/lib/heketi/heketi.db",

"brick_max_size_gb" : 1024,

"brick_min_size_gb" : 1,

"max_bricks_per_volume" : 33,

"_loglevel_comment": [

"Set log level. Choices are:",

" none, critical, error, warning, info, debug",

"Default is warning"

],

"loglevel" : "debug"

}

}

然后运行：

启动 heketi server

nohup ./heketi --config=/opt/heketi/heketi.json &

log都正常的话表示启动成功

创建topology.json文件

topology.json

{

"clusters": [

{

"nodes": [

{

"node": {

"hostnames": {

"manage": [

"192.168.0.211"

],

"storage": [

"192.168.0.211"

]

},

"zone": 1

},

"devices": [

"/dev/sdb"

]

},

{

"node": {

"hostnames": {

"manage": [

"192.168.0.212"

],

"storage": [

"192.168.0.212"

]

},

"zone": 1

},

"devices": [

"/dev/sdb"

]

}

]

}

]

}

然后运行：

./heketi-cli -s $HEKETI_CLI_SERVER（heketi的server接口） --user admin --secret 'hsop' topology load --json=/opt/heketi/topology.json

heketi相关维护接口：

./heketi-cli -s http://192.168.0.210:8080 --user admin --secret 'hsop' cluster list 查看集群

./heketi-cli -s http://192.168.0.210:8080 --user admin --secret 'hsop' cluster info 693b8d4b113033abacc630922948fa86

./heketi-cli -s $HEKETI_CLI_SERVER --user admin --secret 'hsop' volume list

./heketi-cli -s http://192.168.0.210:8080 --user admin --secret 'hsop' node info c057c55c42f408b749db81bb563250a7 查看节点

最后在K8S上创建storageclass

glusterfs-sc.yaml

apiVersion: storage.k8s.io/v1

kind: StorageClass

metadata:

name: gluster-dynmic

provisioner: kubernetes.io/glusterfs

reclaimPolicy: Retain

parameters:

resturl: "http://192.168.0.210:8080"

clusterid: "693b8d4b113033abacc630922948fa86"

restauthenabled: "true"

restuser: "admin"

restuserkey: "hsop"

gidMin: "40000"

gidMax: "50000"

volumetype: "replicate:2"

allowVolumeExpansion: true