k8s跟mysql集群的crd_k8s部署mysql集群master--slave

一、环境信息CentOS Linux release 7.8.2003 (Core)4.4.240-1.el7.elrepo.x86_64k8s 集群版本：1.15docker 版本：Docker version 17.03.3-ce集群信息：NAMESTATUSROLESAGEVERSIONkm-1.11Readymaster7d7h...

自娱自乐的傲娇ho

414人浏览 · 2021-01-27 03:39:05

自娱自乐的傲娇ho · 2021-01-27 03:39:05 发布

一、环境信息

CentOS Linux release 7.8.2003 (Core)

4.4.240-1.el7.elrepo.x86_64

k8s 集群版本：1.15

docker 版本：Docker version 17.03.3-ce

集群信息：

NAME STATUS ROLES AGE VERSION

km-1.11 Ready master 7d7h v1.15.1

kn-1.11 Ready 7d7h v1.15.1

kn1-1.11 Ready 31h v1.15.1

二、MySQL的高可用方案

MySQL高可用方案采用主从复制+读写分离，即由单一的master和多个slave所构成。其中，客户端通过master对数据库进行写操作，通过slave端进行读操作。master出现问题后，可以将应用切换到slave端。此方案是MySQL官方提供的一种高可用解决方案，节点间的数据同步采用MySQL Replication技术。MySQL Replication从一个MySQL数据库服务器(master)的数据复制到一个或多个MySQL数据库服务器(slave)。在默认情况下，复制是异步的；slave不需要一直接收来自主机的更新。根据配置，可以复制数据库中的所有数据库、选定的数据库，或者特定的表。

三、架构说明

通过mysql+xtrabackup的模式来组成数据库master+slave的模式，具体的架构图如下所示：

四、创建StorageClass

因为需要把数据库数据持久化存储，因此需要为pod自动创建pv，pvc

参考：https://blog.51cto.com/lingxudong/2546982

[[email protected] storageclass]# cat nfs-client.yaml

kind: Deployment

apiVersion: apps/v1

metadata:

namespace: blog

spec:

replicas: 1

strategy:

type: Recreate

selector:

matchLabels:

app: nfs-client-provisioner

template:

metadata:

namespace: blog

labels:

app: nfs-client-provisioner

spec:

serviceAccountName: nfs-client-provisioner

containers:

- name: nfs-client-provisioner

image: quay.io/external_storage/nfs-client-provisioner:latest

volumeMounts:

- name: nfs-client-root

mountPath: /persistentvolumes

env:

- name: PROVISIONER_NAME

value: fuseim.pri/ifs

- name: NFS_SERVER

value: 192.168.140.50

- name: NFS_PATH

value: /data/k8s

volumes:

- name: nfs-client-root

nfs:

server: 192.168.140.50

path: /data/k8s

[[email protected] storageclass]# cat nfs-client-sa.yaml

apiVersion: v1

kind: ServiceAccount

metadata:

namespace: blog

---

kind: ClusterRole

apiVersion: rbac.authorization.k8s.io/v1

metadata:

namespace: blog

rules:

- apiGroups: [""]

resources: ["persistentvolumes"]

verbs: ["get", "list", "watch", "create", "delete"]

- apiGroups: [""]

resources: ["persistentvolumeclaims"]

verbs: ["get", "list", "watch", "update"]

- apiGroups: ["storage.k8s.io"]

resources: ["storageclasses"]

verbs: ["get", "list", "watch"]

- apiGroups: [""]

resources: ["events"]

verbs: ["list", "watch", "create", "update", "patch"]

- apiGroups: [""]

resources: ["endpoints"]

verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]

---

kind: ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1

metadata:

namespace: blog

subjects:

- kind: ServiceAccount

namespace: blog

roleRef:

kind: ClusterRole

apiGroup: rbac.authorization.k8s.io

[[email protected] storageclass]# cat nfs-client-class.yaml

apiVersion: storage.k8s.io/v1

kind: StorageClass

metadata:

namespace: blog

provisioner: fuseim.pri/ifs

五、staefulset搭建Mysql集群

1、创建yaml

Yaml配置如下：

[[email protected] mysql]# cat configmap.yaml

apiVersion: v1

kind: ConfigMap

metadata:

namespace: blog

labels:

app: mysql

data:

master.cnf: |

# Apply this config only on the master.

[mysqld]

log-bin

slave.cnf: |

# Apply this config only on slaves.

[mysqld]

super-read-only

部署headless服务，有状态服务都需要，让服务旗下的Pod彼此发现

[[email protected] mysql]# cat service.yaml

# Headless service for stable DNS entries of StatefulSet members.

apiVersion: v1

kind: Service

metadata:

namespace: blog

labels:

app: mysql

spec:

ports:

- name: mysql

port: 3306

clusterIP: None

selector:

app: mysql

---

# Client service for connecting to any MySQL instance for reads.

# For writes, you must instead connect to the master: mysql-0.mysql.

#apiVersion: v1

#kind: Service

#metadata:

# namespace: blog

# name: mysql-read

# labels:

# app: mysql

#spec:

# ports:

#- name: mysql

# port: 3306

#selector:

# app: mysql

[[email protected] mysql]# cat statefulset.yaml

apiVersion: apps/v1

kind: StatefulSet

metadata:

namespace: blog

spec:

selector:

matchLabels:

app: mysql

serviceName: mysql

replicas: 3

template:

metadata:

namespace: blog

labels:

app: mysql

spec:

initContainers:

- name: init-mysql

image: mysql:5.7

command:

- bash

- "-c"

- |

set -ex

# Generate mysql server-id from pod ordinal index.

[[ `hostname` =~ -([0-9]+)$ ]] || exit 1

ordinal=${BASH_REMATCH[1]}

echo [mysqld] > /mnt/conf.d/server-id.cnf

# Add an offset to avoid reserved server-id=0 value.

echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf

# Copy appropriate conf.d files from config-map to emptyDir.

if [[ $ordinal -eq 0 ]]; then

cp /mnt/config-map/master.cnf /mnt/conf.d/

else

cp /mnt/config-map/slave.cnf /mnt/conf.d/

volumeMounts:

- name: conf

mountPath: /mnt/conf.d

- name: config-map

mountPath: /mnt/config-map

- name: clone-mysql

image: hub.c.163.com/combk8s/xtrabackup:1.0

command:

- bash

- "-c"

- |

set -ex

# Skip the clone if data already exists.

[[ -d /var/lib/mysql/mysql ]] && exit 0

# Skip the clone on master (ordinal index 0).

[[ `hostname` =~ -([0-9]+)$ ]] || exit 1

ordinal=${BASH_REMATCH[1]}

[[ $ordinal -eq 0 ]] && exit 0

# Clone data from previous peer.

ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql

# Prepare the backup.

xtrabackup --prepare --target-dir=/var/lib/mysql

volumeMounts:

- name: data

mountPath: /var/lib/mysql

subPath: mysql

- name: conf

mountPath: /etc/mysql/conf.d

containers:

- name: mysql

image: mysql:5.7

env:

- name: MYSQL_ALLOW_EMPTY_PASSWORD

value: "1"

ports:

- name: mysql

containerPort: 3306

volumeMounts:

- name: data

mountPath: /var/lib/mysql

subPath: mysql

- name: conf

mountPath: /etc/mysql/conf.d

resources:

requests:

# cpu: 200m

memory: 1Gi

livenessProbe:

exec:

command: ["mysqladmin", "ping"]

initialDelaySeconds: 30

periodSeconds: 10

timeoutSeconds: 5

readinessProbe:

exec:

# Check we can execute queries over TCP (skip-networking is off).

command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]

initialDelaySeconds: 5

periodSeconds: 2

timeoutSeconds: 1

- name: xtrabackup

image: hub.c.163.com/combk8s/xtrabackup:1.0

ports:

- name: xtrabackup

containerPort: 3307

command:

- bash

- "-c"

- |

set -ex

cd /var/lib/mysql

# Determine binlog position of cloned data, if any.

if [[ -f xtrabackup_slave_info ]]; then

# XtraBackup already generated a partial "CHANGE MASTER TO" query

# because we're cloning from an existing slave.

mv xtrabackup_slave_info change_master_to.sql.in

# Ignore xtrabackup_binlog_info in this case (it's useless).

rm -f xtrabackup_binlog_info

elif [[ -f xtrabackup_binlog_info ]]; then

# We're cloning directly from master. Parse binlog position.

[[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1

rm xtrabackup_binlog_info

echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\

MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in

# Check if we need to complete a clone by starting replication.

if [[ -f change_master_to.sql.in ]]; then

echo "Waiting for mysqld to be ready (accepting connections)"

until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done

echo "Initializing replication from clone position"

# In case of container restart, attempt this at-most-once.

mv change_master_to.sql.in change_master_to.sql.orig

mysql -h 127.0.0.1 <

MASTER_HOST='mysql-0.mysql',

MASTER_USER='root',

MASTER_PASSWORD='',

MASTER_CONNECT_RETRY=10;

START SLAVE;

EOF

# Start a server to send backups when requested by peers.

exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \

"xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"

volumeMounts:

- name: data

mountPath: /var/lib/mysql

subPath: mysql

- name: conf

mountPath: /etc/mysql/conf.d

resources:

#requests:

# cpu: 100m

# memory: 100Mi

volumes:

- name: conf

emptyDir: {}

- name: config-map

configMap:

volumeClaimTemplates:

- metadata:

namespace: blog

annotations:

volume.beta.kubernetes.io/storage-class: course-nfs-storage

spec:

accessModes: ["ReadWriteMany"]

resources:

requests:

storage: 1.5Gi

详细解释看：https://blog.csdn.net/qq_38900565/article/details/102482782?utm_medium=distribute.pc_aggpage_search_result.none-task-blog-2~all~first_rank_v2~rank_v28-12-102482782.nonecase&utm_term=k8s%E9%83%A8%E7%BD%B2mysql&spm=1000.2123.3001.4430

创建成功会显示：

[[email protected] mysql]# kubectl get statefulset/mysql -n blog -o wide

NAME READY AGE CONTAINERS IMAGES

mysql 2/2 29h mysql,xtrabackup mysql:5.7,hub.c.163.com/combk8s/xtrabackup:1.0

[[email protected] mysql]# kubectl get pods -lapp=mysql --watch --namespace=blog

NAME READY STATUS RESTARTS AGE

mysql-0 2/2 Running 9 29h

mysql-1 2/2 Running 7 29h

如果遇到失败，查看日志执行：

kubectl logs -f pod/mysql-0 --all-containers=true -n blog

2、MySQL部署环境验证

通过运行一个临时的容器(使用mysql:5.7镜像)，使用MySQL 客户端发送测试请求给MySQL master节点(主机名为mysql-0.mysql；跨命名空间的话，主机名请使用mysql-0.mysql.blog)