【实战】使用Helm在K8S集群安装MySQL主从-编程知识

文章目录

- 前言
- 技术积累
- - 什么是Helm
  - StorageClass
  - 使用的工具版本
- helm 安装 MySQL 1主2从
- - 1. 添加 bitnami 的仓库
  - 2. 查询 MySQL 资源
  - 3. 拉取 MySQL chart 到本地
  - 4. 对chart 本地 values-test.yaml 修改
  - 5. 对本地 templates 模板修改
  - 6. 安装 MySQL 集群
  - 7. 查看部署的 MySQL 集群
  - 8. 连接 MySQL 集群验证服务
- ingress-nginx开放主从端口

前言

K8S对于云原生部署有着至关重要的作用，几乎所有的应用和中间件都可以部署在K8S，让其帮助我们进行管理。一般情况下我们推荐数据库不使用容器化部署，但是在某些场合为了方便管理也可以采用该种方式。今天我们就用在K8S集群部署MySQL，仅当作学习学习，在生产环境谨慎使用。

技术积累

什么是Helm

Helm是K8S中包管理工具，包管理器类似于我们在 Ubuntu 中使用的apt、Centos中使用的yum一样，能快速查找、下载和安装软件包，能够将一组K8S资源打包统一管理, 是查找、共享和使用为Kubernetes构建的软件的最佳方式。具体介绍和安装使用参见之前博文： Kubernetes包管理工具Helm简介及使用

StorageClass

由于数据库需要使用存储资源，K8S集群中我们可手动创建pv、pvc,StorageClass工具则可以自动车床件pvc。具体简介和使用参见之前博文：【实战】Kubernetes安装持久化工具NFS-StorageClass

使用的工具版本

软件	版本
chart	9.12.0
mysql	8.0.34
kubernetes	version v1.27.2
helm	version v3.12.1

helm 安装 MySQL 1主2从

1. 添加 bitnami 的仓库

$ helm repo add bitnami https://charts.bitnami.com/bitnami

2. 查询 MySQL 资源

$ helm repo update

[root@master k8s]# helm search repo mysql
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/mysql 9.12.1 8.0.34 MySQL is a fast, reliable, scalable, and easy t…
bitnami/phpmyadmin 12.1.0 5.2.1 phpMyAdmin is a free software tool written in P…
bitnami/mariadb 13.1.2 11.0.3 MariaDB is an open source, community-developed …
bitnami/mariadb-galera 9.1.1 11.0.3 MariaDB Galera is a multi-primary database clus…

3. 拉取 MySQL chart 到本地

$ mkdir -p /k8s/mysql && cd /k8s/mysql
#拉取 chart 到本地 /root/mysql 目录
$ helm pull bitnami/mysql --version 9.12.1
$ tar -xvf mysql-9.12.1.tgz
$ cp mysql/values.yaml ./values-test.yaml

#查看当前目录层级
[root@master mysql]# tree -L 2
.
├── mysql
│ ├── Chart.lock
│ ├── charts
│ ├── Chart.yaml
│ ├── README.md
│ ├── templates
│ ├── values.schema.json
│ └── values.yaml
├── mysql-9.12.1.tgz
└── values-test.yaml

3 directories, 7 files

4. 对chart 本地 values-test.yaml 修改

查看集群 storageclasses
[root@master mysql]# kubectl get storageclass

NAME                    PROVISIONER       RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-storage (default)   nfs-provisioner   Delete          Immediate           false                  10d

修改配置
$ vim values-test.yaml

#镜像修改
image:registry: registry.cn-hangzhou.aliyuncs.comrepository: senfel/mysqltag: 8.0.34-debian-11-r31
#架构改为集群复制
architecture: replication
#root密码 主从复制密码
auth:rootPassword: "123456root"#主节点
primary:configuration: |-[mysqld]default_authentication_plugin=caching_sha2_password   ## 加密方式persistence: ## 储存方式storageClass: "nfs-storage"
#从节点副本数
secondary:replicaCount: 2
#从节点
secondary:configuration: |-[mysqld]default_authentication_plugin=caching_sha2_password  ## 加密方式read_only=1 ##限定普通用户只读super_read_only=on ##限定root只读persistence: ##储存方式storageClass: "nfs-storage"

5. 对本地 templates 模板修改

mysql5会有以下问题，mysql8无需处理
[root@master mysql]# kubectl api-versions

# k8s v1.27.2 版本中，statefulsets 版本为：apps/v1  而不是模板中 apps/v1beta1 的版本
# 因此需要替换 master-statefulset.yaml 和 slave-statefulset.yaml 模板
$ sed -i "s#apps/v1beta1#apps/v1#" mysql/templates/master-statefulset.yaml
$ sed -i "s#apps/v1beta1#apps/v1#" mysql/templates/slave-statefulset.yaml未替换模板中 statefulsets 版本，直接安装服务会报错
$ helm install mysql-cluster mysql -f values-test.yaml 
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "StatefulSet" in version "apps/v1beta1"

6. 安装 MySQL 集群

#创建 mysql-cluster 名称空间
$ kubectl create ns mysql-cluster

#安装 MySQL 集群
$ helm install mysql-cluster -n mysql-cluster mysql -f values-test.yaml

##helm -n NAMESAPCE install SERVER_NAME FILE_NAME -f CONFIG_FILE
-n 指定 kubernetes 集群名称空间
-f 指定使用的配置文件，文件中定义的配置可以覆盖 mysql/values.yaml 文件中配置

[root@master mysql]# helm install mysql-cluster -n mysql-cluster mysql -f values-test.yaml

NAME: mysql-cluster
LAST DEPLOYED: Mon Sep  4 15:09:28 2023
NAMESPACE: mysql-cluster
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mysql
CHART VERSION: 9.12.1
APP VERSION: 8.0.34** Please be patient while the chart is being deployed **Tip:Watch the deployment status using the command: kubectl get pods -w --namespace mysql-clusterServices:echo Primary: mysql-cluster-primary.mysql-cluster.svc.cluster.local:3306echo Secondary: mysql-cluster-secondary.mysql-cluster.svc.cluster.local:3306Execute the following to get the administrator credentials:echo Username: rootMYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace mysql-cluster mysql-cluster -o jsonpath="{.data.mysql-root-password}" | base64 -d)To connect to your database:1. Run a pod that you can use as a client:kubectl run mysql-cluster-client --rm --tty -i --restart='Never' --image  registry.cn-hangzhou.aliyuncs.com/senfel/mysql:8.0.34-debian-11-r31 --namespace mysql-cluster --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command -- bash2. To connect to primary service (read/write):mysql -h mysql-cluster-primary.mysql-cluster.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"3. To connect to secondary service (read-only):mysql -h mysql-cluster-secondary.mysql-cluster.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"

7. 查看部署的 MySQL 集群

[root@master mysql]# kubectl get svc,pods -n mysql-cluster

NAME                                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/mysql-cluster-primary              ClusterIP   10.102.253.38   <none>        3306/TCP   2m2s
service/mysql-cluster-primary-headless     ClusterIP   None            <none>        3306/TCP   2m2s
service/mysql-cluster-secondary            ClusterIP   10.102.8.228    <none>        3306/TCP   2m2s
service/mysql-cluster-secondary-headless   ClusterIP   None            <none>        3306/TCP   2m2sNAME                            READY   STATUS    RESTARTS   AGE
pod/mysql-cluster-primary-0     1/1     Running   0          2m2s
pod/mysql-cluster-secondary-0   1/1     Running   0          2m2s
pod/mysql-cluster-secondary-1   1/1     Running   0          80s

#查看 pvc
[root@master mysql]# kubectl get pvc -n mysql-cluster

NAME                             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-mysql-cluster-primary-0     Bound    pvc-1c32a6b6-4b22-4ce0-b45b-82c38fd0d107   8Gi        RWO            nfs-storage    4m12s
data-mysql-cluster-secondary-0   Bound    pvc-5d263bc4-c9d4-4f3d-951c-fd578cad6e32   8Gi        RWO            nfs-storage    4m12s
data-mysql-cluster-secondary-1   Bound    pvc-90072614-6cbb-48f2-8cdf-a8816557e46b   8Gi        RWO            nfs-storage    3m30s

#查看 pv
[root@master mysql]# kubectl get pv | grep mysql

pvc-1c32a6b6-4b22-4ce0-b45b-82c38fd0d107   8Gi        RWO            Delete           Bound         mysql-cluster/data-mysql-cluster-primary-0        nfs-storage             4m45s
pvc-5d263bc4-c9d4-4f3d-951c-fd578cad6e32   8Gi        RWO            Delete           Bound         mysql-cluster/data-mysql-cluster-secondary-0      nfs-storage             4m45s
pvc-90072614-6cbb-48f2-8cdf-a8816557e46b   8Gi        RWO            Delete           Bound         mysql-cluster/data-mysql-cluster-secondary-1      nfs-storage             4m3s
pvc-a5673e1b-e3b2-415f-b2bb-c649eb4cd33d   8Gi        RWO            Delete           Bound         default/data-mysql-cluster-secondary-0            nfs-storage             32m
pvc-a683be89-289c-4088-b46f-f2af4d322a1a   8Gi        RWO            Delete           Bound         default/data-mysql-cluster-primary-0              nfs-storage             32m

8. 连接 MySQL 集群验证服务

#启动一个临时容器
$ kubectl run mysql-cluster-client --rm --tty -i --restart=‘Never’ --image registry.cn-hangzhou.aliyuncs.com/senfel/mysql:8.0.34-debian-11-r31 --namespace mysql-cluster --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command – bash

##登陆 MySQL Master节点
$ mysql -h mysql-cluster-primary.mysql-cluster.svc.cluster.local -uroot -p

Enter password: 123456rootmysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| my_database        |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.01 sec)

#查看主从状态
#查看File和Position的值，在从库配置中会显示。

mysql> show master status\G

*************************** 1. row ***************************File: mysql-bin.000003Position: 157Binlog_Do_DB:Binlog_Ignore_DB:
Executed_Gtid_Set:
1 row in set (0.00 sec)

##登陆从库，查看主从同步状态
$ mysql -h mysql-cluster-secondary.mysql-cluster.svc.cluster.local -uroot -p
Enter password: 123456root

mysql> show slave status\G;

*************************** 1. row ***************************Slave_IO_State: Waiting for source to send eventMaster_Host: mysql-cluster-primaryMaster_User: replicatorMaster_Port: 3306Connect_Retry: 10Master_Log_File: mysql-bin.000003Read_Master_Log_Pos: 157Relay_Log_File: mysql-relay-bin.000006Relay_Log_Pos: 373Relay_Master_Log_File: mysql-bin.000003Slave_IO_Running: YesSlave_SQL_Running: YesReplicate_Do_DB:Replicate_Ignore_DB:Replicate_Do_Table:Replicate_Ignore_Table:Replicate_Wild_Do_Table:Replicate_Wild_Ignore_Table:Last_Errno: 0Last_Error:Skip_Counter: 0Exec_Master_Log_Pos: 157Relay_Log_Space: 799Until_Condition: NoneUntil_Log_File:Until_Log_Pos: 0Master_SSL_Allowed: NoMaster_SSL_CA_File:Master_SSL_CA_Path:Master_SSL_Cert:Master_SSL_Cipher:Master_SSL_Key:Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: NoLast_IO_Errno: 0Last_IO_Error:Last_SQL_Errno: 0Last_SQL_Error:Replicate_Ignore_Server_Ids:Master_Server_Id: 313Master_UUID: ff169bc1-4af1-11ee-b795-f6df04eaa47aMaster_Info_File: mysql.slave_master_infoSQL_Delay: 0SQL_Remaining_Delay: NULLSlave_SQL_Running_State: Replica has read all relay log; waiting for more updatesMaster_Retry_Count: 86400Master_Bind:Last_IO_Error_Timestamp:Last_SQL_Error_Timestamp:Master_SSL_Crl:Master_SSL_Crlpath:Retrieved_Gtid_Set:Executed_Gtid_Set:Auto_Position: 0Replicate_Rewrite_DB:Channel_Name:Master_TLS_Version:Master_public_key_path:Get_master_public_key: 0Network_Namespace:
1 row in set, 1 warning (0.00 sec)

至此，使用Helm在K8S集群安装MySQL主从实战演示完成。

ingress-nginx开放主从端口

ingress-nginx是K8S开放端口的工具，可以直接将端口映射到集群外部。
#修改ingress-nginx tcp配置
$ vim values.yaml

#格式 "ns":"svc:port"
tcp: {"56379":"redis-cluster/redis-cluster:6379","53306": "mysql-cluster/mysql-cluster-primary:3306","54406": "mysql-cluster/mysql-cluster-secondary:3306"
}

#升级releas
$ helm upgrade ingress-nginx -n ingress-nginx .

#验证服务器端口是否开启
[root@node2 ~]# netstat -nplt | grep 06

tcp        0      0 0.0.0.0:53306           0.0.0.0:*               LISTEN      16944/nginx: master
tcp        0      0 0.0.0.0:54406           0.0.0.0:*               LISTEN      16944/nginx: master
tcp6       0      0 :::53306                :::*                    LISTEN      16944/nginx: master
tcp6       0      0 :::54406                :::*                    LISTEN      16944/nginx: master

查看最终的nginx.conf
#获取ingress-nginx pods
[root@node2 ~]# kubectl get pods -n ingress-nginx

NAME                             READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-75x86   1/1     Running   0          8m51s
ingress-nginx-controller-jgzks   1/1     Running   0          8m29s

#查看最终的nginx.conf
[root@master ingress-nginx]# kubectl exec ingress-nginx-controller-75x86 -n ingress-nginx – cat /etc/nginx/nginx.conf
在这里插入图片描述