category
tags
type
status
slug
date
summary
icon
password
前言
实际操作中都是使用KubeSphere作为k8s的默认可视化界面使用。它可以方便地管理k8s环境中的各个元素。

创建各个中间件
1. MySQL
核心三要素
部署方式(StatefulSet),数据存储 (PVC),配置文件(ConfigMap)。

开始前准备
可在官方镜像文档描述查看mysql容器启动的相关配置
传统 Docker的启动方式
docker run -p 3306:3306 --name mysql-01 \ -v /mydata/mysql/log:/var/log/mysql \ -v /mydata/mysql/data:/var/lib/mysql \ -v /mydata/mysql/conf:/etc/mysql/conf.d \ -e MYSQL_ROOT_PASSWORD=root \ --restart=always \ -d mysql:5.7
创建配置configmap
- 在配置项中添加配置

- 添加一个key-value配置,key就是配置文件的文件名 这里是
my.cnf

my.cnf的内容,填写的是数据库服务端的配置
[client] default-character-set=utf8mb4 [mysql] default-character-set=utf8mb4 [mysqld] init_connect='SET collation_connection = utf8mb4_unicode_ci' init_connect='SET NAMES utf8mb4' character-set-server=utf8mb4 collation-server=utf8mb4_unicode_ci skip-character-set-client-handshake skip-name-resolve
创建 挂载卷
- 创建卷

- 设置存储类型和访问模式以及卷容量

添加工作负载
- 在部署工作负载中选择有状态负载

- 镜像 配置为dockerhub中的mysql,并使用镜像默认端口设置

- 镜像环境变量,根据mysql镜像的官方要求,需要添加一个
MYSQL_ROOT_PASSWORD
;创建后点击下一步

- 挂载存储卷


- 挂载配置卷

指定配置卷的挂载目录

对外 提供服务
- KS默认会创建一个内网服务,可以选择修改,或重新创建一个NodePort对外服务

- NodePort类型表示可以从集群中的任意节点IP进行访问

- 访问地址,如图冒号后的地址就是外部访问的端口号

- 访问客户端测试

2. Nacos
开始前准备
可参考nacos官方的部署手册
部署有状态服务
虽然nacos没有存储,但是为了保证nacos的访问地址稳定需要部署成有状态(StatefulSet)

设置镜像
可在官方镜像文档描述查找nacos-server镜像,及容器启动的相关配置

- 环境变量
我这里用到了上面 的数据库 , 需要再添加一个 MODE = standalone 才能支持数据库配置,否则默认是集群模式运行

- 同步 主机时区

暴露外部服务


Nacos的一些问题
负载时的演示效果不同

nacos的默认配置文件
# spring server.servlet.contextPath=${SERVER_SERVLET_CONTEXTPATH:/nacos} server.contextPath=/nacosserver.port=${NACOS_APPLICATION_PORT:8848} spring.datasource.platform=${SPRING_DATASOURCE_PLATFORM:""} nacos.cmdb.dumpTaskInterval=3600 nacos.cmdb.eventTaskInterval=10 nacos.cmdb.labelTaskInterval=300 nacos.cmdb.loadDataAtStart=false db.num=${MYSQL_DATABASE_NUM:1} db.url.0=jdbc:mysql://${MYSQL_SERVICE_HOST}:${MYSQL_SERVICE_PORT:3306}/${MYSQL_SERVICE_DB_NAME}?${MYSQL_SERVICE_DB_PARAM:characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true&useSSL=false} db.url.1=jdbc:mysql://${MYSQL_SERVICE_HOST}:${MYSQL_SERVICE_PORT:3306}/${MYSQL_SERVICE_DB_NAME}?${MYSQL_SERVICE_DB_PARAM:characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true&useSSL=false}db.user=${MYSQL_SERVICE_USER} db.password=${MYSQL_SERVICE_PASSWORD} ### The auth system to use, currently only 'nacos' is supported: nacos.core.auth.system.type=${NACOS_AUTH_SYSTEM_TYPE:nacos} ### The token expiration in seconds: nacos.core.auth.default.token.expire.seconds=${NACOS_AUTH_TOKEN_EXPIRE_SECONDS:18000} ### The default token: nacos.core.auth.default.token.secret.key=${NACOS_AUTH_TOKEN:SecretKey012345678901234567890123456789012345678901234567890123456789} ### Turn on/off caching of auth information. By turning on this switch, the update of auth information would have a 15 seconds delay. nacos.core.auth.caching.enabled=${NACOS_AUTH_CACHE_ENABLE:false} nacos.core.auth.enable.userAgentAuthWhite=${NACOS_AUTH_USER_AGENT_AUTH_WHITE_ENABLE:false} nacos.core.auth.server.identity.key=${NACOS_AUTH_IDENTITY_KEY:serverIdentity} nacos.core.auth.server.identity.value=${NACOS_AUTH_IDENTITY_VALUE:security} server.tomcat.accesslog.enabled=${TOMCAT_ACCESSLOG_ENABLED:false} server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D # default current work dir server.tomcat.basedir= ## spring security config ### turn off security nacos.security.ignore.urls=${NACOS_SECURITY_IGNORE_URLS:/,/error,/**/*.css,/**/*.js,/**/*.html,/**/*.map,/**/*.svg,/**/*.png,/**/*.ico,/console-fe/public/**,/v1/auth/**,/v1/console/health/**,/actuator/**,/v1/console/server/**} # metrics for elastic search management.metrics.export.elastic.enabled=false management.metrics.export.influx.enabled=false nacos.naming.distro.taskDispatchThreadCount=10 nacos.naming.distro.taskDispatchPeriod=200 nacos.naming.distro.batchSyncKeyCount=1000 nacos.naming.distro.initDataRatio=0.9 nacos.naming.distro.syncRetryDelay=5000 nacos.naming.data.warmup=true
3. Redis
4. SEATA
部署前准备
官方文档的Docker部署方式
seata 连接到Nacos,由nacos托管 seata的配置;同时客户端是通过nacos的注册中心中获取seata的地址。
创建配置文件CM
我这里需要将SEATA注册到nacos中,使用nacos进行配置,用mysql进行undo_log的存储,需要修改
application.yml
文件,配置参考如下:server: port: 7091 spring: application: name: seata-server logging: config: classpath:logback-spring.xml file: path: ${user.home}/logs/seata extend: logstash-appender: destination: svc-logstash:4560 kafka-appender: bootstrap-servers: svc-kafka:9092 topic: logback_to_logstash console: user: username: admin password: admin seata: config: # support: nacos, consul, apollo, zk, etcd3 type: nacos nacos: server-addr: nacos-svc:8848 group: SEATA_GROUP namespace: username: nacos password: nacos registry: # support: nacos, eureka, redis, zk, consul, etcd3, sofa type: nacos nacos: application: seata-server server-addr: nacos-svc:8848 group: SEATA_GROUP namespace: username: nacos password: nacos # server: # service-port: 8091 #If not configured, the default is '${server.port} + 1000' security: secretKey: SeataSecretKey0c382ef121d778043159209298fd40bf3850a017 tokenValidityInMilliseconds: 1800000 ignore: urls: /,/**/*.css,/**/*.js,/**/*.html,/**/*.map,/**/*.svg,/**/*.png,/**/*.ico,/console-fe/public/**,/api/v1/auth/login#
创建有状态副本
- 查找镜像

- 开放端口
7091是面板的地址 ,8091是 服务端口地址

- 挂载配置文件,注意,我们这里只修改
application.yml
配置文件,为了不覆盖容器中/seata-server/resources
目录下的其它配置文件,这里要使用子路径配置。

暴露服务

此方案部署后,seata-server虽然部署成功,但是注册到Nacos上的IP是Kubernetes内部集群IP,seata客户端应用是无法访问到seata-server服务的,除非nacos和seata客户端应用也部署在Kubernetes集群内部,但显然目前无法进行这么大的变革。
所以解决问题的方法就是让seata-server的注册IP是宿主机IP.
定义SEATA的外部IP
定义SEATA的外部端口
k8s如果用NodePort方式暴露端口的话端口必须是30000-32765之间的端口号,但是在容器中默认配置的端口是8091,这就会导致连接不上,要从外部访问的折中之计就是先利用hostPort将容器的端口直接在宿主机上暴露。

这样做会占用了宿主机唯一的8091端口,要注意实例数不能大于节点机数,否则多出来的会无法部署。
通过Mysql存储实现Seata的高可用
开始前准备
创建seata的数据库,并且执行官方的初始化脚本 mysql.sql , 数据库建完效果如下,我目前用的是1.5.2版本的脚本:

在NACOS中配置Seata的Store类型为DB
创建
/config.txt
,并填写数据库配置如下:service.vgroupMapping.ruoyi-system-group=default store.mode=db store.db.datasource=druid store.db.dbType=mysql store.db.driverClassName=com.mysql.jdbc.Driver store.db.url=jdbc:mysql://mysql-dev-svc:3306/ry-seata?useUnicode=true store.db.user=root store.db.password=root store.db.minConn=5 store.db.maxConn=30 store.db.globalTable=global_table store.db.branchTable=branch_table store.db.queryLimit=100 store.db.lockTable=lock_table store.db.maxWait=5000
创建一个nacos配置导入脚本 /conf/nacos-config.sh
内容点击展开:
#!/usr/bin/env bash # Copyright 1999-2019 Seata.io Group. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at、 # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. while getopts ":h:p:g:t:u:w:" opt do case $opt in h) host=$OPTARG ;; p) port=$OPTARG ;; g) group=$OPTARG ;; t) tenant=$OPTARG ;; u) username=$OPTARG ;; w) password=$OPTARG ;; ?) echo " USAGE OPTION: $0 [-h host] [-p port] [-g group] [-t tenant] [-u username] [-w password] " exit 1 ;; esac done urlencode() { for ((i=0; i < ${#1}; i++)) do char="${1:$i:1}" case $char in [a-zA-Z0-9.~_-]) printf $char ;; *) printf '%%%02X' "'$char" ;; esac done } if [[ -z ${host} ]]; then host=localhost fi if [[ -z ${port} ]]; then port=8848 fi if [[ -z ${group} ]]; then group="SEATA_GROUP" fi if [[ -z ${tenant} ]]; then tenant="" fi if [[ -z ${username} ]]; then username="" fi if [[ -z ${password} ]]; then password="" fi nacosAddr=$host:$port contentType="content-type:application/json;charset=UTF-8" echo "set nacosAddr=$nacosAddr" echo "set group=$group" failCount=0 tempLog=$(mktemp -u) function addConfig() { curl -X POST -H "${contentType}" "http://$nacosAddr/nacos/v1/cs/configs?dataId=$(urlencode $1)&group=$group&content=$(urlencode $2)&tenant=$tenant&username=$username&password=$password" >"${tempLog}" 2>/dev/null if [[ -z $(cat "${tempLog}") ]]; then echo " Please check the cluster status. " exit 1 fi if [[ $(cat "${tempLog}") =~ "true" ]]; then echo "Set $1=$2 successfully " else echo "Set $1=$2 failure " (( failCount++ )) fi } count=0 for line in $(cat $(dirname "$PWD")/config.txt | sed s/[[:space:]]//g); do (( count++ )) key=${line%%=*} value=${line#*=} addConfig "${key}" "${value}" done echo "=========================================================================" echo " Complete initialization parameters, total-count:$count , failure-count:$failCount " echo "=========================================================================" if [[ ${failCount} -eq 0 ]]; then echo " Init nacos config finished, please start seata-server. " else echo " init nacos config fail. " fi
执行脚本,后面跟上nacos的ip和端口,例如:
./nacos-config.sh -h192.168.31.191 -p31864

配置添加成功的效果如下:当然这些配置你也可以在NACOS中逐一添加

- 导入配置后,重新启动Seata,若日志结尾出现
{dataSource-1} inited
表示Seata 成功连接mysql
06:57:28.361 INFO --- [ main] io.seata.config.ConfigurationFactory : load Configuration from :Spring Configuration [4.265s][info ][gc] GC(8) Pause Young (Allocation Failure) 61M->19M(149M) 15.132ms Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 06:57:29.934 INFO --- [ main] com.alibaba.druid.pool.DruidDataSource : {dataSource-1} inited 06:57:30.007 ERROR --- [ main] i.s.s.s.d.l.DataBaseDistributedLocker : The distribute lock table is not config, please create the target table and config it [5.506s][info ][gc] GC(9) Pause Young (Allocation Failure) 62M->24M(149M) 20.491ms 06:57:30.168 INFO --- [ main] i.s.core.rpc.netty.NettyServerBootstrap : Server started, service listen port: 8091 06:57:30.279 INFO --- [ main] io.seata.server.ServerRunner : seata server started in 1981 millSeconds