一、简介

  实际工作中，按时间分片的比较多，但是也有些特殊的分片，不只是用到一个字段，可能会多个字段，这个时候就要用到复合分片算法来实现了 ComplexKeysShardingAlgorithm 。

  我也是偷懒，不想设计其他的表了，就用以前的表结构来完成，我们就以用户id和金额分表。本文示例大概架构如下图：

例子没很大的实用性，你可以扩展为一个平台很多商户，同一个商户的交易按月分表，也是没问题的，但是我们学习的是思路。

二、maven依赖

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.6.0</version><relativePath/> <!-- lookup parent from repository --></parent><groupId>com.alian</groupId><artifactId>sharding-jdbc</artifactId><version>0.0.1-SNAPSHOT</version><name>sharding-jdbc</name><description>sharding-jdbc</description><properties><java.version>1.8</java.version></properties><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-jpa</artifactId></dependency><dependency><groupId>org.apache.shardingsphere</groupId><artifactId>sharding-jdbc-spring-boot-starter</artifactId><version>4.1.1</version></dependency><dependency><groupId>com.alibaba</groupId><artifactId>druid</artifactId><version>1.2.15</version></dependency><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><version>8.0.26</version><scope>runtime</scope></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.18.20</version></dependency><dependency><groupId>junit</groupId><artifactId>junit</artifactId><version>4.12</version><scope>test</scope></dependency></dependencies><build><plugins><plugin><groupId>org.springframework.boot</groupId><artifactId>spring-boot-maven-plugin</artifactId></plugin></plugins></build></project>

  有些小伙伴的 druid 可能用的是 druid-spring-boot-starter

<dependency><groupId>com.alibaba</groupId><artifactId>druid-spring-boot-starter</artifactId><version>1.2.6</version>
</dependency>

  然后出现可能使用不了的各种问题，这个时候你只需要在主类上添加 @SpringBootApplication(exclude = {DruidDataSourceAutoConfigure.class}) 即可

package com.alian.shardingjdbc;import com.alibaba.druid.spring.boot.autoconfigure.DruidDataSourceAutoConfigure;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;@SpringBootApplication(exclude = {DruidDataSourceAutoConfigure.class})
@SpringBootApplication
public class ShardingJdbcApplication {public static void main(String[] args) {SpringApplication.run(ShardingJdbcApplication.class, args);}}

三、数据库

3.1、创建数据库

CREATE DATABASE `sharding_12` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;

3.2、创建表

  在数据库sharding_12创建表：tb_order_00、tb_order_01、tb_order_02，三者的结构是一样的：

CREATE TABLE `tb_order_00` (`order_id` bigint(20) NOT NULL COMMENT '主键',`user_id` int unsigned NOT NULL DEFAULT '0' COMMENT '用户id',`price` int unsigned NOT NULL DEFAULT '0' COMMENT '价格（单位：分）',`order_status` tinyint unsigned NOT NULL DEFAULT '1' COMMENT '订单状态(1:待付款,2:已付款,3:已取消)',`order_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',`title` varchar(100)  NOT NULL DEFAULT '' COMMENT '订单标题',PRIMARY KEY (`order_id`),KEY `idx_user_id` (`user_id`),KEY `idx_order_time` (`order_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='订单表';CREATE TABLE `tb_order_01` (`order_id` bigint(20) NOT NULL COMMENT '主键',`user_id` int unsigned NOT NULL DEFAULT '0' COMMENT '用户id',`price` int unsigned NOT NULL DEFAULT '0' COMMENT '价格（单位：分）',`order_status` tinyint unsigned NOT NULL DEFAULT '1' COMMENT '订单状态(1:待付款,2:已付款,3:已取消)',`order_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',`title` varchar(100)  NOT NULL DEFAULT '' COMMENT '订单标题',PRIMARY KEY (`order_id`),KEY `idx_user_id` (`user_id`),KEY `idx_order_time` (`order_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='订单表';CREATE TABLE `tb_order_02` (`order_id` bigint(20) NOT NULL COMMENT '主键',`user_id` int unsigned NOT NULL DEFAULT '0' COMMENT '用户id',`price` int unsigned NOT NULL DEFAULT '0' COMMENT '价格（单位：分）',`order_status` tinyint unsigned NOT NULL DEFAULT '1' COMMENT '订单状态(1:待付款,2:已付款,3:已取消)',`order_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',`title` varchar(100)  NOT NULL DEFAULT '' COMMENT '订单标题',PRIMARY KEY (`order_id`),KEY `idx_user_id` (`user_id`),KEY `idx_order_time` (`order_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='订单表';

为了说明 ComplexKeysShardingAlgorithm 算法，我假设有个这样的需求（实际中应该不会有，这里只是演示），生成的订单规则如下：

• userId和price一个奇数一个偶数 tb_order_00
• userId和price都是奇数的订单都存到 tb_order_01
• userId和price都是偶数的订单都存到 tb_order_02

很明显，这样分表的字段就是多个了，userId和price，那我们看怎么实现的

四、配置（二选一）

4.1、properties配置

application.properties

server.port=8899
server.servlet.context-path=/sharding-jdbc# 允许定义相同的bean对象去覆盖原有的
spring.main.allow-bean-definition-overriding=true
# 数据源名称，多数据源以逗号分隔
spring.shardingsphere.datasource.names=ds1
# sharding_1数据库连接池类名称
spring.shardingsphere.datasource.ds1.type=com.alibaba.druid.pool.DruidDataSource
# sharding_1数据库驱动类名
spring.shardingsphere.datasource.ds1.driver-class-name=com.mysql.cj.jdbc.Driver
# sharding_1数据库url连接
spring.shardingsphere.datasource.ds1.url=jdbc:mysql://192.168.0.129:3306/sharding_12?serverTimezone=GMT%2B8&characterEncoding=utf8&useUnicode=true&useSSL=false&zeroDateTimeBehavior=CONVERT_TO_NULL&autoReconnect=true&allowMultiQueries=true&failOverReadOnly=false&connectTimeout=6000&maxReconnects=5
# sharding_1数据库用户名
spring.shardingsphere.datasource.ds1.username=alian
# sharding_1数据库密码
spring.shardingsphere.datasource.ds1.password=123456# 指定库分片策略
spring.shardingsphere.sharding.default-data-source-name=ds1
# 指定tb_order表的数据分布情况，配置数据节点，使用Groovy的表达式
spring.shardingsphere.sharding.tables.tb_order.actual-data-nodes=ds1.tb_order_0$->{0..2}# 采用标准分片策略：ComplexKeysShardingAlgorithm
# 指定tb_order表的分片策略中的分片键
spring.shardingsphere.sharding.tables.tb_order.table-strategy.complex.sharding-columns=user_id,price
# 指定tb_order表的分片策略中的分片算法全类路径的名称
spring.shardingsphere.sharding.tables.tb_order.table-strategy.complex.algorithm-class-name=com.alian.shardingjdbc.algorithm.OrderComplexShardingAlgorithm# 指定tb_order表的主键为order_id
spring.shardingsphere.sharding.tables.tb_order.key-generator.column=order_id
# 指定tb_order表的主键生成策略为SNOWFLAKE
spring.shardingsphere.sharding.tables.tb_order.key-generator.type=SNOWFLAKE
# 指定雪花算法的worker.id
spring.shardingsphere.sharding.tables.tb_order.key-generator.props.worker.id=100
# 指定雪花算法的max.tolerate.time.difference.milliseconds
spring.shardingsphere.sharding.tables.tb_order.key-generator.props.max.tolerate.time.difference.milliseconds=20# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

4.2、yml配置

application.yml

server:port: 8899servlet:context-path: /sharding-jdbcspring:main:# 允许定义相同的bean对象去覆盖原有的allow-bean-definition-overriding: trueshardingsphere:props:sql:# 打开sql输出日志show: truedatasource:# 数据源名称，多数据源以逗号分隔names: ds1ds1:# 数据库连接池类名称type: com.alibaba.druid.pool.DruidDataSource# 数据库驱动类名driver-class-name: com.mysql.cj.jdbc.Driver# 数据库url连接url: jdbc:mysql://192.168.0.129:3306/sharding_12?serverTimezone=GMT%2B8&characterEncoding=utf8&useUnicode=true&useSSL=false&zeroDateTimeBehavior=CONVERT_TO_NULL&autoReconnect=true&allowMultiQueries=true&failOverReadOnly=false&connectTimeout=6000&maxReconnects=5# 数据库用户名username: alian# 数据库密码password: 123456sharding:# 未配置分片规则的表将通过默认数据源定位default-data-source-name: ds1tables:tb_order:# 由数据源名 + 表名组成，以小数点分隔。多个表以逗号分隔，支持inline表达式actual-data-nodes: ds1.tb_order_0$->{0..2}# 分表策略table-strategy:complex:# 分片键sharding-columns: user_id,price# 复合分片算法algorithm-class-name: com.alian.shardingjdbc.algorithm.OrderComplexShardingAlgorithm# key生成器key-generator:# 自增列名称，缺省表示不使用自增主键生成器column: order_id# 自增列值生成器类型，缺省表示使用默认自增列值生成器（SNOWFLAKE/UUID）type: SNOWFLAKE# SnowflakeShardingKeyGeneratorprops:# SNOWFLAKE算法的worker.idworker:id: 100# SNOWFLAKE算法的max.tolerate.time.difference.millisecondsmax:tolerate:time:difference:milliseconds: 20

• actual-data-nodes 使用Groovy的表达式 就是表示上面创建的表
• 通过复合分片策略完成分表，分片键是： user_id 、 price
• table-strategy 采用的是 复合分片策略 ，算法实现类是我们自定义的类 com.alian.shardingjdbc.algorithm.OrderComplexShardingAlgorithm ；
• key-generator ：key生成器，需要指定字段和类型，比如这里如果是SNOWFLAKE，最好也配置下props中的两个属性： worker.id 与 max.tolerate.time.difference.milliseconds 属性

五、复合分片算法

  具体的算法实现如下：

OrderComplexShardingAlgorithm.java

package com.alian.shardingjdbc.algorithm;import lombok.extern.slf4j.Slf4j;
import org.apache.shardingsphere.api.sharding.complex.ComplexKeysShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.complex.ComplexKeysShardingValue;import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Map;@Slf4j
public class OrderComplexShardingAlgorithm implements ComplexKeysShardingAlgorithm<Integer> {private static final DateTimeFormatter FORMATTER = DateTimeFormatter.ofPattern("yyyyMM");@Overridepublic Collection<String> doSharding(Collection<String> availableTargetNames, ComplexKeysShardingValue<Integer> shardingValue) {Collection<String> result = new ArrayList<>();Map<String, Collection<Integer>> shardingValuesMap = shardingValue.getColumnNameAndShardingValuesMap();Collection<Integer> userIds = shardingValuesMap.get("user_id");Collection<Integer> prices = shardingValuesMap.get("price");for (Integer userId : userIds) {for (Integer price : prices) {String suffix;if (userId % 2 == 1 && price % 2 == 1) {// 奇数suffix = "01";} else if (userId % 2 == 0 && price % 2 == 0) {// 偶数suffix = "02";} else {// 1奇数1偶数suffix = "00";}for (String targetName : availableTargetNames) {if (targetName.endsWith("_" + suffix)) { // 按照分片值选择目标表result.add(targetName);}}}}return result;}}

  我们是按照订单时间进行分表的，实际使用也很简单，插入数据时实现接口 OrderComplexShardingAlgorithm <Integer> 。然后重写方法 doSharding ，这个方法会有两个参数，第一个就是物理表的集合，第二个是分片对象。但是有人就会说，你这里正好两个字段都是同一个类型，而我们实际中，可能是多种类型，比如String、Long、Integer、Date等，按照这个写法，不是这个类型的就取不到值了，如果你是你说的这种情况，你就把上面的代码改造下，把类型改成 <Comparable<?>> ，然后就自己转类型，别说你自己都不知道类型，具体见下面代码。

package com.alian.shardingjdbc.algorithm;import lombok.extern.slf4j.Slf4j;
import org.apache.shardingsphere.api.sharding.complex.ComplexKeysShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.complex.ComplexKeysShardingValue;import java.util.ArrayList;
import java.util.Collection;
import java.util.Map;@Slf4j
public class ComparableOrderComplexShardingAlgorithm implements ComplexKeysShardingAlgorithm<Comparable<?>> {@Overridepublic Collection<String> doSharding(Collection<String> availableTargetNames, ComplexKeysShardingValue<Comparable<?>> shardingValue) {Collection<String> result = new ArrayList<>();Map<String, Collection<Comparable<?>>> shardingValuesMap = shardingValue.getColumnNameAndShardingValuesMap();Collection<?> userIds = shardingValuesMap.get("user_id");Collection<?> prices = shardingValuesMap.get("price");
//        Collection<?> orderTimes = shardingValuesMap.get("order_time");for (Object userIdObj : userIds) {for (Object priceObj : prices) {String suffix;int userId = Integer.parseInt(userIdObj + "");int price = Integer.parseInt(priceObj + "");if (userId % 2 == 1 && price % 2 == 1) {// 奇数suffix = "01";} else if (userId % 2 == 0 && price % 2 == 0) {// 偶数suffix = "02";} else {// 1奇数1偶数suffix = "00";}for (String targetName : availableTargetNames) {if (targetName.endsWith("_" + suffix)) { // 按照分片值选择目标表result.add(targetName);}}}}return result;}}

六、实现

6.1、实体层

Order.java

@Data
@Entity
@Table(name = "tb_order")
public class Order implements Serializable {@Id@GeneratedValue(strategy = GenerationType.IDENTITY)@Column(name = "order_id")private Long orderId;@Column(name = "user_id")private Integer userId;@Column(name = "price")private Integer price;@Column(name = "order_status")private Integer orderStatus;@Column(name = "title")private String title;@JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss")@Column(name = "order_time")private LocalDateTime orderTime;}

6.2、持久层

OrderRepository.java

public interface OrderRepository extends PagingAndSortingRepository<Order, Long> {/*** 根据userId和金额查询订单** @param userId* @param price* @return*/List<Order> findByUserIdAndPrice(int userId, int price);
}

6.3、服务层

OrderService.java

@Slf4j
@Service
public class OrderService {@Autowiredprivate OrderRepository orderRepository;public void saveOrder(Order order) {orderRepository.save(order);}public List<Order> findByUserIdAndPrice(int userId, int price) {return orderRepository.findByUserIdAndPrice(userId, price);}
}

6.4、测试类

OrderTests.java

@Slf4j
@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest
public class OrderTests {@Autowiredprivate OrderService orderService;@Testpublic void saveOrder() {// 创建20条订单记录for (int i = 0; i < 20; i++) {Order order = new Order();
//            order.setOrderId(System.currentTimeMillis());// 随机生成1000到1009的用户idint userId = (int) Math.round(Math.random() * (1009 - 1000) + 1000);order.setUserId(userId);// 随机生成50到100的金额int price = (int) Math.round(Math.random() * (10000 - 5000) + 5000);order.setPrice(price);order.setOrderStatus(2);order.setOrderTime(LocalDateTime.now());order.setTitle("");orderService.saveOrder(order);}}@Testpublic void queryOrder() {List<Order> order = orderService.findByUserIdAndPrice(1002,9860);log.info("查询的结果：{}", order);}}

我们插入数据时，采用随机时间插入，具体时间生成见测试类。

效果图：

  从上面的数据来看，满足我们分库分表的要求的，实现都是基于我们自定义的算法实现。

6.4.2、根据时间范围查询订单

13:09:39 403 INFO [main]:Actual SQL: ds1 ::: select order0_.order_id as order_id1_0_, order0_.order_status as order_st2_0_, order0_.order_time as order_ti3_0_, order0_.price as price4_0_, order0_.title as title5_0_, order0_.user_id as user_id6_0_ from tb_order_02 order0_ where order0_.user_id=? and order0_.price=? ::: [1002, 9860]
13:09:39 463 INFO [main]:查询的结果：[Order(orderId=940934509405552641, userId=1002, price=9860, orderStatus=2, title=, orderTime=2023-12-11T11:37:49)]

通过语句查询：

(SELECT *,'tb_order_00' FROM sharding_12.tb_order_00
where user_id=1002  and price=9860)
union all
(SELECT *,'tb_order_01' FROM sharding_12.tb_order_01
where user_id=1002  and price=9860)
union all
(SELECT *,'tb_order_02' FROM sharding_12.tb_order_02
where user_id=1002  and price=9860)

数据库校验：