34 System Integration How to Complete the Seamless Integration of Sharding Sphere Kernel With Spring Spring Boot

34 System Integration How to Complete the Seamless Integration of ShardingSphere Kernel with Spring-SpringBoot #

Today, we will enter into the last module of the entire course - the system integration module. Here, system integration refers to the integration of ShardingSphere with the Spring framework.

So far, ShardingSphere has implemented two system integration mechanisms: one is the namespace mechanism, which integrates with the Spring framework by extending the Spring Schema; and the other is integrating with Spring Boot by writing custom starter components. In this lesson, I will explain these two system integration mechanisms separately.

With the system integration module, developers can seamlessly use ShardingSphere regardless of which Spring framework they choose, resulting in zero learning cost.

Integration with Spring based on namespaces #

From an extensibility perspective, the extension mechanism based on XML Schema is a very common and practical method. In Spring, it allows us to define our own XML structure and parse it using our own Bean Parser. By extending the Spring Schema, ShardingSphere can be effectively integrated with the Spring framework.

1. General development process for integrating with Spring based on namespaces #

Based on the namespace mechanism for integrating with Spring, a fixed process is usually followed in development, consisting of the following five steps:

image

These steps include: writing business objects, writing XSD files, writing BeanDefinitionParser implementation classes, writing NamespaceHandler implementation classes, and writing spring.handlers and spring.schemas configuration files. Let’s take a look at how ShardingSphere implements these steps.

2. Integration of ShardingSphere with Spring #

In ShardingSphere, there are two code projects with names ending in “spring-namespace”, namely sharding-jdbc-spring-namespace and sharding-jdbc-orchestration-spring-namespace. It is obvious that the latter is focused on integrating orchestration-related functions and is relatively simpler. Also, because the implementation process of the namespace mechanism is basically the same, we will discuss it using the sharding-jdbc-spring-namespace project as an example.

In the sharding-jdbc-spring-namespace project, there is a class called SpringShardingDataSource which is specifically used for integrating with Spring. This class is the business object, as shown below:

public class SpringShardingDataSource extends ShardingDataSource {

    public SpringShardingDataSource(final Map<String, DataSource> dataSourceMap, final ShardingRuleConfiguration shardingRuleConfiguration, final Properties props) throws SQLException {

        super(dataSourceMap, new ShardingRule(shardingRuleConfiguration, dataSourceMap.keySet()), props);

    }

}

As you can see, this SpringShardingDataSource class is just a simple wrapper for ShardingDataSource and does not contain any actual operations.

Next, let’s look at the class that defines the configuration tag. This kind of class is a simple utility class that defines the names of the tags. In terms of naming, these classes in ShardingSphere end with “BeanDefinitionParserTag”. For example, the ShardingDataSourceBeanDefinitionParserTag is defined as follows:

public final class ShardingDataSourceBeanDefinitionParserTag {

    public static final String ROOT_TAG = "data-source";
    public static final String SHARDING_RULE_CONFIG_TAG = "sharding-rule";
    public static final String PROPS_TAG = "props";
    public static final String DATA_SOURCE_NAMES_TAG = "data-source-names";
    public static final String DEFAULT_DATA_SOURCE_NAME_TAG = "default-data-source-name";
    public static final String TABLE_RULES_TAG = "table-rules"; 
    
    ...
}

This defines a batch of tags and attributes. We won’t expand on each one. You can refer to the XML-based configuration example shown below to understand the configuration items defined by these definitions:

<sharding:data-source id="shardingDataSource">
    <sharding:sharding-rule data-source-names="ds0,ds1">
        <sharding:table-rules>
            <sharding:table-rule …/>
            <sharding:table-rule …/>
            …
        </sharding:table-rules>
        …
    </sharding:sharding-rule>
</sharding:data-source>

Then, we found the corresponding sharding.xsd file in the META-INF/namespace folder of the sharding-jdbc-spring-namespace code project. Its basic structure is as follows:

<xsd:schema xmlns="http://shardingsphere.apache.org/schema/shardingsphere/sharding"
            xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns:beans="http://www.springframework.org/schema/beans"
            xmlns:encrypt="http://shardingsphere.apache.org/schema/shardingsphere/encrypt"
            targetNamespace="http://shardingsphere.apache.org/schema/shardingsphere/sharding"

            elementFormDefault="qualified" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

            xsi:schemaLocation="http://shardingsphere.apache.org/schema/shardingsphere/encrypt http://shardingsphere.apache.org/schema/shardingsphere/encrypt/encrypt.xsd">

    <xsd:import namespace="http://www.springframework.org/schema/beans" schemaLocation="http://www.springframework.org/schema/beans/spring-beans.xsd" />

    <xsd:import namespace="http://shardingsphere.apache.org/schema/shardingsphere/encrypt" schemaLocation="http://shardingsphere.apache.org/schema/shardingsphere/encrypt/encrypt.xsd"/>

    <xsd:element name="data-source">

        <xsd:complexType>

            <xsd:all>

                <xsd:element ref="sharding-rule" />

                <xsd:element ref="props" minOccurs="0" />

            </xsd:all>

            <xsd:attribute name="id" type="xsd:string" use="required" />

        </xsd:complexType>

	</xsd:element>

	…

</xsd:schema>

You can see that for the “data-source” element, it contains two sub-elements: “sharding-rule” and “props”. The “props” element is not required. In addition, the “data-source” element can also have an “id” attribute, which is required. We have seen this in the previous configuration example. As for the “sharding-rule” element, it can have many nested attributes, which are defined in the sharding.xsd file.

At the same time, we should note that the sharding.xsd file also imports two namespaces using the xsd:import tag. One is the Spring namespace <http://www.springframework.org/schema/beans>, and the other is ShardingSphere’s own namespace <http://shardingsphere.apache.org/schema/shardingsphere/encrypt>, which is defined in the encrypt.xsd file located in the same directory as the sharding.xsd file.

With the business object class and the XSD file definition, let’s take a look at the ShardingNamespaceHandler implementation class, as shown below:

public final class ShardingNamespaceHandler extends NamespaceHandlerSupport {
    
    @Override
    
    public void init() {
    
        registerBeanDefinitionParser(ShardingDataSourceBeanDefinitionParserTag.ROOT_TAG, new ShardingDataSourceBeanDefinitionParser());
    
        registerBeanDefinitionParser(ShardingStrategyBeanDefinitionParserTag.STANDARD_STRATEGY_ROOT_TAG, new ShardingStrategyBeanDefinitionParser());
    
        ...
    
    }
    
}

Here, we directly use the registerBeanDefinitionParser method to establish the correspondence between the tag names and the corresponding BeanDefinitionParser classes. Let’s take a look at the ShardingDataSourceBeanDefinitionParser class, whose core parseInternal method is shown below:

@Override

protected AbstractBeanDefinition parseInternal(final Element element, final ParserContext parserContext) {
    
    // Build BeanDefinitionBuilder for SpringShardingDataSource
    
    BeanDefinitionBuilder factory = BeanDefinitionBuilder.rootBeanDefinition(SpringShardingDataSource.class);
    
    // Parse DataSource parameter in the constructor
    
    factory.addConstructorArgValue(parseDataSources(element));
    
    // Parse ShardingRuleConfiguration parameter in the constructor
    
    factory.addConstructorArgValue(parseShardingRuleConfiguration(element));
    
    // Parse Properties parameter in the constructor
    
    factory.addConstructorArgValue(parseProperties(element, parserContext));
    
    factory.setDestroyMethodName("close");
    
    return factory.getBeanDefinition();
    
}

Here, we define our own BeanDefinitionBuilder and bind it to the business object class SpringShardingDataSource. Then, we use the addConstructorArgValue method to assign values to the dataSourceMap, shardingRuleConfiguration, and props parameters required by the constructor of SpringShardingDataSource.

Let’s take a closer look at the parseDataSources method mentioned in the above code, as shown below:

private Map<String, RuntimeBeanReference> parseDataSources(final Element element) {
    
    Element shardingRuleElement = DomUtils.getChildElementByTagName(element, ShardingDataSourceBeanDefinitionParserTag.SHARDING_RULE_CONFIG_TAG);
    
    List<String> dataSources = Splitter.on(",").trimResults().splitToList(shardingRuleElement.getAttribute(ShardingDataSourceBeanDefinitionParserTag.DATA_SOURCE_NAMES_TAG));
    
    Map<String, RuntimeBeanReference> result = new ManagedMap<>(dataSources.size());
    
    for (String each : dataSources) {
    
        result.put(each, new RuntimeBeanReference(each));
    
    }

Here, we first retrieve the sharding-rule element and extract a list of dataSources from the data-source-names attribute. Then, we create a result map and iterate over the dataSources list. For each dataSource, we create a RuntimeBeanReference and add it to the result map.

public DataSource shardingDataSource(final Map<String, DataSource> dataSourceMap, final SpringBootShardingRuleConfigurationProperties prop, 
                                     final SpringBootMasterSlaveRuleConfigurationProperties masterSlaveProp, 
                                     final SpringBootPropertiesConfigurationProperties propConfig, final Environment environment) throws SQLException {
        return new ShardingDataSource(dataSourceMap, 
                                      createShardingRule(dataSourceMap, prop, environment), 
                                      propConfig.getProps());
    }

这个方法使用了 @Bean 注解，表示它会被实例化成一个 bean，因此可以在其他地方引用。方法参数中的 dataSourceMap 就是之前提到的从 Spring 容器中获取的 DataSource，其他的 prop、masterSlaveProp、propConfig 主要用于构造相应的配置对象，并最终传入 ShardingDataSource 的构造方法中。

在这个方法中，通过 createShardingRule 方法创建了 ShardingRule 对象，然后传入 ShardingDataSource 的构建方法中。

接下来，我们来看一下 createShardingRule 方法的实现：

private ShardingRule createShardingRule(final Map<String, DataSource> dataSourceMap, final SpringBootShardingRuleConfigurationProperties prop,
                                              final Environment environment) {
        ShardingRuleConfiguration config = new SpringBootShardingRuleConfigurationYamlSwapper().swap(prop);
        return StringUtils.isNullOrEmpty(config.getMasterSlaveRule().getMasterDataSourceName()) 
                ? new ShardingRule(config, dataSourceMap.keySet())
                : new ShardingRule(config, dataSourceMap.keySet(), createMasterSlaveRule(dataSourceMap, config.getMasterSlaveRule()), environment);
    }

这个方法通过 SpringBootShardingRuleConfigurationYamlSwapper 类将从配置文件中读取的配置转换为 ShardingRuleConfiguration 对象。

接下来就是判断是否存在主从配置，如果不存在，则调用 ShardingRule 的构造方法，只传入 config 和 dataSourceMap.keySet()；如果存在，则调用另一个构造方法，除了之前的参数外，还需要创建 MasterSlaveRule。

通过这个方法，我们可以创建一个包含 ShardingRule 的 ShardingDataSource 实例。

类似的，SpringBootConfiguration 类中还提供了创建 MasterSlaveDataSource 的方法，也可以通过 @Bean 注解实例化为一个 bean。具体实现过程与创建 ShardingDataSource 类似，就不再赘述。

总体来说，SpringBootConfiguration 类中提供的这些方法，可以很方便地创建出不同类型的 DataSource，并将它们的配置信息封装在相应的配置类中。

接下来我们看一下，ShardingDataSource 是如何处理配置类的参数的。

可以看到，在 ShardingDataSource 的构造方法中，会先调用 ShardingRule 的构造方法传入配置信息，并处理相关业务逻辑。

ShardingRule 是 Sharding-JDBC 的核心规则配置类，它主要包含表路由策略、默认的数据库分片策略、默认的表分片策略以及默认的绑定表的策略。

传入的配置信息在构造方法中被处理后，将被存储在 ShardingRule 中供 ShardingDataSource 使用。

到这里，我们就完成了 ShardingSphere 与 Spring Boot 的集成过程的学习。通过学习本节的内容，我们不仅可以清楚的了解 ShardingSphere 基于命名空间的集成方式以及基于自定义 starter 的集成方式，还可以对 ShardingSphere 主要组件以及内核原理有进一步的认识。希望本节内容对您的学习有所帮助。

小结 #

本节从一个简单的使用示例开始，介绍了 ShardingSphere 中基于命名空间机制的 Spring 项目集成方式以及基于自定义 starter 的 SpringBoot 项目集成方式，并对 ShardingSphere 进行了一定的内核原理阐述。

ShardingSphere 提供了两种主要方式进行 Spring 项目集成，即基于命名空间机制以及基于自定义 starter，二者之间比较而言，基于自定义 starter 可定制化更高，而基于命名空间机制则更加简单且易于理解。

在自定义 starter 集成方式中，ShardingSphere 通过自定义 starter 来实现在 Spring Boot 启动过程中加载相关类并将其添加到配置容器中。

通过学习本节，我们相信大家对于集成 ShardingSphere 到 Spring 项目的方式以及 ShardingSphere 的内核原理有了更深的了解。希望本节的内容能够对您的学习有所帮助。

@Conditional(ShardingRuleCondition.class)

public DataSource shardingDataSource() throws SQLException {

    return ShardingDataSourceFactory.createDataSource(dataSourceMap, new ShardingRuleConfigurationYamlSwapper().swap(shardingRule), props.getProps());

}

This method has two annotations, one is the common @Bean, and the other is @Conditional. The purpose of the @Conditional annotation is to load this Bean only when certain conditions are met. We can see that the @Conditional annotation is set with a ShardingRuleCondition class, which is shown as follows:

public final class ShardingRuleCondition extends SpringBootCondition {

    @Override
    public ConditionOutcome getMatchOutcome(final ConditionContext conditionContext, final AnnotatedTypeMetadata annotatedTypeMetadata) {

        boolean isMasterSlaveRule = new MasterSlaveRuleCondition().getMatchOutcome(conditionContext, annotatedTypeMetadata).isMatch();

        boolean isEncryptRule = new EncryptRuleCondition().getMatchOutcome(conditionContext, annotatedTypeMetadata).isMatch();

        return isMasterSlaveRule || isEncryptRule ? ConditionOutcome.noMatch("Have found master-slave or encrypt rule in environment") : ConditionOutcome.match();

    }

}

It can be seen that ShardingRuleCondition is a standard SpringBootCondition and implements the getMatchOutcome abstract method. We know that the purpose of SpringBootCondition is to represent a condition for registering classes or loading Beans. In the implementation of the ShardingRuleCondition class, MasterSlaveRuleCondition and EncryptRuleCondition are called to determine if they meet these two SpringBootCondition conditions. Obviously, for ShardingRuleCondition, it should only be loaded when neither of the two conditions is met. The logic for the masterSlaveDataSource and encryptDataSource methods is similar and is not repeated here.

Finally, we notice that SpringBootConfiguration also implements the EnvironmentAware interface in Spring. In Spring Boot, when a class implements the EnvironmentAware interface and overrides the setEnvironment method, it can obtain the property values of various configuration items in the application.properties file when the code project is started. The setEnvironment method overridden in SpringBootConfiguration is shown as follows:

@Override

public final void setEnvironment(final Environment environment) {

        String prefix = "spring.shardingsphere.datasource.";

        for (String each : getDataSourceNames(environment, prefix)) {

            try {

                dataSourceMap.put(each, getDataSource(environment, prefix, each));

            } catch (final ReflectiveOperationException ex) {

                throw new ShardingException("Can't find datasource type!", ex);

            } catch (final NamingException namingEx) {

                throw new ShardingException("Can't find JNDI datasource!", namingEx);

            }

        }

}

The logic here is to obtain the spring.shardingsphere.datasource.name or spring.shardingsphere.datasource.names configuration item, and then use the specified DataSource information in the configuration item to construct a new DataSource and load it into the dataSourceMap LinkedHashMap. We can deepen our understanding by combining the configuration items in the course examples:

spring.shardingsphere.datasource.names=ds0,ds1

spring.shardingsphere.datasource.ds0.type=com.alibaba.druid.pool.DruidDataSource

spring.shardingsphere.datasource.ds0.driver-class-name=com.mysql.jdbc.Driver

spring.shardingsphere.datasource.ds0.url=jdbc:mysql://localhost/ds0

spring.shardingsphere.datasource.ds0.username=root

spring.shardingsphere.datasource.ds0.password=root

spring.shardingsphere.datasource.ds1.type=com.alibaba.druid.pool.DruidDataSource

spring.shardingsphere.datasource.ds1.driver-class-name=com.mysql.jdbc.Driver

spring.shardingsphere.datasource.ds1.url=jdbc:mysql://localhost/ds1

spring.shardingsphere.datasource.ds1.username=root

spring.shardingsphere.datasource.ds1.password=root

So far, the implementation process of SpringBootConfiguration is complete.

From Source Code Analysis to Daily Development #

The implementation method of integrating ShardingSphere with Spring introduced today can be directly imported into daily development processes. If we need to implement a custom framework or utility class, from the perspective of developers, it is best to integrate with mainstream development frameworks such as Spring to provide the lowest learning and maintenance costs. The integration process with Spring frameworks has fixed development steps, and we can follow the content introduced in today’s lesson to implement these steps on our own, following the example of ShardingSphere.

Summary and Preview #

This lesson is the last part of ShardingSphere source code analysis. We have focused on the topic of integrating ShardingSphere with the Spring framework to discuss specific implementation methods of ShardingSphere. ShardingSphere provides a template-like implementation method that can be directly referenced, including namespace-based Spring integration and Spring Boot integration based on starters.

Here’s a question for you to think about: What annotations are added to the SpringBootConfiguration class during the integration of ShardingSphere with Spring Boot, and what are their purposes?

After finishing the discussion on the source code analysis of ShardingSphere, the next lesson is the last one of the entire course. We will summarize and look ahead to the future development of ShardingSphere.