20 Rewriting Engine How to Understand the SQL Rewriting Implementation Mechanism Under the Decorator Pattern

20 Rewriting Engine How to Understand the SQL Rewriting Implementation Mechanism under the Decorator Pattern #

In the lesson “17 | Routing Engine: How to Understand the Operation Mechanism of the ShardingRouter Core Class in ShardingSphere?”, we saw another important concept in ShardingSphere, SQL rewriting, in the Shard method of the BaseShardingEngine.

SQL rewriting usually occurs after routing in a database sharding framework and is also an important step in the entire SQL execution process. This is because developers write SQL statements based on logical databases and tables, which cannot be directly executed in the actual database. SQL rewriting is used to convert logical SQL into SQL statements that can be executed correctly in the actual database.

In fact, we have already seen the application of SQL rewriting in the previous examples, particularly in the process of generating distributed primary keys. In relational databases, auto-increment primary keys are a common feature, and for ShardingSphere, this is a typical use case of SQL rewriting.

Today, we will explore the implementation process of SQL rewriting in ShardingSphere based on the scenario of auto-increment primary keys.

Basic Structure of ShardingSphere Rewrite Engine #

Let’s first take a look at the rewriteAndConvert method in the BaseShardingEngine, which is used to execute the rewriting logic:

private Collection<RouteUnit> rewriteAndConvert(final String sql, final List<Object> parameters, final SQLRouteResult sqlRouteResult) { 
    // Construct SQLRewriteContext 
    SQLRewriteContext sqlRewriteContext = new SQLRewriteContext(metaData.getRelationMetas(), sqlRouteResult.getSqlStatementContext(), sql, parameters); 
    // Decorate SQLRewriteContext with ShardingSQLRewriteContextDecorator 
    new ShardingSQLRewriteContextDecorator(shardingRule, sqlRouteResult).decorate(sqlRewriteContext); 
    // Determine if the query is based on cipher columns 
    boolean isQueryWithCipherColumn = shardingProperties.<Boolean>getValue(ShardingPropertiesConstant.QUERY_WITH_CIPHER_COLUMN); 
    // Decorate SQLRewriteContext with EncryptSQLRewriteContextDecorator 
    new EncryptSQLRewriteContextDecorator(shardingRule.getEncryptRule(), isQueryWithCipherColumn).decorate(sqlRewriteContext); 
    // Generate SQLTokens 
    sqlRewriteContext.generateSQLTokens(); 

    Collection<RouteUnit> result = new LinkedHashSet<>(); 
    for (RoutingUnit each : sqlRouteResult.getRoutingResult().getRoutingUnits()) { 
        // Construct ShardingSQLRewriteEngine 
        ShardingSQLRewriteEngine sqlRewriteEngine = new ShardingSQLRewriteEngine(shardingRule, sqlRouteResult.getShardingConditions(), each); 
        // Execute rewriting 
        SQLRewriteResult sqlRewriteResult = sqlRewriteEngine.rewrite(sqlRewriteContext); 
        // Save the rewriting result 
        result.add(new RouteUnit(each.getDataSourceName(), new SQLUnit(sqlRewriteResult.getSql(), sqlRewriteResult.getParameters()))); 
    } 
    return result; 
}

Although this code is not lengthy, it describes the overall process of implementing SQL rewriting. We have added comments to the core code, and there are many core classes involved, which are worth analyzing in depth. The overall structure of the related core classes is as follows:

image.png

In the entire class diagram, SQLRewriteContext is positioned in the middle, and the rewriting engine SQLRewriteEngine and the decorator SQLRewriteContextDecorator both depend on it.

Next, let’s take a look at this SQLRewriteContext and introduce the basic component of the rewrite engine, SQLToken, based on the scenario of auto-increment primary keys.

Core Classes in the Rewrite Engine from the Perspective of Auto-increment Primary Keys #

1. SQLRewriteContext #

From the name itself, just like SQLStatementContext, SQLRewriteContext is also a context object. Let’s take a look at the variable definitions in SQLRewriteContext:

// RelationMetas for table and column relationship metadata
private final RelationMetas relationMetas;
// SQLStatementContext
private final SQLStatementContext sqlStatementContext;
// Raw SQL
private final String sql;
// Parameters
private final List<Object> parameters;
// List of SQLTokens
private final List<SQLToken> sqlTokens = new LinkedList<>();
// Parameter builder

(Note: This translation may have missed some code segments due to limitations in direct translation. Please refer to the original Markdown text for the complete code.) private final ParameterBuilder parameterBuilder; // SQLToken generator private final SQLTokenGenerators sqlTokenGenerators = new SQLTokenGenerators();

Here, we see the previously mentioned SQLStatementContext, as well as the new SQLToken and SQLTokenGenerators. As today's content evolves, these objects will be introduced one by one. Here, let's first clarify that SQLRewriteContext contains various relevant information for SQL rewriting.

2. SQLToken #

Next, let’s take a look at the SQLToken object. This object plays a crucial role in the rewriting engine, as the SQLRewriteEngine implements SQL rewriting based on SQLToken. The definition of the SQLToken class is as follows:

@RequiredArgsConstructor
@Getter
public abstract class SQLToken implements Comparable<SQLToken> {

    private final int startIndex;

    @Override
    public final int compareTo(final SQLToken sqlToken) {
        return startIndex - sqlToken.getStartIndex();
    }
}

SQLToken is actually an abstract class, and in ShardingSphere, there is a large number of subclasses of SQLToken. Most of these SQLTokens are related to SQL rewriting (the package name of this category contains “rewrite”); while some are also related to the data desensitization feature to be discussed later (the package name of this category also contains “encrypt”).

Data desensitization is also a very useful feature provided by ShardingSphere. We will have a dedicated section to introduce it in the “Module 6: Governance and Integration of ShardingSphere Source Code Analysis”.

At the same time, some SQLTokens are located in the shardingsphere-rewrite-engine project, while others are located in the sharding-core-rewrite project. This point should also be noted.

Considering common scenarios for SQL rewriting, the meaning of many SQLTokens can be directly understood from their literal meaning. For example, for the INSERT statement, if a database-generated primary key is used, there is no need to include the primary key field in the SQL statement. However, the database-generated primary key cannot guarantee the uniqueness of the primary key in a distributed scenario. Therefore, ShardingSphere provides a distributed primary key generation strategy, which can automatically replace the existing database-generated primary key.

As an example, let’s assume that the primary key of the health_record table in our case is record_id, and the original SQL is:

INSERT INTO health_record (user_id, level_id, remark) values (1, 1, "remark1")

As can be seen, the above SQL statement does not include the auto-generated primary key, which will be filled by the database. With the configuration of the auto-generated primary key in ShardingSphere, the SQL will be automatically rewritten as:

INSERT INTO health_record (record_id, user_id, level_id, remark) values ("471698773731577856", 1, 1, "Remark1")

Clearly, the rewritten SQL will add the column name of the primary key and the auto-generated primary key value in the INSERT statement.

From the naming, GeneratedKeyInsertColumnToken corresponds to the scenario of filling the auto-generated primary key. This is actually a common SQL rewriting strategy, which is called column filling. The implementation of GeneratedKeyInsertColumnToken is as follows:

public final class GeneratedKeyInsertColumnToken extends SQLToken implements Attachable {

    private final String column;

    public GeneratedKeyInsertColumnToken(final int startIndex, final String column) {
        super(startIndex);
        this.column = column;
    }

    @Override
    public String toString() {
        return String.format(", %s", column);
    }
}

Note that there is an additional column variable here to specify the column where the primary key is located. Let’s trace the constructor call of GeneratedKeyInsertColumnToken and find that this class is created by GeneratedKeyInsertColumnTokenGenerator. Next, let’s take a look at TokenGenerator.

3. TokenGenerator #

As the name implies, the purpose of TokenGenerator is to generate concrete tokens. The interface is defined as follows:

public interface SQLTokenGenerator {
    // Determine whether to generate SQLToken
    boolean isGenerateSQLToken(SQLStatementContext sqlStatementContext);
}

This interface has two sub-interfaces: OptionalSQLTokenGenerator, responsible for generating a single SQLToken, and CollectionSQLTokenGenerator, responsible for generating multiple SQLTokens:

public interface OptionalSQLTokenGenerator extends SQLTokenGenerator {
    // Generate a single SQLToken
    SQLToken generateSQLToken(SQLStatementContext sqlStatementContext);
}

public interface CollectionSQLTokenGenerator extends SQLTokenGenerator {
    // Generate multiple SQLTokens
    Collection<? extends SQLToken> generateSQLTokens(SQLStatementContext sqlStatementContext);
}

In ShardingSphere, just like SQLToken, the class hierarchy of TokenGenerator is also complex. For GeneratedKeyInsertColumnTokenGenerator, it has an abstract base class, BaseGeneratedKeyTokenGenerator, as shown below:

public abstract class BaseGeneratedKeyTokenGenerator implements OptionalSQLTokenGenerator, SQLRouteResultAware {

    // Determine whether to generate SQLToken
    protected abstract boolean isGenerateSQLToken(InsertStatement insertStatement);

    // Generate SQLToken
    protected abstract SQLToken generateSQLToken(SQLStatementContext sqlStatementContext, GeneratedKey generatedKey);

    ...
}

This abstract class provides two template methods, isGenerateSQLToken and generateSQLToken, to be implemented by subclasses. In GeneratedKeyInsertColumnTokenGenerator, the implementation of these two methods is provided:

public final class GeneratedKeyInsertColumnTokenGenerator extends BaseGeneratedKeyTokenGenerator {

    @Override
    protected boolean isGenerateSQLToken(final InsertStatement insertStatement) {
        Optional<InsertColumnsSegment> sqlSegment = insertStatement.findSQLSegment(InsertColumnsSegment.class);
        return sqlSegment.isPresent() && !sqlSegment.get().getColumns().isEmpty();
    }

    @Override
    protected GeneratedKeyInsertColumnToken generateSQLToken(final SQLStatementContext sqlStatementContext, final GeneratedKey generatedKey) {
        Optional<InsertColumnsSegment> sqlSegment = sqlStatementContext.getSqlStatement().findSQLSegment(InsertColumnsSegment.class);
        Preconditions.checkState(sqlSegment.isPresent());
        // Build GeneratedKeyInsertColumnToken
        return new GeneratedKeyInsertColumnToken(sqlSegment.get().getStopIndex(), generatedKey.getColumnName());
    }
}

In the generateSQLToken method above, by using the InsertColumnsSegment obtained from the SQL parsing engine and the corresponding primary key column obtained from the GeneratedKey used to generate distributed primary keys, we can construct a GeneratedKeyInsertColumnToken.

Decorator SQLRewriteContextDecorator #

Now that we have obtained the SQLToken, let’s go back to SQLRewriteContext mentioned earlier. We know that SQLRewriteContext is a context object, which stores a lot of data information related to SQL rewriting. The construction process of this information will vary depending on different application scenarios. Based on these application scenarios, ShardingSphere’s rewriting engine provides the SQLRewriteContextDecorator interface:

public interface SQLRewriteContextDecorator {

    // Decorate the SQLRewriteContext
    void decorate(SQLRewriteContext sqlRewriteContext);
}

As the name suggests, SQLRewriteContextDecorator is a specific application of the decorator pattern. In ShardingSphere, there are only two specific implementations of SQLRewriteContextDecorator: one is ShardingSQLRewriteContextDecorator used for sharding processing, and the other is EncryptSQLRewriteContextDecorator used for data desensitization. We will discuss EncryptSQLRewriteContextDecorator in detail in “30 | Data Desensitization: How to Implement Low-Intrusion Data Desensitization Scheme Based on Rewrite Engine?”. Today, we focus on the implementation process of the former ShardingSQLRewriteContextDecorator:

public final class ShardingSQLRewriteContextDecorator implements SQLRewriteContextDecorator {

    private final ShardingRule shardingRule;

    private final SQLRouteResult sqlRouteResult;

    @Override
    public void decorate(final SQLRewriteContext sqlRewriteContext) {
        // Parameter rewriting
        for (ParameterRewriter each : new ShardingParameterRewriterBuilder(shardingRule, sqlRouteResult).getParameterRewriters(sqlRewriteContext.getRelationMetas())) {
            if (!sqlRewriteContext.getParameters().isEmpty() && each.isNeedRewrite(sqlRewriteContext.getSqlStatementContext())) {
                each.rewrite(sqlRewriteContext.getParameterBuilder(), sqlRewriteContext.getSqlStatementContext(), sqlRewriteContext.getParameters());
            }
        }

        // SQLTokenGenerators initialization
        sqlRewriteContext.addSQLTokenGenerators(new ShardingTokenGenerateBuilder(shardingRule, sqlRouteResult).getSQLTokenGenerators());
    }
}

This code is not long and contains two parts: parameter rewriting and SQLTokenGenerators initialization. I will explain each part separately:

1. Parameter rewriting #

The parameter rewriting part introduces several new classes. First and foremost is ParameterRewriter and the ParameterRewriterBuilder that constructs it.

(1) ParameterRewriter

Let’s start with the definition of ParameterRewriter:

public interface ParameterRewriter {

    // Determine whether rewriting is needed
    boolean isNeedRewrite(SQLStatementContext sqlStatementContext);

    // Perform parameter rewriting
    void rewrite(ParameterBuilder parameterBuilder, SQLStatementContext sqlStatementContext, List<Object> parameters);
}

Based on the auto-increment primary key feature, take ShardingGeneratedKeyInsertValueParameterRewriter as an example to see how ParameterRewriter is implemented. Its isNeedRewrite method is as follows:

@Override
public boolean isNeedRewrite(final SQLStatementContext sqlStatementContext) {
    return sqlStatementContext instanceof InsertSQLStatementContext && sqlRouteResult.getGeneratedKey().isPresent() && sqlRouteResult.getGeneratedKey().get().isGenerated();
}

Obviously, the input SQL should be an InsertSQLStatement, and this rewriting is only performed when the routing result already contains the GeneratedKey and it is generated. （2）ParameterRewriterBuilder

Before introducing the rewrite method, let’s first understand the concept of ParameterBuilder. ParameterBuilder is a parameter builder:

public interface ParameterBuilder {
    List<Object> getParameters(); 
}

ParameterBuilder has two implementation classes: StandardParameterBuilder and GroupedParameterBuilder. Among them, GroupedParameterBuilder saves a collection of StandardParameterBuilder, and is only applicable to InsertSQLStatement.

After understanding this relationship, let’s take a look at the rewrite method of ShardingGeneratedKeyInsertValueParameterRewriter:

@Override 
public void rewrite(final ParameterBuilder parameterBuilder, final SQLStatementContext sqlStatementContext, final List<Object> parameters) { 
    Preconditions.checkState(sqlRouteResult.getGeneratedKey().isPresent()); 
    ((GroupedParameterBuilder) parameterBuilder).setDerivedColumnName(sqlRouteResult.getGeneratedKey().get().getColumnName()); 
    Iterator<Comparable<?>> generatedValues = sqlRouteResult.getGeneratedKey().get().getGeneratedValues().descendingIterator(); 
    int count = 0; 
    int parametersCount = 0; 
    for (List<Object> each : ((InsertSQLStatementContext) sqlStatementContext).getGroupedParameters()) { 
        parametersCount += ((InsertSQLStatementContext) sqlStatementContext).getInsertValueContexts().get(count).getParametersCount(); 
        Comparable<?> generatedValue = generatedValues.next(); 
        if (!each.isEmpty()) { 
            // Use GroupedParameterBuilder for column supplementation and parameter setting 
            ((GroupedParameterBuilder) parameterBuilder).getParameterBuilders().get(count).addAddedParameters(parametersCount, Lists.<Object>newArrayList(generatedValue)); 
        } 
        count++; 
    } 
}

Because this ParameterRewriter is for InsertSQLStatement, it uses GroupedParameterBuilder here, and obtains the GeneratedKey through SQLRouteResult. We set the DerivedColumnName in GroupedParameterBuilder to the primary key column of GeneratedKey, and add the corresponding index and parameter through a loop, which completes the column supplementation required.

This part of the operation can actually be combined with the process of generating the GeneratedKey to deepen the understanding. The createGeneratedKey method mentioned in the lesson “14 | Distributed Primary Key: What are the distributed primary key implementation approaches in ShardingSphere?” also assigns values to GeneratedKey through a loop.

private static GeneratedKey createGeneratedKey(final ShardingRule shardingRule, final InsertStatement insertStatement, final String generateKeyColumnName) { 
    GeneratedKey result = new GeneratedKey(generateKeyColumnName, true); 
    for (int i = 0; i < insertStatement.getValueListCount(); i++) { 
        result.getGeneratedValues().add(shardingRule.generateKey(insertStatement.getTable().getTableName())); 
    } 
    return result; 
}

2. SQLTokenGenerator Initialization #

In the previous content, we focused on the process of parameter rewriting using ParameterRewriter in ShardingSQLRewriteContextDecorator, which is the first part of the decorate method.

Next, we will continue to explain the second part of this method, which is adding SQLTokenGenerator to SQLRewriteContext:

// Initialize SQLTokenGenerators
sqlRewriteContext.addSQLTokenGenerators(new ShardingTokenGenerateBuilder(shardingRule, sqlRouteResult).getSQLTokenGenerators());

This line of code focuses on the creation of SQLTokenGenerator, so a ShardingTokenGenerateBuilder appears:

public interface SQLTokenGeneratorBuilder { 
    // Get the list of SQLTokenGenerator 
    Collection<SQLTokenGenerator> getSQLTokenGenerators(); 
}

In the implementation class ShardingTokenGenerateBuilder of SQLTokenGeneratorBuilder, many TokenGenerators are built-in, including the GeneratedKeyInsertColumnTokenGenerator mentioned earlier:

private Collection<SQLTokenGenerator> buildSQLTokenGenerators() { 
    Collection<SQLTokenGenerator> result = new LinkedList<>(); 
    addSQLTokenGenerator(result, new TableTokenGenerator()); 
    … 
    addSQLTokenGenerator(result, new OffsetTokenGenerator()); 
    addSQLTokenGenerator(result, new RowCountTokenGenerator()); 
    addSQLTokenGenerator(result, new GeneratedKeyInsertColumnTokenGenerator()); 
    … 
    return result; 
}

SQL Rewrite Engine SQLRewriteEngine #

In ShardingSphere, the SQLRewriteEngine interface represents the entry point of the rewrite engine:

public interface SQLRewriteEngine { 
    // Execute SQL rewrite based on SQLRewriteContext 
    SQLRewriteResult rewrite(SQLRewriteContext sqlRewriteContext); 
}

The SQLRewriteEngine interface has only one method, which returns an SQLRewriteResult object based on the input SQLRewriteContext. As we have learned from the previous introduction, the SQLRewriteContext can be decorated by decorator classes to meet different needs in different scenarios.

Note that the SQLRewriteEngine interface has only two implementation classes: DefaultSQLRewriteEngine and ShardingSQLRewriteEngine. We focus on ShardingSQLRewriteEngine, but before introducing this rewrite engine class, we need to introduce the SQLBuilder interface first. From the definition, it can be seen that the purpose of SQLBuilder is to build executable SQL statements:

public interface SQLBuilder { 
    // Generate SQL 
    String toSQL(); 
}

The SQLBuilder interface has an abstract implementation class AbstractSQLBuilder, and its toSQL method is as follows:

@Override 
public final String toSQL() { 
    if (context.getSqlTokens().isEmpty()) { 
        return context.getSql(); 
    } 
    Collections.sort(context.getSqlTokens()); 
    StringBuilder result = new StringBuilder(); 
    result.append(context.getSql().substring(0, context.getSqlTokens().get(0).getStartIndex())); 
    // Assemble the target SQL based on SQLToken 
    for (SQLToken each : context.getSqlTokens()) { 
        result.append(getSQLTokenText(each)); 
        result.append(getConjunctionText(each)); 
    } 
    return result.toString(); 
}

It can be seen that if the sqlTokens of SQLRewriteContext are empty, it directly returns the final SQL saved in SQLRewriteContext; otherwise, it constructs a StringBuilder to store the SQL, then adds each SQLTokenText and conjunctionText one by one to assemble a complete SQL statement. Please note that the method of obtaining SQLTokenText here is a template method and needs to be implemented by the subclasses of AbstractSQLBuilder:

// Get the SQLToken text 
protected abstract String getSQLTokenText(SQLToken sqlToken);

As an implementation class of AbstractSQLBuilder, ShardingSQLBuilder’s getSQLTokenText method includes some scenarios for SQL rewriting as well.

    @Override 
    protected String getSQLTokenText(final SQLToken sqlToken) { 
        if (sqlToken instanceof RoutingUnitAware) { 
            return ((RoutingUnitAware) sqlToken).toString(routingUnit); 
        } 
        if (sqlToken instanceof LogicAndActualTablesAware) { 
            return ((LogicAndActualTablesAware) sqlToken).toString(getLogicAndActualTables()); 
        } 
        return sqlToken.toString(); 
    }

For the input SQLToken, there are two special handling cases here: checking if it implements the RoutingUnitAware interface or the LogicAndActualTablesAware interface. We found that only the ShardingInsertValuesToken implements the RoutingUnitAware interface, while the IndexToken and TableToken implement the LogicAndActualTablesAware interface.

Let’s take the TableToken as an example to discuss table name rewriting. It is the process of rewriting logical table names to actual table names, which is a typical scenario where SQL rewriting is needed. Let’s consider the simplest table name rewriting scenario. If the logical SQL is:

SELECT user_name FROM user WHERE user_id = 1;

Here, the logical table name is user. Assuming we have configured the sharding key user_id, and for the case where user_id = 1, the routing will go to the sharding table user_1. After rewriting, the SQL should be:

SELECT user_name FROM user_1 WHERE user_id = 1;

As we can see, the actual table name here should be user_1, not user. In the TableToken used for rewriting table names, its toString method is as follows:

    @Override 
    public String toString(final Map<String, String> logicAndActualTables) { 
        String actualTableName = logicAndActualTables.get(tableName.toLowerCase()); 
        actualTableName = null == actualTableName ? tableName.toLowerCase() : actualTableName; 
        return Joiner.on("").join(quoteCharacter.getStartDelimiter(), actualTableName, quoteCharacter.getEndDelimiter()); 
    }

The logic is not complicated here. It simply retrieves the actual table name actualTableName from logicAndActualTables based on the logical table name, and then concatenates it into a string. So where does this logicAndActualTables come from? The construction process of logicAndActualTables is in ShardingSQLBuilder:

    private Map<String, String> getLogicAndActualTables() { 
        Map<String, String> result = new HashMap<>(); 
        Collection<String> tableNames = getContext().getSqlStatementContext().getTablesContext().getTableNames(); 
        for (TableUnit each : routingUnit.getTableUnits()) { 
            String logicTableName = each.getLogicTableName().toLowerCase(); 
            result.put(logicTableName, each.getActualTableName()); 
                result.putAll(getLogicAndActualTablesFromBindingTable(routingUnit.getMasterSlaveLogicDataSourceName(), each, tableNames)); 
        } 
        return result; 
    }

The above code is actually only involved in constructing the data structure. If we continue to look at the getLogicAndActualTablesFromBindingTable method, we will find that the process of obtaining the actual table based on the logic table actually happens in BindingTableRule:

    public String getBindingActualTable(final String dataSource, final String logicTable, final String otherActualTable) { 
            int index = -1; 
            for (TableRule each : tableRules) { 
                index = each.findActualTableIndex(dataSource, otherActualTable); 
                if (-1 != index) { 
                    break; 
                } 
            } 
            if (-1 == index) { 
                throw new ShardingConfigurationException("Actual table [%s].[%s] is not in table config", dataSource, otherActualTable); 
            }

for (TableRule each : tableRules) { 
    if (each.getLogicTable().equals(logicTable.toLowerCase())) { 
        return each.getActualDataNodes().get(index).getTableName().toLowerCase(); 
    } 
} 
throw new ShardingConfigurationException("Cannot find binding actual table, data source: %s, logic table: %s, other actual table: %s", dataSource, logicTable, otherActualTable); 
}

And the BindingTableRule depends on the ActualDataNodes stored in the TableRule to calculate the ActualTableIndex and ActualTable. We can again feel the connection between various Rule objects represented by TableRule and BindingTableRule:

image

Once the ShardingSQLBuilder completes the construction of the SQL, let’s go back to ShardingSQLRewriteEngine. At this point, the purpose of its rewrite method becomes clear:

@Override 
public SQLRewriteResult rewrite(final SQLRewriteContext sqlRewriteContext) { 
    return new SQLRewriteResult(new ShardingSQLBuilder(sqlRewriteContext, shardingRule, routingUnit).toSQL(), getParameters(sqlRewriteContext.getParameterBuilder())); 
}

The output SQLRewriteResult object of the rewrite engine now contains the final SQL and the corresponding parameter list:

public final class SQLRewriteResult {
    private final String sql;
    private final List<Object> parameters; 
}

After discussing ShardingSQLRewriteEngine, let’s return to the rewriteAndConvert method of BaseShardingEngine. Except for the part of EncryptSQLRewriteContextDecorator, we should understand the overall execution process of the method. The method ultimately returns a list of RouteUnit objects, and each RouteUnit contains an SQLUnit:

public final class RouteUnit { 
    // target datasource name
    private final String dataSourceName; 
    // SQL unit
    private final SQLUnit sqlUnit; 
} 
    
public final class SQLUnit { 
    // target SQL
    private final String sql; 
    // parameter list
    private final List<Object> parameters; 
}

As we can see, the final result actually includes the target database, target SQL, and related parameters. Once we have obtained this information, we can execute the SQL statement.

From Source Code Analysis to Daily Development #

In today’s content, we can clearly feel the power of the Decorator pattern. The Decorator pattern allows adding new functionality to an existing object without changing its structure. It creates a decorator class that wraps the original class and provides additional functionality while maintaining the method signature integrity.

At the same time, we noticed that in ShardingSphere, the Decorator pattern is applied to an SQLRewriteContext object, which is a valuable practice to learn. In daily development, we can store information that needs to be handled differently according to different scenarios in a context object, and then use the Decorator pattern to decorate this information. The seamless integration between the two can complete functions that cannot be accomplished by subclass implementation alone in many application scenarios, thus dynamically adding additional responsibilities to objects.

Summary and Outlook #

In today’s lesson, we spent a lesson to introduce the basic structure and core classes of the rewriting engine in ShardingSphere. The rewriting engine adopts the Decorator pattern in design, and completes the process of rewriting from logical SQL to target SQL. We also give the implementation principles and source code analysis for two typical application scenarios, including auto-increment primary key and table name rewriting.

Please note that the rewriting engine in ShardingSphere is not only used for these scenarios. In the course “30 | Data Desensitization: How to Implement a Low-intrusive Data Desensitization Solution Based on the Rewriting Engine?”, we will see its application in data desensitization and other scenarios.

Finally, here is a question for you to think about: how does ShardingSphere use the Decorator pattern to decorate the context object of SQL rewriting? Feel free to discuss with everyone in the comments, and I will give comments and answers one by one.

Now that we have obtained the target SQL based on the input logical SQL through the rewriting engine, we can proceed to execute the SQL, which will be the focus of the next lesson on the ShardingSphere execution engine.