17 Demo of Microservices Design Based on Ddd Including a Ddd Technical Backend That Supports Microservices

17 Demo of Microservices Design based on DDD including a DDD Technical Backend that supports Microservices #

In the previous lesson, we explained the code design approach based on DDD. In this lesson, we will continue to discuss the design and development approach of the DDD technical middle platform that supports microservices.

Data Query Using Single Service #

Previously, we discussed how to implement database CRUD operations using a single DAO. However, when it comes to querying data, we have to do the opposite by injecting different DAOs into a single service to perform various queries. This design was forced upon me in my past projects.

A few years ago, I organized a big data team to start designing and developing big data-related products. As we all know, big data-related products involve using big data technologies to analyze and process massive amounts of data, and the ultimate result is to query and present various reports. Therefore, besides various backend analysis and processing, such projects also require presenting numerous and complex reports on the frontend, sometimes even hundreds of them. Furthermore, the users of this system are decision-makers who constantly analyze data in different ways, and each requirement is usually urgent and they want to start using it immediately. Therefore, we needed the ability to develop reports quickly, and traditional approaches of creating reports one by one from scratch simply couldn’t keep up.

Through analyzing existing reports repeatedly, extracting commonalities, and retaining individuality, I found that each report had many similarities or similarities. The code for each report in the service was almost the same, with the only difference being getting query parameters from the frontend and then invoking the DAO for queries, with at most some pagination operations. In that case, why bother designing a service for each functionality? Merge them into one service, inject different DAOs, and then you can perform different queries, right?

Then, how should these DAOs be designed? In the past, MyBatis was used in a way that each DAO had to write its own interface and configure a mapper. However, these DAO interfaces were all identical, with the only difference being the interface name and mapper. Moreover, in the previous design, each service corresponded to one DAO. Now that one service has to correspond to many DAOs, annotation-based configuration alone is not enough. To solve these design challenges, after repeated debugging, I designed the architecture like this.

First, there is only one QueryService for the entire system, which has a QueryDao that can inject different DAOs. However, we don’t need to write an interface for each DAO, as it would be too cumbersome. By injecting different mappers into a QueryDaoImpl, we can create different DAOs, and then assemble the service to be able to be registered as different beans in Spring, allowing for different queries:

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans" ...>

 <description>The application context for query</description>

 <bean id="customerQry" class="com.demo2.support.service.impl.QueryServiceImpl">

  <property name="queryDao">

   <bean class="com.demo2.support.dao.impl.QueryDaoMybatisImpl">

    <property name="sqlMapper" value="com.demo2.trade.query.dao.CustomerMapper"></property>

   </bean>

  </property>

 </bean>

</beans>

In the code, we use XML format and define an applicationContext-qry.xml file. In the bean named customerQry, the class is QueryServiceImpl, and it injects QueryDaoMybatisImpl, but with different sqlMapper configurations, allowing for different queries.

This mapper is the MyBatis mapper, which is placed in the classpath’s mapper directory (as detailed in MyBatis design and development), and its contents are written in the following format:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN"

"http://mybatis.org/dtd/mybatis-3-mapper.dtd">

<mapper namespace="com.demo2.trade.query.dao.CustomerMapper">

 <sql id="select">

  SELECT * FROM Customer WHERE 1 = 1

 </sql>

 <!-- Query conditions section -->

 <sql id="conditions">

  <if test="id != '' and id != null">

   and id = #{id}

  </if>

 </sql>

 <!-- Pagination query section -->

 <sql id="isPage">

  <if test="size != null  and size !=''">

   limit #{size} offset #{firstRow} 

  </if>

 </sql>
 <select id="query" parameterType="java.util.HashMap" resultType="com.demo2.trade.entity.Customer">

      <include refid="select"/>

  <include refid="conditions"/>

  <include refid="isPage"/>

 </select>

 <!-- Count calculation section -->

 <select id="count" parameterType="java.util.HashMap" resultType="java.lang.Long">

  select count(*) from (

   <include refid="select"/>

   <include refid="conditions"/>

  ) count

 </select>

 <!-- Sum calculation section -->

 <select id="aggregate" parameterType="java.util.HashMap" resultType="java.util.HashMap">

  select ${aggregation} from (

   <include refid="select"/>

   <include refid="conditions"/>

  ) aggregation

 </select>

</mapper>

In this configuration, we usually only need to modify the Select and Condition sections.

  • The Select section is the query statement, usually a single-table query instead of using joins to other tables, as it would have poor performance with a large amount of data. Also, the WHERE 1 = 1 part at the end is necessary to avoid errors when there are no query conditions.
  • The Condition section is for query conditions, where a parameter is added if it exists, and removed if not.

Furthermore, the system may execute this mapper multiple times in the backend: once for pagination queries, once for count calculation, and once for sum calculation. Regardless of how many times it is executed, the Select and Condition sections only need to be written once, reducing development effort and avoiding errors.

Ideally, these queries should be single-table queries. But what if we need joins? It is best to use data supplementation, which involves performing supplementation on a page of data based on single-table queries and pagination. If supplementation is required, the QueryService can be configured as follows:

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans" ...>

 <description>The application context for query</description>

 <bean id="productQry" class="com.demo2.support.repository.AutofillQueryServiceImpl">

  <property name="queryDao">

   <bean class="com.demo2.support.dao.impl.QueryDaoMybatisImpl">

    <property name="sqlMapper" value="com.demo2.trade.query.dao.ProductMapper"></property>

   </bean>

  </property>

  <property name="dao" ref="basicDao"></property>

 </bean>

</beans>

In this example, productQry was originally configured as QueryServiceImpl, but it has been replaced with AutofillQueryServiceImpl. In addition, the following configuration was made in vObj.xml:

<?xml version="1.0" encoding="UTF-8"?>

<vobjs>

  <vo class="com.demo2.trade.entity.Product" tableName="Product">

    <property name="id" column="id" isPrimaryKey="true"></property>

    <property name="name" column="name"></property>

    <property name="price" column="price"></property>

    <property name="unit" column="unit"></property>

    <property name="classify" column="classify"></property>

    <property name="supplier_id" column="supplier_id"></property>

    <join name="supplier" joinKey="supplier_id" joinType="manyToOne" class="com.demo2.trade.entity.Supplier"></join>

  </vo>

  <vo class="com.demo2.trade.entity.Supplier" tableName="Supplier">

    <property name="id" column="id" isPrimaryKey="true"></property>

    <property name="name" column="name"></property>

  </vo>

</vobjs>

In the configuration, you can see that in the configuration of Product, a join tag was added, configured with Supplier, and the query for Supplier was also configured. In this way, after completing the query for Product, if a join tag is found, the system will perform a batch query for suppliers based on the configuration of Supplier, and automatically fill in the Product information. Therefore, once the join is configured, the subsequent configuration of Supplier is necessary in order to query supplier data from the database and complete the data filling.

With this design, business developers do not need to implement tedious code for data filling. They only need to configure it during modeling. If filling is required, configure AutofillQueryServiceImpl; if not, configure QueryServiceImpl. The entire system can become flexible and easy to maintain. It is important to note that if AutofillQueryServiceImpl is configured, not only queryDao needs to be configured, but basicDao as well. This is because during data filling, basicDao is used with load() or loadForList().

Support for Microservice with Data Filling #

From the above case, it can be seen that the data of the Product table must be filled with the Supplier in the same database. This is fine in a monolithic application, but not in a microservices architecture. Microservices not only split the application, but also split the database. When the Product microservice needs to fill in the Supplier, it does not have permission to read the database where Supplier is located. It can only remotely call the corresponding interface of the Supplier microservice. In this way, the data filling cannot be completed through the above configuration alone, and the technology platform must provide corresponding support for the microservices.

In a microservices system, the need for data filling through remote interfaces is very common in DDD-based design and development. Therefore, the technology platform must provide support for this situation. To do this, I have provided the ref tag based on the join tag. Assuming that the system splits Product and Supplier into two microservices through microservice splitting. At this time, the following configuration needs to be done in the vObj.xml file of the Product microservice:

<?xml version="1.0" encoding="UTF-8"?>

<vobjs>

  <vo class="com.demo2.product.entity.Product" tableName="Product">

    <property name="id" column="id" isPrimaryKey="true"></property>

    <property name="name" column="name"></property>

    <property name="price" column="price"></property>

    <property name="unit" column="unit"></property>

    <property name="classify" column="classify"></property>

    <property name="supplier_id" column="supplier_id"></property>

    <ref name="supplier" refKey="supplier_id" refType="manyToOne" 
      bean="com.demo2.product.service.SupplierService" method="loadSupplier" listMethod="loadSuppliers">

    </ref>

  </vo>

</vobjs>

The above configuration changes the join tag to ref tag. In the ref tag, bean is the Feign interface for making remote calls to the Supplier microservice from the Product microservice (see Lesson 15 for details). At this point, the Supplier microservice needs to provide 2 query interfaces:

  • Supplier loadSupplier(id): Searches for a supplier by ID;
  • List loadSupplier(List ids): Searches for multiple suppliers by IDs.

Here, the method configuration is for searching for a single ID, while the listMethod configuration is for batch searching for multiple IDs. With these 2 configurations, the Feign interface can be used to implement remote calls between microservices, achieving data integration across microservices. With this design, there is no need to configure the Supplier in the vObj.xml of the Product microservice.

Design of Generic Repository and Factory #

Before adopting DDD, in system design, each service directly injected and used DAO to operate on the database. However, in DDD architecture design, repository and factory are introduced, which not only handle reading and writing to the database, but also mapping and assembling domain objects. So what is the relationship between the repository and factory in DDD and the traditional DAO? How should they be designed?

In traditional DDD design, each module has its own repository and factory. The factory is responsible for creating and assembling domain objects, and is the starting point of their lifecycle. After the domain objects are created, they are placed in the cache of the repository for access by the upper-layer application. After a series of operations on the domain objects, the repository is used to persist the data. For ordinary domain objects, this data persistence process means storing them in a single table. However, for domain objects with aggregate relationships, they need to be stored in multiple tables and placed in the same transaction.

In this process, will there be transactional operation across databases for aggregate relationships? In other words, will multiple domain objects with aggregate relationships be split into multiple microservices? I believe this is not possible, because an aggregate is a form of strongly encapsulated relationship that cannot be split by microservices. If this happens, either it is not an aggregate relationship, or there is a problem with the microservice design. Therefore, it is impossible for the repository to handle transactions across databases.

By understanding traditional DDD design and comparing it with the design of DAO, it becomes clear that the repository and factory are replacements for DAO. However, this replacement is not a simple one-to-one replacement. The repository and factory extend many functions of the DAO while replacing it, such as data integration, mapping and assembly of domain objects, and handling of aggregates, etc. When we have thought through these relationships, the design of a generic repository and factory emerges.

Lark20210108-153942.png

As shown in the above figure, the repository is a DAO that implements the BasicDao interface. However, when the repository reads from or writes to the database, does it copy the BasicDao implementation class code again? No! Doing so would only result in a large amount of duplicate code, which is not conducive to future changes and maintenance. Therefore, the repository encapsulates the BasicDao in a property variable. When the repository needs to read from or write to the database, it actually calls the implementation class of BasicDao, and the repository only implements the additional functions based on the implementation class of BasicDao. This way, the responsibilities and boundaries between the repository and the implementation class of BasicDao are clearly defined.

With this design, to transform a legacy system using the DDD approach, in addition to adding vObj.xml through domain modeling, the injection of DAO is replaced with the injection of the repository, enabling a rapid transformation to domain-driven development. Similarly, to add caching functionality to the repository, instead of directly modifying the repository, a RepositoryWithCache is created based on the repository to solely implement the caching functionality. With this design, the division of responsibilities between classes is very clear, and future changes can be made to the corresponding class based on the reasons for the change, enabling a loosely coupled system that can meet various requirements through component assembly.

Summary #

In this lesson, I have provided you with a practical approach for implementing a technical middle platform. This platform is different from traditional DDD architecture as it eliminates some tedious designs such as TDO, PO, repository, and factory, and encapsulates them in the underlying technical framework. This allows business development personnel to focus more on business modeling and the design and development process based on business modeling. With reduced development workload, it enables quick delivery and makes future changes and maintenance easier, making it possible to design our products better and provide a better user experience that can adapt to changes in the domain model.

In the next lesson, we will discuss event-driven design and implementation in microservices.

Click here for the GitHub link to view the source code.