13 How to Realize a Technical Backend Design That Supports Fast Payments

13 How to realize a technical backend design that supports fast payments #

We mentioned the concept of “front end” earlier, which means that the trend of software team organization is “front end + technical platform”, in order to improve market competitiveness through fast delivery. The so-called “front end + technical platform” means that there is an architecture support team in the development team, which encapsulates many technical architectures in the software development process through a powerful technical platform. With such a technical platform, other development teams can develop their businesses based on it.

This approach can reduce the workload of business development, improve development speed, and lower the technical threshold. Business developers don’t have to focus too much on technology, but can concentrate more on understanding the business and incorporate their deep understanding of the business into the domain modeling process, thus developing software that meets the users’ needs and improving user experience.

Therefore, how to create a powerful and practical technical platform has become an urgent need for various software development teams. Now let’s take a look at how to implement these design ideas in the construction of a technical platform from a practical perspective.

Command and Query Responsibility Segregation (CQRS) is an architectural design pattern proposed by software guru Martin Fowler in his book “Patterns of Enterprise Application Architecture”. This pattern divides the system into command (i.e., create, update, and delete operations) and query parts based on responsibilities.

  • All the create, update, and delete operations in the command part should be based on domain-driven design principles to achieve better scalability in large, complex applications;
  • All query functions, on the other hand, should not use domain-driven design, but instead adopt the transaction script pattern, which means performing queries directly through SQL statements.

Following this design pattern is the best practice we have summarized in many software projects. Therefore, when constructing a technical platform, support for business systems should also be divided into two parts: command and query.

Drawing 0.png

Architectural Design for Command Operations #

Drawing 1.png

Architectural design for command operations in the technical platform

In the command part, the architecture design of a single controller and a single DAO mentioned earlier is employed. As shown in the above diagram, each feature has its own front-end UI. However, unlike previous architectures, when the front-end UI of each feature makes a request to the backend, it no longer calls its own controller, but instead calls a unified controller. However, when the front-end UI of each feature calls this controller, it passes different parameters. First, the front-end passes a bean, but what is this bean? Each feature in the backend has a service, which when injected into the DAO, will be configured as a bean in the Spring framework. At this point, the front-end only knows that it is calling this bean, but it doesn’t know which service it is.

This design ensures both security (the front-end does not know the specific class) and effectively achieves front-end and back-end separation, decoupling the front-end code from the back-end.

Next, the front-end also needs to pass a method, that is, which method to call, and which JSON object. This allows the controller to perform the corresponding operation through reflection. The design philosophy here is that, in the software development process, by specifying rules and contracts, we assume that the front-end developers already know which bean, which method, and what JSON format they need to call on the backend, which greatly simplifies the design of the technical platform.

Design of a Single Controller #

All the create, update, and delete operations of the front-end functions, as well as the get/load operations based on an ID, are accessed through the OrmController.

When accessing the OrmController, the front-end provides the following HTTP request:

http://localhost:9003/orm/{bean}/{method}

For example:

  • GET request

http://localhost:9003/orm/product/deleteProduct?id=P00006

  • POST request

http://localhost:9003/orm/product/saveProduct-d”id=P00006&name=ThinkPad+T220&price=4600&unit=%E4%B8%AA&supplierId=20002&classify=%E5%8A%9E%E5%85%AC%E7%94%A8%E5%93%81”

Here, { bean } refers to the bean.id configured in Spring, and { method } is the method that needs to be called in that bean (note that method overriding is not supported here; if there are multiple methods with the same name, the last method will be called).

  • If the method to be called has a value object, according to the specification, the value object must be placed as the first parameter of the method.
  • If the method to be called has both a value object and other parameters, both the value object properties and the other parameters should be included in the same JSON object. For example, if the method to be called is saveProduct(product, saveMode), the POST request would be:

http://localhost:9003/orm/product/saveProduct -d “id=500006&name=ThinkPad+T220&price=4600&unit=%E4%B8%AA&supplierId=20002&classify=%E5%8A%9E%E5%85%AC%E7%94%A8%E5%93%81&saveMode=1”

It’s important to note that currently the OrmController does not have any permission verification, so all the methods of beans configured in Spring can be called by the front-end. Therefore, in a real project, it is necessary to perform permission verification before the OrmController to regulate the methods that the front-end can call. It is recommended to use a service gateway or a filter for verification.

The flow design of the OrmController is as follows:

  • Get the Service from Spring based on the front-end parameter bean;
  • Use reflection to get the method to be called based on the front-end parameter method;
  • Use reflection to get the first parameter of the calling method as the value object;
  • Create the value object through reflection, get all the properties of the value object through reflection, retrieve the corresponding values from the front-end JSON, and write them to the value object;
  • Get other parameters based on the front-end JSON;
  • Use reflection to invoke the method in the Service with the value object and other parameters.

Design of Single Dao #

When the system performs a series of business operations in the Service and needs to persist data in the end, a single Dao is called consistently. However, before calling the single Dao, each value object should be configured through vObj.xml. In this configuration, the table corresponding to each value object and the field corresponding to each property in the value object are specified using the vObj.xml configuration file. This allows the generic BasicDao to generate SQL statements based on the configuration file, ultimately completing the database persistence operation.

The vObj.xml configuration file is as follows:

<?xml version="1.0" encoding="UTF-8"?>

<vobjs>
  <vo class="com.demo2.customer.entity.Customer" tableName="Customer">
    <property name="id" column="id" isPrimaryKey="true"></property>
    <property name="name" column="name"></property>
    <property name="sex" column="sex"></property>

    <mapper namespace="com.demo2...query.CustomerQry">
     <resultMap id="customerQryResult" type="com.demo2...vo.Customer">
      <!--配置查询返回的结果的字段-->
     </resultMap>
    <!--根据条件查询--}}
     <select id="query" resultMap="customerQryResult">
      select * from customer
     </select>
    </mapper>

查询的实现类 CustomerQueryDao 如下：

@Repository
public class CustomerQueryDao extends BasicDao<CustomerQry> 
                 implements CustomerQueryDao 
{ 
} 

以上配置是采用的是接口继承的方式，配置了查询接口。

配置完，接下来就是查询的实现。但是仍然是先使用 vObj 获取类变量名，再根据类变量名获取要查询的表名。之后，就是利用 SQL 语句进行查询操作了。

查询后，之前的单 Service 的流程设计就可以调动了。

首先会执行 List rs = dao.query(paramMap) 进行查询。

这样，单 Service 的设计如下：

public List<Map> query(Map paramMap) 
{ 
    if (!primaryKey.equals("")) 
    { 
        Map map = new HashMap<>(); 
        map.put(primaryKey, paramMap.get(primaryKey)); 
        Map returnMap = get(map); 
        for (Map.Entry<String, Object> entry : returnMap.entrySet()) 
        { 
            paramMap.put(entry.getKey(), entry.getValue()); 
        } 
    } 
    return dao.query(paramMap); 
}

为什么接下来会需要前端判断结果集的大小呢？

因为如果结果集大小不为 0，就可以进行详细查询了，接着通过各种数据库查询语句，获得更多的信息。

这样，单 Controller 的流程设计就可以调动了。

这时，根据前端请求的次数和后端返回的 count，就可以计算多次分页。

以上就是查询功能的技术中台设计。

整个技术中台设计完成后，即可通过代码生成的方式，将技术中台设计生成代码。到这一步，参与了技术中台的所有角色已经进入到编码的阶段。

新块风控设计 前面介绍了业务功能耦合的实现技术。

我们发现最终耦合的对象都是贷款申请表、贷款申请服务，以及风控审批。

现阶段很多机构可能只需要贷款审批的风控审批功能来实现即可。

因此我们先介绍贷款审批的单风控设计吧。

贷款审批的设计如图所示：

再次申请贷款时，前端输入 HTTP 请求。

贷款申请的单风控设计流程：

数据输入 > ML (Machine Learning) 风控模型 > 审批平台 > 审批动态管理 > 结果通知标准服务 > 发审批邮件 > 数据库管理。

接下来介绍这十个环节各自的功能设计。

数据输入功能设计 数据输入直接关了融资申请的功能。

接下来介绍贷款审批的风控模型设计，后端的技术中台设计感觉比较重要。

由于贷款审批是一个比较复杂的过程，这里分几部分进行介绍。

1.数据输入 等待获得用户的输入，此处暂停。

2.模型训练 将大量的和贷款有关的同类型数据视为"样本"，采用监督学习方法，进行机器学习。 即构建样本的训练集，进行机器学习训练。

3.模型识别 将模型运用到前面大量数据进行建模，预测结果。

4.结果展示 将预测结果以易读的形式展示给用户，以方便用户对其进行判断。

模型文件存储在贷款审批模块中的文件夹下。你可以采用Hadoop平台的存储和机器学习平台中的模型训练模块进行训练和测试。在通用服务模块中，增加唯一可以获得模型训练结果的接口，并从贷款申请中查询模型训练结果。

这样就完成了模型的持久化。

同时，还可以生成模型权值文件和推理文件，并将文件存储在文件存储系统中。

<mapper namespace="com.demo2.customer.query.dao.CustomerMapper">
 
 <!--筛选条件-->
 <sql id="searchParam">
  <if test="id != '' and id != null">
   and id = #{id}
  </if>
 </sql>
 <!--求count判断-->
 <sql id="isCount1">
  <if test="count == null  and notCount ==1">
   select count(*) from (
  </if>
 </sql>
 <sql id="isCount2">
  <if test="count == null  and notCount ==1">
   ) count
  </if>
 </sql> 
 <!--是否分页判断-->
 <sql id="isPage">
  <if test="size != null  and size !=''">
   limit #{size} offset #{firstRow} 
  </if>
  <if test="size ==null  or size ==''">
   <if test="pageSize != null  and pageSize !=''">
    limit #{pageSize} offset #{startNum} 
   </if>
  </if>
 </sql>

 <select id="query" parameterType="java.util.HashMap" resultType="com.demo2.customer.entity.Customer">
     <include refid="isCount1"/>
        SELECT * FROM Customer WHERE 1 = 1
    <include refid="searchParam"/>
    <include refid="isPage"/>
     <include refid="isCount2"/>
 </select>
</mapper>

Then, inject it into Spring and complete the corresponding configuration, and you can start querying:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" ...>
 <description>The application context for query</description>
 <bean id="customerQry" class="com.demo2.support.service.impl.QueryServiceImpl">
  <property name="queryDao">
   <bean class="com.demo2.support.dao.impl.QueryDaoMybatisImpl">
    <property name="sqlMapper" value="com.demo2.customer.query.dao.CustomerMapper.query"></property>
   </bean>
  </property>
 </bean>
</beans>

Each query bean is configured with QueryServiceImpl, but each bean is configured with a different sqlMapper, so different queries will be executed. The sqlMapper here should correspond to the namespace in the MyBatis configuration.

In this way, the single service flow design for querying is as follows:

  • Pass the query parameters map, page, and size to the Dao, and execute the query dao.query(map);
  • Add empty methods beforeQuery() and afterQuery() as hooks before and after the query. When a certain business needs to be processed before or after the query, override these methods in the subclass to implement them;
  • If the count is passed from the front-end, there is no need to calculate the count, otherwise call dao.count() to calculate the “xth page, y pages”;
  • Pack the data into a ResultSet object and return it.

Usually, when executing a query, it is sufficient to execute dao.query(map). Since different beans inject different Daos, executing dao.query(map) will execute different queries. However, in certain businesses, it is necessary to perform certain processing before or after the query, such as converting certain query parameters or converting and supplementing the query results. How can we achieve this with the current design?

First, in QueryService, the beforeQuery() and afterQuery() methods are added. However, these two methods are designed as empty methods in QueryService, with nothing written. Therefore, calling them is the same as not calling them. This design is called “hook”, as shown in the following code:

 /**
  * do something before query. 
  * It just a hook that override the function in subclass if we need do something before query.
  * @param params the parameters the query need
  */
 protected void beforeQuery(Map<String, Object> params) {
  //just a hood
 }
 /**

     * do something after query.
     *
     * It's just a hook that overrides the function in the subclass if we need to do something after the query.
     *
     * @param params the parameters needed for the query
     *
     * @param resultSet the result set after the query.
     *
     * @return
     */
    
     protected ResultSet afterQuery(Map<String, Object> params, ResultSet resultSet) {
    
      //just a hook
    
      return resultSet;
    
     }

This way, if there is no need to add processing before or after the query, you can simply configure the QueryService. When executing the query, it will be as if these two methods do not exist. However, if you need to add some processing before or after the query, you can inherit and write a subclass of QueryService and override beforeQuery() or afterQuery(). In the Spring configuration, configure the subclass, and the processing before and after the query will be implemented.

For example, the ProductQuery requires adding supplier information to the query result set after the query. In this case, create a subclass ProductQueryServiceImpl by inheriting QueryServiceImpl and override afterQuery().

public class ProductQueryServiceImpl extends QueryServiceImpl {

@Autowired

private SupplierService supplierService;

@Override

protected ResultSet afterQuery(Map<String, Object> params,

ResultSet resultSet) {

@SuppressWarnings("unchecked")

List<Product> list = (List<Product>)resultSet.getData();

for(Product product : list) {

String supplierId = product.getSupplierId();

Supplier supplier = supplierService.loadSupplier(supplierId);

product.setSupplier(supplier);

}

resultSet.setData(list);

return resultSet;

}

}

Finally, the query result is returned to the Controller as a ResultSet value object, and the Controller returns it to the front end. In this ResultSet:

  • The data property is the query result set for this page;
  • page and size are pagination information;
  • count is the total number of records.

Using these 3 values, it is possible to display “Page x of y, z records” in the front end. On the first query, in addition to querying the data for this page, the count is also executed. Once the count is recorded, it is no longer necessary to execute count for subsequent pagination queries, thus effectively improving query performance.

The aggregate property is a map. If the query needs to aggregate certain fields at the bottom of a table in the front end, and this aggregation is for the entire query result set, not just for this page, then add the field as the key in aggregate, and the value is the aggregation method, such as count, sum, max, etc. By setting it this way, a summary record can be returned at the last row of the query result set.

Through the design of this technology platform, the coding of various query functions can be greatly simplified. Specifically, designing a normal query only requires creating a MyBatis query statement configuration and creating a bean in the Spring configuration. After that, the query can be performed through the front end, without even having to write any classes. Only when operations need to be added before or after the query, a subclass needs to be created.

In addition, the autofill function of the query result set can also be done using the generic program AutofillQueryServiceImpl, which will be explained in detail in the next section “How to design a DDD-driven technology platform”.

Summary #

This section explains a powerful and practical design practice for a technology platform. Through the encapsulation of this technology platform:

  • When performing insert-update-delete operations, only the front-end interface, Service, and value objects need to be created. More technical details are encapsulated, allowing developers to focus on transforming domain models into business code design and implementation, and constantly meeting user requirements as the domain model changes.
  • When performing query operations, in most cases, only MyBatis and Spring configurations need to be written to complete the query function. The development workload is greatly reduced, and changes become easier and faster.

The design of this technology platform is for a general technology platform. So how about designing a technology platform that supports DDD? Can we write generic repositories and factories? The next section will explain.