12 How to design a technical backend strategy to support fast payments #

In the past, our systems were divided into front-end and back-end. The front-end refers to the UI interface that interacts with users, while the back-end refers to the business logic operations completed by the server. However, in many of our previous business systems, there are some shared components that will also be used in future development. Therefore, if these components can be extracted and turned into reusable components, future system development will be simplified and there will be no need to develop from scratch every time, just reuse these components.

But do these common components belong to the front-end or the back-end? Neither. It includes both the front-end interface and the backend logic, so it is called the “middle platform”. The so-called middle platform is to extract the reusable front-end and back-end code from the previous business systems, remove the individuality and extract the commonality to form reusable components. With these components, system development in the future can reduce costs, increase efficiency and improve delivery speed. Therefore, Alibaba proposed the strategy of “small front-end, big middle platform”, which has been widely recognized by the industry.

In terms of classification, the middle platform is divided into business middle platform, technology middle platform, and data middle platform.

• The business middle platform is to abstract business components, such as user permissions, member management, warehouse management, logistics management, etc., and turn them into microservices that can be used by various business systems.
• The technology middle platform is to encapsulate the technology framework that each business system needs to adopt and design a unified API to reduce the technical threshold for upper-level business development, reduce development workload and improve delivery speed.
• The data middle platform organizes the data of various business systems, establishes a platform for data storage and calculations, and provides support for the analysis and utilization of data from various systems.

Once you understand these concepts, you will understand the design ideas of the technology middle platform that supports DDD and microservices. It encapsulates the underlying technologies of DDD and microservices to support development teams in achieving rapid delivery in the future and to cope with the fiercely competitive market. Therefore, first of all, it is necessary to understand the pain points of technology in achieving rapid delivery in order to understand how to build the technology middle platform.

Building a Rapid Delivery Team #

Many teams have encountered such a situation: at the beginning of a project, due to simple business requirements and few participants, they can often achieve a relatively fast delivery speed. However, as the project progresses, the business becomes more and more complex, and more and more people participate, resulting in a slower delivery speed. This makes the team unable to adapt to the rapid changes in the market, putting them at a disadvantage in competition. However, the scale development of software is an inevitable trend for all software development. Therefore, solving the contradiction between scaled teams and rapid software delivery has become a problem that we must face.

Silo-based development team

Why does the delivery speed slow down as the team grows bigger? The above figure shows the entire process from requirement to delivery. In this process, we need to go through interactions between multiple departments to complete the final delivery, and a lot of time is wasted on departmental communication and coordination. Such a team is called a “silo-based development team”.

Chimney-style software development

Chimney-style development teams can lead to chimney-style software development. As shown in the diagram above, in most software projects, each feature requires its own page, Controller, Service, and DAO. This requires writing a large amount of code, much of which is repetitive. The more code you write, the more bugs there are, and the more difficult it becomes to make changes in the future.

Finally, a unified release also limits the speed of delivery. As shown in the diagram, when the business owner assigns a requirement to multiple teams, Team A may only need 1 week to complete their work. But after Team A completes their work, can they deliver it to the customer immediately? The answer is no, because Team B needs 2 weeks to develop. Team A can only wait until Team B completes their development before a unified release can be made. A unified release restricts the speed of system delivery, so even if Team A’s development speed is fast, it cannot generate user value if it cannot be delivered to users immediately.

As the system grows larger, with more and more features becoming more complex, the size of the development team also increases. This leads to decreased efficiency of the development team, longer delivery cycles, and increased difficulty in technology transformation.

Organizational structure of feature teams

How can this problem be solved? As shown in the diagram above, first, the organizational structure of the teams needs to be adjusted. The cylindrical architecture needs to be turned upright, creating “feature teams”. In feature teams, each team directly faces the end customer. For example, the shopping team is responsible for the shopping functionality, including everything from requirements to development, from UI to application to database. Finally, after testing, this team is responsible for deployment. In this way, the entire delivery process is handled by this team, eliminating the need for so much communication and coordination between teams, naturally increasing the delivery speed.

Front-end + Technology Middleware #

With the organizational structure of feature teams, if a unified release is still used, the delivery speed will not improve. Therefore, based on feature teams, the software architecture adopts a microservices architecture, where each feature team maintains its own microservices. In this way, when a team completes development, they can independently package and release without waiting for other teams. This significantly improves delivery speed, as shown in the diagram below:

Organizational structure of Front-end + Technology Middleware

Feature teams + microservices architecture can effectively improve the delivery speed of large-scale teams. However, upon closer examination, it is astonishing to find that building such feature teams incurs very high costs. Each member of the team must understand both the business and the development process; they need to understand UI, applications, databases, and even big data to become full-stack engineers. If each feature team is built this way, with every member being a full-stack engineer, the cost would be too high and it would not be feasible to implement. So, how can this problem be solved?

The key to solving the problem lies in the underlying architecture team. The architecture team is no longer just one architect, but a whole team.

• The architecture team, through technology selection, creates a technology middleware, encapsulating various technologies in software development, such as UI, applications, databases, and even big data.
• Then exposed to the upper-layer business in the form of API interface.

With this organizational structure, the technical threshold for business development will be reduced, and the workload will also decrease. As a result, the main responsibility of feature teams will change, freeing them from software technology and allowing them to focus more on understanding requirements and implementing business solutions, thus improving user experience. This is what we call “front-end development”. The concept of front-end development involves a shift in roles, where business developers no longer focus on technology but instead concentrate on understanding the business, responding quickly to changes in market demands.

By adopting the “front-end development + technology platform” strategy, the architecture team’s support is required to enhance the design capabilities and delivery speed of the team. The architecture team refines from the perspective of business development, identifies commonalities, retains individuality, and consolidates these commonalities into the technology platform. Such a technology platform needs the support of Domain-Driven Design (DDD) and microservices architecture. By encapsulating the various technical components involved in DDD and microservices into the technology platform and abstracting the technical details, it can effectively support rapid business development and delivery, enabling the team to gain a competitive advantage in the market.

Based on the above analysis, we have clarified the requirements for building a technology platform. In order to improve the delivery speed of development teams and enhance market competitiveness, it is necessary to build a technology platform at the bottom layer of the system. A technology platform that supports rapid delivery should have the following characteristics:

(1) Simple and user-friendly technology platform

It should significantly reduce the workload of software development, making the software system easy to change, maintain, and upgrade, thereby reducing the technical threshold for business developers. By using the architectural design of single Controller and single DAO mentioned earlier, this goal can be achieved. The key lies in how this design concept is implemented.

(2) Support for technical architecture evolution

The technology platform we build can help development teams adjust their technical architecture, facilitating architectural evolution and effectively reducing the cost of architectural changes. This requires the system’s architectural design to decouple the technical framework from the business code effectively. Using design patterns such as clean architecture, hexagonal architecture, and CQRS can help achieve this decoupling.

(3) Support for domain-driven design and microservices architecture

As mentioned earlier, although domain-driven design is a valuable concept, implementing it in a software project and ultimately in a microservices architecture requires a technology platform that supports domain-driven design and microservices architecture.

Building a simple and user-friendly technology platform #

Firstly, let’s take a look at how to build a simple and user-friendly technology platform, that is, how to simplify development. In the past, software projects required a large amount of code writing during the development process. This not only increased the workload of software development, resulting in delayed software delivery, but also increased the complexity of future maintenance and modifications. One important rule in software development is:

• The more code you write, the higher the probability of bugs and the more difficult future maintenance and modifications will be.
• The less code you write, the fewer bugs there will be, and the easier future maintenance and modifications will become. As the saying goes: a small boat is easy to turn around, but why did the Titanic hit the iceberg when it saw it? Because it was too big and couldn’t turn around in time. The same goes for writing code. It’s easy to make changes to a 10-line code, but it becomes very complex when making changes to hundreds or thousands of lines of code. Therefore, when designing software, we should adhere to the following attitude: spend more time on analysis and design, simplify software design to the extreme, and spend less time on coding. As the saying goes: sharpen the knife before cutting wood. Code written with such an attitude is both fast and easy to maintain.

Next, let’s take a look at the problems that exist in the traditional software development process. In the past, software projects required writing too much code during development. Each functionality required its own UI, Controller, Service, and DAO. Moreover, there were different data formats at each level, so a large amount of the code we wrote was for data format conversion between different levels. As shown in the following figure:

For example, the frontend data is transmitted to the backend in the form of a Form. At this time, the backend gets it from the Model or Request in the MVC layer, then converts it into a value object, and then calls the Service. However, after obtaining the data from the Model or Request, due to writing too many judgments and operations in the Controller of the MVC layer, it takes a lot of code to put it into the value object.

Then, in the Service, after various business operations, when it is time to persist the data, the value object needs to be converted into a persistent object (PO) and stored in the database. At this time, a DAO needs to be written for each functionality. The more code we write, the more difficult it is to maintain and make changes in the future. So, can we unify these conversions and sink them into a common codebase in the technical platform? Based on this idea, the system architecture is adjusted as follows:

In this architecture, the data at each level is unified into value objects. How is this achieved? Firstly, the frontend data is increasingly passed in the form of JSON with JSON being a key-value pair data format. Therefore, we can establish a development specification that requires the design of the JSON object on the frontend to correspond one-to-one with the format of the value objects in the backend. In this way, when the JSON object is passed to the backend, the MVC layer only needs a universal program to convert the JSON object into value objects in a unified format. Do we still need to write controllers for each functionality? No, the entire system only needs one controller, which can be moved down to the technical platform.

Similarly, when the Service goes through a series of business operations and needs to persist the data, it can be done like this: create a configuration file called vObj.xml to establish the correspondence relationship. Each value object corresponds to a table in the database, and each attribute corresponds to a field. In this way, when the DAO gets a value object, it knows which table in the database the data in that object should be saved to. Do we still need to write a DAO for each functionality? No, the entire system only needs one DAO.

By designing a system based on the above design ideas, the development workload will be greatly reduced. In business development, there is no need to write the MVC layer for each functionality. The business code will not be written in the Controller but rather in a standardized manner in the Service or value objects. Next, the system only has one DAO, and each Service injects this DAO. In this way, the only things that need to be written by business developers are the frontend UI, Service, and value objects. And both the Service and value objects are mapped from the domain model. Therefore, business developers will spend more energy on functional design and frontend UI, providing users with a better user experience and improving the delivery speed.

Summary #

We adopt DDD in order to have a deeper understanding of the business and create products that satisfy users. We also need to deliver products quickly to cope with the competitive and rapidly changing market. Both aspects need to be taken into account in order to gain a competitive advantage in the market. Therefore, we not only need to learn DDD, but also how to build a technical platform that supports DDD and enables rapid delivery.

This lecture explains how to build a simple and user-friendly technical platform. The next lecture will further explain the design and implementation of this platform, including how to implement a single controller, a single DAO, and how to create generic repositories and factories, and so on.