11 Key Architecture Clean-up for Resolving Technical Refactoring Dilemmas #

There is a saying about the “master and space”: Masters are usually at a very high level, with a broad vision. How high are they standing? They are standing in space, so it’s easy for people to suffocate when trying to follow them. Therefore, when we learn theories from masters, we need to ground them in project applications, to “land” them on the “ground”, in order for these theories to be useful. The same applies to the implementation of Domain-Driven Design (DDD).

So, how do we implement DDD? Besides practicing DDD and domain modeling in projects, we also need a technical middleware that supports DDD and microservices. During the implementation of DDD, this technical middleware should be able to encapsulate tedious aggregate operations, repository and factory designs, and various related technologies. With the support of this technical middleware, development teams can focus more on understanding user business and pain points, developing satisfying features quickly, and delivering them rapidly, rather than being constrained by tedious technical details. This not only reduces the amount of code written and lowers the technical threshold, but also makes future changes easier and facilitates technological updates.

Then, how do we design such a technical middleware? We should start by analyzing the pain points in the design of the existing system.

Technological Changes at the Lower Levels #

In recent years, with the emergence of new technologies such as the internet, big data, and artificial intelligence, the technological architecture of the entire IT industry has been rapidly evolving.

• In the past, we used to say “architecture is the most stable and unchanging part of a software system”;
• Now, we say “good architecture comes from constant evolution”.

Therefore, the design of today’s architecture needs to consider how to make the lower-level architecture more adaptable to technological changes and adjustments, in order to gain a technological advantage in the industry’s competition against constantly evolving new technologies and frameworks.

However, in actual projects, especially in many old projects that have been running for seven or eight years or more, upgrading the technology can be extremely difficult, just like peeling off a layer of skin. Why is technological upgrading so difficult? The reason is that a large amount of business code in the system depends on the underlying technology framework, forming a coupling.

For example, in the past, we used Hibernate for data persistence, and each module’s DAO had to inherit from HibernateDaoSupport. As a result, every DAO became dependent on Hibernate. When the system architecture is upgraded from Hibernate2 to Hibernate3, or even to MyBatis, it is not as simple as changing a jar file.

When the technology framework is changed, the underlying classes, interfaces, and package names change, which means that all modules in the upper layer need to be modified, and after the modifications, testing is also necessary. The cost of such technological upgrades is extremely high, and the risks are great. We need to carefully consider the solution.

In summary, the root cause of the high cost of upgrading the technological architecture of an old system lies in the coupling between business code and the underlying technology framework. Therefore, the solution is to decouple them. How do we decouple? By establishing an “interface layer” between the upper-layer business code and the underlying technology framework.

How do we establish the “interface layer” between business code and the underlying framework? As shown in the above diagram, when performing persistence in the upper-layer business code, the DAOs of each module no longer call the underlying framework. Instead, they call the DaoSupport interface of the interface layer. DaoSupport is a self-designed interface that should meet all the business needs of the upper-layer, such as various types of insert, update, delete, get, load, and find operations, and keep this interface stable. The design and implementation of the upper-layer business code depend on the DaoSupport interface. As long as it remains stable, the upper-layer business code will also remain stable.

Next, based on the DaoSupport interface, write implementation classes to call the underlying technology framework and achieve actual persistence.

• Initially, we used Hibernate2 as the underlying framework, so we wrote an implementation class for Hibernate2.
• When Hibernate2 was upgraded to Hibernate3, we wrote an implementation class for Hibernate3.
• When the underlying framework needed to be upgraded to MyBatis, we wrote an implementation class for MyBatis.

With this design, when the system undergoes a technological architecture upgrade, the impact will no longer extend to the business layer code, but will be limited to adjusting the implementation classes of the interface layer. The cost of technological upgrades will be greatly reduced.

Designing a Clean Architecture #

Through the analysis of the problems and the design of the interface layer mentioned earlier, we can come to a very important conclusion: how can we easily achieve technological evolution and ensure rapid delivery by the development team? The key is to decouple the business code from the technology framework. As shown in the diagram above, in the system’s layered structure, based on domain-driven design, the business code is integrated into the business domain layer. The business domain layer includes the services in the Business (BUS) layer, as well as the business entities and value objects associated with them. The essence of the design in the domain layer is to implement the domain model through the design of anemic and rich models and finally translate it into code design. Based on this, the business domain layer is decoupled from other layers of the technical framework through layering, which is the core design idea of “Clean Architecture”.

Clean Architecture is an architectural design concept proposed by Robert C. Martin in the book “Clean Architecture”. As shown in the figure above, it divides the system into several different layers in the form of a circular ring, hence it is also known as “Onion Architecture”. In the center of the Clean Architecture are the business entities (yellow portion) and the business applications (red portion), where the business entities are the core business logic and the business applications are the services that are user-facing. Together, they form the business domain layer, which is the implementation of business code formed through the domain model.

The outermost layer of Clean Architecture consists of various technology frameworks, including:

• Interaction with user interfaces (UI);
• Network interaction between clients and servers;
• Interaction with hardware devices and databases;
• Interaction with other external systems.

The essence of Clean Architecture lies in the adapter layer, which decouples the core business code from the peripheral technology frameworks through adapters. Therefore, the core of implementing Clean Architecture lies in designing the adapter layer, which allows the business code to be decoupled from the technology frameworks, and enables independent work between the business development team and the technical architecture team.

The refined diagram of Clean Architecture, image from “The Architecture Chronicles”

As shown in the figure, Clean Architecture is further refined and divided into two parts: the active adapter and the passive adapter.

• The active adapter, also known as the “North-bound adapter”, is used for processing business requests initiated by front-end users in different forms. These requests are then received by the application layer and handled by the domain layer. Users can initiate requests through various forms such as browsers, clients, mobile apps, WeChat, and specialized devices for the Internet of Things. However, through the North-bound adapter, the requests are ultimately processed in the same form by calling the application layer.
• Passive adapter, also known as “southbound adapter”, is responsible for persistently storing the final result data in some form after completing various business processing in the business domain layer. The final data can be stored in relational databases, NoSQL databases, NewSQL databases, Redis caches, or sent to other application systems in the form of message queues. Regardless of the form used, there is only one set of business domain layers, but there can be various forms of persistent storage. The southbound adapter decouples the business logic from the storage technology.

Implementation of Clean Architecture #

How can we implement the architecture design based on clean architecture? As shown in the above diagram, in this architecture, the adapter layer is decoupled from the business through the design of several parts such as the data access layer, data persistence layer, and interface layer.

Firstly, users can access the system through different front-end forms such as browsers, clients, mobile apps, WeChat, and IoT devices, obtaining multi-channel access to the system. The access forms of different channels are different. Through the data access layer, the front-end multi-channel access is decoupled, and then the upper-layer business code is called in the same way to decouple the various front-end access channels from the back-end business logic. In this way, no matter how the front-end changes or how many types of channels there are, the back-end business only needs to write one set of code, which greatly reduces the maintenance cost.

Next, the business logic is decoupled from the database through the data persistence layer. As mentioned earlier, in the next 3-5 years, we will undergo another round of big data transformation. After the transformation, the design of data storage may no longer be limited to relational databases and 3NF design thinking, but will change greatly by storing data in NoSQL databases using JSON, adding redundancy, designing wide tables, etc. However, no matter how it changes, it is only a change in the storage form, and the unchanged part is the business entity in the business logic layer. Therefore, through the decoupling of the data persistence layer, when the system transforms to big data technology in the future, the business logic layer does not need to make any changes, only the implementation of the data access layer needs to be rewritten to transform to big data technology. The cost of transformation will be greatly reduced, making the transformation easier.

Finally, it is the underlying technology architecture. Nowadays, when we talk about architecture, we are increasingly talking about architecture evolution, that is, the underlying technology architecture needs to evolve constantly with changes in the market and technology. However, many systems undergo painful changes in technical architecture, especially when the underlying framework and business code are tightly coupled. Once the underlying framework changes, it will require a lot of changes to the business code, causing all business modules to undergo changes, resulting in high cost and risk in architecture adjustments.

Since the problem here is coupling, the solution is decoupling. In the process of platform construction, in addition to integrating various technologies into the system through technical selection, an interface layer should be established through encapsulation. By encapsulating the implementation of many technologies through the interface layer, it can be provided to upper-level business developers through simpler interfaces. This not only reduces the technical threshold for business development, allowing them to focus more on business and improve development speed, but also decouples the business code from the technical framework. With decoupling, it will be possible to evolve with lower cost and accelerate the evolution of technical architecture to catch up with this rapidly changing era.

Summary #

The essence of clean architecture is based on DDD business implementation, including services, entities, and value objects guided by the domain model design and development. The outermost layer of clean architecture consists of various hardware, devices, and technology frameworks. The core idea of clean architecture is to decouple the business implementation from the technical framework through the adapter layer, which is also the best practice of DDD in architecture design.

Therefore, a technology middle platform that supports DDD and microservices is based on the thinking of clean architecture, encapsulating the tedious aggregation operations, repositories, and factory designs in DDD’s bottom layer, as well as the technical frameworks in microservices, and the adapters in clean architecture, all in the technology middle platform. With this technology middle platform, upper-level business developers can more easily apply the thinking of DDD, iteratively and quickly deliver user requirements, and gain advantages in fierce market competition.

The next lecture will further explain the design of such a technology middle platform.