06 Domain Events the Key to Decoupling Microservices

06 Domain Events - The Key to Decoupling Microservices #

Hello, I’m Ou Chuangxin. Today, let’s talk about “Domain Events”.

During Event Storming, we found that in addition to business behaviors such as commands and operations, there is another very important type of event. This type of event usually leads to further business operations. In Domain-Driven Design (DDD), this type of event is called a domain event.

This is just a simple definition that does not really help us understand it. So, what exactly is a domain event? What is the technical implementation mechanism of a domain event? In this lecture, we will focus on solving these two major questions.

Domain Events #

Domain events are a very important part of the domain model, used to represent events that occur in the domain. A domain event will lead to further business operations, helping to achieve both business decoupling and the formation of a complete business loop.

For example, a domain event can be a step in a business process, such as triggering the conversion of an insurance application to a policy after the payment of the insurance premium; or it can be an event that occurs during a batch processing process, such as generating quarterly premium payment notices and triggering the sending of payment notification emails; or it can be a subsequent action triggered by an event, such as locking an account after three consecutive failed password attempts.

So how do we identify domain events?

It’s simple, it’s strongly related to the definition above. When doing user journey or scenario analysis, we need to capture keywords mentioned by business or domain experts such as “If… happens, then…,” “When… is completed, please notify…,” “If… happens, then…,” and so on. In these scenarios, if a certain event triggers further operations, then this event is likely a domain event.

Why use eventual consistency for domain events instead of the traditional approach of direct invocation in SOA?

Let’s review one of the design principles of aggregates mentioned in [Lesson 05]: use eventual consistency outside the boundaries. A transaction can only modify the state of one aggregate at most. If a business operation involves the modification of multiple aggregate states, domain events with eventual consistency should be used.

Domain-driven event-driven design can break strong dependencies between domain models. After an event is published, the publisher does not need to concern itself with the success of the subsequent event handling by subscribers. This can achieve decoupling of the domain model and maintain the independence of the domain model and the consistency of the data. When mapping the domain model to a microservices system architecture, domain events can decouple microservices and do not require strong consistency of data between microservices, instead relying on eventual consistency based on events.

Returning to specific business scenarios, we find that some domain events occur between aggregates within a microservice, some occur between microservices, and some occur in both scenarios. Generally speaking, cross-microservice domain event processing is more common in microservice design.

1. Domain Events within a Microservice #

When domain events occur between aggregates within a microservice, after the domain event occurs, the event entity construction and event data persistence are completed. The publisher aggregate publishes the event to the event bus, and the subscriber receives the event data and completes the subsequent business operations.

Most of the event integration within microservices occurs within the same process, and the process itself can control transactions well, so there is no need to introduce messaging middleware necessarily. However, if an event updates multiple aggregates at the same time, considering the DDD principle of “one transaction updates only one aggregate,” you need to consider whether to introduce an event bus. However, the use of an event bus within a microservice may increase development complexity, so you need to consider overall complexity and benefits.

Application services within a microservice can use cross-aggregate service orchestration and composition to access cross-aggregates using service invocation. This approach is usually used in scenarios with high real-time requirements and data consistency. This process will involve distributed transactions to ensure that the publisher and subscriber data are updated successfully.

2. Domain Events between Microservices #

Cross-microservice domain events facilitate business collaboration between different bounded contexts or domain models, with the main purpose being to achieve decoupling of microservices and reduce the pressure of real-time service access between microservices.

There are more scenarios where domain events occur between microservices, and the event handling mechanism is more complex. Cross-microservice events can drive the flow of business processes or the flow of data between different sub-domains or microservices.

The mechanism for handling cross-microservice events needs to consider event construction, event publishing and subscribing, event data persistence, message middleware, and even the introduction of distributed transaction mechanisms during event data persistence.

Access between microservices can also be achieved through direct service invocation, enabling real-time access to data and services. The downside is that simultaneous data updates across microservices require the introduction of distributed transactions to ensure data consistency. The distributed transaction mechanism will affect system performance and increase coupling between microservices, so we should try to avoid using distributed transactions as much as possible.

Let me introduce you a case related to domain events in the insurance underwriting process.

The generation of an insurance policy involves many subdomains, changes in business status, and the transfer of business data across microservices. This process generates many domain events, which facilitate the flow and role transformation of insurance business data and objects among different microservices and subdomains.

In the diagram below, I have listed several key processes to illustrate how to drive the underwriting business process using domain event-driven design.

Event Start: Customer purchases insurance - Business personnel completes policy entry - Generate application form - Initiate payment action.

The underwriting microservice generates a payment notice and publishes the first event: Payment notice has been generated and the payment notice data is published to the message middleware. The payment microservice subscribes to the payment notice event and completes the payment process. Payment notice has been generated, and the domain event ends.
After the payment is completed by the payment microservice, the second domain event is published: Payment has been completed, and the payment data is published to the message middleware. The original subscriber, the payment microservice, becomes the publisher. The original event publisher, the underwriting microservice, becomes the subscriber. The underwriting microservice, after receiving the payment information and confirming the completion of payment, completes the conversion from application form to policy. Payment has been completed, and the domain event ends.
After the underwriting microservice completes the conversion from application form to policy, the third domain event is published: Policy has been generated, and the policy data is published to the message middleware. The policy microservice receives the policy data and completes the saving operation. Policy has been generated, and the domain event ends.
After the policy microservice completes the saving operation, a series of domain events will occur. The policy data will be sent to commission, payment collection, reinsurance, and other microservices concurrently via the message middleware, until it reaches the finance department and completes all subsequent business processes. I won’t go into details here.

In summary, the asynchronous mechanism driven by domain events can promote the flow of business processes and data among different microservices, achieve decoupling of microservices, reduce the pressure of service calls between microservices, and improve user experience.

Overall Architecture of Domain Events #

The execution of domain events requires a series of components and technologies to support them. Let’s take a look at this overall technical architecture diagram of domain events. Domain event processing includes event construction and publishing, event data persistence, event bus, message middleware, event receiving and processing, etc. Let’s discuss each of them.

1. Event Construction and Publishing #

The basic attributes of an event include at least the event’s unique identifier, occurrence time, event type, and event source. The unique identifier of the event should be globally unique, so that the event can be transmitted across multiple bounded contexts unambiguously. The basic attributes of the event mainly record the event itself and the background data of the event’s occurrence.

In addition, there is another important attribute in the event, which is the business attribute used to record the business data at the moment of the event’s occurrence. These data will be transmitted with the event to the subscribers for further business operations.

The basic attributes and business attributes together constitute the event entity, which depends on the aggregate root. After a domain event occurs, the business data in the event will no longer be modified, so the business data can be saved in the form of serialized value objects, which can be easily parsed and obtained in the message middleware.

To ensure the uniformity of event structure, we also create a base event class DomainEvent (see the figure below), and subclasses can expand their own attributes and methods. Since events do not have much business behavior, the implementation of methods is generally straightforward.

Before an event is published, it needs to be constructed and persisted. There are many ways to publish events. You can use application services or domain services to publish events to an event bus or a message middleware. You can also capture incremental event data from the event table using scheduled programs or database log capture technology and publish them to the message middleware.

2. Event Data Persistence #

Event data persistence can be used for reconciliation between systems or for auditing event data between publishers and subscribers. When encountering issues such as message middleware or subscriber system downtime or network interruption, the subsequent business workflows can continue after the issues are resolved, ensuring data consistency.

There are two solutions for event data persistence, and you can choose according to your own business scenarios during implementation.

Persist the event data to the local business database’s event table and use local transactions to ensure the consistency of business and event data.

Persist the event data to a shared event database. It should be noted that the business database and event database are not located in the same database, so their data persistence operations will span databases. Therefore, distributed transaction mechanisms are needed to ensure strong consistency of business and event data, which may affect system performance to some extent.

3. Event Bus #

The event bus is an important component for implementing domain events between aggregates within a microservice. It provides services such as event distribution and receiving. The event bus is an in-process model that traverses the subscriber list between aggregates within the microservice and passes data in a synchronous or asynchronous mode. The event distribution process is roughly as follows:

If the subscriber is within the microservice (other aggregates), the event is directly delivered to the specified subscriber.

If the subscriber is outside the microservice, the event data is saved to the event repository (table) and then asynchronously sent to the message middleware.

If there are both internal and external subscribers within the microservice, the event is first distributed to internal subscribers, the event message is saved to the event repository (table), and then asynchronously sent to the message middleware.

4. Message Middleware #

Most cross-microservice domain events will use message middleware to implement event publishing and subscription across microservices. Message middleware products are very mature, and there are many technologies available on the market, such as Kafka, RabbitMQ, etc.

5. Event Receiving and Processing #

Microservice subscribers adopt a listening mechanism at the application layer to receive event data in the message queue. After persisting the event data, further business processing can be started. Domain event processing can be implemented in domain services.

Domain Event Operation Mechanism: Case Study #

Here, I will explain the operation mechanism of domain events using the example of the payment notice event in the underwriting process. This domain event occurs between the insurance application and the payment collection microservices. The domain event that occurs is: Payment notice has been generated. The next business operation is: Payment.

Event Trigger: The policyholder generates the insurance policy, and after it is approved, initiates the operation to generate the payment notice.

The insurance microservice application calls the domain services createPaymentNotice and createPaymentNoticeEvent in the aggregate, respectively, to create the payment notice and the payment notice event. The PaymentNoticeEvent class, which inherits from the base class DomainEvent, is used for the payment notice event.
The repository service is used to persist the business and event data related to the payment notice. To avoid distributed transactions, these business and event data are all persisted in the local insurance microservice database.
Through database log capture technology or scheduled programs, the incremental data of the events is obtained from the database event table and published to the message middleware. Note: Event publishing can also be done through application services or domain services.
The payment collection microservice subscribes to the payment notice event message topic from the message middleware at the application layer, listens for and retrieves the event data, and then calls the domain service in the domain layer to persist the event data in the local database.
The payment collection microservice calls the domain service PayPremium in the domain layer to complete the payment.
The event ends.

Note: After the payment is completed, subsequent microservices in the process will generate many new domain events, such as payment completed, policy saved, and so on. The processing mechanism for these subsequent events is similar to steps 1-6.

Summary #

Today we mainly talked about domain events and the mechanism for handling domain events. Domain event-driven is a mature technology that has been widely used in many distributed architectures. Domain events are an important concept in DDD, and we should pay special attention to them when designing. By using domain events to drive the flow of business, we can achieve eventual consistency based on events, reduce the pressure of direct access between microservices, decouple microservices, and maintain the independence and data consistency of the domain model.

In addition, the domain event-driven mechanism can achieve a pattern of one publisher to multiple subscribers, which is basically impossible in traditional direct service call design.