43 Service Downgrading Design New Versions New Solutions

43 Service Downgrading Design New Versions New Solutions #

With the continuous development and popularization of microservices architecture, RPC framework has become an indispensable and important role in microservices architecture. Dubbo, as a mature RPC framework in the Java ecosystem, has also been continuously developed and grown with the update of technology. Of course, the traditional Dubbo architecture also faces challenges brought by new ideas, new ecosystems, and new technologies.

In microservices architecture, the service is the basic unit, and in the Dubbo architecture, the basic unit of service is the Java interface. This architectural difference will bring a series of challenges. Starting from version 2.7.5, Dubbo introduced the service introspection architecture to cope with the challenges brought by microservices architecture. What are the specific challenges? Let’s explain in detail below.

Challenges faced by the Registry #

Before we start introducing the challenges faced by the registry, let’s review the traditional Dubbo architecture and its core components introduced in the previous lessons:

Drawing 0.png

Dubbo Core Architecture

Based on the above architecture diagram, let’s review the functions of these core components together.

  • Registry: Registry Center. Responsible for registering and discovering service addresses, the Provider and Consumer of services only interact with the registry center when they start. The registry center is aware of the existence of the Provider through long connections, and when the Provider goes offline, the registry center immediately pushes relevant event notifications to the Consumer.
  • Provider: Service Provider. When it starts, it registers itself with the Registry and adds its service address and relevant configuration information encapsulated into a URL to ZooKeeper.
  • Consumer: Service Consumer. When it starts, it subscribes to the Registry. The subscription operation fetches the URLs registered by the Provider from ZooKeeper and adds corresponding listeners to ZooKeeper. After obtaining the Provider URLs, the Consumer selects LoadBalance, Router, and Cluster implementation according to the corresponding parameters in the URLs, creates corresponding Invoker objects, and then encapsulates them into proxy objects for the service interface, which are returned to the upper-level business. When the upper-level business calls methods on this proxy object, remote invocation will be executed.
  • Monitor: Monitoring Center. Used to count the number of service invocations and the invocation time. During the runtime of the Provider and Consumer, they will count the number of invocations and the invocation time in memory, and send statistical data to the monitoring center once a minute. The monitoring center is not a necessary role in the above architecture diagram. The failure of the monitoring center will not affect the functions of the Provider, Consumer, and Registry, only the loss of monitoring data.

Through the introduction of the entire Dubbo implementation system, we know that URL is the core that runs through the registration and discovery of Dubbo. The format of the Provider URL registered in ZooKeeper is roughly as follows:

dubbo://192.168.0.100:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&group=groupA&interface=org.apache.dubbo.demo.DemoService&metadata-type=remote&methods=sayHello,sayHelloAsync&pid=59975&release=&side=provider&timestamp=1601390276192

The URL includes the IP and Port of the Provider, the full name of the service interface, the version number of the Dubbo protocol, the group information, the process ID, and so on.

The common registry centers we use, such as ZooKeeper, Nacos, or etcd, are all centralized infrastructures. The registry center is essentially based on memory as the core storage, and its memory usage is proportional to the number of service interfaces and the number of Provider nodes. A Dubbo Provider node can register multiple service interfaces. As the business develops, the number of service interfaces will increase, and in order to support the traffic growth of the entire system, the number of deployed Dubbo Provider and Consumer nodes will also increase, which leads to the increasing memory pressure on the registry center.

In the production environment, to avoid single points of failure, high availability solutions are used when building the registry center. The essence of these high availability solutions is the underlying consensus protocol, such as Zab protocol used by ZooKeeper, and Raft protocol used by etcd. When the registration data frequently changes, the network overhead of the internal nodes of the registry center cluster used for data synchronization will also increase.

From the perspective of the registry center, both Dubbo Provider and Dubbo Consumer can be considered as clients of the registry center, and they will maintain long connections with the registry center cluster, which will also cause a certain amount of network overhead and resource consumption.

When using registry center implementation solutions like ZooKeeper, the registry center actively pushes the changes in registration data to the client. Suppose a Dubbo Consumer subscribes to N service interfaces, and each service interface is provided by an M-node cluster. When a machine migration occurs in a Provider node, it will involve updating M * N URLs, and these change events will be notified to each Dubbo Consumer node, which puts pressure on the registry center in terms of processing notifications.

In summary, in large-scale microservice implementation practices, both the registry center and the traditional Dubbo architecture face many challenges and pressures from multiple perspectives, including memory, network overhead, and notifications.

Dubbo’s Improvement Plan #

Starting from version 2.7.0, Dubbo added the feature of simplified URL, and the extracted data from the URL is stored in the metadata center. However, this optimization only reduced the length of the URL and reduced the pressure on the registry center from the perspectives of memory usage and reducing notification frequency, but it did not reduce the number of URLs in the registry center. Therefore, the pressure on the registry center is still quite significant.

Dubbo version 2.7.5 introduced the service introspection architecture, which further reduces the pressure on the registry center. In this optimization, Dubbo changed to an application-level service registration and discovery model, minimizing the number of Dubbo service metadata registrations. The core process is shown in the following diagram:

Lark20201222-120323.png

Service Introspection Architecture Diagram

The above diagram shows the core process of Dubbo service registration and discovery after introducing service introspection. Dubbo will execute these operations in order (if one of the operations fails, subsequent operations will not be executed).

Let’s first look at the process on the Provider side:

  1. Publish all service interfaces defined in the business. The specific process is the same as the publishing process introduced in [Lesson 41].

  2. Publish the MetadataService interface, and the publishing of this interface is autonomously done by the Dubbo framework.

  3. Register the Service Instance to the registry center.

  4. Establish the mapping between Service ID and Service Name, and synchronize it to the configuration center.

Next, let’s take a look at the execution flow on the Consumer side:

  1. Register the Service Instance of the current Consumer. Dubbo allows Consumers to not register services, so this step is optional.

  2. Retrieve the mapping between Service ID and Service Name from the configuration center.

  3. Get the collection of Service Instances from the registry center based on the Service ID.

  4. Randomly select a Service Instance and retrieve the metadata of the MetadataService. This step involves invoking the MetadataService to get the URL list of the business interfaces exposed by the Service Instance. From this URL list, we can filter out the URLs of the currently subscribed Service.

  5. Initiate a remote call based on the business interface URL obtained in step 8.

As for some new concepts mentioned in the diagram, for your convenience, we will provide a brief introduction to their implementations.

  • Service Name: The name of the service. For example, in an e-commerce system, there could be services such as User Service, Product Service, Inventory Service, etc.
  • Service Instance: Represents a single Dubbo application process. Multiple Service Instances make up a service cluster and have the same Service Name.
  • Service ID: Uniquely identifies a Dubbo service, consisting of four parts: ${protocol}:${interface}:${version}:${group}.

In some scenarios, we may deploy two sets of service nodes with different configurations to verify the effectiveness of certain configurations. For example, there are 100 service nodes divided into two groups A and B. Group A has a timeout set to 3000 ms, while Group B has a timeout set to 2000 ms. This way, the service will have two sets of different metadata.

Following the optimization strategy mentioned earlier, when subscribing to services, we will get 100 Service Instances. Since the metadata published by each Service Instance may be different, we need to call the MetadataService of each Service Instance to get the metadata.

To reduce the number of MetadataService invocations, Dubbo proposes the Service Revision Version Optimization strategy. The core idea is to calculate a hash value (revision value) for the service URLs published by each Service Instance and publish them to the registry center along with the Service Instance. When the Consumer subscribes, if the revision values of the Service Instances are the same, the MetadataService will not be called again. Instead, the Consumer will directly reuse the URLs. The following diagram shows the core logic of Dubbo’s service revision:

Lark20201222-120318.png

Diagram of Consumer-side interaction with Dubbo’s service revision

Based on this diagram, we can see the process of implementing service revision on the Dubbo Consumer side:

  1. The Consumer retrieves the list of Provider-side ServiceInstances from the registry center through the service discovery API.
  2. The registry center returns 100 service instances, with Service Instances numbered 0 to 49 having a revision of 1, and Service Instances numbered 50 to 99 having a revision of 2.
  3. The Consumer randomly selects one Service Instance from these 100 instances, for example, it selects ServiceInstance number 68.
  4. The Consumer calls the MetadataService exposed by ServiceInstance 68 to obtain the list of Dubbo service URLs it publishes and establishes a local memory cache for the service URLs with a revision of 2.
  5. The Consumer then randomly selects another Service Instance from the remaining 99 instances, for example, it selects ServiceInstance number 30. It discovers that its revision is 1, and there is no local cache for the service URLs with a revision of 1. At this point, the Consumer will make a MetadataService call similar to step 4 to retrieve the service URL list from ServiceInstance 30 and update the cache.

At this point, the local cache has covered all revision values in the current scenario. Any ServiceInstance selected randomly in the future will either have a revision of 1 or 2, and it will fall into the local cache without making another MetadataService call. The remaining Service Instances will reuse the two URL lists in the local cache and replace the corresponding parameters (such as host, port, etc.) based on the Service Instance, resulting in the complete list of service URLs published by the Service Instance.

In general, the number of revisions is not large, so the number of MetadataService invocations made by the Consumer is limited and does not increase with the expansion of Service Instances. This avoids metadata expansion caused by different versions of the same service.

Summary #

In this lesson, we mainly introduced the self-introspection architecture of Dubbo services.

First, we reviewed the traditional architecture of Dubbo and the core functions and interaction flow of basic components in the traditional architecture. Then we analyzed the various challenges and pressures faced by Dubbo’s traditional architecture in large-scale microservice implementations. Finally, we focused on the service introspection solution introduced in Dubbo version 2.7.5 or later. The service introspection solution effectively addresses many challenges faced by Dubbo and alleviates the pressure on large-scale microservices systems built on Dubbo. Based on this, we also specifically explained the principles behind Dubbo’s service revision scheme and how it avoids metadata expansion.