01 Dubbo Source Code Environment Setup The Journey of a Thousand Miles Begins With a Single Step #
A good start is half the battle won, and the same goes for reading source code. Many students, after making up their minds to read an open-source framework, dive straight into it and get lost in the “maze” of code. At this point, some students realize that they need to debug while reading, while others end up going round in circles and give up when setting up the source code environment. Finally, only a few students manage to set up the source code environment, but they don’t know how to simulate requests to make the code execute in the place they want to debug.
Are you familiar with these pain points? Have you encountered them before? Don’t worry, in this lesson, I will guide you step by step to set up the Dubbo source code environment.
- Before we start setting up the source code environment, we will have an overall look at the architecture of Dubbo, which will help you understand the basic functionality and core roles of Dubbo.
- Then we will get hands-on and set up the Dubbo source code environment, building the simplest environment where a Demo example can be run.
- After setting up the source code environment, we will dive deeper into the functionality of each core module in the Dubbo source code. This will lay the groundwork for analyzing the implementation of each module in the next step.
- Finally, we will analyze in detail the three Demo examples that come with the Dubbo source code, briefly reviewing the basics of Dubbo usage. These three examples will also serve as the entry points for debugging the source code in the subsequent steps.
Introduction to Dubbo Architecture #
To help you better understand and learn, before we start setting up the Dubbo source code environment, let’s briefly introduce the core roles in the Dubbo architecture. This will help you review the Dubbo architecture and help those who are not familiar with Dubbo quickly understand it. The following figure shows the core architecture of Dubbo:
Dubbo Core Architecture Diagram
- Registry: Registry Center. Responsible for registering and looking up service addresses. Service providers and consumers only interact with the registry center when they start up. The registry center is aware of the existence of providers through long connections. When a provider goes down, the registry center will immediately push relevant event notifications to consumers.
- Provider: Service Provider. When a provider starts up, it registers with the registry center by adding its service address and related configuration information encapsulated as a URL to ZooKeeper.
- Consumer: Service Consumer. When a consumer starts up, it subscribes to the registry center. The subscription retrieves the provider’s URL from ZooKeeper and adds relevant listeners to ZooKeeper. Once the provider URL is obtained, the consumer selects a provider from multiple providers based on the load balancing algorithm, establishes a connection with it, and finally makes an RPC call to the provider. If the provider URL changes, the consumer will obtain the latest provider URL information and make the necessary adjustments, such as disconnecting from the down provider and establishing a connection with the new provider, through the listeners added during the previous subscription process of the registry center. The consumer establishes a long-lived connection with the provider and caches the provider information, so even if the registry center goes down, it will not affect the running providers and consumers.
- Monitor: Monitoring Center. Used to record the number and time of service calls. The provider and consumer will record the number and time of calls in memory during runtime and send statistics to the monitoring center every minute. The monitoring center is not a necessary role in the architecture diagram above. If the monitoring center goes down, it will not affect the functionality of the provider, consumer, and registry, only the loss of monitoring data.
Setting up the Dubbo Source Code Environment #
Of course, to set up the Dubbo source code environment, you first need to download the source code. Here, you can directly fork the official repository https://github.com/apache/dubbo to your own repository, and then execute the following command to download the code:
git clone [[email protected]](/cdn-cgi/l/email-protection):xxxxxxxx/dubbo.git
Then switch to the branch. The latest version is Dubbo 2.7.7 at the moment, so we will use this new version:
git checkout -b dubbo-2.7.7 dubbo-2.7.7
Next, use the mvn command to compile:
mvn clean install -Dmaven.test.skip=true
Finally, execute the following command to convert to an IDEA project:
mvn idea:idea // If an error occurs during execution, execute "mvn idea:workspace" instead
After that, import the source code in IDEA. This import process will download the required dependencies, so it may take some time.
### Dubbo Core Modules
After successfully importing the Dubbo source code in IDEA, the project structure should look like the following:
Now, let's briefly introduce the functionalities of these core modules. We will provide detailed analysis in later lessons.
* **dubbo-common module:** This is a common module of Dubbo, which contains various utility classes and common logic. For example, Dubbo SPI implementation, timer implementation, dynamic compiler, etc.
* **dubbo-remoting module:** This is the remote communication module of Dubbo. Its sub-modules depend on various open-source components for remote communication. The dubbo-remoting-api sub-module defines the abstract concept of this module, while other sub-modules depend on different open-source components for implementation. For example, the dubbo-remoting-netty4 sub-module uses Netty 4 for remote communication, and the dubbo-remoting-zookeeper sub-module interacts with ZooKeeper cluster using Apache Curator.
* **dubbo-rpc module:** This module abstracts the remote invocation protocol in Dubbo. It provides abstractions for various protocols and depends on the remote invocation functionality of the dubbo-remoting module. The dubbo-rpc-api sub-module is the core abstraction, while other sub-modules provide specific protocol implementations. For example, the dubbo-rpc-dubbo sub-module is the implementation for the Dubbo protocol and depends on dubbo-remoting-netty4 and other dubbo-remoting sub-modules. The implementation of the dubbo-rpc module only supports one-to-one invocation and does not involve cluster-related content.
* **dubbo-cluster module:** This module is responsible for managing clusters in Dubbo. It provides a series of functionalities like load balancing, fault tolerance, and routing. The ultimate goal is to mask multiple providers as a single provider, so consumers can invoke the provider cluster as if they are calling a single provider.
* **dubbo-registry module:** This module is responsible for interacting with various open-source registry centers in Dubbo. It provides the capability to work with different registry center implementations. The dubbo-registry-api sub-module is the top-level abstraction, while other sub-modules provide specific implementations for different registry center components. For example, the dubbo-registry-zookeeper sub-module is the specific implementation for integrating Dubbo with ZooKeeper.
* **dubbo-monitor module:** This module is responsible for monitoring services in Dubbo. It is mainly used for statistics of service invocation count, invocation time, and implementing service invocation trace.
* **dubbo-config module:** This module handles the configuration exposed by Dubbo. For example, the dubbo-config-api sub-module handles the configuration when using the API approach, and the dubbo-config-spring sub-module handles the configuration when integrating with Spring. With the dubbo-config module, users only need to understand the configuration rules of Dubbo without the need to know the internal details of Dubbo.
* **dubbo-metadata module:** This module is about Dubbo's metadata (which will be discussed in detail in later lessons). Similarly, the dubbo-metadata module follows the pattern of having an api sub-module for abstraction and other sub-modules for specific implementations.
- dubbo-configcenter module: The dynamic configuration module of Dubbo, mainly responsible for storing and notifying external configurations and service governance rules, and providing multiple sub-modules to integrate with various open-source service discovery components.
Demo Examples in Dubbo Source Code #
In the Dubbo source code, we can find a dubbo-demo module, which includes three very basic Dubbo example projects: Demo Example with XML Configuration, Demo Example with Annotation Configuration, and Demo Example using API directly. In the following, we will briefly introduce the basic usage of Dubbo from the perspective of these three examples. At the same time, these three projects will be used as the entry point for debugging the Dubbo source code, and we will modify them as needed. However, before that, you need to start ZooKeeper as the registration center and write a business interface as the contract for the Provider and Consumer.
Start ZooKeeper #
In the Dubbo architecture diagram, you can see that the address and configuration information of the Provider are passed to the Consumer through the registration center. Although Dubbo supports many registration centers, ZooKeeper is commonly used as the registration center in production environments. Therefore, when debugging the Dubbo source code, you naturally need to start ZooKeeper locally.
So how do you start ZooKeeper?
First, you need to download the zookeeper-3.4.14.tar.gz package (download link: https://archive.apache.org/dist/zookeeper/zookeeper-3.4.14/). After downloading, execute the following command to decompress it:
tar -zxf zookeeper-3.4.14.tar.gz
After decompression, go to the zookeeper-3.4.14 directory, copy the conf/zoo_sample.cfg file and rename it as conf/zoo.cfg, and then execute the following command to start ZooKeeper:
>./bin/zkServer.sh start
# The output is as follows
ZooKeeper JMX enabled by default
Using config: /Users/xxx/zookeeper-3.4.14/bin/../conf/zoo.cfg // configuration file
Starting zookeeper ... STARTED // start successfully
Business Interface #
Before using Dubbo, you also need a business interface, which can be considered as the contract between the Dubbo Provider and Dubbo Consumer, reflecting a lot of information:
- For the Provider: how to provide the service, what is the name of the service, what parameters need to be received, and what responses need to be returned.
- For the Consumer: how to use the service, what is the name of the service, what parameters need to be passed in, and what responses will be obtained.
The dubbo-demo-interface module is where the business interface is defined, as shown in the following figure:
In the interface DemoService, two methods are defined:
public interface DemoService {
String sayHello(String name); // synchronous call
// asynchronous call
default CompletableFuture<String> sayHelloAsync(String name) {
return CompletableFuture.completedFuture(sayHello(name));
}
}
Demo 1: XML Configuration based #
In the dubbo-demo-xml module under the dubbo-demo module, it provides a Provider and a Consumer based on Spring XML configuration.
Let’s first look at the dubbo-demo-xml-provider module, whose structure is shown in the following figure:
In the pom.xml file of this module, besides a lot of Dubbo dependencies, it also depends on the DemoService interface:
<dependency>
<groupId>org.apache.dubbo</groupId>
<artifactId>dubbo-demo-interface</artifactId>
<version>${project.parent.version}</version>
</dependency>
DemoServiceImpl implements the DemoService interface, where the sayHello() method directly returns a string, and the sayHelloAsync() method returns a CompletableFuture object.
In the dubbo-provider.xml configuration file, DemoServiceImpl is configured as a Spring Bean and exposed as a DemoService service:
<!-- Configure as a Spring Bean -->
<bean id="demoService" class="org.apache.dubbo.demo.provider.DemoServiceImpl"/>
<!-- Expose as a Dubbo service -->
<dubbo:service interface="org.apache.dubbo.demo.DemoService" ref="demoService"/>
Also, specify the address of the registry center (the ZooKeeper address earlier). This allows Dubbo to register the exposed DemoService service in ZooKeeper:
<!-- ZooKeeper address -->
<dubbo:registry address="zookeeper://127.0.0.1:2181"/>
Finally, write a main() method in the Application, specify the Spring configuration file, and start the ClassPathXmlApplicationContext.
Next, let's take a look at the dubbo-demo-xml-consumer module, the structure of which is shown in the following figure:
In the pom.xml file, we also have a dependency on the dubbo-demo-interface public module.
In the dubbo-consumer.xml configuration file, specify the address of the registry center (the ZooKeeper address earlier). This allows Dubbo to retrieve the list of services exposed by the Provider from ZooKeeper:
<!-- ZooKeeper address -->
<dubbo:registry address="zookeeper://127.0.0.1:2181"/>
It will also use the [dubbo:reference] tag to import the DemoService service, which can be used as a Spring Bean later:
<!-- Import DemoService service and configure it as a Spring Bean -->
<dubbo:reference id="demoService" check="false"
interface="org.apache.dubbo.demo.DemoService"/>
Finally, write a main() method in the Application, specify the Spring configuration file, and start the ClassPathXmlApplicationContext. After that, you can remotely invoke the sayHello() method of the DemoService on the Provider side.
#### Demo 2: Annotation-based Configuration
The dubbo-demo-annotation module is an example of configuration based on Spring annotations, which simply moves the configuration information from XML to annotations.
Let's first look at the dubbo-demo-annotation-provider module:
public class Application {
public static void main(String[] args) throws Exception {
// Initialize Spring container using AnnotationConfigApplicationContext,
// getting relevant configuration information from the annotations of ProviderConfiguration
AnnotationConfigApplicationContext context =
new AnnotationConfigApplicationContext(
ProviderConfiguration.class);
context.start();
System.in.read();
}
@Configuration // Configuration class
// The @EnableDubbo annotation specifies that the Bean in the specified package will be scanned and exposed as Dubbo services
@EnableDubbo(scanBasePackages = "org.apache.dubbo.demo.provider")
// The @PropertySource annotation specifies other configuration information
@PropertySource("classpath:/spring/dubbo-provider.properties")
static class ProviderConfiguration {
@Bean
public RegistryConfig registryConfig() {
RegistryConfig registryConfig = new RegistryConfig();
registryConfig.setAddress("zookeeper://127.0.0.1:2181");
return registryConfig;
}
}
}
Here, there is also a DemoServiceImpl that implements DemoService, which can be scanned and exposed as a Dubbo service under the org.apache.dubbo.demo.provider directory.
Next, let's look at the dubbo-demo-annotation-consumer module. In the Application, the Spring container is also initialized using AnnotationConfigApplicationContext, and the Bean in the specified directory is scanned, including the DemoServiceComponent Bean, which is injected with Dubbo service-related Beans using the @Reference annotation:
@Component("demoServiceComponent")
public class DemoServiceComponent implements DemoService {
@Reference // Inject Dubbo service
private DemoService demoService;
@Override
public String sayHello(String name) {
return demoService.sayHello(name);
}
// Other methods
}
#### Demo 3: API-based Configuration
In some cases, it is not possible to rely on the Spring framework and only API can be used to build Dubbo Providers and Consumers, as is typical when writing SDKs.
First, let's look at the dubbo-demo-api-provider module, where the Application.main() method is the entry point:
// Create an instance of ServiceConfig, with the generic parameter being the implementation class of the business interface, i.e., DemoServiceImpl
ServiceConfig<DemoServiceImpl> service = new ServiceConfig<>();
// Specify the business interface
service.setInterface(DemoService.class);
// Specify the implementation of the business interface, which will handle the Consumer's requests
service.setRef(new DemoServiceImpl());
// Get the instance of DubboBootstrap, which is a singleton object
DubboBootstrap bootstrap = DubboBootstrap.getInstance();
// Create an instance of ApplicationConfig and specify the ZK address and ServiceConfig instance
bootstrap.application(new ApplicationConfig("dubbo-demo-api-provider"))
.registry(new RegistryConfig("zookeeper://127.0.0.1:2181"))
.service(service)
.start()
.await();
Here, there is also a DemoServiceImpl that implements the DemoService interface, which can be scanned and exposed as a Dubbo service under the org.apache.dubbo.demo.provider directory.
Next, let's look at the dubbo-demo-api-consumer module. The Application contains a regular main() method entry:
// Create a ReferenceConfig, specifying the referenced interface DemoService
ReferenceConfig<DemoService> reference = new ReferenceConfig<>();
reference.setInterface(DemoService.class);
reference.setGeneric("true");
// Create DubboBootstrap, specifying ApplicationConfig and RegistryConfig
DubboBootstrap bootstrap = DubboBootstrap.getInstance();
bootstrap.application(new ApplicationConfig("dubbo-demo-api-consumer"))
.registry(new RegistryConfig("zookeeper://127.0.0.1:2181"))
.reference(reference)
.start();
// Get the instance of DemoService and invoke its methods
DemoService demoService = ReferenceConfigCache.getCache()
.get(reference);
String message = demoService.sayHello("dubbo");
System.out.println(message);
### Summary
In this lesson, we first introduced the core architecture of Dubbo and the functions of each core component. Then we set up the Dubbo source code environment and explained the functions of the core modules in detail, laying the foundation for analyzing the Dubbo source code later. Finally, we analyzed the three Demo examples included in the Dubbo source code in detail. Now you can debug the Dubbo source code starting from these three Demo examples.
In the following lessons, we will address several questions: How does Dubbo interact with registry centers such as ZooKeeper? How do Providers and Consumers interact with each other? Why don't we feel any network interaction when writing business code? What data does Dubbo Provider publish to the registry center? How does the Consumer correctly identify it? What is the unified contract between the two? How is this contract extensible? Where else is this contract used in Dubbo? You can think about these questions in advance, and I will answer them one by one in the later lessons.