02 Dubbo Configuration Mainline Understanding Half of Dubbo by Grasping URL

02 Dubbo Configuration Mainline Understanding Half of Dubbo by Grasping URL #

Hello, I’m Yang Sizheng, and today I will share with you the topic of Dubbo Configuration Bus: Grasping the URL for Half Understanding of Dubbo.

In the field of the Internet, each information resource has a unified and unique address on the web, which is called a URL (Uniform Resource Locator), a unified resource locator on the Internet, which refers to a network address.

URL is essentially a special format of string. A standard URL format can include the following parts:

protocol://username:password@host:port/path?key=value&key=value
  • protocol: The protocol of the URL. The commonly known ones are HTTP and HTTPS protocols, but there are also other protocols such as FTP and SMTP.
  • username/password: Username/password. This is often used in HTTP Basic Authentication by directly carrying the username and password after the URL protocol.
  • host/port: Host/port. In practice, domain names are used instead of specific host and port.
  • path: The path of the request.
  • parameters: Parameter key-value pairs. Parameters are usually included in the URL for GET requests, while POST requests include parameters in the request body.

URL is a fundamental and essential component in Dubbo. When reading the source code, you will find that many methods take URL as a parameter and extract useful parameters by parsing the incoming URL. That’s why URL is called the Dubbo Configuration Bus.

For example, in the next lesson introducing the core implementation of Dubbo SPI, you will see that URL is involved in determining the extension implementation. In the subsequent lesson on registry center implementation, you will also see that providers encapsulate their own information into a URL and register it in ZooKeeper to expose their services. Consumers use URL to determine which providers they subscribe to.

Therefore, URL is very important to Dubbo, so “grasping the URL is half understanding of Dubbo”. In this article, we will introduce the application of URL in Dubbo and the importance of URL as a unified contract of Dubbo. Finally, we will illustrate the specific application of URL in Dubbo through examples.

URL in Dubbo #

Any implementation in Dubbo can be abstracted as a URL. Dubbo uses URL to uniformly describe all objects and configuration information, and it runs through the entire Dubbo framework. Let’s take a look at an example of a typical URL in Dubbo:

dubbo://172.17.32.91:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=dubbo-demo-api-provider&dubbo=2.0.2&interface=org.apache.dubbo.demo.DemoService&methods=sayHello,sayHelloAsync&pid=32508&release=&side=provider&timestamp=1593253404714dubbo://172.17.32.91:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=dubbo-demo-api-provider&dubbo=2.0.2&interface=org.apache.dubbo.demo.DemoService&methods=sayHello,sayHelloAsync&pid=32508&release=&side=provider&timestamp=1593253404714

This is the URL information for a Demo Provider registered on ZooKeeper. Let’s analyze the different parts of this URL:

  • protocol: dubbo protocol.
  • username/password: No username and password.
  • host/port: 172.17.32.91:20880
  • path: org.apache.dubbo.demo.DemoService
  • parameters: Key-value pair parameters, which are the parameters after the question mark.

Below is the constructor of URL, and you can see that its core fields are basically consistent with the analysis of the URL earlier:

public URL(String protocol,
           String username,
           String password,
           String host,
           int port,
           String path,
           Map<String, String> parameters,
           Map<String, Map<String, String>> methodParameters) {
    if (StringUtils.isEmpty(username) 
            && StringUtils.isNotEmpty(password)) {
        throw new IllegalArgumentException("Invalid url");
    } 
    this.protocol = protocol;
    this.username = username;
    this.password = password;
    this.host = host;
    this.port = Math.max(port, 0);
    this.address = getAddress(this.host, this.port);
    while (path != null && path.startsWith("/")) {
        path = path.substring(1);
    }
    this.path = path;
    if (parameters == null) {
        parameters = new HashMap<>();
    } else {
        parameters = new HashMap<>(parameters);
    }
    this.parameters = Collections.unmodifiableMap(parameters);
    this.methodParameters = Collections.unmodifiableMap(methodParameters);
}

In addition, the dubbo-common package also provides a helper class for URL:

  * **URLBuilder**, helps to construct URLs;
  * **URLStrParser**, parses a string into a URL object.

### The Power of Contracts

For URL in Dubbo, many people call it a "configuration bus" or a "unified configuration model". Although the terms are different, they all express the same idea: URL is treated as a "**public contract**" in Dubbo. A URL can contain a lot of extension parameters, and URLs as contextual information permeate the entire extension design system.

In fact, a good open-source product has a set of flexible and clear extension contracts. Third parties can extend the product based on this contract, and its own kernel can also be built according to this contract. Without a common contract, only conventions for each interface or method, different interfaces or even different methods in the same interface may be passed different parameter types. Sometimes a Map is passed, sometimes a string is passed, and the format of the string is also uncertain, requiring you to parse it yourself. This adds an implicit convention that is not clearly manifested.

Therefore, there are many benefits to using URL in Dubbo, increasing convenience:

  * Using URL as a means of passing contextual information makes the code more readable and understandable. There is no need to spend a lot of time guessing the format and meaning of the transmitted data, thus forming a unified specification, making the code easier to write and read.
  * Using URL as a method parameter (equivalent to a Map with Key/Value both being String), it expresses more meaning than individual parameters. When the code needs to be extended, new parameters can be appended to the URL in the form of Key/Value without changing the structure of the input or return values.
  * Using this "public contract" can simplify communication. The cost of communication between people is very high, and the efficiency of information transmission is very low. Using a unified contract, terminology, and vocabulary can eliminate a lot of communication costs and maximize communication efficiency.



### URL Examples in Dubbo

After understanding the structure of the URL and the reasons why Dubbo uses URL, let's take a look at three real examples in Dubbo to further appreciate the importance of URL.

#### 1\. Application of URL in SPI

In Dubbo SPI, there is an important scenario that relies on URL - adaptive methods annotated with @Adaptive. A very important role of URL is to select the appropriate extension implementation together with the @Adaptive annotation.

For example, in the dubbo-registry-api module, we can see the RegistryFactory interface. The getRegistry() method is annotated with @Adaptive({"protocol"}), indicating that this is an adaptive method. At runtime, Dubbo dynamically generates the corresponding "$Adaptive" type for it, as shown below:

```java
public class RegistryFactory$Adaptive implements RegistryFactory {

    public Registry getRegistry(org.apache.dubbo.common.URL arg0) {

        if (arg0 == null) throw new IllegalArgumentException("...");

        org.apache.dubbo.common.URL url = arg0;

        // Try to get the Protocol of URL. If the Protocol is empty, use the default value "dubbo"
        String extName = (url.getProtocol() == null ? "dubbo" :
             url.getProtocol());

        if (extName == null)
            throw new IllegalStateException("...");

        // Select the corresponding extension implementation based on the extension name. The core principle of Dubbo SPI will be analyzed in the next class.
        RegistryFactory extension = (RegistryFactory) ExtensionLoader
          .getExtensionLoader(RegistryFactory.class)

    public static String getExtension(String extName) {
        if (extName == null || extName.length() == 0) {
            throw new IllegalArgumentException("Extension name cannot be null or empty");
        }
        ExtensionLoader<T> loader = (ExtensionLoader<T>) getExtensionLoader(type);
        return loader.getExtension(extName).getExtension();
    }

    public static Registry getRegistry(URL url) {
        if (url == null) throw new IllegalArgumentException("url == null");
        ExtensionLoader<RegistryFactory> loader = ExtensionLoader.getExtensionLoader(RegistryFactory.class);
        return loader.getAdaptiveExtension().getRegistry(url);
    }
}

We will see that the generated “RegistryFactory$Adaptive” class will automatically implement the “getRegistry()” method, in which, the extension name will be determined based on the URL’s protocol, thus determining the specific extension implementation class to be used. We can find the “RegistryProtocol” class and set a breakpoint in its “getRegistry()” method, then debug any Provider from the previous lesson’s demo example. We will get the following content:

Drawing 0.png

The “registryUrl” value passed in here is:

zookeeper://127.0.0.1:2181/org.apache.dubbo...

So the extension name obtained in “RegistryFactory$Adaptive” is “zookeeper”, and the Registry extension implementation class to be used this time is “ZookeeperRegistryFactory”. As for the full content of Dubbo SPI, we will give a detailed introduction in the next lesson, so we will not elaborate on it here.

2. The Application of URL in Service Exporting #

Let’s take a look at another example related to URL. In the previous lesson, when we introduced the simplified architecture of Dubbo, we mentioned that when the Provider starts, it will register the services it exposes to ZooKeeper. What information is registered to ZooKeeper? Let’s take a look at the “ZookeeperRegistry.doRegister()” method, set a breakpoint there, and then debug the Provider. We will get the following figure:

Drawing 1.png

The passed-in URL contains the Provider’s address (172.18.112.15:20880), the exposed interface (org.apache.dubbo.demo.DemoService), and other information. The “toUrlPath()” method will determine the node path created on ZooKeeper based on the passed-in URL parameters, and the “dynamic” parameter value in the URL will determine whether the created ZNode is a temporary node or a persistent node.

3. The Application of URL in Service Subscription #

After the Consumer starts, it will subscribe to the registry and listen to the Providers it is interested in. So how does the Consumer tell the registry which Providers it is interested in?

Let’s take a look at the “ZookeeperRegistry” implementation class, which is created by the “ZookeeperRegistryFactory” factory class. The “doSubscribe()” method in it is the core implementation of the subscription operation. Set a breakpoint at line 175 and debug the Consumer in the Demo. We will get the content shown in the figure below:

Lark20200731-183202.png

We can see that the URL parameters passed in are as follows:

consumer://...?application=dubbo-demo-api-consumer&category=providers,configurators,routers&interface=org.apache.dubbo.demo.DemoService...

Where the Protocol is “consumer”, indicating the subscription protocol used by the Consumer. The “category” parameter indicates the categories to be subscribed to. Here, it subscribes to the “providers”, “configurators”, and “routers” categories. The “interface” parameter indicates which service interface to subscribe to. Here, it subscribes to the Provider that exposes the “org.apache.dubbo.demo.DemoService” implementation.

Based on the above parameters in the URL, the “ZookeeperRegistry” will organize them into a ZooKeeper path in the “toCategoriesPath()” method, and then add a listener on top of it using the zkClient.

Through the above examples, I believe you have already felt the reason why URL is called the “bus” or “contract” in the Dubbo system. In the subsequent source code analysis, we will see more about the implementation of URL.

Conclusion #

In this lesson, we mainly introduced Dubbo’s encapsulation of URL and related utility classes. We also explained the benefits of unified contracts, and of using URL as the unified configuration bus in Dubbo. Finally, we introduced the implementation of URL in scenarios such as Dubbo SPI, Provider registration, and Consumer subscription. All of these can help you better understand the role of URL in Dubbo.

Here, you can think about whether there is a similar unified contract like Dubbo URL in other frameworks or in actual work. I welcome you to share your thoughts in the comments section.