41 Add Meal One Key Through Train Service Deployment Full Process

41 Add Meal One-Key Through Train Service Deployment Full Process #

In previous lessons, we have analyzed the core implementation of Dubbo. In the next two lessons, we will connect these core implementations within Dubbo and analyze the complete process of service publishing and service referencing in Dubbo. This will help you connect the independent knowledge points introduced in the previous lessons and form a complete whole.

In this lesson, we will focus on the process of publishing services on the Provider node, which involves many core components introduced previously. We will start with the entry class DubboBootstrap and analyze the assembly of the Provider URL and the process of service publishing. We will provide a detailed introduction to the core flow of local and remote publishing.

Entry: DubboBootstrap #

In the implementation of Provider in the [Lesson 01] dubbo-demo-api-provider example, we can see that the startup entry point of the Provider node is the DubboBootstrap.start() method. In this method, some initialization operations are performed, as well as updates to some state control fields. The specific implementation is as follows:

public DubboBootstrap start() {

    if (started.compareAndSet(false, true)) { // CAS operation, ensure only one startup

        ready.set(false); // Used to determine whether the current node has been started, will be used in Dubbo QoS later

        // Initialize some basic components, such as configuration center related components, event listening, metadata related components, which will be introduced later

        initialize(); 

        // Key: Publish services

        exportServices();

        if (!isOnlyRegisterProvider() || hasExportedServices()) {

            // Used to expose local metadata services, will be introduced in detail when explaining metadata

            exportMetadataService();

            // Used to register service instances to the registry dedicated to service discovery

            registerServiceInstance();
        }

        // Process ReferenceConfig of Consumer

        referServices();

        if (asyncExportingFutures.size() > 0) {

            // Asynchronously publish services, start a thread to listen for completion of publishing, and set ready to true after completion

            new Thread(() -> {

                this.awaitFinish();

                ready.set(true);

            }).start();
        } else { // For synchronous publishing, set ready to true after successful publishing

            ready.set(true);
        }
    }

    return this;
}

The DubboBootstrap is not only used directly to start the Provider through the API. When integrating Spring with Dubbo, DubboBootstrap is also used as the entry point for service publishing. The specific logic is in the Spring Context listener DubboBootstrapApplicationListener, as shown below:

public class DubboBootstrapApplicationListener extends OneTimeExecutionApplicationContextEventListener
        implements Ordered {

    private final DubboBootstrap dubboBootstrap;

    public DubboBootstrapApplicationListener() {
        // Initialize DubboBootstrap object
        this.dubboBootstrap = DubboBootstrap.getInstance();
    }

    @Override
    public void onApplicationContextEvent(ApplicationContextEvent event) {

        // Listen for ContextRefreshedEvent and ContextClosedEvent
        if (event instanceof ContextRefreshedEvent) {
            onContextRefreshedEvent((ContextRefreshedEvent) event);
        } else if (event instanceof ContextClosedEvent) {
            onContextClosedEvent((ContextClosedEvent) event);
        }

    }

    private void onContextRefreshedEvent(ContextRefreshedEvent event) {
        dubboBootstrap.start(); // Start DubboBootstrap
    }

    private void onContextClosedEvent(ContextClosedEvent event) {
        dubboBootstrap.stop();
    }

    @Override
    public int getOrder() {
        return LOWEST_PRECEDENCE;
    }
}

Here, we focus on the exportServices() method, which is the entry point for the core logic of service publishing. Each service interface will be converted to the corresponding ServiceConfig instance, then transformed into an Invoker through proxying, and finally transformed into an Exporter for publishing. The core object conversions involved in the service publishing process are shown in the following diagram:

Core process flow of service publishing

The specific implementation of the exportServices() method is as follows:

private void exportServices() {
    // Get all service configurations to be exposed from the configuration manager, where each interface corresponds to a `ServiceConfigBase` instance
    configManager.getServices().forEach(sc -> {
        ServiceConfig serviceConfig = (ServiceConfig) sc;
        serviceConfig.setBootstrap(this);
        if (exportAsync) { // Asynchronous mode, get a thread pool to execute the service exporting logic asynchronously
            ExecutorService executor = executorRepository.getServiceExporterExecutor();
            Future<?> future = executor.submit(() -> {
                sc.export();
                exportedServices.add(sc);
            });
            // Record asynchronously exported Future instances
            asyncExportingFutures.add(future);
        } else { // Synchronous publishing
            sc.export(); 
            exportedServices.add(sc);
        }
    });
}

ServiceConfig #

In the ServiceConfig.export() method, the first step of service publishing is to check the parameters. The second step is to decide whether to delay the publishing or immediately call the doExport() method based on the current configuration. The third step is to callback the relevant listeners via the exported() method. The specific implementation is as follows:

public synchronized void export() {
    if (!shouldExport()) {
        return;
    }
    if (bootstrap == null) {

            map.put(TOKEN_KEY, UUID.randomUUID().toString());
        }
        map.put(ALIVE_KEY, String.valueOf(provider.isAlive()));
        map.put(INTERFACE_KEY, interfaceName);
        map.put(SERVICE_KEY, path);
        map.put(TIMEOUT_KEY, String.valueOf(provider.getTimeout()));
        map.put(TOKEN_KEY, provider.getToken());
        if (StringUtils.isNotEmpty(stub)){
            map.put(STUB_KEY, stub);
        }
        if (StringUtils.isNotEmpty(cluster)){
            if (!cluster.contains(":") && ExtensionLoader.getExtensionLoader(Cluster.class).hasExtension(cluster)){
                cluster = cluster+":";
            }
        }
        map.put(CLUSTER_KEY, cluster);
        map.put(PROTOCOL_KEY, protocolConfig.getName());
        map.put(CONFIG_VERSION_KEY, Long.toString(System.currentTimeMillis()));
        if (CollectionUtils.isNotEmpty(registryURLs)){
            StringBuilder registryURLsBuilder = new StringBuilder();
            for (URL url : registryURLs) {
                registryURLsBuilder.append(url.toFullString());
                registryURLsBuilder.append(",");
            }
            registryURLsBuilder.deleteCharAt(registryURLsBuilder.length() - 1); // 删除最后一个逗号
            map.put(REGISTRY_KEY, registryURLsBuilder.toString());
        }
        if (metadataReportConfig != null && metadataReportConfig.isValid()) {
            map.put(REGISTRY_KEY, EMPTY_PROTOCOL); // 元数据参数
        }
        if (CollectionUtils.isNotEmpty(provider.getParams())){
            for (Map.Entry<String, String> entry : provider.getParams().entrySet()) {
                map.put(entry.getKey(), entry.getValue());
            }
        }
        // Configurator
        List<Configurator> configurators = getConfigurators();
        if (CollectionUtils.isNotEmpty(configurators)){
            StringBuilder configuratorsBuilder = new StringBuilder();
            for (Configurator configurator : configurators) {
                configuratorsBuilder.append(configurator.getUrl().toFullString());
                configuratorsBuilder.append(",");
            }
            configuratorsBuilder.deleteCharAt(configuratorsBuilder.length() - 1); // 删除最后一个逗号
            map.put(CONFIGURATORS_SUFFIX, configuratorsBuilder.toString());
        }
        if (CollectionUtils.isNotEmpty(getProtocols())){
            List<ProtocolConfig> protocolConfigs = new ArrayList<>(getProtocols());
            for (ProtocolConfig protocol : protocolConfigs) {
                //添加动态配置
                map.put(CLUSTER_KEY,"failover");
                if (StringUtils.isNotEmpty(ha)){
                    map.put(HA_KEY, ha);
                }
                if (StringUtils.isNotEmpty(loadbalance)){
                    map.put(LOADBALANCE_KEY, loadbalance);
                }
                map.put(RIDES_KEY, "");
                appendExportParameters(map, protocolConfig);
                String contextPath = protocolConfig.getContextpath();
                if (StringUtils.isEmpty(contextPath)) {
                    contextPath = "";
                } else if (!contextPath.startsWith("/")) {
                    contextPath = "/" + contextPath;
                }
                map.put(CONTEXT_PATH_KEY, contextPath);
                if (protocolConfig.getServer() != null && protocolConfig.getServer().size() > 0) {
                    for (String key : protocolConfig.getServer().keySet()) {
                        map.put("server." + key, StringUtils.isNotEmpty(protocolConfig.getServer().get(key)) ? protocolConfig.getServer().get(key) : "");
                    }
                }
                String[] addresses = StringUtils.isNotEmpty(internalConfig.getEndpoint()) ? internalConfig.getEndpoint().split("\\s*[;]+\\s*") : null;
                if(null != addresses){
                    for (int i = 0; i < addresses.length; i++){
                        String address = addresses[i];
                        if (StringUtils.isEmpty(address)){
                            continue;
                        }
                        String[] ipAddr = address.split(":");
                        String oneIp = ipAddr[0].trim();
                        String port = ipAddr.length > 1 ? ipAddr[1].trim() : null;
                        map.put("ip." + i, oneIp);
                        if (null != port && port.length() > 0){
                            map.put("port." + i, port);
                        }
                    }
                    if (map.containsKey("ip.0") && (addresses.length > 1)) {
                        map.put("port.0", "-1");
                    }
                }
                // 计算权重
                String conStr = StringUtils.isNotEmpty(internalConfig.getEndpoint())?internalConfig.getEndpoint():internalConfig.getHost() +(StringUtils.isNotEmpty(internalConfig.getServerPort()) ? (":" + internalConfig.getServerPort()) :);
                Integer weight = defaultPortWeightForLocal(conStr);
                if(null != weight && weight > 0){
                    map.put(WEIGHT_KEY, weight.toString());
                }
                doExportUrlsFor1Protocol(protocol, map);
            }
        }else{
            String conStr = StringUtils.isNotEmpty(internalConfig.getEndpoint())?internalConfig.getEndpoint():internalConfig.getHost() +(StringUtils.isNotEmpty(internalConfig.getServerPort()) ? (":" + internalConfig.getServerPort()) :);
            Integer weight = defaultPortWeightForLocal(conStr);
            if(null != weight && weight > 0){
                map.put(WEIGHT_KEY, weight.toString());
            }
            doExportUrlsFor1Protocol(protocolConfig, map);
        }
        token = provider.getToken();
    }
    
    if (!ConfigUtils.isEmpty(token)) {
    
        if (ConfigUtils.isDefault(token)) {
    
            map.put(TOKEN_KEY, UUID.randomUUID().toString());
    
        } else {
    
            map.put(TOKEN_KEY, token);
    
        }
    
    }
    
    // Put the map data into serviceMetadata, which is related to metadata, and its function will be detailed later.
    
    serviceMetadata.getAttachments().putAll(map);
    
    // Get the values of host and port
    
    String host = findConfigedHosts(protocolConfig, registryURLs, map);
    
    Integer port = findConfigedPorts(protocolConfig, name, map);
    
    // Assemble URL based on the host, port obtained above, and the map collection obtained earlier
    
    URL url = new URL(name, host, port, getContextPath(protocolConfig).map(p -> p + "/" + path).orElse(path), map);
    
    // Use Configurator to override or add new parameters
    
    if (ExtensionLoader.getExtensionLoader(ConfiguratorFactory.class)
    
        .hasExtension(url.getProtocol())) {
    
        url = ExtensionLoader.getExtensionLoader(ConfiguratorFactory.class)
    
            .getExtension(url.getProtocol()).getConfigurator(url).configure(url);
    
    }
    
    ... ...
    
}

After the above preparation, the obtained service URL is as follows (for easier viewing, here each URL parameter is displayed separately on a line):

    dubbo://172.17.108.185:20880/org.apache.dubbo.demo.DemoService?
    
    anyhost=true
    
    &application=dubbo-demo-api-provider
    
    &bind.ip=172.17.108.185
    
    &bind.port=20880
    
    &default=true
    
    &deprecated=false
    
    &dubbo=2.0.2
    
    &dynamic=true
    
    &generic=false
    
    &interface=org.apache.dubbo.demo.DemoService
    
    &methods=sayHello,sayHelloAsync
    
    &pid=3918
    
    &release=
    
    &side=provider
    
    &timestamp=1600437404483
    

### Service Publishing Entry

After the service URL is assembled, the doExportUrlsFor1Protocol() method begins to execute the service publishing. Service publishing can be divided into **remote publishing** and **local publishing**, and the specific publishing method depends on the scope parameter in the service URL.

The scope parameter has three optional values, namely none, remote, and local, which represent not publishing, publishing locally, and publishing to the remote registry, respectively. You can see from the code of the doExportUrlsFor1Protocol() method below:

- The condition for publishing locally is scope != remote;
- The condition for publishing to the registry is scope != local.

The default value of the scope parameter is null, which means the service will be published locally and to the registry by default. Let's take a look at the specific implementation of the service publishing in the doExportUrlsFor1Protocol() method:

private void doExportUrlsFor1Protocol(ProtocolConfig protocolConfig, List<URL> registryURLs) {
    
    ... ...// Omit the process of assembling the service URL
    
    // Get the scope parameter from the URL, which can be none, remote, or local
    
    // respectively, representing not publishing, publishing locally, and 
    
    // publishing to the remote registry. The specific meaning will be explained later.
    
    String scope = url.getParameter(SCOPE_KEY);
    
    if (!SCOPE_NONE.equalsIgnoreCase(scope)) { // Only publish if scope is not none
    
        if (!SCOPE_REMOTE.equalsIgnoreCase(scope)) {// Publish locally
    
            exportLocal(url);
    
        }
    
        if (!SCOPE_LOCAL.equalsIgnoreCase(scope)) { // Publish to the remote registry
    
            if (CollectionUtils.isNotEmpty(registryURLs)) { // If at least one registry is configured
    
                for (URL registryURL : registryURLs) { // Publish the service to each registry
    
                    // The injvm protocol is only used in exportLocal() and will not publish the service to the registry
    
                    // So, ignore the injvm protocol here
    
                    if (LOCAL_PROTOCOL.equalsIgnoreCase(url.getProtocol())){
    
                        continue;
    
                    }
    
                    // Set the dynamic parameter of the service URL
    
                    url = url.addParameterIfAbsent(DYNAMIC_KEY, registryURL.getParameter(DYNAMIC_KEY));
    
                    // Create monitorUrl and add it to the service URL as the monitor parameter
    
                    URL monitorUrl = ConfigValidationUtils.loadMonitor(this, registryURL);
    
                    if (monitorUrl != null) {
    
                        url = url.addParameterAndEncoded(MONITOR_KEY, monitorUrl.toFullString());
    
                    }
    
                    // Set the proxy parameter of the service URL, which is the method (jdk or javassist) for generating dynamic proxies, added as a parameter to the RegistryURL
    
                    String proxy = url.getParameter(PROXY_KEY);
    
                    if (StringUtils.isNotEmpty(proxy)) {
    
                        registryURL = registryURL.addParameter(PROXY_KEY, proxy);
    
                    }
    
                    // Create the corresponding Invoker for the service implementation object. In the third parameter of the getInvoker() method, the service URL is added as the export parameter to the RegistryURL
    
                    // Here PROXY_FACTORY is an adapter of the ProxyFactory interface
    
                    Invoker<?> invoker = PROXY_FACTORY.getInvoker(ref, (Class) interfaceClass, registryURL.addParameterAndEncoded(EXPORT_KEY, url.toFullString()));
    
                    // DelegateProviderMetaDataInvoker is a simple decorator that associates the current ServiceConfig with the Invoker, and its invoke() method is delegated to the underlying Invoker object
    
                    DelegateProviderMetaDataInvoker wrapperInvoker = new DelegateProviderMetaDataInvoker(invoker, this);
    
                    // Call the Protocol implementation to publish the service
    
                    // Here PROTOCOL is an adapter of the Protocol interface
    
                    Exporter<?> exporter = PROTOCOL.export(wrapperInvoker);
    
                    exporters.add(exporter);
    
                }
    
            } else {
    
                // If there is no registry, only publish the service, and the service information will not be published to the registry. The Consumer cannot find the information of this service in the registry, but it can directly connect to it.
    
                // The publishing process is similar to the above process, but it will not be published to the registry
    
                Invoker<?> invoker = PROXY_FACTORY.getInvoker(ref, (Class) interfaceClass, url);
    
                DelegateProviderMetaDataInvoker wrapperInvoker = new DelegateProviderMetaDataInvoker(invoker, this);
    
                Exporter<?> exporter = PROTOCOL.export(wrapperInvoker);
```java
    exporters.add(exporter);

}

// Metadata related operations

WritableMetadataService metadataService = WritableMetadataService.getExtension(url.getParameter(METADATA_KEY, DEFAULT_METADATA_STORAGE_TYPE));

if (metadataService != null) {

    metadataService.publishServiceDefinition(url);

}

}

}

this.urls.add(url);
}

Local Export #

After understanding the entry logic of local and remote export, let’s dive into the logic of local export.

In the exportLocal() method, the Protocol is replaced with the injvm protocol, the host is set to 127.0.0.1, and the port is set to 0, resulting in a new LocalURL, as follows:

injvm://127.0.0.1/org.apache.dubbo.demo.DemoService?anyhost=true

&application=dubbo-demo-api-provider

&bind.ip=172.17.108.185

&bind.port=20880

&default=true

&deprecated=false

&dubbo=2.0.2

&dynamic=true

&generic=false

&interface=org.apache.dubbo.demo.DemoService

&methods=sayHello,sayHelloAsync

&pid=4249

&release=

&side=provider

&timestamp=1600440074214

After that, the corresponding ProxyFactory implementation is found by the ProxyFactory interface adapter (JavassistProxyFactory is used by default), and the getInvoker() method is called to create an Invoker object. Finally, the implementation of InjvmProtocol is found by the Protocol interface adapter, and the export() method is called to publish the service. The implementation of the exportLocal() method is as follows:

private void exportLocal(URL url) {

    URL local = URLBuilder.from(url) // Create a new URL

            .setProtocol(LOCAL_PROTOCOL)

            .setHost(LOCALHOST_VALUE)

            .setPort(0)

            .build();

    // Local export

    Exporter<?> exporter = PROTOCOL.export(

            PROXY_FACTORY.getInvoker(ref, (Class) interfaceClass, local));

    exporters.add(exporter);

}

The implementation of InjvmProtocol is relatively simple, and it is not shown here. If you are interested, you can study it in the source code.

Remote Export #

After introducing local export, let’s take a look at the core logic of remote export. The process of publishing a remote service is much more complex than that of local export.

In the doExportUrlsFor1Protocol() method, when publishing a remote service, it will iterate through all the RegistryURLs and select the corresponding Protocol extension to publish the service based on the RegistryURL. We know that the RegistryURL is the registry:// protocol, so the RegistryProtocol implementation is used here.

Let’s take a look at the core flow of the RegistryProtocol.export() method:

public <T> Exporter<T> export(final Invoker<T> originInvoker) throws RpcException {

    // Convert "registry://" protocol to "zookeeper://" protocol

    URL registryUrl = getRegistryUrl(originInvoker);

    // Get the export parameter, which stores a ProviderURL of the "dubbo://" protocol

    URL providerUrl = getProviderUrl(originInvoker);

    // Get the configuration directory to be monitored. Here, the category=configurators parameter is added based on the ProviderURL and encapsulated into an OverrideListener to record it in the overrideListeners collection

    final URL overrideSubscribeUrl = getSubscribedOverrideUrl(providerUrl);

    final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);

    overrideListeners.put(overrideSubscribeUrl, overrideSubscribeListener);

    // When initialized, it will check the Override configuration once and override the ProviderURL

    providerUrl = overrideUrlWithConfig(providerUrl, overrideSubscribeListener);

    // Export the service, which will start the corresponding Server by executing the DubboProtocol.export() method at the bottom

    final ExporterChangeableWrapper<T> exporter = doLocalExport(originInvoker, providerUrl);

    // Get the Registry object corresponding to the RegistryURL, which depends on the RegistryFactory introduced in the previous lesson

    final Registry registry = getRegistry(originInvoker);

    // Get the Provider URL to be published to the registry, which will delete some redundant parameter information

    final URL registeredProviderUrl = getUrlToRegistry(providerUrl, registryUrl);

    // Determine whether to register the service based on the value of the "register" parameter

    boolean register = providerUrl.getParameter(REGISTER_KEY, true);

    if (register) { // Call the register() method of the Registry to publish the registeredProviderUrl to the registry

        register(registryUrl, registeredProviderUrl);

    }

    // Record Provider-related information in the ProviderModel

    registerStatedUrl(registryUrl, registeredProviderUrl, register);

    // Subscribe to override data from the registry, mainly by monitoring the configurators node of this service

    registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);

    exporter.setRegisterUrl(registeredProviderUrl);

    exporter.setSubscribeUrl(overrideSubscribeUrl);

    // Trigger the RegistryProtocolListener listener

    notifyExport(exporter);

    return new DestroyableExporter<>(exporter);

}

As we can see, the process of remote export can be roughly divided into the following 5 steps.

Prepare URLs, such as ProviderURL, RegistryURL, and OverrideSubscribeUrl.
Export the Dubbo service. Call the DubboProtocol.export() method in the doLocalExport() method to start the Server on the Provider side.
Register the Dubbo service. In the register() method, call the ZookeeperRegistry.register() method to register the service with ZooKeeper.
Subscribe to the Override configuration of the Provider side. Call the ZookeeperRegistry.subscribe() method to subscribe to the configuration changes of the configurators node in the registry.
Trigger the RegistryProtocolListener listener.

The detailed process of remote export is shown in the following diagram:

Drawing 1.png

Summary

In this lesson, we focused on the core process of Dubbo service export.

First, we introduced the methods related to service export in the DubboBootstrap, focusing on the start() and exportServices() methods. Then, we detailed the three core steps of the ServiceConfig class: parameter checking, immediate (or delayed) execution of the doExport() method for publishing, and callback to the relevant listeners for service publishing.

Next, we analyzed the doExportUrlsFor1Protocol() method, which is the entry point for publishing a service and determines the service publishing process. It involves the assembly of the Provider URL, the local service publishing process, and the remote service publishing process, all of which we have analyzed in detail.