48 Configuration Center Design and Implementation Centralized Configuration and I Want Localization Too Below

48 Configuration Center Design and Implementation Centralized Configuration and I Want Localization Too Below #

In the previous lesson, we analyzed in detail the implementation of the Configuration interface and the DynamicConfiguration interface. The implementation of the DynamicConfiguration interface is the foundation of the dynamic configuration center. So how does Dubbo’s dynamic configuration center start? We will discuss this in detail in this lesson.

Base Configuration Class #

During the initialization of DubboBootstrap, the ApplicationModel.initFrameworkExts() method is called to initialize all implementations of the FrameworkExt interface, as shown in the following diagram:

Image 1.png

FrameworkExt Inheritance Diagram

The relevant code snippet is as follows:

public static void initFrameworkExts() {
    Set<FrameworkExt> exts = ExtensionLoader.getExtensionLoader(FrameworkExt.class).getSupportedExtensionInstances();
    for (FrameworkExt ext : exts) {
        ext.initialize();
    }
}

ConfigManager is used to manage all AbstractConfig objects in the current Dubbo node, including the implementation object of ConfigCenterConfig. The configuration center-related information we add through XML, Annotation, or API (such as the address, port, and protocol of the configuration center) will be converted into ConfigCenterConfig objects.

Multiple Configuration objects introduced in the previous lesson are maintained in the Environment, with the following specific meanings:

  • propertiesConfiguration (PropertiesConfiguration type): Configuration information provided by all OrderedPropertiesProvider implementations and configuration information specified in environment variables or -D parameters.
  • systemConfiguration (SystemConfiguration type): Configuration information added through -D parameters.
  • environmentConfiguration (EnvironmentConfiguration type): Configuration information added directly in environment variables.
  • externalConfiguration, appExternalConfiguration (InmemoryConfiguration type): When using the Spring framework and configuration of include-spring-env is set to true, the configuration will be automatically read from the Spring Environment. By default, the key values dubbo.properties and application.dubbo.properties will be read into these two InmemoryConfiguration objects in order.
  • globalConfiguration (CompositeConfiguration type): Used to combine the above configuration sources.
  • dynamicConfiguration (CompositeDynamicConfiguration type): Used to combine all the DynamicConfiguration corresponding to the current configuration center.
  • configCenterFirst (boolean type): Used to identify whether the configuration from the configuration center has the highest priority.

The Configuration objects mentioned above are initialized in the constructor of Environment, and in the initialize() method, the configurations obtained from the Spring Environment are filled into externalConfiguration and appExternalConfiguration. The relevant implementation code is as follows:

public Environment() {
    // Create the above Configuration objects
    this.propertiesConfiguration = new PropertiesConfiguration();
    this.systemConfiguration = new SystemConfiguration();
    this.environmentConfiguration = new EnvironmentConfiguration();
    this.externalConfiguration = new InmemoryConfiguration();
    this.appExternalConfiguration = new InmemoryConfiguration();
}

public void initialize() throws IllegalStateException {
    // Read the corresponding configurations and fill the above Configuration objects
    ConfigManager configManager = ApplicationModel.getConfigManager();
    Optional<Collection<ConfigCenterConfig>> defaultConfigs = configManager.getDefaultConfigCenter();
    defaultConfigs.ifPresent(configs -> {
        for (ConfigCenterConfig config : configs) {
            this.setExternalConfigMap(config.getExternalConfiguration());
            this.setAppExternalConfigMap(config.getAppExternalConfiguration());
        }
    });
    this.externalConfiguration.setProperties(externalConfigurationMap);
    this.appExternalConfiguration.setProperties(appExternalConfigurationMap);
}

Starting the Configuration Center #

After Environment is initialized, DubboBootstrap calls the startConfigCenter() method to start one or more configuration center clients. The core operations involve: calling the ConfigCenterConfig.refresh() method to refresh the configuration of the configuration center; and creating a DynamicConfiguration object based on the configuration in ConfigCenterConfig through the prepareEnvironment() method.

private void startConfigCenter() {
    Collection<ConfigCenterConfig> configCenters = configManager.getConfigCenters();
    if (CollectionUtils.isEmpty(configCenters)) { // No configuration center specified
        // omitted logic
    } else {
        for (ConfigCenterConfig configCenterConfig : configCenters) { // Multiple configuration centers may be configured
            configCenterConfig.refresh(); // Refresh the configuration
            // Check if the configuration of the configuration center is valid
            ConfigValidationUtils.validateConfigCenterConfig(configCenterConfig);

} } if (CollectionUtils.isNotEmpty(configCenters)) { // Create a CompositeDynamicConfiguration object to assemble multiple DynamicConfiguration objects CompositeDynamicConfiguration compositeDynamicConfiguration = new CompositeDynamicConfiguration(); for (ConfigCenterConfig configCenter : configCenters) { // Create the corresponding DynamicConfig object based on the ConfigCenterConfig and add it to the CompositeDynamicConfiguration compositeDynamicConfiguration.addConfiguration(prepareEnvironment(configCenter)); } // Record the CompositeDynamicConfiguration to the dynamicConfiguration field in the Environment environment.setDynamicConfiguration(compositeDynamicConfiguration); } configManager.refreshAll(); // Refresh all AbstractConfig configurations }

1. Refresh configuration center configuration #

First, let’s look at the ConfigCenterConfig.refresh() method. This method combines all the initialized Configuration objects in the Environment, and then iterates through the setter methods of all fields in ConfigCenterConfig, retrieving the final value of the corresponding field from the Environment. The specific implementation is as follows:

public void refresh() {
    // Get the Environment object
    Environment env = ApplicationModel.getEnvironment();
    // Merge all initialized Configuration and return
    CompositeConfiguration compositeConfiguration = env.getPrefixedConfiguration(this);
    Method[] methods = getClass().getMethods();
    for (Method method : methods) {
        if (MethodUtils.isSetter(method)) { // Get the setter method of each field in ConfigCenterConfig
            // Get the final value of this field based on the relevant configuration center configuration and the various Configuration in Environment
            String value = StringUtils.trim(compositeConfiguration.getString(extractPropertyName(getClass(), method)));
            // Call the setter method to update the corresponding field of ConfigCenterConfig
            if (StringUtils.isNotEmpty(value) && ClassUtils.isTypeMatch(method.getParameterTypes()[0], value)) {
                method.invoke(this, ClassUtils.convertPrimitive(method.getParameterTypes()[0], value));
            }
        } else if (isParametersSetter(method)) { // Set the parameters field, the logic is basically similar to setting other fields, but the implementation is different
            String value = StringUtils.trim(compositeConfiguration.getString(extractPropertyName(getClass(), method)));
            if (StringUtils.isNotEmpty(value)) {
                // Get the current parameters field
                Map<String, String> map = invokeGetParameters(getClass(), this);
                map = map == null ? new HashMap<>() : map;
                // Override the parameters collection
                map.putAll(convert(StringUtils.parseParameters(value), ""));
                // Set the parameters field
                invokeSetParameters(getClass(), this, map);
            }
        }
    }
}

Here we focus on the Environment.getPrefixedConfiguration() method. This method combines the existing Configuration objects in the Environment and the current ConfigCenterConfig in order, to obtain a CompositeConfiguration object that determines the final configuration information of the configuration center. The specific implementation is as follows:

public synchronized CompositeConfiguration getPrefixedConfiguration(AbstractConfig config) {
    // Create a CompositeConfiguration object, the prefix and id here are determined by ConfigCenterConfig
    CompositeConfiguration prefixedConfiguration = new CompositeConfiguration(config.getPrefix(), config.getId());
    // Wrap ConfigCenterConfig into ConfigConfigurationAdapter
    Configuration configuration = new ConfigConfigurationAdapter(config);
    if (this.isConfigCenterFirst()) { // Determine the location of the ConfigCenterConfig configuration based on the configuration
        // The sequence would be: SystemConfiguration -> AppExternalConfiguration -> ExternalConfiguration -> AbstractConfig -> PropertiesConfiguration
        // Combine the existing Configuration objects and ConfigCenterConfig in order
        prefixedConfiguration.addConfiguration(systemConfiguration);
        prefixedConfiguration.addConfiguration(environmentConfiguration);
        prefixedConfiguration.addConfiguration(appExternalConfiguration);
        prefixedConfiguration.addConfiguration(externalConfiguration);
        prefixedConfiguration.addConfiguration(configuration);
        prefixedConfiguration.addConfiguration(propertiesConfiguration);
    } else {
        // The priority is as follows: SystemConfiguration -> AbstractConfig -> AppExternalConfiguration -> ExternalConfiguration -> PropertiesConfiguration
        prefixedConfiguration.addConfiguration(systemConfiguration);
        prefixedConfiguration.addConfiguration(environmentConfiguration);
        prefixedConfiguration.addConfiguration(configuration);
        prefixedConfiguration.addConfiguration(appExternalConfiguration);
        prefixedConfiguration.addConfiguration(externalConfiguration);
        prefixedConfiguration.addConfiguration(propertiesConfiguration);
    }

        cc.setHighestPriority(true);
        cc.setPreferred(true);
        cc.setCheck(false);

然后将转换后的 ConfigCenterConfig 对象添加到 configManager 的配置中心列表中。最后，在 DubboBootstrap.startConfigCenter() 方法的最后，会调用 configManager.refreshAll() 方法，将所有的 ConfigCenterConfig 对象添加到 configManager 的缓存中，并触发它们的初始化。因此，如果注册中心被当做配置中心使用，那么它的初始化流程和明确指定配置中心是一样的。

配置中心初始化的最终流程 #

接下来看一下，DubboBootstrap 是如何启动配置中心初始化的。在 DubboBootstrap.initialize() 方法中，会调用 startConfigCenter() 方法，将启动配置中心初始化和相关的逻辑：

private void initialize() throws IllegalStateException {
    if (initialized) {
        throw new IllegalStateException("DubboBootstrap has already been initialized!");
    }
    initialized = true;
    // 省略部分代码...
    startConfigCenter(); // 启动配置中心初始化
    // 省略部分代码...
}

在 startConfigCenter() 方法中，会按照如下的顺序初始化配置中心：

private void startConfigCenter() {
    if (!CollectionUtils.isEmpty(configManager.getConfigCenters())) {
        // 初始化明确指定的配置中心
        configManager.getConfigCenters().forEach(cc -> prepareEnvironment(cc));
    }
    useRegistryAsConfigCenterIfNecessary(); // 将注册中心作为配置中心使用
    registerProviderUrls(); // 注册 Provider URL 到注册中心
}

可以看到，DubboBootstrap 会首先初始化明确指定的配置中心，然后再将注册中心作为配置中心使用。最后，会调用 registerProviderUrls() 方法，向注册中心注册 Provider URL。

综上所述，Dubbo 的配置中心初始化流程如下：

  1. 按照配置文件中配置的顺序初始化明确指定的配置中心。
  2. 如果没有明确指定配置中心，并且注册中心被设置为可以作为配置中心使用，在此时将注册中心作为配置中心。
  3. 初始化完成之后，将配置中心封装成 CompositeDynamicConfiguration 对象，记录到 Environment.dynamicConfiguration 字段中，以供后续使用。
  4. 配置中心初始化完成后，会调用 AbstractConfig 的 refresh() 方法，根据最新的配置更新 Dubbo 的配置。

特殊配置说明 #

关于配置中心，有一些特殊的配置需要特别说明一下。

首先是关于配置中心的配置文件名，默认使用的是 “dubbo.properties”。如果在 ConfigCenterConfig 中没有明确指定配置文件名，那么会优先使用 “dubbo.properties”，如果文件不存在，再使用 “application.properties”。

其次是关于配置中心的分组，默认使用的是应用名。具体来说，如果 ConfigCenterConfig 中的 group 字段没有配置，那么会使用 ApplicationConfig 中的 name 字段作为配置中心的分组。而对于注册中心，group 字段使用的是 “dubbo”。所以，在配置中心获取配置的时候，需要注意配置文件名和分组的问题。

配置中心的缓存 #

Dubbo 在初始化配置中心的时候，会将配置中心的相关信息缓存到 ConfigManager 中。这里的 ConfigManager 是一个全局唯一的对象，负责管理所有的配置，包括配置中心、注册中心、应用配置等。通过 ConfigManager 可以方便地获取指定类型的配置。

这样设计的目的是为了方便 Dubbo 的其他组件（比如注册中心、服务发现等）可以获取相关的配置信息。这样可以降低 Dubbo 的耦合性，各个组件之间不需要直接引用对方，只需要通过 ConfigManager 获取需要的配置即可。

配置中心的更新 #

在应用启动的过程中，Dubbo 会定时检查配置中心的更新情况，并根据需要更新相关的配置。Dubbo 通过 CompositeDynamicConfiguration 对象来管理配置中心，可以同时管理多个配置中心。

具体的更新流程如下：

  1. 首先，Dubbo 会获取所有的 DynamicConfiguration 对象，遍历它们并调用 getProperties() 方法获取最新的配置内容。
  2. 然后，Dubbo 会将获取到的最新配置内容转化成 Properties 对象，并与旧的配置进行对比。
  3. 如果新旧配置不相同，Dubbo 会将最新的配置反序列化成对应的配置对象，并触发相应的配置更新逻辑。
  4. 如果新旧配置相同，Dubbo 不会进行任何操作。

从上述流程可以看出，Dubbo 更新配置的基本原理是比较旧配置和新配置是否相同，如果不同，则触发对应的配置更新逻辑。这样设计的目的是为了避免不必要的配置更新，提高配置更新的效率。

综上所述，Dubbo 的配置中心是一个非常重要的组件，它提供了统一管理和动态更新配置的能力。通过配置中心，我们可以将配置集中存储，避免散落在各个应用中，从而大大降低了配置管理的难度。在实际应用中，我们可以根据不同的需求使用不同的配置中心，比如使用 ZooKeeper 作为配置中心，使用 Apollo 作为配置中心等。同时，Dubbo 的配置中心也提供了一些特殊的配置，比如文件名和分组等，需要注意配置的正确性。最后，Dubbo 的配置中心支持动态更新配置，会定期检查配置中心的更新情况，并将最新的配置应用到相应的组件中。通过配置中心，我们可以实现配置的动态变更，无需重启应用，提高了系统的可维护性和可扩展性。 if (registryConfig.getTimeout() != null) { cc.setTimeout(registryConfig.getTimeout().longValue()); } cc.setHighestPriority(false); // Lower priority here configManager.addConfigCenter(cc); }); startConfigCenter(); // Invoke the startConfigCenter() method again to initialize the configuration center }

After the initialization of the configuration center is completed, subsequent access to DynamicConfiguration can be directly obtained from the Environment. For example, DynamicConfigurationServiceNameMapping relies on DynamicConfiguration to manage the mapping between Service ID and Service Name.

Next, DubboBootstrap executes the loadRemoteConfigs() method to determine whether additional registry centers or protocols have been configured based on the updated externalConfigurationMap and appExternalConfigurationMap configuration information from the previous step. If so, they are converted into RegistryConfig and ProtocolConfig here and recorded in ConfigManager for subsequent use.

Then, DubboBootstrap executes the checkGlobalConfigs() method to perform a series of checks and initializations of AbstractConfig, such as ProviderConfig, ConsumerConfig, and MetadataReportConfig. The implementation is relatively simple and is not shown here.

Next, DubboBootstrap initializes MetadataReport, MetadataReportInstance, MetadataService, and MetadataServiceExporter through the initMetadataService() method. These components related to metadata have been deeply analyzed in previous lessons. The initialization process here is not complicated. If you are interested, you can refer to the source code for further study.

At the end of the initialization of DubboBootstrap, the initEventListener() method is called to add DubboBootstrap as an EventListener listener to the EventDispatcher. DubboBootstrap inherits the GenericEventListener abstract class as shown in the figure below:

(Insert image here)

GenericEventListener is a generic listener that allows subclasses to listen to any interested Event events by defining the corresponding onEvent() method. In GenericEventListener, the handleEventMethods collection is maintained. The Key is the subclass of Event, the event that the listener is interested in, and the Value is the corresponding onEvent() method for processing the Event.

In the constructor of GenericEventListener, all onEvent() methods implemented by the current GenericEventListener are found through reflection and recorded in the handleEventMethods field. The specific search logic is implemented in the findHandleEventMethods() method:

private Map<Class<?>, Set<Method>> findHandleEventMethods() {
        Map<Class<?>, Set<Method>> eventMethods = new HashMap<>();
        of(getClass().getMethods()) // Iterate through all methods in the current GenericEventListener subclass
                // Filter to get the onEvent() method, the specific filtering conditions are in the isHandleEventMethod() method:
                // 1. The method must be public
                // 2. The method has only one parameter, and the parameter is a subclass of Event
                // 3. The return type of the method is void and does not declare throwing exceptions
                .filter(this::isHandleEventMethod) 
                .forEach(method -> {
                    Class<?> paramType = method.getParameterTypes()[0];
                    Set<Method> methods = eventMethods.computeIfAbsent(paramType, key -> new LinkedHashSet<>());
                    methods.add(method);
                });
        return eventMethods;
    }

In the onEvent() method of GenericEventListener, the corresponding onEvent() method is found and called from the handleEventMethods collection based on the specific type of the received Event event:

public final void onEvent(Event event) {
        // Get the actual type of the Event
        Class<?> eventClass = event.getClass(); 
        // Find the corresponding onEvent() method based on the type of the Event and call it
        handleEventMethods.getOrDefault(eventClass,  emptySet()).forEach(method -> {
            ThrowableConsumer.execute(method, m -> {
                m.invoke(this, event);
            });
        });
    }

We can check all the methods of DubboBootstrap, and we currently don’t find any methods that meet the isHandleEventMethod() conditions. However, in another implementation of GenericEventListener-LoggingEventListener, we can see multiple methods that meet the isHandleEventMethod() conditions in the onEvent() method overload shown in the figure below. In these overloaded onEvent() methods, INFO logs are outputted.

(Insert image here)

This concludes the entire initialization process of DubboBootstrap and the logic related to the configuration center during this process.

Summary #

In this lesson, we focused on the core process of initializing the Dubbo dynamic configuration center and introduced the important component classes involved in this process.

First, we introduced the ConfigManager and Environment classes, which are very basic configuration classes. Then, we explained the core process of initializing the dynamic configuration center for DubboBootstrap, as well as the process of starting the dynamic configuration center. Finally, we analyzed the relevant content of the GenericEventListener listener.

If you have any questions or good experiences regarding this topic, please feel free to share them in the comments section.