03 Dubbo SPI Analysis Interface Implementation Polar Turnaround Above #

In order to better achieve the OCP principle (which stands for “Open for extension, Closed for modification”), Dubbo adopts a “microkernel + plugin” architecture. What is a microkernel architecture? Microkernel architecture, also known as plugin architecture, is a scalable architecture that separates functionality. The kernel functionality is relatively stable and only responsible for managing the lifecycle of plugins, without constantly modifying it due to the expansion of system functionality. The functional extensions are all encapsulated in plugins, which are independent modules containing specific functionality to extend the functionality of the kernel system.

In the microkernel architecture, the kernel usually uses Factory, IoC, OSGi, and other methods to manage the lifecycle of plugins. Dubbo ultimately decided to use the SPI mechanism to load plugins. Dubbo SPI is designed with reference to the native SPI mechanism of JDK, and it has been optimized for performance and enhanced functionality. Therefore, before explaining Dubbo SPI, it is necessary to first introduce the working principle of JDK SPI.

JDK SPI #

SPI (Service Provider Interface) is mainly used by framework developers. For example, when accessing a database using the Java language, we use the java.sql.Driver interface. Different database products have different underlying protocols and provide different implementations of java.sql.Driver. When developing the java.sql.Driver interface, developers are not aware of which database the user will ultimately use. In this case, the Java SPI mechanism can be used to find the concrete implementation class for the java.sql.Driver interface during runtime.

1. JDK SPI mechanism #

When a service provider provides an implementation of an interface, they need to create a file named after the service interface in the META-INF/services/ directory in the classpath. This file records the specific implementation classes of the service interface provided by the jar package. When an application introduces the jar package and needs to use the service, the JDK SPI mechanism can obtain the specific implementation class names by looking up the configuration file in the META-INF/services/ directory of the jar package, and load and instantiate the implementation classes, ultimately using the implementation classes to accomplish the business functionality.

Next, let’s demonstrate the basic usage of JDK SPI through a simple example:

First, we need to create a Log interface to simulate the logging functionality:

public interface Log { 

    void log(String info); 

}

Next, we provide two implementations - Logback and Log4j, representing implementations of two different logging frameworks, as shown below:

public class Logback implements Log { 

    @Override 

    public void log(String info) { 

        System.out.println("Logback:" + info); 

    } 

} 

public class Log4j implements Log { 

    @Override 

    public void log(String info) { 

        System.out.println("Log4j:" + info); 

    } 

}

In the resources/META-INF/services directory of the project, add a file named com.xxx.Log. This is the configuration file that JDK SPI needs to read. The specific content is as follows:

com.xxx.impl.Log4j 

com.xxx.impl.Logback

Finally, create the main() method, where the above configuration file will be loaded, all implementations of the Log interface will be instantiated, and their log() methods will be executed, as shown below:

public class Main { 

    public static void main(String[] args) { 

        ServiceLoader<Log> serviceLoader =  

                ServiceLoader.load(Log.class); 

        Iterator<Log> iterator = serviceLoader.iterator(); 

        while (iterator.hasNext()) { 

            Log log = iterator.next(); 

            log.log("JDK SPI");  

        } 

    } 

} 

// Output: 

// Log4j:JDK SPI 

// Logback:JDK SPI

2. Analysis of JDK SPI source code #

From the above example, we can see that the entry method of JDK SPI is the ServiceLoader.load() method. Next, let’s analyze its specific implementation in depth.

In the ServiceLoader.load() method, it first tries to obtain the currently used ClassLoader (get the ClassLoader bound to the current thread, and if it fails, use the SystemClassLoader), and then calls the reload() method. The call relationship is shown in the following diagram:

In the reload() method, the providers cache (a LinkedHashMap collection) is first cleaned up. This cache is used to cache the implementation objects created by ServiceLoader, where the Key is the fully qualified class name of the implementation class, and the Value is the object of the implementation class. Next, a LazyIterator iterator is created to read the SPI configuration file and instantiate the implementation class objects.

The specific implementation of the ServiceLoader.reload() method is as follows:

// Cache, used to cache the implementation objects created by ServiceLoader

private LinkedHashMap<String,S> providers = new LinkedHashMap<>(); 

public void reload() { 

    providers.clear(); // Clear the cache 

    lookupIterator = new LazyIterator(service, loader); // Iterator 

}

In the previous example, the iterator used in the main() method is implemented by ServiceLoader.LazyIterator. The Iterator interface has two key methods: hasNext() and next(). The next() method in this case ultimately calls its nextService() method, and the hasNext() method ultimately calls the hasNextService() method. The call relationship is shown in the following diagram:

First, let’s take a look at the LazyIterator.hasNextService() method, which is mainly responsible for finding the SPI configuration file in the META-INF/services directory and iterating over it. The rough implementation is as follows:

private static final String PREFIX = "META-INF/services/"; 

Enumeration<URL> configs = null; 

Iterator<String> pending = null; 

String nextName = null; 

private boolean hasNextService() { 

    if (nextName != null) { 

        return true; 

    } 

    if (configs == null) { 

        // Concatenate the PREFIX prefix with the name of the service interface, this is the SPI configuration file defined in the META-INF directory (e.g., META-INF/services/com.xxx.Log)

        String fullName = PREFIX + service.getName(); 

        // Load the configuration file 

        if (loader == null) 

            configs = ClassLoader.getSystemResources(fullName); 

        else 

            configs = loader.getResources(fullName); 

    } 

    // Traverse the content of the configuration file line by line

    while ((pending == null) || !pending.hasNext()) {  

        if (!configs.hasMoreElements()) { 

            return false; 

        } 

        // Parse the configuration file 

        pending = parse(service, configs.nextElement());  

    }

nextName = pending.next(); // update the nextName field

return true;
}

After completing the parsing of the SPI configuration file in the hasNextService() method, let’s take a look at the LazyIterator.nextService() method. This method is responsible for instantiating the implementation class read by the hasNextService() method, and it caches the instantiated objects in the providers collection. The core implementation is as follows:

private S nextService() {

String cn = nextName;

nextName = null;

// Load the class specified in the nextName field

Class<?> c = Class.forName(cn, false, loader);

if (!service.isAssignableFrom(c)) { // Check the type

fail(service, "Provider " + cn + " not a subtype");

}

S p = service.cast(c.newInstance()); // Create an object of the implementation class

providers.put(cn, p); // Add the implementation class name and corresponding instance object to the cache

return p;

}

The above is the underlying implementation of the iterator used in the main() method. Finally, let’s take a look at how the iterator obtained using the ServiceLoader.iterator() method in the main() method is implemented. This iterator is implemented as an anonymous inner class that depends on the LazyIterator. The core implementation is as follows:

public Iterator<S> iterator() {

return new Iterator<S>() {

// knownProviders is used to iterate through the providers cache

Iterator<Map.Entry<String,S>> knownProviders

= providers.entrySet().iterator();

public boolean hasNext() {

// First check the cache, if not found, load using LazyIterator

if (knownProviders.hasNext())

return true;

return lookupIterator.hasNext();

}

public S next() {

// First check the cache, if not found, load using LazyIterator

if (knownProviders.hasNext())

return knownProviders.next().getValue();

return lookupIterator.next();

}

// Remove method is omitted

};

}

3. Application of JDK SPI in JDBC #

After understanding the principle of JDK SPI, let’s take a look at how JDBC uses the JDK SPI mechanism to load implementation classes provided by different database vendors.

In JDK, only the java.sql.Driver interface is defined, and the specific implementation is provided by different database vendors. Let’s analyze the JDBC implementation package provided by MySQL as an example.

In the META-INF/services directory of the mysql-connector-java-*.jar package, there is a java.sql.Driver file with only one line of content, as shown below:

com.mysql.cj.jdbc.Driver

When connecting to a MySQL database using the mysql-connector-java-*.jar package, we use the following statement to create a database connection:

String url = "jdbc:xxx://xxx:xxx/xxx";

Connection conn = DriverManager.getConnection(url, username, pwd);

DriverManager is the JDBC driver manager provided by JDK, and the code segment inside is as follows:

static {

loadInitialDrivers();

println("JDBC DriverManager initialized");

}

When calling the getConnection() method, the DriverManager class is loaded, parsed, and triggers the execution of the static block by the Java virtual machine. In the loadInitialDrivers() method, all the implementation classes of the java.sql.Driver interface are scanned and instantiated using the JDK SPI mechanism. The core implementation is as follows:

private static void loadInitialDrivers() {

String drivers = System.getProperty("jdbc.drivers")

// Load all java.sql.Driver implementation classes using the JDK SPI mechanism

ServiceLoader<Driver> loadedDrivers =

ServiceLoader.load(Driver.class);

Iterator<Driver> driversIterator = loadedDrivers.iterator();

while(driversIterator.hasNext()) {

driversIterator.next();

}

String[] driversList = drivers.split(":");

for (String aDriver : driversList) { // Initialize Driver implementation classes

Class.forName(aDriver, true,

ClassLoader.getSystemClassLoader());

}

}

In the com.mysql.cj.jdbc.Driver implementation class provided by MySQL, there is also a static block that creates a com.mysql.cj.jdbc.Driver object and registers it in the DriverManager.registeredDrivers collection (of type CopyOnWriteArrayList). It is shown below:

static {

java.sql.DriverManager.registerDriver(new Driver());

}

In the getConnection() method, DriverManager gets the corresponding Driver object from the registeredDrivers collection and creates a Connection. The core implementation is as follows:

private static Connection getConnection(String url, java.util.Properties info, Class<?> caller) throws SQLException {

// Omit try/catch block and permission handling logic

for(DriverInfo aDriver : registeredDrivers) {

Connection con = aDriver.driver.connect(url, info);

return con;

}

}

Summary #

In this article, we started with an example to introduce the basic usage of the JDK SPI mechanism, then analyzed the core principles and underlying implementations of the JDK SPI, and conducted an in-depth analysis of its source code. Finally, we used the MySQL JDBC implementation as an example to analyze the usage of the JDK SPI in practice.

Although the JDK SPI mechanism is simple and easy to use, it does have some small flaws. You can think about it first, and I will answer this question in the next lesson when analyzing the Dubbo SPI mechanism.