23 Core Interface Introduction RPC Layer Skeleton Decapsulation #

In the previous lessons, we delved into the Dubbo Remoting layer in the Dubbo architecture, and learned about the underlying network model and thread model of Dubbo. Starting from this lesson, we will introduce a layer on top of Dubbo Remoting, the Protocol layer (as shown in the figure below). The Protocol layer is the user of the Remoting layer, creating ExchangeClients and ExchangeServers through the Exchangers facade class, and creating corresponding ChannelHandler and Codec2 implementations and handing them over to the Exchange layer for decoration.

Position of the Protocol layer in the Dubbo architecture

The Protocol layer corresponds to the dubbo-rpc module in the Dubbo source code. The structure of this module is shown in the figure below:

Structure of the dubbo-rpc module

We can see that there are many modules, similar to the dubbo-remoting module. Among them, dubbo-rpc-api abstracts specific protocols, service exposure, service reference, and proxying, etc. It is the core of the entire Protocol layer. The remaining modules, such as dubbo-rpc-dubbo, dubbo-rpc-grpc, dubbo-rpc-http, etc., are specific protocols supported by Dubbo and can be regarded as implementations of the dubbo-rpc-api module.

dubbo-rpc-api #

First, let’s take a look at the package structure of the dubbo-rpc-api module, as shown in the figure below:

Package structure of the dubbo-rpc-api module

According to the structure of the dubbo-rpc-api module shown in the above figure, we can see that the dubbo-rpc-api module includes the following core packages.

• filter package: When performing service reference, a series of filters will be applied, including many filters.
• listener package: In the process of service publishing and service reference, we can add listeners to listen to corresponding events. The interfaces related to listeners, such as Adapter and Wrapper implementations, are in this package.
• protocol package: Some abstract classes that implement the Protocol interface and the Invoker interface are located in this package. They mainly provide some common logic for concrete implementations of the Protocol interface and the Invoker interface.
• proxy package: Provides the ability to create proxies. In this package, it supports generating local proxy classes using JDK dynamic proxies and Javassist bytecode.
• support package: includes the RpcUtils utility class, Protocol implementations related to Mock, and Invoker implementations.

Interfaces and classes that are not in the above packages are more core abstract interfaces, and the classes in the above packages are mostly implementations of these interfaces. Now let’s introduce the core interfaces in the org.apache.dubbo.rpc package.

Core Interfaces #

The core interfaces involved in the Dubbo RPC layer are Invoker, Invocation, Protocol, Result, Exporter, ProtocolServer, and Filter. These interfaces abstract different concepts of the Dubbo RPC layer. They seem to be independent of each other, but they work together to build the skeleton of the Dubbo RPC layer. Below, we will introduce the meanings of these core interfaces one by one.

First, let’s introduce a very important interface in Dubbo - the Invoker interface. It can be said that the Invoker interface permeates the entire implementation of Dubbo code, and many design ideas in Dubbo tend to converge on the concept of Invoker. However, for those who have just started to read this part of the code, it may not be very friendly.

Here, with the help of the following simplified diagram, we will compare and explain the two most critical types of Invokers: the service provider Invoker and the service consumer Invoker.

Core diagram of Invoker

Taking the Consumer in the dubbo-demo-annotation-consumer example project as an example, it obtains a DemoService object, as shown below. This is actually a proxy (i.e., Proxy) that will complete the network call through the Invoker at the bottom:

@Component("demoServiceComponent")

public class DemoServiceComponent implements DemoService {

    @Reference

    private DemoService demoService;

    @Override

    public String sayHello(String name) {

        return demoService.sayHello(name);

    }

}

Next, let’s look at the Provider implementation in the dubbo-demo-annotation-provider example:

@Service

public class DemoServiceImpl implements DemoService {

    @Override

    public String sayHello(String name) {

        return "Hello " + name + ", response from provider: " + RpcContext.getContext().getLocalAddress();

    }

}

The DemoServiceImpl class will be wrapped as an instance of AbstractProxyInvoker and a corresponding Exporter instance will be generated. When the Dubbo Protocol layer receives a request, it will find this Exporter instance and call the corresponding AbstractProxyInvoker instance to complete the invocation of the Provider logic. Here, I have identified the two most important types of Invokers and briefly introduced their working scenarios. Of course, there are other types of Invokers in Dubbo, which we will introduce one by one later.

Next, let’s take a look at the specific definition of the Invoker interface, as shown below:

public interface Invoker<T> extends Node {

    // Service interface

    Class<T> getInterface();

    // Perform a call, also known as a "session" by some people, you can understand it as a call

    Result invoke(Invocation invocation) throws RpcException;

}

Invocation Interface is the parameter of the Invoker.invoke() method, which abstracts the target service and method information, related parameter information, specific parameter values, and some additional information of an RPC call. The specific definition is as follows:

public interface Invocation {

    // Unique identifier of the invoked service

    String getTargetServiceUniqueName();

    // Name of the invoked method

    String getMethodName();

    // Name of the invoked service

    String getServiceName();

    // Collection of parameter types

    Class<?>[] getParameterTypes();

    // Collection of parameter signatures

    default String[] getCompatibleParamSignatures() {

        return Stream.of(getParameterTypes())

                .map(Class::getName)

                .toArray(String[]::new);

    }

    // Specific parameter values of this call

    Object[] getArguments();

    // Invoker object associated with this call

    Invoker<?> getInvoker();

    // Invoker object can set some key-value attributes,
    // these attributes will not be passed to the provider

    Object put(Object key, Object value);

    Object get(Object key);

    Map<Object, Object> getAttributes();

    // Invocation can carry key-value information as additional
    // information, which will be passed to the provider together,
    // note the difference with attributes

    Map<String, String> getAttachments();

    Map<String, Object> getObjectAttachments();

    void setAttachment(String key, String value);

    void setAttachment(String key, Object value);

    void setObjectAttachment(String key, Object value);

    void setAttachmentIfAbsent(String key, String value);

    void setAttachmentIfAbsent(String key, Object value);

    void setObjectAttachmentIfAbsent(String key, Object value);

    String getAttachment(String key);

    Object getObjectAttachment(String key);

    String getAttachment(String key, String defaultValue);

    Object getObjectAttachment(String key, Object defaultValue);

}

Result Interface is the return value of the Invoker.invoke() method, which abstracts the return value (or exception) and additional information of an invocation. We can also add callback methods which will be triggered when the RPC invocation method ends. The specific definition of the Result Interface is as follows:

public interface Result extends Serializable {

    // Get/set the return value of this invocation

    Object getValue();

    void setValue(Object value);

    // If this invocation throws an exception, it can be accessed
    // through the following three methods

    Throwable getException();

    void setException(Throwable t);

    boolean hasException();

    // The recreate() method is a composite operation. If an exception occurs
    // during this invocation, the exception is thrown directly. If there is
    // no exception, the result is returned.
}

Object recreate() throws Throwable;

// Add a callback function to be triggered when the RPC call is completed
Result whenCompleteWithContext(BiConsumer<Result, Throwable> fn);

<U> CompletableFuture<U> thenApply(Function<Result, ? extends U> fn);

// Block the thread and wait for the RPC call to complete (or timeout)
Result get() throws InterruptedException, ExecutionException;

Result get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException;

// The Result can also carry additional information
Map<String, String> getAttachments();

Map<String, Object> getObjectAttachments();

void addAttachments(Map<String, String> map);

void addObjectAttachments(Map<String, Object> map);

void setAttachments(Map<String, String> map);

void setObjectAttachments(Map<String, Object> map);

String getAttachment(String key);

Object getObjectAttachment(String key);

String getAttachment(String key, String defaultValue);

Object getObjectAttachment(String key, Object defaultValue);

void setAttachment(String key, String value);

void setAttachment(String key, Object value);

void setObjectAttachment(String key, Object value);

In the previous introduction of the Provider-side Invoker, it was mentioned that our implementation of the business interface will be wrapped in an AbstractProxyInvoker object, and then exposed by the Exporter for Consumers to call the service. Exposer exposes the implementation of Invoker, which essentially allows the Provider to find the corresponding Invoker based on the various information in the request. We can maintain a Map, where the Key can be constructed based on the information in the request, and the Value is the Exporter object that encapsulates the corresponding service Bean, thus meeting the requirements of service publishing.

Let’s first take a look at the definition of the Exporter interface:

public interface Exporter<T> {

    // Get the underlying wrapped Invoker object
    Invoker<T> getInvoker();

    // Cancel the exposure of the underlying Invoker object
    void unexport();

}

To listen for service publication events and unpublication events, Dubbo defines an SPI extension interface called ExporterListener interface, which is defined as follows:

@SPI
public interface ExporterListener {

    // This method is triggered when a service is exported
    void exported(Exporter<?> exporter) throws RpcException;

    // This method is triggered when a service is unexported
    void unexported(Exporter<?> exporter);

}

Although ExporterListener is an extension interface, Dubbo itself does not provide any useful extension implementations, so we need to provide our own specific implementation to listen for the events we are interested in.

Similarly, we can add InvokerListener listeners to listen for events triggered when Consumer references a service. The definition of the InvokerListener interface is as follows:

@SPI
public interface InvokerListener {

    // This method is triggered when a service is referred
    void referred(Invoker<?> invoker) throws RpcException;

    // This method is triggered when a reference to a service is destroyed
    void destroyed(Invoker<?> invoker);

}

The Protocol interface is one of the core interfaces at the Dubbo Protocol layer. It defines the export() and refer() methods. The specific definition is as follows:

@SPI("dubbo") // Default implementation uses DubboProtocol
public interface Protocol {

    // Default port
    int getDefaultPort();

    // Export an Invoker, the export() method needs to be idempotent,
    // meaning that multiple invocations of export() have the same effect as a single invocation
    @Adaptive
    <T> Exporter<T> export(Invoker<T> invoker) throws RpcException;

    // Refer to an Invoker, the refer() method returns an Invoker object based on the parameters,
    // which can be used by the Consumer to invoke the service on the Provider side
    @Adaptive
    <T> Invoker<T> refer(Class<T> type, URL url) throws RpcException;

    // Destroy the Exporter and Invoker objects used by the export() and refer() methods,
    // and release the resources occupied by the current Protocol object
    void destroy();

    // Return all the ProtocolServers used by the current Protocol
    default List<ProtocolServer> getServers() {
        return Collections.emptyList();
    }

}

In the implementation of the Protocol interface, the export() method is not simply wrapping the Invoker object into an Exporter object and returning it. It also involves creating proxy objects and starting underlying Servers. The refer() method not only queries Invokers based on the type and URL parameters, but also involves creating related Clients.

Dubbo specifically defines a ProxyFactory interface at the Protocol layer as a factory for creating proxy objects. The ProxyFactory interface is an extension interface, which defines the getProxy() method for creating proxy objects from Invoker, as well as the getInvoker() method for wrapping the proxy object back into an Invoker object.

@SPI("javassist")
public interface ProxyFactory {

    // Create a proxy object for the given Invoker object
    @Adaptive({PROXY_KEY})
    <T> T getProxy(Invoker<T> invoker) throws RpcException;

    @Adaptive({PROXY_KEY})
    <T> T getProxy(Invoker<T> invoker, boolean generic) throws RpcException;

    // Wrap the given proxy object as an Invoker object, it can be considered as the inverse operation of getProxy()
    @Adaptive({PROXY_KEY})
    <T> Invoker<T> getInvoker(T proxy, Class<T> type, URL url) throws RpcException;

}

Seeing the @SPI annotation on ProxyFactory, we know that its default implementation uses javassist to create code objects. Of course, Dubbo also provides other ways to create code, such as JDK dynamic proxy.

The ProtocolServer interface is a simple wrapper for the previously introduced RemotingServer, and its implementation is also very simple, so it will not be further explained here.

The last core interface to be introduced is the Filter interface. Regarding Filter, it is believed that those who have done Java Web programming should be very familiar with this basic concept. In Java Web development, a Filter is used to intercept HTTP requests. The Filter interface in Dubbo is similar in function, which is used to intercept Dubbo requests.

In the Filter interface of Dubbo, the invoke() method passes the request to the subsequent Invoker for processing (the subsequent Invoker object may be wrapped by a Filter). The specific definition of the Filter interface is as follows:

@SPI
public interface Filter {

    // Pass the request to the subsequent Invoker for processing
    Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException;

    interface Listener { // Used to listen for responses or exceptions
        void onResponse(Result appResponse, Invoker<?> invoker, Invocation invocation);

        void onError(Throwable t, Invoker<?> invoker, Invocation invocation);
    }

}

Filter is also an extension interface, and Dubbo provides a rich set of Filter implementations for functional extension. Of course, we can also provide our own Filter implementation to extend Dubbo’s functionality.

Summary #

In this lesson, we first introduced the position of the Dubbo RPC layer in the entire Dubbo framework, and then explained the structure of the dubbo-rpc-api layer and the basic functions provided by each package. Next, we provided a detailed introduction to the core interfaces involved in the Dubbo RPC layer, including Invoker, Invocation, Protocol, Result, ProxyFactory, ProtocolServer, and other core interfaces, as well as the interfaces of the extension classes such as ExporterListener and Filter.