02 Take a glance at the overall picture, grasp the Netty framework structure #

In the previous lesson, I introduced the features and advantages of Netty. Today, we will officially begin studying the technical principles of Netty.

When learning any technology, it is important to have a holistic view. When starting out, it is not advisable to get caught up in trivial technical details, which can lead to a dead end. In this lesson, we will use the overall architecture design of Netty as a starting point to help you establish clear learning objectives and build a learning framework for Netty. This framework will guide us throughout the entire learning process.

In this lesson, we will use Netty version 4.1.42 as the reference. I will introduce its overall structure, logical architecture, and source code structure.

Netty Overall Structure #

Netty is a meticulously designed network foundation component. The Netty official website provides a structure diagram but does not provide further explanations. From the diagram, we can clearly see that the Netty structure is divided into three modules:

1. Core Layer #

The Core layer is the essence of Netty. It provides common abstractions and implementations for low-level network communication, including an extensible event model, a generic communication API, support for zero-copy ByteBuf, and more.

2. Protocol Support Layer #

The Protocol Support layer covers the implementation of mainstream protocols, such as HTTP, SSL, Protobuf, compression, large file transmission, WebSocket, text, binary, and other mainstream protocols. In addition, Netty also supports custom application layer protocols. Netty’s rich protocol support reduces development costs for users, allowing us to quickly develop services such as HTTP and WebSocket based on Netty.

3. Transport Service Layer #

The Transport Service layer provides definitions and implementation methods for network transport capabilities. It supports different transport methods such as Socket, HTTP tunneling, and virtual machine pipeline. Netty abstracts and encapsulates data transmission for TCP, UDP, and other protocols, allowing users to focus more on implementing business logic without worrying about the details of low-level data transmission.

Netty’s module design is highly versatile and extensible. It is not only an excellent network framework but also a toolbox for network programming. Netty’s design philosophy is very elegant and worth learning from.

Now, we have a general understanding of the overall structure of Netty. Next, let’s take a look at the logical architecture of Netty and learn how it decomposes its functionality.

Netty Logical Architecture #

The following diagram shows the logical processing architecture of Netty. Netty’s logical processing architecture is a typical network layered architecture, divided into network communication layer, event dispatch layer, and service orchestration layer, with each layer having its own responsibilities. The diagram includes the core components used in each layer of Netty. I will introduce each core component used in each logical layer of Netty and how these components coordinate and operate together.

Network Communication Layer #

The network communication layer is responsible for performing network I/O operations. It supports various network protocols and connection operations for different I/O models. When network data is read into the kernel buffer, it will trigger various network events, which are then dispatched to the event dispatch layer for processing.

The core components of the network communication layer include the BootStrap, ServerBootStrap, and Channel.

• BootStrap & ServerBootStrap

Bootstrap means “to boot” and is mainly responsible for the start-up, initialization, and server connection processes of the entire Netty program. It acts as the main thread, connecting other core components of Netty.

As shown in the diagram below, Netty’s bootstrappers are divided into two types: Bootstrap for client-side bootstrapping and ServerBootStrap for server-side bootstrapping. They both inherit from the abstract class AbstractBootstrap.

Bootstrap and ServerBootStrap are very similar, with a key difference being that Bootstrap is used to connect to a remote server and only binds to one EventLoopGroup. ServerBootStrap, on the other hand, is used for server startup and binds to a local port, employing two EventLoopGroups, typically known as the Boss and the Worker.

What roles do the Boss and Worker play in ServerBootStrap? What is their relationship? Here, the Boss and Worker can be understood as the “manager” and “employees”. Each server has one Boss and a group of Workers who handle connections. The Boss continuously receives new connections and assigns them to Workers for processing. With Bootstrap components, we can conveniently configure and launch Netty applications. It serves as the entry point for the entire Netty framework, handling the initialization of all core components.

• Channel

The literal meaning of Channel is “channel,” which serves as the carrier for network communication. Channel provides basic APIs for network I/O operations such as register, bind, connect, read, write, and flush. Netty implements its own Channel based on JDK NIO Channel. Compared to JDK NIO, Netty’s Channel provides a higher level of abstraction and shields the complexity of underlying Socket. It gives Channel more powerful functionalities, so you don’t need to directly work with Java Socket classes when using Netty.

The following diagram shows the family of Channel:

Common Channel implementation classes include:

• NioServerSocketChannel: Asynchronous TCP server.
• NioSocketChannel: Asynchronous TCP client.
• OioServerSocketChannel: Synchronous TCP server.
• OioSocketChannel: Synchronous TCP client.
• NioDatagramChannel: Asynchronous UDP connection.
• OioDatagramChannel: Synchronous UDP connection.

Of course, Channel can have various states such as connection established, connection registered, data read/write, and connection destroyed. As the state changes, Channel goes through different lifecycles. Each state is associated with corresponding event callbacks. The table below lists the most common states of Channel and their corresponding event callbacks.

That’s all for the network communication layer. Let’s summarize briefly. Bootstrap and ServerBootstrap are responsible for starting client and server respectively. They are powerful auxiliary classes. Channel serves as the carrier for network communication, providing the ability to interact with underlying sockets. So how are events within the lifecycle of a Channel handled? That’s the responsibility of Netty’s event dispatching layer.

Event Dispatching Layer #

The event dispatching layer is responsible for aggregating and processing various events through the Reactor thread model. It integrates multiple types of events (I/O events, signal events, timer events, etc.) through the Selector main loop threads. The actual business logic is handled by relevant handlers in the service orchestration layer.

The core components of the event dispatching layer include EventLoopGroup and EventLoop.

• EventLoopGroup & EventLoop

EventLoopGroup is essentially a thread pool that mainly receives I/O requests and assigns threads to handle these requests. The following diagram explains the relationship between EventLoopGroups, EventLoops, and Channels.

From the above diagram, we can summarize several points about the relationship between EventLoopGroup, EventLoop, and Channel:

1. An EventLoopGroup often contains one or more EventLoops. EventLoop is responsible for handling all I/O events during the lifecycle of a Channel, such as accept, connect, read, write, etc.
2. At any given time, an EventLoop is bound to a thread, and each EventLoop handles multiple Channels.
3. Each time a new Channel is created, the EventLoopGroup selects an EventLoop to bind it to. This Channel can bind and unbind with EventLoop multiple times during its lifecycle. The following diagram is the family tree of EventLoopGroup. It can be seen that Netty provides multiple implementations of EventLoopGroup, and EventLoop is a subinterface of EventLoopGroup, so EventLoop can also be understood as EventLoopGroup, but it only contains one EventLoop.

The implementation class of EventLoopGroup is NioEventLoopGroup, which is also the most recommended thread model in Netty. NioEventLoopGroup inherits from MultithreadEventLoopGroup and is developed based on the NIO model. NioEventLoopGroup can be understood as a thread pool, where each thread is responsible for handling multiple channels, and a channel only corresponds to one thread.

EventLoopGroup is the core processing engine of Netty. So what is the relationship between EventLoopGroup and the Reactor thread model mentioned in previous courses? In fact, EventLoopGroup is the specific implementation of the Netty Reactor thread model. By creating different EventLoopGroup configurations, Netty can support three types of Reactor thread models:

1. Single thread model: EventLoopGroup contains only one EventLoop, where the boss and worker use the same EventLoopGroup.
2. Multi-thread model: EventLoopGroup contains multiple EventLoops, where the boss and worker use the same EventLoopGroup.
3. Master-slave multi-thread model: EventLoopGroup contains multiple EventLoops, where the boss is the master reactor and the worker is the slave reactor. They use different EventLoopGroups. The master reactor is responsible for creating new network connection channels and then registering them to the slave reactor.

After introducing the event scheduling layer, we can say that the Netty engine has started. The event scheduling layer is responsible for listening to network connections and read/write operations, and then triggering various types of network events. We need a mechanism to manage these complex events and execute them in an orderly manner. Next, let’s learn about the responsibilities of the core components in the Netty service orchestration layer.

Service orchestration layer #

The responsibility of the service orchestration layer is to assemble various services. It is the core processing chain of Netty, used to dynamically arrange and propagate network events in an orderly manner.

The core components of the service orchestration layer are ChannelPipeline, ChannelHandler, and ChannelHandlerContext.

• ChannelPipeline

ChannelPipeline is the core orchestration component in Netty, responsible for assembling various ChannelHandlers. The actual data encoding, decoding, and processing operations are completed by ChannelHandlers. ChannelPipeline can be understood as a list of ChannelHandler instances–internally, different ChannelHandlers are connected together through a bidirectional linked list. When I/O read/write events are triggered, the ChannelPipeline will sequentially call the ChannelHandler list to intercept and process the data of the Channel.

ChannelPipeline is thread-safe because each new Channel will be associated with a new ChannelPipeline. A ChannelPipeline is associated with an EventLoop, and an EventLoop is bound to a single thread.

Both ChannelPipeline and ChannelHandler are highly customizable components. Developers can control the manipulation of channel data through these two core components. Let’s take a look at the structure diagram of ChannelPipeline:

From the above diagram, it can be seen that ChannelPipeline contains two types of handlers: inbound ChannelInboundHandlers and outbound ChannelOutboundHandlers. Let’s understand these two concepts with the data transmission process between the client and the server.

Both the client and the server have their own ChannelPipeline. Taking the client as an example, when data is sent from the client to the server, this process is called “outbound”. On the contrary, it is called “inbound”. Inbound data is processed by a series of InboundHandlers, and then outbound data is processed by OutboundHandlers in the opposite direction. The encoding operation we often use is an outbound operation, and the decoding operation is an inbound operation. After receiving data from the client, the server needs to first process it with an inbound decoder and then send it back to the client with an outbound encoder. Therefore, a complete request-reply process between the client and the server can be divided into three steps: client outbound (sending data), server inbound (parsing data and executing business logic), and server outbound (responding with the result).

• ChannelHandler & ChannelHandlerContext

While introducing ChannelPipeline, you should have a basic understanding of ChannelHandlers. Data encoding/decoding and other transformation operations are actually handled by ChannelHandlers. From the perspective of developers, the most important thing to focus on is ChannelHandler. We rarely manipulate Channel directly and usually perform operations indirectly through ChannelHandlers.

The following diagram describes the relationship between Channel and ChannelPipeline. From the diagram, we can see that each time a Channel is created, a new ChannelPipeline is bound to it, and each time a ChannelHandler is added to the ChannelPipeline, a new ChannelHandlerContext is bound to it. Therefore, the relationship between ChannelPipeline, ChannelHandlerContext, and ChannelHandler is closely related. You may wonder, what is the purpose of binding a ChannelHandlerContext to each ChannelHandler?

ChannelHandlerContext is used to save the context of the ChannelHandler. Through ChannelHandlerContext, we can know the association between ChannelPipeline and ChannelHandler. ChannelHandlerContext enables interaction between ChannelHandlers. It contains all the events related to the lifecycle of ChannelHandlers, such as connect, bind, read, flush, write, close, etc. In addition, imagine a scenario where each ChannelHandler needs to implement some common logic. Without the abstraction of ChannelHandlerContext, would you need to write a lot of repetitive code?

These are the logical processing architectures of Netty. It can be seen that the architectural design of Netty is very reasonable, shielding the underlying NIO and framework implementation details. For business developers, they only need to focus on the arrangement and implementation of business logic. Once you understand the concepts of each core component in Netty, you may wonder how these components collaborate with each other. Combining the interaction process between the client and the server, I have drawn a diagram to give you a complete overview of the internal logic flow of Netty.

• When the server starts initializing, there are two components: the Boss EventLoopGroup and the Worker EventLoopGroup. The Boss is responsible for listening to network connection events. When a new network connection event arrives, the Channel is registered to the Worker EventLoopGroup.
• The Worker EventLoopGroup will be assigned an EventLoop responsible for handling the read and write events of the Channel. Each EventLoop is single-threaded and performs event looping through the Selector.
• When the client initiates I/O read and write events, the server EventLoop will read the data and then trigger various listeners for data processing through the Pipeline.
• The client data will be passed to the first ChannelInboundHandler in the ChannelPipeline, and after the data processing is completed, the processed data will be passed to the next ChannelInboundHandler.
• When the data is written back to the client, the processing result will be propagated in the ChannelOutboundHandler of the ChannelPipeline and finally reach the client.

The above is the overall interaction process of Netty components. You only need to have a general understanding of the responsibilities of each component and think of them as a production line. The detailed implementation principles of each component will be discussed in future courses.

Netty Source Code Structure #

The Netty source code is divided into multiple modules, and the responsibilities between the modules are clearly defined. As mentioned in the previous section about the overall functional modules, the division of the Netty source code modules is also in line with it.

We can not only use the all-in-one Jar package of Netty but also use individual utility packages. Below, I will introduce the commonly used utility packages in Netty based on its layered structure and actual business scenarios.

Core Modules #

The netty-common module is the core foundation of Netty, providing various utility classes that are required by other modules. In the common module, commonly used packages include general utility classes and custom concurrent packages.

• General utility classes: For example, TimerTask, HashedWheelTimer, etc.
• Custom concurrent packages: For example, asynchronous model Future & Promise, the enhanced FastThreadLocal compared to JDK, etc.

In the netty-buffer module, Netty has implemented a more complete ByteBuf utility class for the data carrier in network communication. With its human-friendly Buffer API design, it has become a perfect replacement for Java ByteBuffer. The dynamic design of ByteBuf not only solves the problem of memory waste caused by the fixed length of ByteBuffer but also allows for safer modification of the Buffer capacity. In addition, Netty has made many optimizations for ByteBuf, such as buffer pooling, CompositeByteBuf to reduce data copying, etc.

The netty-resolver module mainly provides parsing utilities for infrastructure-related things, including IP Address, Hostname, DNS, etc.

Protocol Support Modules #

The netty-codec module is mainly responsible for the encoding and decoding work, which converts raw byte data into business entity objects and vice versa. As shown in the diagram below, Netty supports decoders and encoders for most mainstream protocols in the industry, such as HTTP, HTTP2, Redis, XML, etc., which saves developers a lot of effort. In addition, this module provides abstract decoding and encoding classes ByteToMessageDecoder and MessageToByteEncoder, and by inheriting these classes, we can easily implement custom decoding and encoding logic.

The netty-handler module is mainly responsible for data processing. The part of data processing in Netty fundamentally consists of a sequence of ordered handlers. The netty-handler module provides out-of-the-box implementation classes for ChannelHandler, such as logging, IP filtering, traffic shaping, etc. If you need these functionalities, you only need to add the corresponding ChannelHandler to the pipeline.

Transport Service Modules #

The netty-transport module can be said to be the core module that provides data processing and transport in Netty. This module provides many important interfaces, such as Bootstrap, Channel, ChannelHandler, EventLoop, EventLoopGroup, ChannelPipeline, etc. The Bootstrap is responsible for the startup work of clients or servers, including the creation and initialization of Channels. The EventLoop is responsible for initiating I/O read and write operations to registered Channels. The ChannelPipeline is responsible for the ordered arrangement of ChannelHandlers. These components are mentioned when introducing the logical architecture of Netty.

The above only introduces the commonly used functional modules of Netty, and there are many other modules not listed one by one. Interested students can query the source code of Netty on GitHub (https://github.com/netty/netty).

Summary #

In this lesson, we have provided a preliminary introduction to the overall architecture of Netty from its overall structure, logical architecture, and source code structure. It can be seen that the layered architecture design of Netty is very reasonable, decoupling the logic between layers. For developers, they only need to extend the business logic.

When I first started with Netty, I was overwhelmed by the many core components, so I organized the relationships between the core components of Netty in the logical architecture to help you get started quickly. Starting from the next lesson, we will provide detailed introductions to the core components of Netty’s logical architecture.