11 Firing Up the Furnace, a Detailed Explanation of Netty's Data Transmission Carrier Byte Buf

11 Firing up the furnace, a detailed explanation of Netty’s data transmission carrier ByteBuf #

In the process of learning the encoding and decoding section, we have seen that Netty extensively uses its own implementation of the ByteBuf utility class. ByteBuf is the data container of Netty, and all byte stream transmissions in network communication are done through ByteBuf. However, the JDK NIO package already provides a similar ByteBuffer class, so why does Netty have to reinvent the wheel? In this lesson, I will explain ByteBuf in detail.

Why Choose ByteBuf #

Let’s first introduce the JDK NIO ByteBuffer so that we can understand its shortcomings and pain points. The following figure shows the internal structure of ByteBuffer:

Netty11

From the figure, it can be seen that ByteBuffer has the following four basic attributes:

  • mark: a mark that has been set for a key position during reading, for easy rewinding to that position;
  • position: the current reading position;
  • limit: the length of valid data in the buffer;
  • capacity: the initial capacity of the buffer.

The relationship between the four basic attributes is: mark <= position <= limit <= capacity. With the basic attributes of ByteBuffer in mind, it is not difficult to understand some of its shortcomings when used.

First, the length allocated by ByteBuffer is fixed and cannot be dynamically expanded or contracted. Therefore, it is difficult to control how much capacity needs to be allocated. If the capacity is allocated too large, it can easily lead to memory waste. If the capacity is allocated too small, attempting to store too much data will throw a BufferOverflowException. When using ByteBuffer, to avoid insufficient capacity issues, you must check the capacity every time you store data. If it exceeds the maximum capacity of ByteBuffer, then you need to allocate a larger ByteBuffer and migrate the existing data to it. The whole process is relatively cumbersome and unfriendly to developers.

Second, ByteBuffer can only access the current operable position through the position attribute. Since the position pointer is shared for reading and writing, you need to frequently call the flip and rewind methods to switch between reading and writing states. Developers must handle ByteBuffer data reading and writing with great care, otherwise they may make mistakes.

ByteBuffer, as a frequently used data carrier in network communication, obviously cannot meet the needs of Netty. Netty has re-implemented a more efficient and user-friendly ByteBuf. Compared to ByteBuffer, ByteBuf provides many cool features:

  • Capacity can be dynamically expanded as needed, similar to StringBuffer;
  • Reading and writing use different pointers and the read/write mode can be freely switched without the need to call the flip method;
  • Zero-copy can be achieved through built-in composite buffer types;
  • Supports reference counting;
  • Supports buffer pooling.

Here we only have a simple understanding of ByteBuf. Next, let’s see how ByteBuf is implemented.

Internal Structure of ByteBuf #

Similarly, let’s compare the internal structure of ByteBuf with ByteBuffer.

Netty11（2）.png

From the figure, it can be seen that ByteBuf contains three pointers: readerIndex , writeIndex , and maxCapacity . Based on the positions of these pointers, the internal structure of ByteBuf can be divided into four parts:

The first part is the discarded bytes which represents invalid byte data that has been discarded.

The second part is the readable bytes which represents the bytes in ByteBuf that can be read. The number of bytes that can be read can be calculated as writeIndex - readerIndex. When N bytes are read from ByteBuf, the readerIndex will be increased by N. The readerIndex will not be greater than the writeIndex. When readerIndex == writeIndex, it means that ByteBuf is no longer readable.

The third part is the writable bytes where data written to ByteBuf is stored. When N bytes of data are written to ByteBuf, the writeIndex will be increased by N. When the writeIndex exceeds the capacity, it means that the capacity of ByteBuf is insufficient and needs to be expanded.

The fourth part is the expandable bytes which represents the maximum number of bytes that can be expanded in ByteBuf. When the writeIndex exceeds the capacity, it will trigger ByteBuf expansion. The maximum expansion is limited to the maxCapacity. If it exceeds the maxCapacity, an error will occur when writing to ByteBuf.

As can be seen, the ByteBuf re-designed by Netty effectively distinguishes the readable, writable, and expandable data, solving the shortcomings of ByteBuffer’s inability to expand and the complexity of switching reading and writing modes. Next, let’s learn about the core API of ByteBuf together. You can use it as a substitute for ByteBuffer.

Reference Counting #

ByteBuf is designed based on reference counting. It implements the ReferenceCounted interface, and the lifecycle of ByteBuf is managed by reference counting. As long as the reference count is greater than 0, it means that ByteBuf is still being used. When ByteBuf is no longer referenced by other objects, the reference count becomes 0, indicating that the object can be released.

When a new ByteBuf object is created, its initial reference count is 1. When the release() method of ByteBuf is called, the reference count is reduced by 1. Therefore, do not mistakenly think that calling release() guarantees the ByteBuf object will be released. You can verify this with the following code example:

ByteBuf buffer = ctx.alloc().directbuffer();

assert buffer.refCnt() == 1;

buffer.release();

assert buffer.refCnt() == 0;

Reference counting is of great help to Netty’s cache pool design. When the reference count is 0, the ByteBuf can be put into an object pool, avoiding the need to repeatedly create a ByteBuf for each use. This has great significance for implementing high-performance memory management.

In addition, Netty can use the characteristics of reference counting to implement a memory leak detection tool. The JVM does not know how Netty’s reference counting is implemented. When a ByteBuf object becomes unreachable, it will still be garbage collected. However, if the reference count of the ByteBuf is not 0 at this time, the object will not be released or put into the object pool, resulting in a memory leak. Netty will perform sample analysis on allocated ByteBuf to detect whether the ByteBuf has become unreachable and has a reference count greater than 0. It determines the location of the memory leak and outputs it to the log. You need to pay attention to the keyword “LEAK” in the log.

ByteBuf Classification #

ByteBuf has multiple implementation classes, each with different characteristics. The following diagram shows the family tree of ByteBuf, which can be divided into three different dimensions: Heap/Direct, Pooled/Unpooled, and Unsafe/Non-Unsafe. I will introduce the different characteristics of these three dimensions one by one.

image

Heap/Direct refers to on-heap and off-heap memory. Heap means memory allocated on the JVM heap, relying on byte arrays as the underlying data structure. Direct, on the other hand, refers to off-heap memory, which is not subject to JVM restrictions and relies on the underlying ByteBuffer implementation provided by the JDK.

Pooled/Unpooled indicates whether the memory is pooled or unpooled. Pooled means the memory is taken from a pre-allocated memory pool and can be returned to the ByteBuf memory pool after use, waiting for future allocations. Unpooled, on the other hand, means the memory is directly allocated and managed by the system API, ensuring that it can be managed and reclaimed by the JVM garbage collector.

The difference between Unsafe and Non-Unsafe lies in the safety of the operation. Unsafe indicates that physical memory is accessed by calling the Unsafe object provided by the JDK, using the offset + index approach to manipulate data. Non-Unsafe, on the other hand, does not rely on the JDK’s Unsafe object and directly manipulates data through array indices.

ByteBuf Core API #

I will divide the core API of ByteBuf into three aspects: Pointer Operations, Data Read and Write, and Memory Management. Before explaining the usage of the API, let’s review the custom decoder we implemented earlier to deepen our understanding of the ByteBuf API.

public final void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) {

    // Check if ByteBuf has enough readable bytes

    if (in.readableBytes() < 14) { 

        return;

    }

    in.markReaderIndex(); // Mark the current readerIndex

    in.skipBytes(2); // Skip magic number

    in.skipBytes(1); // Skip protocol version

    byte serializeType = in.readByte();

    in.skipBytes(1); // Skip message type

    in.skipBytes(1); // Skip status field

    in.skipBytes(4); // Skip reserved field

    int dataLength = in.readInt();

    if (in.readableBytes() < dataLength) {

        in.resetReaderIndex(); // Reset the readerIndex

        return;

    }

    byte[] data = new byte[dataLength];

    in.readBytes(data);

    SerializeService serializeService = getSerializeServiceByType(serializeType);

    Object obj = serializeService.deserialize(data);

    if (obj != null) {

        out.add(obj);

    }

}

Pointer Operations API #

  • readerIndex() & writeIndex()

readerIndex() returns the current reader index position, and writeIndex() returns the current write index position.

  • markReaderIndex() & resetReaderIndex()

markReaderIndex() is used to save the position of readerIndex, while resetReaderIndex() resets the current readerIndex to the previously saved position.

These two APIs are commonly used when implementing protocol decoding. For example, in the source code of the custom decoder mentioned above, before reading the protocol content length field, markReaderIndex() is used to save the position of readerIndex. If the number of readable bytes in the ByteBuf is less than the value of the length field, it means that the ByteBuf does not yet have a complete data packet. In this case, resetReaderIndex() is used to reset the position of readerIndex.

In addition, there are corresponding write index operations markWriterIndex() and resetWriterIndex(), which are similar to the read index operations, and I will not go into details about them.

Data Read and Write APIs #

  • isReadable()

isReadable() is used to determine if the ByteBuf is readable. If writerIndex is greater than readerIndex, then the ByteBuf is readable; otherwise, it is in a non-readable state.

  • readableBytes()

readableBytes() can be used to get the number of bytes that can be read from the ByteBuf. It can be calculated by subtracting readerIndex from writerIndex.

  • readBytes(byte[] dst) & writeBytes(byte[] src)

readBytes() and writeBytes() are two of the most commonly used methods. readBytes() reads the corresponding number of bytes from the ByteBuf into the byte array dst. readBytes() is often used in conjunction with readableBytes(), and the size of the byte array dst is usually equal to the size of readableBytes().

  • readByte() & writeByte(int value)

readByte() reads a byte from the ByteBuf, and increments readerIndex by 1. Similarly, writeByte() writes a byte to the ByteBuf, and increments writerIndex by 1. Netty also provides read and write methods for 8 primitive data types, such as readChar(), readShort(), readInt(), readLong(), writeChar(), writeShort(), writeInt(), writeLong(), but I won’t go into detail here.

  • getByte(int index) & setByte(int index, int value)

In addition to readByte() and writeByte(), there are also getByte() and setByte(). The get/set series of methods also provide read and write operations for 8 primitive types. What is the difference between these two series of methods? The read/write methods change the readerIndex and writerIndex pointers when reading and writing, while the get/set methods do not change the position of the pointers.

Memory Management APIs #

  • release() & retain()

As introduced earlier, the basic concept of reference counting has been introduced. Each time release() is called, the reference count is decremented by 1, and each time retain() is called, the reference count is incremented by 1.

  • slice() & duplicate()

slice() is equivalent to slice(buffer.readerIndex(), buffer.readableBytes()), which by default cuts the data between readerIndex and writerIndex, with a maximum capacity of the number of readable bytes in the original ByteBuf. The allocated memory and reference count are shared with the original ByteBuf.

duplicate() differs from slice() in that it cuts the entire original ByteBuf. The allocated memory and reference count are also shared. If data is written to the ByteBuf allocated by duplicate(), it will affect the underlying data of the original ByteBuf.

  • copy()

copy() copies all the information from the original ByteBuf. All the data is independent, and writing data to the ByteBuf allocated by copy() will not affect the original ByteBuf.

So far, we have covered almost all the core API of ByteBuf. The design of separating the read and write pointers in ByteBuf does bring many practical and convenient functions. In the development process, you no longer need to worry about the headache of flip and rewind operations.

ByteBuf Practical Exercises #

After learning about the internal structure and core APIs of ByteBuf, let’s demonstrate how ByteBuf should be used through a simple example. The code is shown below.

public class ByteBufTest {

    public static void main(String[] args) {

        ByteBuf buffer = ByteBufAllocator.DEFAULT.buffer(6, 10);

        printByteBufInfo("ByteBufAllocator.buffer(5, 10)", buffer);

        buffer.writeBytes(new byte[]{1, 2});

        printByteBufInfo("write 2 Bytes", buffer);

        buffer.writeInt(100);

        printByteBufInfo("write Int 100", buffer);

        buffer.writeBytes(new byte[]{3, 4, 5});

        printByteBufInfo("write 3 Bytes", buffer);

        byte[] read = new byte[buffer.readableBytes()];

        buffer.readBytes(read);

        printByteBufInfo("readBytes(" + buffer.readableBytes() + ")", buffer);

        printByteBufInfo("BeforeGetAndSet", buffer);

        System.out.println("getInt(2): " + buffer.getInt(2));

        buffer.setByte(1, 0);

        System.out.println("getByte(1): " + buffer.getByte(1));

        printByteBufInfo("AfterGetAndSet", buffer);

    }

    private static void printByteBufInfo(String step, ByteBuf buffer) {

        System.out.println("------" + step + "-----");

        System.out.println("readerIndex(): " + buffer.readerIndex());

        System.out.println("writerIndex(): " + buffer.writerIndex());

        System.out.println("isReadable(): " + buffer.isReadable());

        System.out.println("isWritable(): " + buffer.isWritable());

        System.out.println("readableBytes(): " + buffer.readableBytes());

        System.out.println("writableBytes(): " + buffer.writableBytes());

        System.out.println("maxWritableBytes(): " + buffer.maxWritableBytes());

        System.out.println("capacity(): " + buffer.capacity());

        System.out.println("maxCapacity(): " + buffer.maxCapacity());

    }

}

I won’t paste the output of the program here. I suggest you try to think about how readerIndex and writerIndex are changed, and then run the above code to verify the results.

Based on the code example, let’s summarize some key points to note when using the ByteBuf API:

  • write series methods will change the position of writerIndex. When writerIndex is equal to the capacity, the buffer is set to a non-writable state.
  • When writing data to a non-writable buffer, the buffer will attempt to expand, but the maximum capacity after expansion cannot exceed the maxCapacity. If the data written exceeds the maxCapacity, the program will throw an exception.
  • read series methods will change the position of readerIndex, while get/set methods will not change the positions of readerIndex/writerIndex.

Summary #

In this lesson, we introduced Netty’s powerful data container - ByteBuf. It not only solves the shortcomings of ByteBuffer in JDK NIO, but also provides a more user-friendly interface. Many developers have replaced ByteBuffer with ByteBuf, even if they are not writing a network application. As the most basic data structure in Netty, you must master it proficiently. This is the way to become proficient in Netty. In the next lessons, we will introduce the memory management and related design of Netty around ByteBuf.