28 How to Handle Network Requests Using Rust in Web Development

28 Network Development (Part 1): How to Handle Network Requests with Rust? #

Hello, I’m Chen Tian. Today we’re going to learn how to handle network requests in network development with Rust.

In the internet age, when we talk about network development, what often comes to mind is web development as well as the related HTTP and WebSocket protocols.

The reason we say “related” is because many aspects of the protocols are not well-known to most web developers, such as concurrency control during updates. When we talk about gRPC, many would consider it a mysterious “low-level” protocol, when in fact, it is merely an encapsulation of a binary message format under HTTP/2.

So for network development, this vast topic, it is neither possible nor necessary for us to cover everything. Today, we will briefly discuss the big picture of network development before focusing on how to use the Rust standard library and libraries in the ecosystem to handle networking, including methods for network connections and data processing. Finally, we’ll introduce several typical network communication models.

However, even with that focus, the content will still be extensive, so we’ll divide the topic into two lectures. If you’ve only focused on web development before, you may find much of the content challenging, and it’s recommended to brush up on related knowledge and concepts for a better understanding.

Let’s briefly review the ISO/OSI seven-layer model and the corresponding protocols. We won’t dwell on the physical layer, which mainly involves PHY chips:

Seven-layer model figure

In the seven-layer model, the link layer and network layer are usually built into operating systems, so we don’t need to deal with them directly. However, the presentation layer and application layer are closely related, so in practice, most applications only concern themselves with the network, transport, and application layers.

The network layer currently sees IPv4 and IPv6 competing side by side, with IPv6 not yet fully replacing IPv4. Apart from applications highly sensitive to latency (such as games), the vast majority use TCP in the transport layer. As for the application layer, the user-friendly and firewall-friendly HTTP protocol family—HTTP, WebSocket, HTTP/2, and the still-evolving HTTP/3—has emerged as the mainstream choice for applications.

Let’s look at the Rust ecosystem’s support for network protocols：

Rust ecosystem’s network protocol support figure

The Rust standard library provides std::net, which wraps the use of the entire TCP/IP stack. However, std::net is synchronous, so if you want to build a high-performance asynchronous network, you can use tokio. tokio::net offers nearly identical functionality to std::net, and once you’re familiar with std::net, tokio::net’s features won’t feel strange. So, we’ll start with std::net.

std::net #

Under std::net, we have data structures for dealing with TCP/UDP and some auxiliary structures:

  • TCP: TcpListener/TcpStream, handling server listening and client connections
  • UDP: UdpSocket, dealing with UDP sockets
  • Others: IpAddr is a wrapper for IPv4 and IPv6 addresses; SocketAddr represents an IP address + port structure

Let’s focus on TCP handling, and we’ll also touch on IpAddr/SocketAddr.

TcpListener/TcpStream #

To create a TCP server, we can use TcpListener to bind to a port, then loop over to process incoming client requests. Once a request is received, we get a TcpStream, which implements the Read/Write traits, and we can read from and write to the socket like a file:

use std::{
    io::{Read, Write},
    net::TcpListener,
    thread,
};

fn main() {
    let listener = TcpListener::bind("0.0.0.0:9527").unwrap();
    loop {
        let (mut stream, addr) = listener.accept().unwrap();
        println!("Accepted a new connection: {}", addr);
        thread::spawn(move || {
            let mut buf = [0u8; 12];
            stream.read_exact(&mut buf).unwrap();
            println!("data: {:?}", String::from_utf8_lossy(&buf));
            // A total of 17 bytes written
            stream.write_all(b"glad to meet you!").unwrap();
        });
    }
}

For the client, we can use TcpStream::connect() to obtain a TcpStream. Once the client request is accepted by the server, data can be sent or received:

use std::{
    io::{Read, Write},
    net::TcpStream,
};

fn main() {
    let mut stream = TcpStream::connect("127.0.0.1:9527").unwrap();
    // A total of 12 bytes written
    stream.write_all(b"hello world!").unwrap();

    let mut buf = [0u8; 17];
    stream.read_exact(&mut buf).unwrap();
    println!("data: {:?}", String::from_utf8_lossy(&buf));
}

In this example, after establishing a connection, the client sends 12 bytes of “hello world!” to the server. After the server reads and replies, the client tries to receive a complete 17-byte message from the server, “glad to meet you!”.

Currently, both the client and server have to hardcode the size of the data to receive, which is not flexible. We’ll see later how to better handle this using message framing.

We can see in the client’s code that we don’t need to explicitly close TcpStream, as TcpStream’s implementation also handles the Drop trait, which closes it when it goes out of scope.

But if you look at the TcpStream documentation, you’ll find it doesn’t implement Drop. That’s because TcpStream internally wraps sys_common::net::TcpStream, which in turn wraps Socket. Socket is a platform-dependent structure, such as FileDesc in Unix, which contains an OwnedFd, ultimately calling libc::close(self.fd) to close the file descriptor, and thus closing TcpStream.

General Methods for Handling Network Connections #

If you’re using a web framework to handle web traffic, then you don’t need to worry about network connections—the framework takes care of everything, and you just need to focus on handling specific routes or RPC logic. But if you’re building your own protocol on top of TCP, you need to consider how to properly handle network connections.

We’ve seen in our previous listener code that the main loop for network handling continuously accepts() a new connection:

fn main() {
    ...
    loop {
        let (mut stream, addr) = listener.accept().unwrap();
        println!("Accepted a new connection: {}", addr);
        thread::spawn(move || {
            ...
        });
    }
}

But the process of handling connections must take place in another thread or asynchronous task, rather than in the main loop, as this would block the main loop from accepting new connections until the current one is processed.

Thus, a loop + spawn is the basic way to handle network connections:

Network connection handling figure

However, using threads to handle frequent connections and disconnections can be inefficient, and figuring out how to share common data between threads can be a headache. Let’s take a closer look.

How to Handle a Large Number of Connections? #

Creating threads continuously can quickly exhaust available thread resources in the system when connection numbers rise. Additionally, since threads are scheduled by the operating system, each scheduling process involves a complex and not-so-efficient save and load context switch, so if threads are used, encountering a C10K bottleneck—when connection numbers reach the tens of thousands—can push the system to the limits in terms of resources and processing power.

From the resource perspective, too many threads take up too much memory; with a default stack size of 2MB in Rust, 10k connections can eat up 20GB of memory (although the default stack size can be modified as needed). From the processing power perspective, too many threads switching back and forth when connection data arrives keeps the CPU too busy, preventing it from handling more connection requests.

So, threads are not a good way to handle network services that potentially have a large number of connections.

To break the C10K bottleneck and reach C10M, we can only use user-space coroutines, whether stackful coroutines like Erlang/Golang or stackless coroutines like Rust’s asynchronous handling.

That’s why in most Rust code for network handling, there’s seldom direct use of std::net; instead, an asynchronous network runtime, such as tokio, is used.

How to Handle Shared Information? #

The second issue is that, when building a server, we often have some shared state available to all connections, such as database connections. In such scenarios, for shared data that doesn’t need modification, we can consider using Arc. If modifications are required, we can use Arc<Mutex>.

Network shared information figure

But using locks means that once a resource that’s locked needs to be accessed on a critical path, the entire system’s throughput will take a significant hit.

One idea is to reduce the granularity of locks, which will reduce conflicts. For instance, in a kv server, we can hash a key, modulo N, and distribute different keys across N memory stores, which reduces the lock granularity to 1/N of what it was:

Network lock granularity figure

Another idea is to change how shared resources are accessed so that they’re only accessed by one specific thread; other threads or coroutines can only interact with it via message-sending. If you’re familiar with Erlang/Golang, this should be familiar. In Rust, we can use the channel data structure.

Rust channels, whether from the standard library or third-party libraries, are very well implemented. Synchronous channels include the standard library’s mpsc:channel and the third-party crossbeam_channel, while asynchronous channels include tokio’s mpsc:channel and flume.

General Methods for Handling Network Data #

Let’s look at how to handle network data. Most of the time, we can use existing application layer protocols, such as HTTP.

Under the HTTP protocol, it’s generally agreed upon in the industry to use JSON to construct REST APIs/JSON APIs. Both the client and server have solid ecosystems to support this approach. You just need to use serde to make your defined Rust data structures Serializable/Deserializable and then use serde_json to generate serialized JSON data.

Here’s an example using rocket to handle JSON data. First, include it in Cargo.toml:

rocket = { version = "0.5.0-rc.1", features = ["json"] }

Then add the following code in main.rs:

#[macro_use]
extern crate rocket;

use rocket::serde::json::Json;
use rocket::serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
#[serde(crate = "rocket::serde")]
struct Hello {
    name: String,
}

#[get("/", format = "json")]
fn hello() -> Json<Hello> {
    Json(Hello { name: "Tyr".into() })
}

#[launch]
fn rocket() -> _ {
    rocket::build().mount("/", routes![hello])
}

Rocket is a full-featured web framework for Rust, similar to Django for Python. You can see that with rocket, a web server can be up and running with just over 10 lines of code.

If, for performance or other reasons, you need to define your own protocol between client and server, you can use the traditional TLV (Type-Length-Value) to describe the protocol data, or the more efficient and concise protobuf.

Using protobuf to Define a Custom Protocol #

protobuf is a very convenient tool for defining backward-compatible protocols. It can be used not only for building gRPC services but also for other network services.

In previous practical sessions, whether it was the implementation of thumbor or the kv server, we used protobuf. In the kv server session, we built a protocol based on protobuf over TCP, supporting a series of HXXX commands. We won’t go into how to use protobuf again here.

However, when building protocol messages with protobuf, keep in mind that protobuf generates variable-length messages, so you need to agree between the client and server how to define a message frame.

Common methods for defining message frames include appending “\r\n” at the end of the message and adding a length at the beginning of the message.

Appending “\r\n” to the message tail is generally used for text-based protocols, such as the HTTP header/POP3/Redis’s RESP protocol, etc. However, for binary protocols, it is better to add a fixed length to the message’s beginning, because the data in a protobuf message might coincidentally include consecutive “\r\n” characters. If we use “\r\n” to mark the end of the message, the messages might get mixed up, so it’s not an option.

But the two methods can be mixed. For example, the HTTP protocol uses “\r\n” to define the header, but its body uses length definition, albeit declared in the HTTP header’s Content-Length.

We mentioned that gRPC uses protobuf, so how does gRPC define message frames?

gRPC uses a five-byte Length-Prefixed-Message, including a compression flag byte and a four-byte message length. This way, when processing a gRPC message, we first read five bytes, extract the length N, and then read N bytes to obtain a complete message.

We can also use this method to handle custom protocols using protobuf.

Since this method of handling is very common, tokio provides a length_delimited codec for dealing with length-separated message frames, which can be used in conjunction with the Framed structure. If you check its documentation, you’ll find that in addition to simply supporting length prefixing, it also supports various complex scenarios.

For instance, if a message has a fixed header with 3 bytes for length and 5 bytes for other content, after processing by LengthDelimitedCodec, you’ll get complete data. You can also discard the length by using num_skip(3), which is very flexible:

LengthDelimitedCodec use cases figure

Here’s the server and client code I wrote using tokio/tokio_util. You can see that both the server and client use LengthDelimitedCodec for message framing.

Server code:

use anyhow::Result;
use bytes::Bytes;
use futures::{SinkExt, StreamExt};
use tokio::net::TcpListener;
use tokio_util::codec::{Framed, LengthDelimitedCodec};

#[tokio::main]
async fn main() -> Result<()> {
    let listener = TcpListener::bind("127.0.0.1:9527").await?;
    loop {
        let (stream, addr) = listener.accept().await?;
        println!("accepted: {:?}", addr);
        // LengthDelimitedCodec defaults to 4-byte length
        let mut stream = Framed::new(stream, LengthDelimitedCodec::new());

        tokio::spawn(async move {
            // Incoming messages will only include message body (excluding length)
            while let Some(Ok(data)) = stream.next().await {
                println!("Got: {:?}", String::from_utf8_lossy(&data));
                // Messages to be sent need only include message body; not length
                // Framed/LengthDelimitedCodec will automatically calculate and add length
                stream.send(Bytes::from("goodbye world!")).await.unwrap();
            }
        });
    }
}

And the client code:

use anyhow::Result;
use bytes::Bytes;
use futures::{SinkExt, StreamExt};
use tokio::net::TcpStream;
use tokio_util::codec::{Framed, LengthDelimitedCodec};

#[tokio::main]
async fn main() -> Result<()> {
    let stream = TcpStream::connect("127.0.0.1:9527").await?;
    let mut stream = Framed::new(stream, LengthDelimitedCodec::new());
    stream.send(Bytes::from("hello world")).await?;

    // Receive data returned by the server
    if let Some(Ok(data)) = stream.next().await {
        println!("Got: {:?}", String::from_utf8_lossy(&data));
    }

    Ok(())
}

Compared to the TcpListener/TcpStream code we used earlier, neither side needs to know the length of the data being sent, thanks to the StreamExt trait’s next() interface to get the next message; when sending, they just need to use the SinkExt trait’s send() interface, and the corresponding length will be automatically calculated and added at the beginning of the message frame to be sent.

Of course, if you want to run these two pieces of code, remember to add the following to your Cargo.toml:

[dependencies]
anyhow = "1"
bytes = "1"
futures = "0.3"
tokio = { version = "1", features = ["full"] }
tokio-util = { version = "0.6", features = ["codec"] }

Complete code can be found in the directory for this lecture in the course GitHub repo.

For simplicity, I didn’t directly use protobuf in the code. You can think of the content inside the Bytes sent and received as if it was serialized into binary by protobuf (if you want to see how protobuf is handled, you can review the source code for [thumbor] and [kv server]). We can see that using LengthDelimitedCodec, building a custom protocol becomes very simple. Just twenty lines of code complete a very complicated task.

Conclusion #

Today we discussed the ecosystem for network development with Rust, briefly learned about the Rust standard library’s std::net and the tokio library with excellent support for asynchronous tasks, and how to use them for handling network connections and data.

In most cases, we should favor asynchronous network development, so you’ll often see tokio in network-related code. As the main asynchronous network runtime in Rust, you can spend more time getting familiar with its features.

In the upcoming KV server implementation, we’ll see more detailed network handling. You’ll also see how we implement our own Stream to handle message frames.

Thought Exercise #

In the examples from the kv server, we