29 Network Development (Part 2): How to Handle Network Requests with Rust? #
Hello, I’m Chen Tian.
In the last lecture, we introduced how to do network development based on TCP using Rust, listening with TcpListener and connecting with TcpStream. On the *nix operating system level, a TcpStream is essentially a file descriptor. It’s important to note that when dealing with network applications, there are some issues that must be faced:
- Networks are unreliable
- Network latency can be very high
- Bandwidth is limited
- Networks are very insecure
We can use TCP and protocols built on top of TCP to deal with network unreliability; use queues and timeouts to deal with network latency; use streamlined binary structures, compression algorithms, and certain tricks (such as HTTP 304) to reduce bandwidth usage and unnecessary network transmission; finally, we need to use secure protocols like TLS or the noise protocol to protect data in transit.
Today, we continue looking at the network communication models mainly involved in network development.
Bidirectional Communication #
In the example of a TCP server in the last lecture, what was done was bidirectional communication. This is the most typical type of communication:
Once the connection is established, both the server and the client can initiate transmission to each other as needed. The entire network operates in full duplex mode. Protocols that we are familiar with, such as TCP/WebSocket, run on this model.
The advantage of bidirectional communication is that the direction of data flow is unrestricted, and one end does not have to wait for the other to send data, allowing for relatively real-time network processing.
Request-Response #
In the world of web development, the request-response model is the most familiar model. The client sends a request, and the server returns a response based on the request. The entire network operates in half-duplex mode. HTTP/1.x runs on this model.
Generally, in the request-response model, until a client initiates a request, the server will not and cannot actively send data to the client. Also, the order of request sending and response returning is one-to-one and cannot be out of order, leading to Head-Of-Line blocking on the application layer.
The request-response model is very simple to handle. Despite its limitations, due to the popularity of the HTTP protocol, the request-response model is broadly used.
Control Plane/Data Plane Separation #
Sometimes, however, servers and clients conduct complex communications involving control signals and data streams. Because TCP has inherent network layer head-of-line blocking, when control signals and data are mixed in the same connection, large data streams can block control signals, causing increased latency and the inability to respond in a timely manner.
For example, using FTP, if a user transfers a 1GB file and then issues an ls command, if the file transfer and ls command are conducted in the same connection, the user will only see the result of the ls command after the file transfer is completed. This is obviously not user-friendly.
Therefore, we adopt the approach of separating the control plane from the data plane for network processing.
The client will first connect to the server to establish a control connection; the control connection is a long-lasting connection that exists until the interaction terminates. Then, as needed, they will create new temporary data connections for the transfer of large volumes of data, which will automatically close after completing their respective tasks.
In addition to FTP, many other protocols adopt a similar approach, such as the multimedia communication protocol SIP.
HTTP/2, and the Yamux protocol that draws on HTTP/2 for multiplexing, although running on the same TCP connection, also construct a control plane and data plane on the application layer.
Taking HTTP/2 as an example, the control plane (ctrl stream) can create many new streams for parallel processing of multiple application layer requests. For instance, in gRPC using HTTP/2, various requests can be processed in parallel, and data between different streams can be returned out of order without being restricted by the request-response model. Although HTTP/2 is still confined by TCP layer head-of-line blocking, it has addressed the application layer’s head-of-line blocking.
P2P Networks #
So far, we have discussed traditional client/server interaction models (C/S or B/S), where the roles of clients and servers within the network are not equal; the client is always the side initiating the connection, while the server is the side handling the connection.
The non-equal network model has many benefits. For example, clients do not need a public network address, can be hidden behind NAT devices (like NAT gateways, firewalls), and as long as the server has a public address, the network can be connected. Therefore, the client/server model is inherently centralized, with all connections requiring a middleman, the server, even for data exchanges between two clients. This model has become mainstream in the network world with the large-scale use of the internet.
However, many application scenarios require that the two ends of a communication can interact directly without an intermediary. For example, suppose A and B share a 1GB file. If it is transferred via a server, the data is effectively transferred twice, which is very inefficient.
The P2P model breaks this inequality, theoretically allowing any two nodes to connect directly. Each node is both a client and a server.
How to Build a P2P Network #
However, due to the historical lack of IPv4 addresses, as well as concerns about privacy and network security, internet service providers have used NAT devices heavily on the access side, rendering ordinary network users without a directly accessible public IP. As a result, building a P2P network must first address network connectivity.
The mainstream solution is for each node in the P2P network to first discover their public IP/port through a STUN server, then register their public IP/port at a bootstrap/signaling server, allowing others to discover them and establish connections with potential “neighbors”.
In a large P2P network, a node often has dozens of neighbors. Through these neighbors and the network information they hold, each node can build a routing table on how to locate a particular node (or data). Furthermore, nodes can join one or more topics, and then spread messages across the whole topic using certain protocols (like gossip):
Building a P2P network is usually more complex than a client/server network because the connections between nodes need to support many protocols: node discovery (mDNS, bootstrap, Kad DHT), node routing (Kad DHT), content discovery (pubsub, Kad DHT) and application layer protocols. The security of connections is also faced with different challenges.
Hence, we see P2P protocol connections often use multiplexing protocols similar to yamux within a single TCP connection to carry many other protocols:
Regarding network security, TLS can protect the client/server model effectively, but the creation, issuance, and trust of certificates are problematic for P2P networks. As a result, P2P networks tend to use their security protocols or noise protocol to build security protocols that can rival the security level of TLS 1.3.
Handling P2P Networks with Rust #
In Rust, there’s a mature library called libp2p for handling P2P networks.
Below is a simple P2P chat application that uses MDNS for node discovery and floodpub for message propagation on the local network. Comments are written in key places:
// Actual Rust code is presented here, demonstrating how to create a simple P2P chat application using the libp2p library.
To run this code, you need to include futures and libp2p in your Cargo.toml:
futures = "0.3"
libp2p = { version = "0.39", features = ["tcp-tokio"] }
The complete code can be found in the directory for this lecture on the GitHub repo of this course.
If you open window A and run:
❯ cargo run --example p2p_chat --quiet
And then windows B/C run:
❯ cargo run --example p2p_chat --quiet
Finally, window D uses a topic parameter, making it distinct from the other topics:
❯ cargo run --example p2p_chat --quiet -- hello
You will see that each node broadcasts via MDNS to discover existing P2P nodes on the local network. Now A/B/C/D are forming a P2P network, where A/B/C are subscribed to ’lobby’, while D is subscribed to ‘hello’.
Type “Hello from X” in each of the A/B/C/D windows to see:
Window A:
hello from A
(and so on; the results show messages exchanged between peers in the P2P network.)
It’s fine if you don’t understand this libp2p chat code. P2P involves a lot of new concepts and protocols that need to be grasped in advance, and this class isn’t specially dedicated to P2P, so if you’re interested in these concepts and protocols, you can read the libp2p documentation and its example code on your own.
Summary #
From the code in these two lectures, we can see that whether it’s handling high-level HTTP protocols or dealing with lower-level networks, Rust has a rich set of tools available for your use.
Through Rust’s network ecosystem, we can build a complete TCP server in just dozens of lines of code, or a simple P2P chat tool with just over a hundred lines. Rust is well equipped to both build your high-performance network server to handle known protocols or to construct your protocols.
We need to use various means to deal with the four problems in network development: networks are unreliable, network latency can be very high, bandwidth is limited, and networks are very insecure. Similarly, in the later development of a KV server, we will spend a lecture on how to use TLS to build secure networks.
Thinking Questions #
- Take a look at the libp2p documentation and example code, and clone libp2p to your local machine to run each example code.
- Read the NetworkBehaviour trait of libp2p and the corresponding implementation for floodsub.
- If you have the capacity and interest, try replacing the floodsub in this example with the more efficient and bandwidth-saving gossipsub.
Congratulations on completing the 29th check-in of your Rust studies. If you feel you have gained something, feel free to share with your friends and invite them to discuss together. See you in the next class!