41 Case Study Optimizing NAT Performance Part One

41 Case Study Optimizing NAT Performance Part One #

Hello, I am Ni Pengfei.

In the previous section, we explored the analysis method for network latency increase issues. Through a case study, we learned how to use tools such as hping3, tcpdump, Wireshark, and strace to debug and locate the root cause of the problem.

To briefly recap, network latency is the most crucial performance indicator for a network. Due to various factors such as network transmission and packet processing, network latency is inevitable. However, excessive network latency directly affects the user experience.

Therefore, when encountering an increase in network latency, you can start by analyzing network latency from different levels, such as routers, packet transmission and reception, packet processing, and even the application layer. Once you have identified the source of the network latency at a certain level, you can further pinpoint the bottleneck.

Today, I will show you another factor that may cause network latency, known as Network Address Translation (NAT).

Next, let’s learn about the working principle of NAT and understand how to optimize potential performance issues caused by NAT.

NAT Principles #

NAT technology can rewrite the source IP or destination IP of IP packets, and is widely used to solve the problem of shortage of public IP addresses. Its main principle is that multiple hosts in the network access external resources by sharing the same public IP address. At the same time, since NAT shields the internal network, it naturally provides security isolation for machines in the LAN.

NAT can be configured in a router that supports Network Address Translation (NAT), also known as NAT gateway, or in a Linux server. If the second method is used, the Linux server actually acts as a “soft” router.

The main purpose of NAT is to achieve address translation. Depending on the implementation method, NAT can be divided into three types:

  • Static NAT, where the internal IP and the public IP have a one-to-one permanent mapping relationship;

  • Dynamic NAT, where the internal IP is dynamically selected from a pool of public IPs for mapping;

  • Network Address and Port Translation (NAPT), which maps internal IPs to different ports of a public IP, allowing multiple internal IPs to share the same public IP address.

NAPT is the most popular type of NAT currently, and it is also the type of NAT we configure in Linux. According to different translation methods, we can further divide NAPT into three types.

The first type is Source Network Address Translation (SNAT), where the destination address remains unchanged, only the source IP or source port is replaced. SNAT is mainly used in scenarios where multiple internal IPs share the same public IP to access external resources.

The second type is Destination Network Address Translation (DNAT), where the source IP remains unchanged, only the destination IP or destination port is replaced. DNAT is mainly used to access various services in the internal network through different port numbers of the public IP, while hiding the real IP addresses of the backend servers.

The third type is bidirectional address translation, which uses both SNAT and DNAT simultaneously. When receiving network packets, DNAT is performed to convert the destination IP to the internal IP; when sending network packets, SNAT is performed to replace the source IP with an external IP.

Bidirectional address translation is actually a one-to-one mapping relationship between the external IP and the internal IP, so it is commonly used in virtualized environments to allocate floating public IP addresses to virtual machines.

To help you understand NAPT, I have drawn a diagram. Let’s assume:

  • The internal IP address of the local server is 192.168.0.2;

  • The public IP address in the NAT gateway is 100.100.100.100;

  • The address of the destination server baidu.com to be accessed is 123.125.115.110.

The process of SNAT and DNAT is shown in the following diagram:

From the diagram, you can see that:

  • When the server accesses baidu.com, the NAT gateway replaces the source address from the server’s internal IP 192.168.0.2 to the public IP address 100.100.100.100 before sending it to baidu.com;

  • When baidu.com responds with a packet, the NAT gateway replaces the destination address from the public IP address 100.100.100.100 to the server’s internal IP 192.168.0.2 before sending it to the server in the internal network.

Now that you understand the principles of NAT, let’s take a look at how to implement NAT functionality in Linux.

iptables and NAT #

The Netfilter framework provided by the Linux kernel allows for modification (such as NAT) and filtering (such as firewall) of network packets. Based on this framework, tools like iptables, ip6tables, and ebtables provide a more user-friendly command-line interface for system administrators to configure and manage NAT and firewall rules.

Among them, iptables is the most commonly used configuration tool. To master the principles and usage of iptables, it is essential to understand the working flow of network packets through Netfilter. The following diagram illustrates this process.

- (Image source: Wikipedia)

In this diagram, the square boxes with a green background represent tables, which are used to manage chains. Linux supports 4 types of tables, including filter (for filtering), nat (for NAT), mangle (for modifying packet data), and raw (for raw packets).

The square boxes with a white background accompanying the tables represent chains, which are used to manage specific iptables rules. Each table can contain multiple chains, such as:

  • The filter table has built-in INPUT, OUTPUT, and FORWARD chains.

  • The nat table has built-in PREROUTING, POSTROUTING, OUTPUT, etc.

Of course, you can also create your own chains as needed.

The gray conntrack represents the connection tracking module. It records the state of network connections in the connection tracking table (hash table) in the kernel and serves as the foundation for iptables state filtering (-m state) and NAT implementation.

All iptables rules are placed in these tables and chains and executed according to the order and priority of rules shown in the diagram.

To achieve NAT functionality, you mainly operate within the nat table. The nat table has three built-in chains:

  • PREROUTING, which executes rules before routing decisions, such as DNAT on incoming packets.

  • POSTROUTING, which executes rules after routing decisions, such as SNAT or MASQUERADE on outgoing or forwarded packets.

  • OUTPUT, like PREROUTING, but only handles packets being sent from the local machine.

Once you are familiar with the tables and chains in iptables, the corresponding NAT rules become relatively simple. Let’s take the three categories of NAPT as examples to explain in more detail.

SNAT #

According to the previous content, we know that SNAT needs to be configured in the POSTROUTING chain of the nat table. We commonly use two methods to configure it.

The first method is to configure SNAT for a subnet uniformly, and let Linux choose the default outgoing IP. This is actually what we often refer to as MASQUERADE:

$ iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j MASQUERADE

The second method is to configure SNAT for a specific IP address and specify the translated source address:

$ iptables -t nat -A POSTROUTING -s 192.168.0.2 -j SNAT --to-source 100.100.100.100

DNAT #

Now let’s look at DNAT. It is evident that DNAT needs to be configured in the PREROUTING or OUTPUT chain of the nat table, with PREROUTING being more commonly used (because it can also be used for forwarded packets).

$ iptables -t nat -A PREROUTING -d 100.100.100.100 -j DNAT --to-destination 192.168.0.2

Bidirectional Address Translation #

Bidirectional address translation means adding both SNAT and DNAT rules to establish a one-to-one mapping relationship between a public IP and an internal IP, i.e.:

$ iptables -t nat -A POSTROUTING -s 192.168.0.2 -j SNAT --to-source 100.100.100.100
$ iptables -t nat -A PREROUTING -d 100.100.100.100 -j DNAT --to-destination 192.168.0.2

When configuring NAT rules using iptables, Linux needs to forward network packets from other IPs, so don’t forget to enable Linux’s IP forwarding feature.

You can execute the following command to check if this feature is enabled. If the output is 1, it means that IP forwarding is already enabled:

$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

If it is not enabled yet, you can manually enable it by executing the following command:

$ sysctl -w net.ipv4.ip_forward=1
net.ipv4.ip_forward = 1

Of course, to avoid losing the configuration after a reboot, don’t forget to write the configuration to the /etc/sysctl.conf file:

$ cat /etc/sysctl.conf | grep ip_forward
net.ipv4.ip_forward=1

After explaining all these principles, what should we do when encountering performance issues with NAT? Based on what we have learned about NAT today, you can try to think of a solution and give it a try. We will continue “breaking it down” in the next lesson.

Summary #

Today, we have learned about the principle of Network Address Translation (NAT) in Linux.

NAT technology is capable of rewriting the source IP or destination IP of IP packets, making it commonly used to solve the problem of shortage of public IP addresses. It enables multiple hosts in a network to access external resources through sharing the same public IP address. At the same time, NAT also serves as a security measure by isolating the internal network from the outside.

NAT in Linux is implemented based on the kernel’s connection tracking module. As a result, it incurs high performance costs as it maintains the state of each connection. We will continue learning about the specific methods for analyzing NAT performance issues in the next lesson.

Reflections #

Finally, I’ll leave you with a question to ponder. MASQUERADE is the most commonly used SNAT rule, often used to provide a shared outgoing IP for multiple internal IP addresses.

Let’s assume we have a Linux server that uses MASQUERADE to provide outgoing access for all internal IPs. In that case,

  • Can MASQUERADE still function properly if multiple internal IP addresses have the same port number?

  • If there are a large number of internal IP addresses or requests, are there any risks with this approach?

Feel free to discuss this in the comments section or share this article with your colleagues and friends. Let’s practice in real-world scenarios and improve through communication.