10 Small Trials and Bold Moves Ii How to Break Through the Speed Limit of Vpc Networks

10 Small Trials and Bold Moves II How to break through the speed limit of VPC networks #

Hello, I’m Jingyuan.

In the lesson on Cold Start, I talked to you about how there are many factors that affect the speed of the function’s first execution. One key factor is: the connection time of the VPC network.

We know that VPC is a secure and isolated network environment provided by cloud vendors to users. Users can deploy their own services, such as Redis, databases, and message queues, in the VPC space. And our functions are usually created in a dedicated VPC for functions. A production-level function service application inevitably needs to interact with other resources and middleware. Therefore, the communication efficiency between VPCs becomes particularly important.

In today’s lesson, I will analyze the technical points for functions across VPCs through case studies, focusing on communication efficiency between VPCs.

I hope that through this lesson, you can gain some inspiration for practical application and subsequent optimization work in the Serverless field. Additionally, I hope you can take a systematic approach to discover more optimization methods and further break through the speed limit of function access.

Evolution of Cross-VPC Issues #

Regarding the ways of network connectivity in the cloud, I believe you are already familiar with them, such as dedicated connections (Express Connect), VPN gateways, and peering connections. These methods are used to establish connections between IDC and VPC or between VPCs.

Among these methods, peering connections can indeed solve the problem of cross-VPC communication. However, let’s imagine a scenario where each user uses this method to solve the problem. In that case, the platform would need to build connections on both ends each time, which may also lead to IP subnet conflicts. This approach is obviously not universal and is too costly.

Therefore, for serverless function computing scenarios, we usually establish communication using Elastic Network Interfaces (ENIs). ENIs are attached to cloud hosts or container instances. In other words, we associate the user’s VPC with the user’s device instance using ENIs, allowing communication between the device and the VPC.

However, in fact, let’s put aside the difference in network speed between these two methods. Whether it is a container instance or a host, they will face the same problems:

Long cold-start time: The creation of ENIs cannot be avoided, which takes time.
Resource limitations: Because of resource virtualization, it means it relies on hardware parameters and is limited by factors such as the kernel and CPU. Therefore, almost all cloud vendors limit the number of ENIs in VPCs, which limits the scalability of this method to a certain extent.
Resource waste: Since ENIs need to be attached to container instances or hosts, it means that multiple instances will occupy multiple ENI resources, resulting in resource waste.

As we can see, although they solve the problem of cross-VPC communication, they still bring performance and resource cost pressures. So, can we explore a way to share resources without having to create so many ENIs? One solution that comes to mind immediately is the egress proxy.

What is a proxy? #

How to proxy? Let’s take a look at the overall process.

Simply put, we add a Proxy Node between the Function Compute cluster and the user’s VPC. On the Proxy Node, we create a network interface (e.g., eth1), which is pre-created in the subnet of the user’s VPC, but the tenant’s permissions belong to the Function Compute VPC.

With the elastic network interface, the next step is to establish a connection between the function and the proxy node, which means building a link between the host of the function and the host of the proxy to pass the data packets, while keeping the source IP and destination IP unchanged.

We usually use tunneling technology. After the tunnel is established, there are two things to note. One is the forwarding handling of multiple network interfaces in the Proxy Node, and the other is the container network forwarding process, as functions usually run in containers.

Next, I will introduce how they work step by step.

Hands-on Practice #

In this lesson, I will guide you through a practical case of connecting VPC networks and demonstrate the whole process of accessing resources in a user VPC through a proxy server by using a Baidu Intelligent Cloud Server (BCC).

Before the experiment, we need to prepare two VPC subnets to simulate the creation of function clusters and user clusters. We also need to purchase three BCC cloud servers, with two in VPC1 and one in VPC2. The following diagram clearly describes the deployment logic:

Tunneling Technology #

After the preparation is completed, let’s take a look at tunneling technology.

What is a tunnel? In networking, a tunnel is a method of efficiently and securely connecting networks using protocols that are not supported by the network itself.

Tunnels in the kernel can be seen through ip tunnel help. There are currently five types: IPIP, GRE, SIT, ISATAP, and VTI.

In this lesson, we choose to use GRE (Generic Routing Encapsulation). It is a further version of the IPIP (IP in IP) protocol and can replace IPIP according to official descriptions. The concept and differences of tunneling technology itself are not the focus of this lesson. However, if you are interested, you can read more about it through the additional reading I provide at the end of this article.

Next, let’s create a tunnel to establish a connection between the func bcc1 and proxy bcc servers via GRE. We first execute the following commands on func bcc1 to create the GRE tunnel:

# Load the GRE module
modprobe ip_gre
lsmod | grep gre
# Create a GRE tunnel in gre0 mode, associate it with an IP, and assign the virtual local IP as 10.0.0.3
ip tunnel add gre0 mode gre local 192.168.80.8 remote 192.168.4.4 dev eth0
ip addr add 10.0.0.3 peer 10.0.0.4 dev mgre0
ip link set dev mgre0 up

Execute the following commands on proxy bcc to establish the GRE tunnel:

modprobe ip_gre
lsmod | grep gre
ip tunnel add mgre0 mode gre local 192.168.4.4 dev eth0
ip link set dev mgre0 up
ip address add 10.0.0.4/24 dev mgre0
ip neigh replace 10.0.0.3 lladdr 192.168.80.8 dev mgre0

Finally, we can verify whether the GRE tunnel between proxy bcc and func bcc1 is established by using the ping command.

At this point, we have completed half of the task of establishing the tunnel. Next, let’s see how proxy bcc can connect with user bcc.

Forwarding Technology #

We have noticed that proxy bcc and user bcc are not in the same VPC cluster. How do we solve this problem?

The logic we build is to create a virtual network card eth1 in the user VPC, which is subordinate to proxy bcc. This method is consistent with the “dynamic creation of network cards” method we mentioned in the “Evolution of Cross-VPC” above.

Here I need to mention that if you are a developer of business functions and the cloud provider does not provide the corresponding privileged account and the ability to create elastic network cards, you will not be able to perform the actual operation. However, you only need to understand the process. If you are a cloud platform maintenance worker, you can experience the operation in this procedure.

After creating and binding the elastic network card, we will find that there are two network cards in proxy bcc.

We can use the ping command to verify whether we can connect to user bcc. It is important to note that the proxy bcc and user bcc can communicate with each other because the elastic network interface is created based on the VPC subnet where the user bcc is located, but the tenant still belongs to the proxy bcc.

After establishing the connection, we need to use forwarding technology to route the packets from func bcc1 to user bcc through the proxy bcc.

First, we need to enable forwarding capability on the proxy bcc by setting ip_forward to 1.

echo 1 > /proc/sys/net/ipv4/ip_forward

It is important to note that the Linux system disables packet forwarding by default for security reasons. Packet forwarding refers to the process of receiving a packet on one network interface and sending it to another network interface of the same host based on the destination IP address of the packet, according to the routing table.

In our case, we need to enable forwarding capability because we receive data from the cloud host on eth0 and send it out through eth1. However, in order for the routing of func bcc1 to reach user bcc, we still need to refine the routing rules.

On func bcc1, we can add a route to the user bcc subnet 172.16.101.0/24 using the ip route add command:

ip route add 172.16.101.0/24 dev mgre0

At this point, func bcc1 is still unable to access user bcc 172.16.101.4. By monitoring the traffic on eth1 of the proxy bcc, we can see that the source IP of the packets sent from eth1 to the user VPC is still the IP of the func1 bcc gre0 device, which is actually only available internally within our bcc. Therefore, after the user bcc receives the request, its return traffic cannot be routed to this IP.

To solve this issue, we need to add an iptables rule on the proxy bcc to set the source IP SNAT of the traffic exiting eth1 in the POSTROUTING chain to the IP address of eth1, which is 172.16.101.4.

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

With this, we can now ping user bcc from func bcc1.

Container Network #

On virtual machines, we have already experienced the operations, but typically our functions are run within container instances, which have some differences when accessing user VPC services. We can also verify this and find that the requests initiated by pods are properly routed to the tunnel device under the proxy bcc, but are not further forwarded to eth1.

So why does this happen? Usually, it is because the Pod’s IP and the routing rules we built on the proxy bcc are inconsistent. As for how to solve this, I will not answer immediately. You can think about it first. Let’s take a look at a concept: ip-masq-agent.

ip-masq-agent is an extension used to manage IP masquerading, which manages SNAT rules for IP ranges within Kubernetes cluster nodes. It is deployed as a DaemonSet in the Kubernetes cluster, starting a proxy program on each node. The proxy program configures iptables rules and chooses the container’s own IP address or masquerades as the node’s IP address as the source address for outgoing traffic when sending container traffic to destinations outside the cluster IP range or outside the IP range of the cluster node.

After reading the explanation of ip-masq-agent, I believe you already have an answer. The solution is to set the Pod IP SNAT to the IP of the gre0 device and then forward it to the proxy bcc, hiding the Pod’s IP from the proxy bcc.

The configuration of ip-masq-agent comes from a configMap, which can be viewed or modified using the kubectl edit cm command.

With this, we can now access user bcc services from within Pods.

High Availability #

If we are deploying to a production environment, we also need to consider the high availability of the services. In the case diagram we have repeatedly mentioned in this course, I only drew one Proxy Node, which is fine for experimenting, but it is not sufficient for a production environment. To address this, we need to establish a pair of active-standby proxies for each VPC/Subnet. The active-standby proxies should use HaVip for disaster recovery and failover.

HaVip is an internal IP that can be independently created and released. It can be used in conjunction with high availability software (such as KeepAlived) to build active-standby services and improve the availability of the services.

In addition, from a serverless perspective, you need to automatically scale up or down the Proxy Nodes based on the traffic to functions. When the traffic increases, the Proxy Nodes need to be automatically scaled up, and when the traffic decreases, the Proxy Nodes need to be automatically scaled down.

Let’s further improve the architecture diagram after adding the active-standby Proxy Nodes and container Pods. The updated architecture is shown below:

Conclusion #

Finally, let me summarize the content of today.

This class mainly focused on explaining the optimization of function access speed in a cross-VPC scenario. We learned about the evolution process of cross-VPC connectivity as well as the four essential elements of proxies. Finally, with the help of a practical case study, we understood the implementation process of function proxy technology.

In the review of the evolution process, we found that peering connections are not universally applicable in function computing and are only used in a few scenarios with large clients. Although the creation of elastic network interfaces has certain versatility, we do not recommend using it due to its high latency and resource wastage.

These two technologies have given us some thoughts: if we can prepare an accessible route in advance and make it reusable, can we solve such problems? Yes, for this reason, we propose the function proxy technology.

In fact, there are still many technical bottlenecks that proxy technology needs to address, mainly including the tunneling path, network interface forwarding, container network forwarding technology, and the issue of high availability in production environments. We can solve these issues by using highly available virtual IPs and a primary/backup proxy cluster. At the same time, from an elastic perspective, I also suggest introducing the ability to scale up and down. This capability can be integrated into your entire Serverless learning journey and can be said to be the essence of Serverless.

Finally, due to the nature of function computing as a glue language, cross-medium and cross-network access cannot be ignored. Although we have been continuously evolving and solving these issues, function proxy technology also has its drawbacks, such as increasing resource consumption to some extent. Therefore, I hope that through the content and thought process of this class, you can broaden your horizons and come up with more optimization methods to deepen the technical accumulation of Serverless.

Thinking question #

Alright, this lesson comes to an end here, and finally, I have a thinking question for you.

If we use the IPIP tunneling protocol in the tunneling breakthrough, what problems would we encounter?

Feel free to write down your thoughts and answers in the comment section. Let’s discuss and exchange ideas together. Thank you for reading, and you are also welcome to share this lesson with more friends for discussion and learning.