47 Three-Step Melody for Building Microservice API Gateways - Part One #
Hello, I’m Wen Ming.
In today’s lesson, we will enter the practical section of the OpenResty column. I will use the content of three lessons to introduce how to implement a microservice API gateway. In this process, we will not only involve the OpenResty knowledge we have learned before, but I will also show you how to create a new product and open-source project from scratch from various dimensions such as industry, product, and technology selection.
What is the use of a microservices API gateway? #
Let’s first look at the role of a microservices API gateway. The following diagram provides a brief explanation:
As we all know, the concept of an API gateway is not new and has existed for more than a decade. Its main purpose is to serve as the entry point for traffic, centrally handling requests related to business, ensuring that requests are processed in a secure, fast, and accurate manner. It has the following traditional functionalities:
- Reverse proxy and load balancing, which is consistent with the positioning and functionalities of Nginx.
- Dynamic upstream, dynamic SSL certificates, and dynamic rate limiting are runtime dynamic functionalities that are not provided by the open-source version of Nginx.
- Active and passive health checks of upstream servers, as well as service circuit-breaking functionality.
- Expansion on top of the API gateway to become a full lifecycle API management platform.
In recent years, business-related traffic is no longer solely initiated by PC clients and browsers. More traffic is coming from mobile devices, IoT devices, and in the future, with the widespread adoption of 5G, this traffic will increase even more. At the same time, with the structural transformation of microservices architecture, the traffic between services is also growing explosively. In this new business scenario, it has naturally given rise to more advanced and extensive functionalities of the API gateway:
- Cloud-native friendly, architecture needs to become lightweight and containerization-friendly.
- Integration with statistics and monitoring components such as Prometheus, Zipkin, and SkyWalking.
- Support for gRPC proxy and protocol conversion between HTTP and gRPC, converting users’ HTTP requests into internal service gRPC requests.
- Playing the role of an OpenID Relying Party, integrating with identity authentication providers such as Auth0 and Okta, and prioritizing traffic security.
- Implementing serverless by dynamically executing user functions at runtime, making edge nodes of the gateway more flexible.
- Not locking users, supporting hybrid cloud deployment architectures.
- Finally, the gateway nodes should be stateless, capable of scaling up or down arbitrarily.
When a microservices API gateway encompasses the above functionalities, it allows users’ services to focus only on the business itself. Functionalities unrelated to business implementation, such as service discovery, circuit-breaking, identity authentication, rate limiting, statistics, and performance analysis, can be addressed at an independent gateway level.
From this perspective, an API gateway can replace all functionalities of Nginx, handling North-South traffic. It can also serve as the Istio control plane and Envoy data plane, handling East-West traffic.
Why do we need to reinvent the wheel? #
The reason for reinventing the wheel is that the current microservice API gateway solutions are not sufficient to meet our needs. Let’s first look at closed-source commercial products. They have complete functionality, covering the full lifecycle management of APIs, such as design, multi-language SDKs, documentation, testing, and deployment. They also provide SaaS services and some even integrate with public clouds, making them very convenient to use. However, they also bring two pain points.
The first pain point is platform lock-in. The API gateway is the entry point for business traffic and is not as easy to migrate as non-business traffic like images and videos that can be accelerated using CDNs. The API gateway is bound with many business-related logics. Once you use a closed-source solution, it becomes difficult and costly to smoothly migrate to another platform.
The second pain point is the inability to customize and develop. Medium to large enterprises usually have unique requirements and need customization. However, in such cases, you can only rely on vendors and cannot develop your own customizations.
This is also why open-source API gateway solutions have become popular. However, the existing open-source products are not perfect and have their own shortcomings.
First, they rely on relational databases like PostgreSQL and MySQL. This means that when configurations change, gateway nodes can only query the database in a round-robin manner. This not only slows down configuring effectiveness but also adds complexity to the code, making it difficult to understand. Moreover, the database becomes a single point of failure and a performance bottleneck, compromising overall high availability. If you use the API gateway in a Kubernetes environment, relational databases become even more cumbersome and hinder rapid scalability.
Second, plugins cannot be hot-loaded. When you add a new plugin or modify the code of an existing plugin, you must reload the service for the changes to take effect. This is similar to reloading Nginx configuration, which obviously affects user requests.
Third, the code structure is complex and difficult to grasp. Some open-source projects have multiple layers of object-oriented encapsulation, making even simple logic unclear. However, for the API gateway scenario, a more straightforward and direct approach is more clear, efficient, and conducive to customization.
Therefore, we need a more lightweight and cloud-native, developer-friendly API gateway. Of course, we can’t build it from scratch. We need to first deeply understand the characteristics of existing API gateways. In this case, the Cloud Native Computing Foundation’s (CNCF) landscape diagram is a valuable reference:
This diagram filters out commonly used API gateways in the industry, primarily focusing on open-source solutions, and provides valuable content for our technology selection below.
Core Components and Concepts of API Gateway #
Before diving into the implementation details, let’s familiarize ourselves with the core components of an API gateway. Based on the functionalities we mentioned earlier, an API gateway needs several components to start running.
Firstly, there is the routing component. It matches client requests based on defined rules, executes corresponding plugins, and forwards the requests to designated upstreams. These routing rules can be composed of hosts, URIs, request headers, etc. The familiar location directive in Nginx is an implementation of routing.
Secondly, there are plugins. They are the soul of an API gateway, responsible for functionalities such as authentication, rate limiting, IP blacklisting/whitelisting, Prometheus, Zipkin, etc. Since they are plugins, they should be easy to plug-and-play. Additionally, plugins should not interfere with each other. It’s like building with Lego blocks, requiring a unified and well-defined development interface to interact with the underlying system.
Next, there is the schema. As an API gateway deals with APIs, it needs to validate the format of APIs, such as data types, allowed field contents, required fields, etc. A schema layer is essential to provide unified and independent definition and validation.
Lastly, there is storage. It is used to store various user configurations and is responsible for pushing the configurations to all gateway nodes when changes occur. This is a crucial underlying component, as its selection determines how the upper-level plugins are written, whether the system maintains high availability and scalability, etc. Therefore, we need to make cautious decisions when selecting the storage technology.
In addition, on top of these core components, we need to abstract several common concepts of an API gateway, which are universally applicable across different API gateways.
Let’s start with Route. A route consists of three parts: matching conditions, bound plugins, and upstreams, as shown in the following diagram:
We can configure everything directly in a Route, which is the simplest approach. However, when there are many APIs and upstreams, this approach leads to repeated configurations. In such cases, we need two additional concepts: Service and Upstream to provide abstraction.
Next, let’s talk about Service. It represents an abstraction of a certain type of API, or it can be understood as a group of Routes. It is usually a one-to-one correspondence with an upstream service. The relationship between Route and Service is often N:1. I have also prepared a diagram to illustrate this:
With the abstraction of Service, we can separate the repeated plugins and upstreams. Therefore, when plugins or upstreams need to be changed, we only need to modify the corresponding Service instead of multiple Routes.
Lastly, let’s discuss Upstream. Continuing with the previous example, if two Routes share the same upstream but have different bound plugins, we can abstract the upstream separately, as shown in the following diagram:
This way, when there are changes in the upstreams, the Routes are completely unaware of it, as the changes are handled within the Upstream.
In fact, from the derivation process of these three main concepts, we can see that they are based on real-life scenarios, rather than being artificially created. As a result, these abstractions are applicable to all API gateways and are independent of specific technical solutions.
Conclusion #
In today’s lesson, we introduced the purpose, functionality, core components, and abstract concepts of a microservices API gateway. These are the foundations of an API gateway.
Here, I leave you with a question to ponder: Do you think API gateways can handle both traditional north-south traffic and east-west traffic between microservices? If you are already using an API gateway, feel free to share your thoughts on the technical selection process. I welcome your comments and discussions in the comment section. Also, feel free to share this article with your colleagues and friends to learn and progress together.