49 Three Step Melody for Building Microservice API Gateways Part Three

49 Three-Step Melody for Building Microservice API Gateways - Part Three #

Hello, I’m Wen Ming.

In today’s lesson, we will conclude the setup of the microservice API gateway. Let’s assemble and run the selected components according to the blueprint we designed using a minimal example.

Nginx Configuration and Initialization #

We know that API Gateway is used to handle incoming traffic, so we first need to do some simple configuration in Nginx.conf to ensure that all traffic is processed through the Lua code of the gateway.

server {
    listen 9080;

    init_worker_by_lua_block {
        apisix.http_init_worker()
    }

    location / {
        access_by_lua_block {
            apisix.http_access_phase()
        }
        header_filter_by_lua_block {
            apisix.http_header_filter_phase()
        }
        body_filter_by_lua_block {
            apisix.http_body_filter_phase()
        }
        log_by_lua_block {
            apisix.http_log_phase()
        }
    }
}

In this example, we use the open-source API Gateway APISIX as an example, so the above code example contains the keyword apisix. In this example, we listen on port 9080 and intercept all requests to this port by using location /. The requests are then processed in order through the access, rewrite, header filter, body filter, and log phases, with corresponding plugin functions being called in each phase. In this case, the rewrite phase is handled by the apisix.http_access_phase function.

As for the system initialization work, we handle it in the init_worker phase, which includes reading various configuration parameters, initializing directories in etcd, retrieving the list of plugins from etcd, and sorting the plugins by priority. I have listed the key parts of the code and explained them, and of course, you can see a more complete init_worker function on GitHub.

function _M.http_init_worker()
    -- Initialize the three most important parts: router, service, and plugin
    router.init_worker()
    require("apisix.http.service").init_worker()
    require("apisix.plugin").init_worker()
end

By reading this code, you can see that the initialization of the router and plugin parts is relatively more complex, mainly involving reading configuration parameters and making some choices based on the different parameters. Because this involves reading data from etcd, we use the ngx.timer method to bypass the limitation of “cannot use cosocket in the init_worker phase”. If you are interested in this part and have the ability to learn, I recommend reading the source code to deepen your understanding.

Route Matching #

In the access phase, the first thing we need to do is match the route. We match the incoming request against the pre-configured route rules based on the URI, host, args, cookie, etc.:

router.router_http.match(api_ctx)

The only thing exposed externally is the line of code above. The api_ctx object contains the information of the URI, host, args, cookie, and so on. The implementation of the match function can be found here. It uses the lua-resty-radixtree mentioned earlier. If no match is found, it means that there is no corresponding upstream for this request, and a 404 response will be returned.

local router = require("resty.radixtree")

local match_opts = {}

function _M.match(api_ctx)
    -- Get the request parameters from ctx as matching conditions for routing
    match_opts.method = api_ctx.var.method
    match_opts.host = api_ctx.var.host
    match_opts.remote_addr = api_ctx.var.remote_addr
    match_opts.vars = api_ctx.var
    -- Call the routing match function
    local ok = uri_router:dispatch(api_ctx.var.uri, match_opts, api_ctx)
    -- Return a 404 response if no match is found
    if not ok then
        core.log.info("not find any matched route")
        return core.response.exit(404)
    end

    return true
end

Loading Plugins #

Of course, if the route can be matched, the filtering plugins and loading plugins steps will be executed, which is the core of the API Gateway. Let’s first take a look at the following code:

local plugins = core.tablepool.fetch("plugins", 32, 0)
-- Perform an intersection operation on the plugin lists in etcd and the local configuration file
api_ctx.plugins = plugin.filter(route, plugins)

-- Run the functions mounted by the plugins in the rewrite and access phases one by one
run_plugin("rewrite", plugins, api_ctx)
run_plugin("access", plugins, api_ctx)

In this code snippet, we first fetch a table with a length of 32 using table pool, which is a performance optimization technique we previously introduced. Then we have the plugin filtering function. You might wonder why we need this step. Haven’t we already obtained the plugin list from etcd and completed the sorting in the “init worker” phase?

In fact, this filtering is done in comparison with the local configuration file for the following two reasons.

  • First, newly developed plugins need to be gradually released. At this time, the new plugin exists in the etcd list, but it is only enabled on some gateway nodes. So, we need to perform an additional intersection operation.
  • Second, to support the debug mode. Which plugins have handled the terminal request? What is the order in which these plugins are loaded? These information will be useful for debugging, so the filtering function also checks whether the debug mode is enabled and records this information in the response headers.

Therefore, at the end of the access phase, we will run these filtered plugins one by one according to their priorities, as shown in the following code snippet:

local function run_plugin(phase, plugins, api_ctx)
    for i = 1, #plugins, 2 do
        local phase_fun = plugins[i][phase]
        if phase_fun then
            -- The most core invocation code
            phase_fun(plugins[i + 1], api_ctx)
        end
    end

    return api_ctx
end

As you can see, when iterating over the plugins, we do it with a step of 2, because each plugin consists of two parts: the plugin object and the configuration parameters of the plugin. Now let’s take a look at the most core line of code in the above example:

phase_fun(plugins[i + 1], api_ctx)

Looking at this line of code in isolation may be a bit abstract, so let’s replace it with a specific limit_count plugin, which will make it much clearer:

limit_count_plugin_rewrite_function(conf_of_plugin, api_ctx)

Up to this point, we have implemented the overall process of the API Gateway. The code is all in the same file, which contains over 400 lines of code, but the core code is just the few dozen lines we introduced above.

Writing a Plugin #

Now, there is one more thing missing before a complete demo can be achieved, and that is writing a plugin to make it work. Let’s take the limit-count plugin as an example, which limits the number of requests. You can view its complete implementation by clicking here. Now, I will explain the key code in detail.

Firstly, we need to import lua-resty-limit-traffic as the underlying library for request limiting:

local limit_count_new = require("resty.limit.count").new

Next, we use JSON Schema from rapidjson to define the parameters of this plugin:

local schema = {
    type = "object",
    properties = {
        count = {type = "integer", minimum = 0},
        time_window = {type = "integer", minimum = 0},
        key = {type = "string",
        enum = {"remote_addr", "server_addr"},
        },
        rejected_code = {type = "integer", minimum = 200, maximum = 600},
    },
    additionalProperties = false,
    required = {"count", "time_window", "key", "rejected_code"},
}

These plugin parameters correspond to most of the parameters of resty.limit.count, including the key for limiting, the size of the time window, and the number of requests to be limited. In addition, a parameter rejected_code is added to specify the status code to return when a request is rate-limited.

Finally, we mount the plugin’s processing function to the rewrite phase:

function _M.rewrite(conf, ctx)
    -- Retrieve the limit count object from the cache, if it doesn't exist, create it using the `create_limit_obj` function and cache it
    local lim, err = core.lrucache.plugin_ctx(plugin_name, ctx,  create_limit_obj, conf)

    -- Retrieve the value of the key from the `ctx.var` and combine it with the configuration type and version to form a new key
    local key = (ctx.var[conf.key] or "") .. ctx.conf_type .. ctx.conf_version

    -- Enter the limiting judgment function
    local delay, remaining = lim:incoming(key, true)
    if not delay then
        local err = remaining
        -- If the threshold is exceeded, return the specified status code
        if err == "rejected" then
            return conf.rejected_code
        end

        core.log.error("failed to limit req: ", err)
        return 500
    end

    -- If the threshold is not exceeded, continue with the next plugin according to its priority and set the corresponding response header
    core.response.set_header("X-RateLimit-Limit", conf.count,
                             "X-RateLimit-Remaining", remaining)
end

In the above code, the logic for limiting is only one line, the rest is preparation and setting response headers. If the threshold is not exceeded, the request will be allowed to proceed, and the corresponding response header will be set.

Final Thoughts #

In today’s lesson, we have completed a demo of an API gateway by writing the overall framework and plugins. Furthermore, with the OpenResty knowledge learned in this column, you can continue to build on it and create more sophisticated features.

Lastly, I would like to leave you with a question to ponder. We know that an API gateway can handle both layer 7 and layer 4 traffic. Based on this, can you think of some use cases for it? Feel free to leave a comment sharing your thoughts and also share this article with others for learning and exchange.