41 'Luaresty' Wrappers Let You Stay Away From the Pain of Multilevel Caching

41 ’luaresty’ Wrappers Let You Stay Away from the Pain of Multilevel Caching #

Hello, I’m Wen Ming.

In the previous two lessons, we have learned about caching in OpenResty, as well as the problematic cache storms that can occur. These are all basic knowledge. In actual project development, developers naturally hope to have a ready-to-use library that has already taken care of various details and hidden them away, so they can directly develop business code.

This is actually one of the benefits of division of labor. The developers of basic components focus on flexible architecture, extreme performance, and stable code, without needing to concern themselves with higher-level business logic. On the other hand, application engineers are more concerned about business implementation and rapid iteration, and don’t want to be distracted by various technical details at the lower level. This gap needs to be bridged with some encapsulation libraries.

Cache in OpenResty faces the same problem. Shared dictionaries and LRU caches are stable and efficient enough, but they require handling too many details. Without some useful encapsulation, reaching the “last mile” of application development can become quite painful. This is where the importance of the community comes in. An active community will proactively discover and quickly bridge these gaps.

lua-resty-memcached-shdict #

Let’s go back to the topic of cache encapsulation. lua-resty-memcached-shdict is an official project of OpenResty. It wraps memcached with shared dict and handles details such as cache storms and expired data. If your cached data is stored in the backend memcached, you can try using this library.

Although it is developed by OpenResty, it is not included in the default OpenResty package. If you want to test it locally, you need to download its source code to the search path of your local OpenResty.

This wrapper library is actually the same as the solution mentioned in the previous lesson. It uses lua-resty-lock to achieve mutual exclusion. When the cache is invalidated, only one request goes to memcached to retrieve the data, avoiding cache storms. If the latest data is not obtained, stale data is returned to the client.

However, although this lua-resty library is an official project of OpenResty, it is not perfect. First of all, it lacks test case coverage, which means that the code quality cannot be continuously guaranteed. Secondly, it exposes too many interface parameters, with 11 required parameters and 7 optional parameters:

local memc_fetch, memc_store =
    shdict_memc.gen_memc_methods{
        tag = "my memcached server tag",
        debug_logger = dlog,
        warn_logger = warn,
        error_logger = error_log,

        locks_shdict_name = "some_lua_shared_dict_name",

        shdict_set = meta_shdict_set,  
        shdict_get = meta_shdict_get,  

        disable_shdict = false,  -- optional, default false

        memc_host = "127.0.0.1",
        memc_port = 11211,
        memc_timeout = 200,  -- in ms
        memc_conn_pool_size = 5,
        memc_fetch_retries = 2,  -- optional, default 1
        memc_fetch_retry_delay = 100, -- in ms, optional, default to 100 (ms)

        memc_conn_max_idle_time = 10 * 1000,  -- in ms, for in-pool connections,optional, default to nil

        memc_store_retries = 2,  -- optional, default to 1
        memc_store_retry_delay = 100,  -- in ms, optional, default to 100 (ms)

        store_ttl = 1,  -- in seconds, optional, default to 0 (i.e., never expires)
    }

Most of the exposed parameters can actually be simplified by “creating a new memcached processing function”. The current approach of throwing all the parameters to the users to fill in at once is not user-friendly, so I also welcome interested developers to contribute PRs to optimize this aspect.

In addition, the documentation of this wrapper library mentions further optimization directions:

  • The first is to use lua-resty-lrucache to increase the cache at the worker level, not just at the server-level shared dict cache.
  • The second is to use ngx.timer to perform asynchronous cache update operations.

The first direction is actually a very good suggestion because the cache performance within the worker will naturally be better; as for the second suggestion, you need to consider it based on your actual situation. However, generally, I do not recommend using it because not only is the number of timers limited, but if there is an error in the update logic here, the cache will never be updated again, which has a relatively large impact.

lua-resty-mlcache #

Next, let’s introduce lua-resty-mlcache, a widely used cache wrapper in OpenResty. It uses shared dict and lua-resty-lrucache to implement a multi-level caching mechanism. Let’s take a look at how to use this library through two code examples:

local mlcache = require "resty.mlcache"

local cache, err = mlcache.new("cache_name", "cache_dict", {
    lru_size = 500,    -- size of the L1 (Lua VM) cache
    ttl = 3600,   -- 1h ttl for hits
    neg_ttl  = 30,     -- 30s ttl for misses
})
if not cache then
    error("failed to create mlcache: " .. err)
end

First, let’s look at the first code snippet. This code imports the mlcache library and sets the initialization parameters. We usually put this code in the initialization phase and only need to do it once.

In addition to the required parameters cache_name and cache_dict, the third parameter is a dictionary with 12 optional options. If the options are not provided, their default values will be used. This approach is obviously more elegant than lua-resty-memcached-shdict. In fact, it is best to adopt this approach, like mlcache, when designing your own interface - keep the interface as simple as possible while maintaining enough flexibility.

Now let’s look at the second code snippet, which is the logic code for request processing:

local function fetch_user(id)
    return db:query_user(id)
end

local id = 123
local user , err = cache:get(id , nil , fetch_user , id)
if err then
    ngx.log(ngx.ERR , "failed to fetch user: ", err)
    return
end

if user then
    print(user.id) -- 123
end

As you can see, the multiple layers of cache have been hidden here. You only need to use the mlcache object to get the cache and set the callback function for cache expiration. The complex logic behind it is completely hidden.

Speaking of which, you may be curious about how this library is implemented internally. Next, let’s take a look at the architecture and implementation of this library. The following diagram is from a presentation by thibault, the author of mlcache, at the 2018 OpenResty Conference:

From the diagram, you can see that mlcache divides the data into three layers: L1, L2, and L3.

L1 cache is lua-resty-lrucache. Each worker has its own copy, so there will be N copies of data if there are N workers, resulting in data redundancy. Since operating on lrucache within a single worker does not trigger locks, its performance is higher, making it suitable as the first-level cache.

L2 cache is a shared dict. All workers share the same cache data. If a cache miss occurs in L1 cache, the L2 cache will be queried. The API provided by ngx.shared.DICT uses spin locks to ensure atomicity of operations, so we don’t have to worry about competition here.

L3 cache is used when there is also a cache miss in the L2 cache. It calls the callback function to query data from external databases or other data sources, and caches the result in the L2 cache. In order to avoid cache storms, lua-resty-lock is used to ensure that only one worker goes to the data source to retrieve data.

Overall, from the perspective of a request:

  • First, it queries the L1 cache within the worker. If L1 cache is hit, it returns the result directly.
  • If the L1 cache misses or expires, it queries the L2 cache shared among workers. If L2 cache is hit, it returns the result and caches it in L1.
  • If L2 cache also misses or expires, it calls the callback function to query the data from the data source and writes it to the L2 cache. This is the role of the L3 data layer.

From this process, you can also see that cache updates are passively triggered by end-user requests. Even if a certain request fails to retrieve the cache, subsequent requests can still trigger the update logic to ensure the safety of the cache as much as possible.

However, although mlcache has been implemented quite well, there is still a pain point in practical use - the serialization and deserialization of data. This is not a problem with mlcache itself, but rather the difference between lrucache and shared dict that we have repeatedly mentioned before. In lrucache, we can store various Lua data types, including tables, but in shared dict, we can only store strings.

L1 cache, which is lrucache, is the layer of data that users really interact with. Naturally, we want to cache various types of data in it, including strings, tables, and cdata. However, the problem is that L2 cache can only store strings. Therefore, when data is promoted from L2 to L1, we need to do a conversion, namely from a string to a data type that can be directly provided to users.

Fortunately, mlcache has taken this into consideration, and in the new and get interfaces, it provides an optional function l1_serializer specifically for handling data when promoting from L2 to L1. Let’s look at the example code below, which I selected from the test case suite:

local mlcache = require "resty.mlcache"

local cache, err = mlcache.new("my_mlcache", "cache_shm", {
l1_serializer = function(i)
    return i + 2
end,
})

local function callback()
    return 123456
end

local data = assert(cache:get("number", nil, callback))
assert(data == 123458)

Let me explain briefly. In this example, the callback function returns the number 123456. In the new function, we set the l1_serializer function, which adds 2 to the input number before setting the L1 cache, resulting in 123458. With this serialization function, the conversion between data types when transitioning between L1 and L2 caches can be more flexible.

It can be said that mlcache is a powerful caching library, and its documentation is very detailed. Today, I only mentioned some of the core principles. I recommend that you take the time to explore and practice more on your own to learn more about how to use it.

Conclusion #

With multiple layers of caching, the performance of the server can be maximized, although there are many hidden details in between. At this point, having a stable and efficient encapsulation library can save us a lot of trouble. I also hope that through the introduction of these two encapsulation libraries today, I can help you better understand caching.

Finally, I would like to leave you with a question to ponder: Is the shared dictionary cache layer necessary? Can we just use lrucache? Feel free to leave a comment and share your opinions with me. You are also welcome to share this article with others and engage in discussions and progress together.