19 the Core and Essence of Open Resty Cosocket

19 The Core and Essence of OpenResty- cosocket #

Hello, I’m Wen Ming. Today we will learn about the core technology in OpenResty: cosocket.

In fact, we have mentioned it multiple times in previous lessons. Cosocket is the foundation of various lua-resty-* non-blocking libraries. Without cosocket, developers cannot quickly connect various external network services using Lua.

In earlier versions of OpenResty, if you wanted to interact with services like Redis or memcached, you needed to use C modules such as redis2-nginx-module, redis-nginx-module, and memc-nginx-module. These modules are still included in the OpenResty distribution.

However, with the introduction of cosocket functionality, they have been replaced by lua-resty-redis and lua-resty-memcached. Basically, no one uses C modules to connect to external services anymore.

What is cosocket? #

So what exactly is cosocket? In fact, cosocket is a proprietary term in OpenResty that combines the words “coroutine” and “socket” in English, which means cosocket = coroutine + socket. Therefore, you can translate cosocket as “coroutine socket”.

Cosocket not only requires support for Lua coroutine features, but also relies on the important event mechanism in Nginx. The combination of these two enables non-blocking network I/O. In addition, cosocket supports TCP, UDP, and Unix Domain Socket.

If we call a cosocket-related function in OpenResty, the internal implementation looks like the diagram below:

Observant students may have noticed that I used this diagram in the previous lesson on the principles and basic concepts of OpenResty. From the diagram, you can see that whenever the Lua script triggers a network operation, it involves coroutine yield and resume.

When encountering network I/O, it relinquishes control (yield) and registers the network event in the Nginx event listening list, transferring control to Nginx. When an Nginx event reaches its triggering condition, it wakes up the corresponding coroutine to continue processing (resume).

OpenResty encapsulates and implements operations like connect, send, and receive based on this blueprint, forming the cosocket API we see today. Now, let’s take TCP API as an example to introduce it. The APIs for UDP and Unix Domain Socket are similar to TCP.

Introduction to cosocket API and instructions #

The cosocket API related to TCP can be classified into the following categories:

  • Creating objects: ngx.socket.tcp.
  • Setting timeouts: tcpsock:settimeout and tcpsock:settimeouts.
  • Establishing connection: tcpsock:connect.
  • Sending data: tcpsock:send.
  • Receiving data: tcpsock:receive, tcpsock:receiveany, and tcpsock:receiveuntil.
  • Connection pooling: tcpsock:setkeepalive.
  • Closing connection: tcpsock:close.

We also need to pay special attention to the contexts in which these APIs can be used:

rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*_

I would like to emphasize that due to various limitations of the Nginx core, the cosocket API cannot be used in set_by_lua*, log_by_lua*, header_filter_by_lua*, and body_filter_by_lua*. Currently, it is also not supported in init_by_lua* and init_worker_by_lua*, although the Nginx core does not restrict these two phases and support for them may be added in the future.

In addition, there are 8 Nginx instructions starting with lua_socket_ that are related to these APIs. Let’s take a brief look at them:

  • lua_socket_connect_timeout: Connection timeout, default is 60 seconds.
  • lua_socket_send_timeout: Send timeout, default is 60 seconds.
  • lua_socket_send_lowat: Send low water threshold, default is 0.
  • lua_socket_read_timeout: Read timeout, default is 60 seconds.
  • lua_socket_buffer_size: Buffer size for reading data, default is 4k/8k.
  • lua_socket_pool_size: Connection pool size, default is 30.
  • lua_socket_keepalive_timeout: Idle time for cosocket objects in the connection pool, default is 60 seconds.
  • lua_socket_log_errors: Whether to log errors when cosocket encounters an error, default is “on”.

You can see that some instructions have the same functionality as the APIs, such as setting timeouts and connection pool size. However, if there is a conflict between the two, the APIs have a higher priority and will override the values set by the instructions. Therefore, it is generally recommended to use APIs for configuration, as it provides more flexibility.

Next, let’s look at a concrete example to understand how to use these cosocket APIs. The following code snippet has a simple function: it sends a TCP request to a website and prints the returned content:

$ resty -e 'local sock = ngx.socket.tcp()
            sock:settimeout(1000)  -- one second timeout
            local ok, err = sock:connect("www.baidu.com", 80)
            if not ok then
                ngx.say("failed to connect: ", err)
                return
            end

            local req_data = "GET / HTTP/1.1\r\nHost: www.baidu.com\r\n\r\n"
            local bytes, err = sock:send(req_data)
            if err then
                ngx.say("failed to send: ", err)
                return
            end

            local data, err, partial = sock:receive()
            if err then
                ngx.say("failed to receive: ", err)
                return
            end

            sock:close()
            ngx.say("response is: ", data)'

Now let’s analyze this code snippet:

  • First, it creates a TCP cosocket object named sock using ngx.socket.tcp().
  • Then, it sets the timeout to 1 second using settimeout(). Note that the timeout here is not differentiated between connect and receive operations and is set uniformly.
  • Next, it uses connect() to establish a connection to port 80 of a specified website. If the connection fails, it exits directly.
  • If the connection is successful, it uses send() to send the constructed data. If the send operation fails, it exits.
  • If sending the data is successful, it uses receive() to receive the data returned by the website. The default parameter value for receive() is *l, which means it only returns the first line of data. If the parameter is set to *a, it will continuously receive data until the connection is closed.
  • Finally, it calls close() to close the socket connection proactively.

You see, it only takes a few steps to complete with the cosocket API for network communication, it’s that simple. However, we should not be satisfied with this. Next, let’s make some adjustments to this example.

The first action is to set timeout values for socket connection, sending, and reading.

Previously, we used settimeout() to set a unified timeout value. If we want to set different timeout values, we need to use the settimeouts() function, like this:

sock:settimeouts(1000, 2000, 3000)

This line of code represents a connection timeout of 1 second, sending timeout of 2 seconds, and reading timeout of 3 seconds.

In OpenResty and lua-resty libraries, most of the time-related APIs use milliseconds as the unit, but there are exceptions, so you need to pay special attention when making the calls.

The second action is to receive a specific size of content.

As mentioned before, the receive() method can receive a line of data or continuously receive data. But if you only want to receive data of size 10K, how should you set it?

This is where receiveany() shines. It is designed specifically to meet this requirement. Take a look at the following code:

local data, err, partial = sock:receiveany(10240)

This code means that at most, 10K of data will be received.

Of course, there is another common user requirement regarding receive, which is to keep receiving data until a specified string is encountered.

receiveuntil() is specifically designed to solve this kind of problem. It does not return a string like receive() or receiveany(), but instead returns an iterator. This allows you to call it in a loop to read the matched data in segments. When the reading is completed, it returns nil. Here’s an example:

local reader = sock:receiveuntil("\r\n")

while true do
    local data, err, partial = reader(4)
    if not data then
        if err then
            ngx.say("failed to read the data stream: ", err)
            break
        end

        ngx.say("read done")
        break
    end
    ngx.say("read chunk: [", data, "]")
end

In this code, receiveuntil will return the data before \r\n and read 4 bytes at a time using the iterator. This achieves the functionality we want.

The third action is to not close the socket directly, but instead put it into the connection pool.

We know that without a connection pool, each incoming request will create a new connection, resulting in frequent creation and destruction of cosocket objects, causing unnecessary performance loss.

To avoid this problem, after you finish using a cosocket, you can call setkeepalive() to put it into the connection pool, like this:

local ok, err = sock:setkeepalive(2 * 1000, 100)
if not ok then
    ngx.say("failed to set reusable: ", err)
end

This code sets the idle time of the connection to 2 seconds and the size of the connection pool to 100. So, when calling the connect() function, it will first get the cosocket object from the connection pool.

However, there are two points to note about using the connection pool.

  • First, we cannot put a connection with an error into the connection pool, otherwise, it will cause data reception and transmission failures when using it next time. This is also why we need to check whether each API call is successful.
  • Second, we need to understand the number of connections. The connection pool is at the worker level, each worker has its own connection pool. So, if you have 10 workers and set the connection pool size to 30, it means there will be 300 connections for the backend service.

Conclusion #

In summary, today we learned about the basic concepts of cosocket, as well as the relevant commands and APIs. Through a practical example, we familiarized ourselves with how to use the TCP-related APIs. The usage of UDP and Unix Domain Sockets is similar to TCP, so once you understand what we learned today, you should be able to handle them easily as well.

From this, you should also understand that cosocket is relatively easy to get started with, and once you master it, you can connect to various external services. It can be said that cosocket gives OpenResty wings to imagine.

Finally, I have two homework questions for you.

Firstly, in today’s example, we sent a string using tcpsock:send. What if we need to send a table composed of strings? How should we handle it?

Secondly, as you noticed, there are certain stages in which cosocket cannot be used. Can you think of any workarounds?

Feel free to leave a comment and share your thoughts with me. You are also welcome to share this article with your colleagues and friends. Let’s exchange ideas and progress together.