08 Runtime How Do Functions Execute in Containers Under Different Programming Language Conditions

08 Runtime How do functions execute in containers under different programming language conditions #

Hello, I’m Jingyuan. In this lesson, we will continue to explore the principles and practice of runtime.

In the previous lesson, I shared with you the basic concepts of runtime and the differences in runtime for different languages. I also used the example of GoLang to give you a concrete understanding of runtime.

Today, I will focus on the second half: the implementation and similarities and differences of interpreted languages. Through today’s lesson, I believe you will have a comprehensive understanding of runtime from 1 to N and abstract a set of building ideas for your own practical applications.

Python Runtime #

Through learning the GoLang Runtime, I believe you have understood the work that the runtime needs to complete and the entire processing flow.

As I mentioned earlier, the business logic and dependencies of compiled languages usually have a strong binding relationship, and the runtime and user-written function code need to be compiled into a binary file or a Jar package or a War package before being executed. However, interpreted languages do not require this. This also partly explains why many function computing platforms only open source the runtime of compiled languages.

Next, let’s take Python as an example to introduce the execution mechanism of interpreted languages, and further deepen your understanding of the function computing runtime.

If you have developed function computing code using Python or Node.js, you may find that the uploaded files do not require any dependencies at all, and even two lines of code can define your function entry. For example, consider this Python case:

def handler(event, context):
    return 'hello world'

In this case, it is not conducive for us to analyze the runtime, but some cloud vendors provide the ability to log into function instances. Let’s continue to use Alibaba Cloud Function Compute (FC) as an example. FC provides the instance login capability, which allows us to directly view the execution process of the runtime in the instance.

Path #

After logging into the function instance, the first thing we need to do is to find out where the runtime is mounted. Think about it, where can we find it?

As a process in a container, the runtime continuously waits for requests. Its lifecycle is consistent with the function container instance. Therefore, we can directly use the “ps” command in the current path to view the running state of processes in the current instance.

root@s-62c8e50b-80f87072536d411b8914:/code# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
user100+     1     0  0 02:16 pts/0    00:00:00 /var/fc/runtime/rloader
user100+     4     1  0 02:31 pts/0    00:00:00 /var/fc/lang/python3/bin/python3 -W ignore /var/fc/runtime/python3/bootstrap.py
root         6     0  0 02:31 ?        00:00:00 /bin/sh -c cd /code > /dev/null 2>&1 && COLUMNS=167; export COLUMNS; LINES=27; export LINES; TERM=xterm-256color; expor
root         7     6  0 02:31 ?        00:00:00 /bin/sh -c cd /code > /dev/null 2>&1 && COLUMNS=167; export COLUMNS; LINES=27; export LINES; TERM=xterm-256color; expor
root         8     7  0 02:31 ?        00:00:00 /usr/bin/script -q -c /bin/bash /dev/null
root         9     8  0 02:31 pts/0    00:00:00 sh -c /bin/bash
root        10     9  0 02:31 pts/0    00:00:00 /bin/bash

Process 1 is the initial process started by the instance, and process 4 is forked from process 1. It is actually the Python runtime, and bootstrap.py is the entry point of the entire runtime. You can know that the path of the Python runtime is actually under /var/fc/runtime/python3/ by looking at the startup parameters of the process.

Core Flow #

For this part of the Python runtime, I will only extract part of the code in the main function to introduce its main flow, and you can also understand the differences and similarities between Python and GoLang runtimes in handling requests. You can find the complete code in /var/fc/runtime/python3/.

def main():
    ......
    try:
        _init_log() # Initialize logging
        ...
        _set_global_concurrent_params() # Set global concurrent parameters
        fc_api_addr = os.environ['FC_RUNTIME_API']
        fc_runtime_client = FcRuntimeClient(fc_api_addr) # Create runtime client
        _sanitize_envs()
    except Exception as e:
        ...
        return
    # Wait for requests
    ....
    while True:
        try:
            if global_concurrent_limit == 1:
                request = fc_runtime_client.wait_for_invocation()
            else:
                request = fc_runtime_client.wait_for_invocation_unblock()
            ...
            handler = FCHandler(request, fc_runtime_client) # Get the entry function of the function
            if global_concurrent_limit == 1:
                handler.handle_request() # Execute the business logic
            else:
                # If executed with multiple concurrency, start the task execution through a thread
                from threading import Thread
                Thread(target=handler.handle_request).start()
        except Exception as e:
             ...

From the code structure, the core processing flow of the entire runtime is similar to the logic in the GoLang runtime, including the following three aspects.

  • Service Initialization: It includes initializing the log and setting global parameters in concurrent mode. At the same time, the client that retrieves the request is constructed based on the service address provided by the environment variable FC_RUNTIME_API.
  • Retrieve Requests: Allow the client to continuously retrieve the request information.
  • Execute User Code Based on Request Information: The only difference here between Python and GoLang is that in the case of multiple concurrency, Python assigns threads to handle task processing instead of coroutines.

Here, let me briefly mention how the user’s handler is associated and the process of invoking the user’s handler to process requests, that is, the handle_request method of FCHandler:

def handle_request(self):
    ...
    event, ctx, handler = self.get_handler()
    # Load user handler
    valid_handler, request_handler = _concurrent_get_handler(handler)
    # Execute handler
    if self.function_type == Constant.HANDLE_FUNCTION and self.request.http_params:
        # HTTP function
        succeed, resp = wsgi_wrapper(self.request_id, request_handler, event, ctx, self.request.http_params)
        if succeed:
            self.print_end_log(True)
            self.fc_runtime_client.return_http_result(self.request_id, resp[0], resp[1], Constant.CONTENT_TYPE)
            return
    else:
        succeed, resp = execute_request_handler(self.function_type, request_handler, event, ctx)
        if succeed:
            self.print_end_log(True)
            self.fc_runtime_client.return_invocation_result(self.request_id, resp, Constant.CONTENT_TYPE)
            return
    ...

First, the program calls get_handler to obtain the userHandler information from the request. This information is actually the entry function you configured in the function metadata, such as index.handler. The subsequent process is similar to the handling concept in GoLang, which uses reflection to obtain the specific userHandler and then invoke it.

We can use a diagram to outline the main flow above to help you trace the execution process later:

Flowchart

Tips for Analyzing Runtime #

For runtimes like GoLang and Java, there are usually open-source codes available for inspection. However, for interpreted languages like Python, the runtime code is usually not directly accessible, and many function computing platforms, like Alibaba Cloud Function Compute (FC), do not provide users with a login entrance to the function instance. In such cases, analyzing the runtime can be inconvenient.

Here’s a little trick I can teach you. Normally, users have read permissions to the files inside the function container instance. You can view the runtime code by injecting commands into the code.

I’ll continue to use Python runtime as an example to demonstrate. By executing some shell commands within the function code, the desired runtime code can be printed. You can achieve this by using common commands like ps, ls, cat, etc. Here’s an example:

import logging
import os
 
def handler(event, context):
  print(os.system("ps -ef"))
  print(os.system("ls -l  /var/fc/runtime"))
  print(os.system('cat /var/fc/runtime/python3/bootstrap.py'))
  return 'hello world'

It should be noted that different cloud providers may have different implementations. Some providers may have hidden the runtime code path, allowing you to see only one entrance layer.

Approach to Building a Runtime #

After learning about the implementation principles of two different types of runtimes, I believe you now have a relatively clear understanding of the entire workflow of a runtime. Next, we can utilize the knowledge we’ve acquired to build our own runtime.

Let’s recollect the key steps that a request goes through, from reaching the function instance to being executed:

  1. An initialization process is required to load the runtime into memory and start running it.
  2. The runtime needs to obtain detailed information about the request. In GoLang and Python, this is typically obtained through a designated interface for client requests, such as RUTIME_API. Other methods, such as pipelines, can also be used to obtain this information.
  3. User code is called to execute the request, and after processing, the response is returned.

When building our own function platform, we can follow this approach. However, if you are a function business developer, you can also define your own runtime based on the tools and platforms provided by cloud vendors.

Hands-on Experience #

Next, I will guide you to manually build a custom runtime using the development tool Serverless Devs provided by Alibaba Cloud Function Compute (FC) to help you deepen your understanding of the runtime concept.

First of all, we choose python37 as the custom runtime and create an initial project.

s init fc-custom-python37-event

During the project initialization, you will be prompted to make some configurations. Just follow the prompts to input the required information. Here I named the new project “GeekBang”.

图片

After entering the project folder, we can see what has been created:

$ tree .
.
├── code
│   ├── gunicorn_conf.py
│   ├── requirements.txt
│   └── server.py
├── readme.md
├── s.yaml
└── s_en.yaml

As you can see, it includes some YAML format configuration files and a “code” folder. These two YAML files are essentially the same. The only difference is that the comments in “s_en.yaml” are in English. Let’s open “s.yaml” to see its content. Besides the common parameters such as timeout and memory limit, it also has a custom runtime startup parameter.

customRuntimeConfig:
  command:
    - gunicorn
  args:
    - '-c'
    - 'gunicorn_conf.py'
    - 'server:app'

Among them, gunicorn is a Python HTTP server. When the function instance is initialized for the first time, it will execute the following command. And this command will ultimately start the server.py file under the code path.

gunicorn -c gunicorn_conf.py server:app

Looking at the code in server.py, we can see that it is essentially an HTTP server.

@app.route('/invoke', methods=['POST'])
def event_invoke():
    rid = request.headers.get(REQUEST_ID_HEADER)
    print("FC Invoke Start RequestId: " + rid)

    data = request.stream.read()
    print(data)

    try:
        # do your things, for example:
        evt = json.loads(data)
        print(evt)
    except Exception as e:
        exc_info = sys.exc_info()
        trace = traceback.format_tb(exc_info[2])
        errRet = {
            "message": str(e),
            "stack": trace
        }
        print(errRet)
        print("FC Invoke End RequestId: " + rid +
              ", Error: Unhandled function error")
        return errRet, 404, [("x-fc-status", "404")]

    print("FC Invoke End RequestId: " + rid)

    return data

When you execute the call in the console or development tool, it will eventually call the /invoke interface to implement your business logic. In the try block, you can directly call a pre-defined function. Here, I simply print the event data.

Then, we deploy the code to the cloud one by one through build and deploy, and finally, we can call it using the invoke command. Below is the result of our experience.

$ s invoke -e '{"key":"value"}'

========= FC invoke Logs begin =========
FC Invoke Start RequestId: 5c44f71b-eadb-4bbf-8969-396ecf4e25a4
b'{"key":"value"}'
{'key': 'value'}
FC Invoke End RequestId: 5c44f71b-eadb-4bbf-8969-396ecf4e25a4
21.0.0.1 - - [27/Jun/2022:07:17:14 +0000] "POST /invoke HTTP/1.1" 200 15 "-" "Go-http-client/1.1"

Duration: 2.60 ms, Billed Duration: 3 ms, Memory Size: 1536 MB, Max Memory Used: 80.95 MB
========= FC invoke Logs end =========

You will find that the invocation result is basically the same as that of the regular runtime execution.

Of course, we can still log in to the cloud console to check the process status. It can be found that the server started by gunicorn is providing HTTP services.

root@c-62ca6e0c-3721605419294647aac9:/code# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  4 06:13 pts/0    00:00:00 /usr/local/bin/python /code/.s/python/bin/gunicorn -c gunicorn_conf.py server:app
root         4     1  3 06:13 pts/0    00:00:00 /usr/local/bin/python /code/.s/python/bin/gunicorn -c gunicorn_conf.py server:app
root         5     1  3 06:13 pts/0    00:00:00 /usr/local/bin/python /code/.s/python/bin/gunicorn -c gunicorn_conf.py server:app
root        11     0  0 06:13 ?        00:00:00 /bin/sh -c cd /code > /dev/null 2>&1 && COLUMNS=167; export COLUMNS; LINES=27; export LINES; TERM=xterm-256color; expor
root        12    11  0 06:13 ?        00:00:00 /bin/sh -c cd /code > /dev/null 2>&1 && COLUMNS=167; export COLUMNS; LINES=27; export LINES; TERM=xterm-256color; expor
root        13    12  0 06:13 ?        00:00:00 /usr/bin/script -q -c /bin/bash /dev/null
root        14    13  0 06:13 pts/0    00:00:00 sh -c /bin/bash
root        15    14  0 06:13 pts/0    00:00:00 /bin/bash
root        17    15  0 06:13 pts/0    00:00:00 ps -ef

Through this process, you should have understood that the implementation mechanism and execution process of the runtime are actually the same. The only difference lies in the coding style based on different language types and their characteristics.

Summary #

Today I have introduced you to the runtime execution process of Python in different ways. During this time, I have also shared with you a method to view the running status of function compute instances - injecting commands into the code.

Based on the introduction in these two courses, we can summarize that the runtime mainly does three things:

  • Get request information ；
  • Associate user code ；
  • Invoke user code to process the request .

Here we need to note that when obtaining request information, it is usually obtained from the agreement interface or pipeline on the single machine side. After obtaining the request, the entry information of the user code can be obtained through the reflection mechanism, and finally called.

If we only consider the simplest running process, the runtime does not even need to do some complicated type conversion work (such as reflection mechanism), just like the example in the custom runtime, define the execution command for the function instance startup, and provide a simple http service.

I hope that through this course, you will have a certain understanding of the language runtime under the function compute form, not only knowing how to use it, but also knowing how it is implemented. When encountering problems and developing more complex functions in the future, you will have a clear understanding.

Thought Questions #

Alright, this class is coming to an end, and I have a thought question for you.

How often do you use custom runtimes in your scenarios? Have you encountered any problems? And how did you solve them?

Feel free to write down your thoughts and answers in the comments section. Let’s discuss and exchange ideas together. Thank you for reading, and please feel free to share this class with more friends for further discussions and learning.