07 Runtime How Do Functions Execute in Containers Under Different Programming Language Conditions

07 Runtime How do functions execute in containers under different programming language conditions #

Hello, I’m Jingyuan.

In the lesson on Lifecycle, I mentioned that one of the key elements supporting the execution of functions is the runtime. We can also choose different runtimes for different languages on the function computing platform, which is actually a great advantage of functions: multi-language runtimes can greatly reduce the language barrier for developers.

So what is the runtime at the function computing level? Do the runtimes for different languages work in the same way? If you were to customize a runtime, how would you do it?

With these questions in mind, today we will discuss the implementation behind this. I will analyze the mechanism of the function computing runtime layer by layer from the perspective of source code, taking Golang as a representative of compiled languages and Python as a representative of interpreted languages. I will guide you to abstract the general ideas and experience how to build a custom runtime.

I hope that through these two lessons, you will have a certain understanding of the principles and features of runtimes, clarify how the function computing platform breaks the limits of programming language technology stacks, and provides multiple development environments for developers. At the same time, I believe this lesson will also help you become more proficient in the subsequent use and development of function computing.

In today’s lesson, I will focus on introducing the basic features and implementation principles of runtimes, and I will use Golang as an entry point to explain its execution process, allowing you to have a cognitive process from 0 to 1.

What is Runtime? #

We are no stranger to the term “runtime”, as every programming language has its own runtime. For example, Java has its Java Runtime, which allows the machine to understand and execute Java code. In other words, it enables the code to “interact” with the machine and implement your business logic.

Similarly, in the context of Function Compute, runtime is the execution environment that enables functions to run on machines or containers, implementing the execution of business logic. It is typically composed of frameworks specific to a particular language . The runtime of Function Compute relies on the existence of the runtime of the language. However, due to its closer alignment with the upper-level application, it is relatively easier to analyze its working principles.

The above diagram illustrates the relationship between the runtime, function composition, and initialization process. During the initialization of a function instance, the function runtime is usually loaded by an initialization process, and the runtime can then internally communicate with service requests, receive and process requests normally. When a request arrives, your code will be loaded and processed in the corresponding language runtime.

Therefore, we can simply understand the runtime as a service framework environment in a specific language context. This service will run as a process in the user’s container and be associated with the user’s code. After the service starts, it will continuously wait for incoming requests. Once a request arrives, the runtime will associate it with your code for execution. After the execution is completed, it will continue to process the next request.

I want to emphasize that this execution does not necessarily occur sequentially. In order to improve concurrency, some architectures may also use coroutines or threads for execution.

Does this process seem relatively easy to understand? Now, let’s take a closer look at the process and think about whether different language runtimes are implemented in the same way.

Implementation Principle #

At runtime, it boils down to a specific program written in a programming language. So for the problem mentioned above, let’s first look at the differences in the programming languages themselves.

Language Types #

We know that computers can only execute binary instructions, and we classify different programming languages into interpreted languages and compiled languages based on when they are converted into binary instructions.

Compiled languages, such as C, C++, and GoLang, need to bundle all static dependencies and source code together at compilation time and can be executed directly after compilation. On the other hand, interpreted languages like Python and Node.js only need to be executed through interpreters, so they can completely separate business code from dependencies.

Here, it is worth noting that Java, though it requires compilation, needs to convert the compiled machine code into binary instructions again through the JVM, so it has both interpreted and compiled characteristics. Personally, I tend to classify it as a compiled language because when you use Java to develop functions, you will find that we usually package all dependencies into a Jar or War file for uploading, which is more in line with the style of compiled languages.

Additionally, if you have used function computing platforms from different cloud vendors, you may notice that compiled languages like GoLang and Java usually require a strong dependency on a package provided by the platform, while Python and Node.js do not. Why is that?

In the difference between language types mentioned earlier, we mentioned that compiled languages need to bundle all associated static code dependencies together, so in the specific case of function instances, your business code and runtime generate a complete binary file, Jar file, or War file.

Having understood the differences in languages, you should be mentally prepared that the implementation of these two types of function computing runtimes will also differ. Next, I will discuss the implementation of runtimes from the perspectives of compiled languages and interpreted languages.

GoLang Runtime #

As mentioned earlier, for compiled languages, user code usually needs to be compiled together with the runtime, so most cloud vendors open-source the runtime of compiled languages.

Using the GoLang runtime of Alibaba Cloud Function Compute (FC) as an example, let’s take a look at its implementation.

The GoLang runtime mainly performs three tasks.

Retrieving requests

In the GoLang runtime, the platform will write the RUNTIME API to environment variables in advance, and the runtime will retrieve the requests through the initialized client object runtimeAPIClient.

Associating the user’s entry function

The user’s entry function, UserHandler in the diagram, is obtained through reflection in the GoLang runtime and encapsulated into a uniform structure Handler. Then, it is assigned as a property of the Function structure type so that users can define it according to their own programming habits without imposing any restrictions on the UserHandler’s structure.

Here I would like to explain that in the source code, the author uses handler, while the UserHandler in the diagram is a name I replaced to distinguish it from the handler in the main process. Throughout the rest of the article, we will use UserHandler consistently to differentiate it from the handler defined by users.

Invoking UserHandler to process requests

After retrieving the request and UserHandler (e.g., HandleRequest(ctx context.Context, event string)), the request can be executed with the Function object created in the second step.

Next, I will go through the processing steps in detail, starting with the entry function (the main function) of GoLang user code.

Entry #

After the entire binary is loaded, the program first enters the main function and passes the user-defined function entry method, userHandler, as a parameter to the Start method and calls the method.

The parameter of Start is interface{}, which allows your userHandler to be defined as any type.

/**
* base function type
* eventFunction functionType = 101
* httpFunction  functionType = 102
**/
func Start(userhandler interface{}) {
    log.SetFlags(log.LstdFlags | log.Lmicroseconds)
    log.Println("start")
    StartWithContext(context.Background(), userhandler, eventFunction)
}
 
func StartWithContext(ctx context.Context, userhandler interface{}, funcType functionType) {
    StartHandlerWithContext(ctx, userhandler, funcType)
}

Upon entering the Start method and going deeper, the StartHandlerWithContext method will eventually be called. Then, the global variable runtimeAPIStartFunction will be assigned to the local variable startFunction.

Regarding runtimeAPIStartFunction, I have listed the related code below. You will notice that it contains an environment variable name env and a subsequent method startRuntimeAPILoop that processes requests in a loop.

func StartHandlerWithContext(ctx context.Context,
     userhandler interface{}, funcType functionType) {
    startFunction := runtimeAPIStartFunction
    // Retrieve RUNTIME API
    config := os.Getenv(startFunction.env)
    ...
    err := startFunction.f(ctx, config, handlerWrapper{userhandler, funcType}, lifeCycleHandlers)
    ...
}
 
// runtimeAPIStartFunction is a predefined global variable
runtimeAPIStartFunction = &startFunction{
    env: "FC_RUNTIME_API",
    f:   startRuntimeAPILoop,
}

Finally, when the actual value of the environment variable startFunction.env is obtained, you will find that the parameters passed earlier, userHandler, ctx, and the environment variable just obtained, are all passed to startRuntimeAPILoop for invocation. These are the key pieces of information required for request processing.

Preparations #

After obtaining the parameters required for function request, it is necessary to pull the request and process it. You can understand how startRuntimeAPILoop works first through the code:

func startRuntimeAPILoop(ctx context.Context, api string, baseHandler handlerWrapper, lifeCycleHandlers []handlerWrapper) (e error) {
    ...
    // Create a client to connect to the Runtime API, from which the client obtains request information
    client := newRuntimeAPIClient(api)
    // Convert the UserHandler passed-in into the standard Function structure in the runtime
    function := NewFunction(baseHandler.handler, baseHandler.funcType).withContext(ctx)
    ...
    for {
        // Obtain request information

    req, err := client.next()
    ...
    // Launch a new goroutine to handle the request with the function
    go func(req *invoke, f *Function) {
        err = handleInvoke(req, function)
    ...
    }(req, function)
}

First, the program creates a client based on the RUNTIMEAPI to ensure the source of obtaining the request. Then, it creates a Function object based on the userHandler and functionType passed in earlier.

The creation of this Function object depends on the value of the function type passed down from the start entry point, which determines whether to create an event handler or an HTTP handler, corresponding to handling event requests and HTTP requests respectively.

In the code I provided, we transmit an event function, and the following analysis will be traced based on this method invocation flow.

func NewFunction(handler interface{}, funcType functionType) *Function {
    f := &Function{
        funcType: funcType,
    }
    // The type of handler to construct is determined based on the passed in funcType
    if f.funcType == eventFunction {
        f.handler = NewHandler(handler)
    } else {
        f.httpHandler = NewHttpHandler(handler)
    }
    return f
}

The event type handler is obtained through NewHandler, which requires the return value to be an interface of type Handler, which needs to implement the standard Invoke method.

type Handler interface {
    Invoke(ctx context.Context, payload []byte) ([]byte, error)
}
func NewHandler(userhandler interface{}) Handler {
    ...
    // Obtain the dynamic value of userhandler
    handler := reflect.ValueOf(userhandler)
    // Obtain the type information of userhandler
    handlerType := reflect.TypeOf(userhandler)
    ...
    return fcHandler(func(ctx context.Context, payload []byte) (interface{}, error) {
        ...
        // Call userhandler using the dynamic value
        response := handler.Call(args)
        ...
        return val, err
    })
}

In NewHandler, it also uses the reflection mechanism in GoLang to obtain the type and dynamic value of the userhandler, and constructs an fcHandler type method with standard parameters and return values based on the reflection information. In fcHandler, since handler in the code itself is of type Value, we can call the function it represents through the Call method. If you are interested in the details of reflection, you can also refer to the official documentation on GoLang Reflection.

fcHandler is special. It is a function type itself and already implements the Invoke method, so it is also a Handler type, which explains why fcHandler is used as the return value mentioned earlier. In fcHandler, Invoke calls the corresponding function of itself to handle the request.

After the preparation work is ready, the program starts to handle the request. Based on the above code analysis, it is not difficult to find that the main goroutine in this function mainly does three things:

Prepare the client to obtain user requests (newRuntimeAPIClient) and the Function (wrapped by the Handler);
Continuously obtain new requests through the client, i.e., the client.next() method in the code;
Allocate a new goroutine and let the Function handle the obtained request in the new goroutine.

Execution Flow #

So, when entering the new goroutine, the request is actually executed by the userHandler initially passed in. Let’s dive into the handleInvoke method in the goroutine, and we can find the following call relationship:

-> handleInvoke 1
    -> function.Invoke 2
        -> function.invokeEventFunc || function.invokeHttpFunc 3
            -> function.handler.Invoke 4

In step 3 of the numbered code, if we pass in an httpFunction type at the start, function.invokeHttpFunc will be called here. Of course, we will continue to trace along the previously mentioned event event request, so function.invokeEventFunc will be called in this function, and fn.handler.Invoke will be called in this function.

Based on the above function call relationship, when f.handler.Invoke is executed, it actually calls fcHandler once. Finally, fcHandler completes the call to userHandler through handler.Call.

type fcHandler func(context.Context, []byte) (interface{}, error)

func (handler fcHandler) Invoke(ctx context.Context, payload []byte) ([]byte, error) {

    response, err := handler(ctx, payload)
    if err != nil {
        return nil, err
    }

    responseBytes, err := json.Marshal(response)
    if err != nil {
        return nil, err
    }

    return responseBytes, nil
}

I have summarized the above process into the following diagram. You can refer to it while reviewing the main flow of GoLang runtime:

This is the main flow of the event invocation in the GoLang runtime. For more detailed flow and definitions, you can download the code from Github and understand it step by step based on this idea.

Through the study of GoLang runtime, I believe you have gained a clear understanding of the work that the runtime needs to complete and its overall processing flow.

That’s it for today’s lesson. You can take some time to digest. How does an interpreted language runtime work? How do we analyze it when cloud vendors do not open-source it? We will continue to discuss these questions in the next lesson.

Summary #

Finally, let me summarize today’s content. In this class, I introduced the implementation principle of the runtime of compiled languages, represented by Golang, in the form of serverless function computing. The function computing runtime is essentially a code framework that allows functions to be executed in containers.

The runtime is usually loaded by an initialization process, and then it communicates internally to receive and process the requests received by the function.

Depending on the type of programming language, the implementation of the runtime may vary slightly. The runtime of compiled languages needs to be packaged with user code into a binary file or other specific language package (such as Jar or War packages), while the runtime of interpreted languages can be separated from user code. Therefore, vendors generally open source the code of compiled runtimes and provide them to developers in the form of SDK.

From the code framework of the Golang runtime, we can see that the runtime mainly involves obtaining requests, associating the user’s function entry handler, and executing the user’s implementation.

I hope that through today’s lesson, you will have a certain understanding of the language runtime in the form of function computing. Not only will you know how to use it, but you will also know how it is implemented. When encountering problems or developing more complex functionalities in the future, you will have a better understanding.

Thought Question #

Alright, that’s the end of this lesson, and I have a thought question for you.

In our previous lesson, we talked about Knative. Does Knative involve the concept of runtime? Does runtime only exist on cloud platforms?

Feel free to write down your thoughts and answers in the comments section. Let’s exchange and discuss together. Thank you for reading, and feel free to share this lesson with more friends for mutual learning and discussion.