17 The Powerful Decorators #

Hello, I’m Jingxiao. In this lesson, let’s learn about decorators together.

Decorators have always been a very useful and classic feature in Python, with a wide range of applications in engineering, such as logging, caching, and more. However, in everyday work and life, I have found that many people, especially beginners, often feel intimidated by decorators due to their relatively complex syntax, thinking that they are “too fancy to learn”. But in reality, it’s not the case.

In today’s lesson, I will start with the concepts of functions and closures that we discussed earlier, and then introduce the concept, syntax, and basic usage of decorators. Finally, I will deepen your understanding through practical examples in real-world engineering.

Now, let’s get into the main content and learn together!

Functions -> Decorators #

Function Core Review #

Before we introduce decorators, let’s first review some core concepts of functions that we must grasp.

First, we need to know that in Python, functions are first-class citizens. Functions are also objects. We can assign a function to a variable, like in the code below:

def func(message):
    print('Got a message: {}'.format(message))
    
send_message = func
send_message('hello world')

# Output
Got a message: hello world

In this example, we assign the function func to the variable send_message. So when you call send_message, it is equivalent to calling the function func().

Second, we can pass functions as arguments to another function, like in the code below:

def get_message(message):
    return 'Got a message: ' + message


def root_call(func, message):
    print(func(message))

root_call(get_message, 'hello world')

# Output
Got a message: hello world

In this example, we pass the function get_message as an argument to the function root_call() and then call it.

Third, we can define a function inside another function, which is called function nesting. Here is an example:

def func(message):
    def get_message(message):
        print('Got a message: {}'.format(message))
    return get_message(message)

func('hello world')

# Output
Got a message: hello world

In this code, we define a new function get_message() inside the function func(), and then call it as the return value of func().

Fourth, functions can also return function objects (closures). Here is an example:

def func_closure():
    def get_message(message):
        print('Got a message: {}'.format(message))
    return get_message

send_message = func_closure()
send_message('hello world')

# Output
Got a message: hello world

Here, the return value of the function func_closure() is the function object get_message itself. After that, we assign it to the variable send_message, and then call send_message('hello world') to output 'Got a message: hello world'.

Simple Decorators #

After a simple review, let’s move on to today’s new knowledge - decorators. As usual, let’s start with a simple example of a decorator:

def my_decorator(func):
    def wrapper():
        print('wrapper of decorator')
        func()
    return wrapper

def greet():
    print('hello world')

greet = my_decorator(greet)
greet()

# Output
wrapper of decorator
hello world

In this code, the variable greet points to the inner function wrapper(), and the inner function wrapper() will call the original function greet() again. Therefore, when greet() is called, it will first print 'wrapper of decorator', and then output 'hello world'.

The function my_decorator() here is a decorator. It wraps the original function greet() inside it and changes its behavior, but the original function greet() remains unchanged.

In fact, the above code can be expressed in a simpler and more elegant way in Python:

def my_decorator(func):
    def wrapper():
        print('wrapper of decorator')
        func()
    return wrapper

@my_decorator
def greet():
    print('hello world')

greet()

The @ here is called syntax sugar. @my_decorator is equivalent to the previous statement greet = my_decorator(greet), but it is more concise. Therefore, if you need to decorate other functions in your program in a similar way, you only need to add @decorator above them, which greatly improves the reusability of functions and the readability of the program.

Decorators with Parameters #

You might wonder, what if the original function greet() needs to pass arguments to the decorator?

One simple way is to add the corresponding parameters to the wrapper function in the decorator, like this:

def my_decorator(func):
    def wrapper(message):
        print('wrapper of decorator')
        func(message)
    return wrapper

@my_decorator
def greet(message):
    print(message)

greet('hello world')

# Output
wrapper of decorator
hello world

However, a new problem arises. What if I have another function that needs to be decorated by my_decorator(), but this new function has two parameters? What should we do? For example:

@my_decorator
def celebrate(name, message):
    ...

In fact, in most cases, we use *args and **kwargs as the parameters for the inner function wrapper() in the decorator. *args and **kwargs accept any number and type of arguments, so the decorator can be written in the following form:

def my_decorator(func):
    def wrapper(*args, **kwargs):
        print('wrapper of decorator')
        func(*args, **kwargs)
    return wrapper

Decorators with Custom Parameters #

Actually, decorators have even greater flexibility. As mentioned before, decorators can accept any type and number of parameters of the original function. In addition, they can also accept custom parameters.

For example, if I want to define a parameter to indicate the number of times the inner function of the decorator is executed, I can write it in the following form:

def repeat(num):
    def my_decorator(func):
        def wrapper(*args, **kwargs):
            for i in range(num):
                print('wrapper of decorator')
                func(*args, **kwargs)
        return wrapper
    return my_decorator

@repeat(4)
def greet(message):
    print(message)

greet('hello world')

# Output:
wrapper of decorator
hello world
wrapper of decorator
hello world
wrapper of decorator
hello world
wrapper of decorator
hello world

Is it still the original function? #

Now, let’s take a look at an interesting phenomenon. In the previous example, let’s try printing some metadata of the greet() function:

greet.__name__
# Output
'wrapper'

help(greet)
# Output
Help on function wrapper in module __main__:

wrapper(*args, **kwargs)

You will find that after the greet() function is decorated, its metadata has changed. The metadata tells us “it is no longer the previous greet() function, but has been replaced by the wrapper() function”.

To solve this problem, we can typically use the built-in decorator @functools.wraps, which helps preserve the metadata of the original function (i.e., copies the metadata of the original function to the corresponding decorator function):

import functools

def my_decorator(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print('wrapper of decorator')
        func(*args, **kwargs)
    return wrapper

@my_decorator
def greet(message):
    print(message)

greet.__name__

# Output
'greet'

Class decorators #

Previously, we mainly discussed the use of functions as decorators, but in fact, classes can also be used as decorators. Class decorators mainly rely on the __call__() function, which is executed whenever you call an instance of the class.

Let’s take a look at the following code:

class Count:
    def __init__(self, func):
        self.func = func
        self.num_calls = 0

    def __call__(self, *args, **kwargs):
        self.num_calls += 1
        print('num of calls is: {}'.format(self.num_calls))
        return self.func(*args, **kwargs)

@Count
def example():
    print("hello world")

example()

# Output
num of calls is: 1
hello world

example()

# Output
num of calls is: 2
hello world

...

Here, we define a class Count, which takes the original function func() as an argument during initialization. The __call__() function increments the variable num_calls by 1, then prints it, and finally calls the original function. Therefore, when we call the function example() for the first time, the value of num_calls is 1, and it becomes 2 when we call it for the second time.

Nested decorators #

Recalling the previous example, most of them involved a single decorator. However, Python actually supports multiple decorators. For example, you can write it in the following form:

@decorator1
@decorator2
@decorator3
def func():
    ...

The execution order is from the inside out, so the above statement is equivalent to the following line of code:

decorator1(decorator2(decorator3(func)))

In this way, the 'hello world' example can be rewritten as follows:

import functools

def my_decorator1(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print('execute decorator1')
        func(*args, **kwargs)
    return wrapper

def my_decorator2(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print('execute decorator2')
        func(*args, **kwargs)
    return wrapper

@my_decorator1
@my_decorator2
def greet(message):
    print(message)

greet('hello world')

# Output
execute decorator1
execute decorator2
hello world

Examples of Decorator Usage #

So far, I have finished discussing the basic concept and usage of decorators. Next, I will provide several examples from practical work to help deepen your understanding of decorators.

Authentication #

First, let’s talk about the most common application of authentication. This is easy to understand. Let’s take the most common example: when you log in to WeChat, you need to enter your username and password, and then click “Confirm”. In this process, the server will check whether your username exists and whether it matches the password. If the authentication passes, you can log in successfully; if not, an exception will be thrown and you will be prompted with a login failure message.

For example, on some websites, you can browse content without logging in, but if you want to publish an article or leave a comment, the server will check if you are logged in when you click “Publish”. If you are not logged in, this operation is not allowed.

Let’s take a look at a general code example:

import functools

def authenticate(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        request = args[0]
        if check_user_logged_in(request): # If the user is logged in
            return func(*args, **kwargs) # Execute the function post_comment() 
        else:
            raise Exception('Authentication failed')
    return wrapper
    
@authenticate
def post_comment(request, ...):
    ...

In this code, we defined the decorator authenticate. The function post_comment() represents a user’s comment on an article. Every time this function is called, it first checks whether the user is logged in. If the user is logged in, this operation is allowed. If not, it is not allowed.

Logging #

Logging is another common use case. In practical work, if you suspect that a certain function takes too long and increases the overall system latency, so you want to test the execution time of certain functions online, decorators are a common means to achieve this.

Typically, we represent it in the following way:

import time
import functools

def log_execution_time(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        res = func(*args, **kwargs)
        end = time.perf_counter()
        print('{} took {} ms'.format(func.__name__, (end - start) * 1000))
        return res
    return wrapper
    
@log_execution_time
def calculate_similarity(items):
    ...

Here, the log_execution_time decorator records the running time of a function and returns its execution result. If you want to calculate the execution time of any function, simply add @log_execution_time above that function.

Input Validation #

Now let’s talk about the third application I want to discuss today, input validation.

In large companies’ machine learning frameworks, before we use a machine cluster for model training, we often use decorators to validate the input (often a long JSON file) for its reasonableness. This can greatly avoid the huge overhead caused by incorrect input.

The code is usually written in the following format:

import functools

def validation_check(input):
    @functools.wraps(func)
    def wrapper(*args, **kwargs): 
        ... # Check if the input is valid
        
@validation_check
def neural_network_training(param1, param2, ...):
    ...

In fact, in work, many situations often involve unreasonable input. Because the models we call for training are often complex, the input files have thousands of lines, and it is often difficult to find errors.

Imagine a scenario where, without input validation, it is easy to encounter a situation where “the model training has been running for several hours, but the system reports an error saying that one of the input parameters is incorrect, resulting in wasted efforts”. Such “disaster” greatly reduces development efficiency and wastes machine resources.

Caching #

Finally, let’s look at the application of caching. The usage of caching decorators is actually quite common. Here, I will use the built-in LRU cache in Python as an example to explain (if you are not familiar with LRU cache, please click the link for reference).

The representation of LRU cache in Python is @lru_cache. @lru_cache caches the function’s arguments and results in the process. When the cache is full, it will delete the least recently used data.

Correct use of caching decorators can greatly improve program execution efficiency. Why is that? Let me provide a common example to illustrate.

In the server-side code of large companies, there are often many checks related to devices. For example, whether you are using an Android or iPhone device and what version you are using. One of the reasons for this is that some new features often only exist on certain specific phone systems or versions (e.g., Android v200+).

In this case, we usually use caching decorators to wrap these check functions to avoid repeated invocations and improve program execution efficiency. It can be written as follows:

@lru_cache
def check(param1, param2, ...): # Check user device type, version number, etc.
    ...

Summary #

In this lesson, we learned about the concept and usage of decorators. Decorators, in essence, modify the behavior of a function using a decorator function so that the original function doesn’t need to be modified.

Decorators is to modify the behavior of the function through a wrapper so we don’t have to actually modify the function.

In practice, decorators are commonly used in various domains such as authentication, logging, input validation, and caching. Proper use of decorators can greatly improve the readability and efficiency of a program.

Discussion Questions #

So, in your usual work, in which situations do you usually use decorators? Feel free to leave a comment and discuss with me. You are also welcome to share this article with your colleagues and friends, and progress together through communication.