Special Delivery a Core Makefile for Go Project Common Usage

Special Delivery A Core Makefile for Go Project Common Usage #

Hello, I’m Kong Lingfei. Today, we have a special broadcast as an “extra dish” for those of you who are eager for more content. Enjoy!

In the 14th lesson, I emphasized the importance of mastering Makefile syntax and recommended that you study “Write Makefile with Me” written by Chen Hao (PDF Remastered Edition). Perhaps you’ve already clicked on the link and felt a bit overwhelmed by the many Makefile syntax.

Actually, in my opinion, although Makefile has many syntax rules, not all of them need to be mastered. Some syntax rules are rarely used in Go projects. To write a high-quality Makefile, you should first master some core and commonly used syntax knowledge. In this lesson, I will specifically introduce the commonly used Makefile syntax and rules in Go projects, helping you quickly establish a solid foundation in the most important aspects.

A Makefile consists of three parts: Makefile rules, Makefile syntax, and Makefile commands (these commands can be Linux commands or executable script files). In this lesson, I will introduce some core syntax knowledge from Makefile rules and Makefile syntax. Before introducing this syntax knowledge, let’s first see how to use a Makefile script.

How to Use Makefile #

In practical use, we usually start by writing a Makefile to specify the compilation rules for the entire project. We then use the Linux make command to parse the Makefile and automate project compilation and management.

By default, the make command will search for a Makefile in the current directory, in the order of GNUmakefile, makefile, and Makefile. Once found, it will read and execute this file.

Most make programs support both “makefile” and “Makefile” file names, but I recommend using “Makefile”. Because the first character of this file name is capitalized, it is obvious and easy to identify. make also supports the -f and --file parameters to specify other file names, such as make -f golang.mk or make --file golang.mk.

Introduction to Makefile Rules #

When learning Makefile, the most important thing is to understand the rules in Makefile. Rules are a crucial concept in Makefile, and they typically consist of targets, prerequisites, and commands, used to specify the order of compiling source files. The popularity of Makefile is mainly due to its rules, as Makefile rules can automatically determine whether a target needs to be recompiled, ensuring that the target is only compiled when necessary.

In this lesson, we will mainly focus on the syntax of rules in Makefile, as well as phony targets and order-only dependencies.

Rule Syntax #

The syntax of rules in Makefile mainly includes targets, prerequisites, and commands, as shown below:

target ... : prerequisites ...
    command
    ...
    ...

  • target: It can be an object file, an executable file, or a label. Wildcards can be used for targets, and when there are multiple targets, they are separated by spaces.

  • prerequisites: It represents the dependencies required to generate the target. When there are multiple dependencies, they are separated by spaces.

  • command: It represents the command(s) to be executed for the target (can be any shell command).

    • By default, before executing a command, the command itself is printed and then the result of the command is printed. To suppress the printing of the command, prefix each command with @.
    • Multiple commands can be specified, either on separate lines or on the same line separated by semicolons. If the second command depends on the first command, they need to be written on the same line and separated by a semicolon.
    • To ignore errors in a command, prefix each command with a minus sign -.

If the targets do not exist, or if any of the files in prerequisites are newer than the targets file, the command defined will be executed to generate the required file or perform the desired operation.

Let’s understand the rules in Makefile through an example.

Step 1: First, create a hello.c file.

#include <stdio.h>
int main() {
  printf("Hello World!\n");
  return 0;
}

Step 2: Create a Makefile file in the current directory.

hello: hello.o
    gcc -o hello hello.o

hello.o: hello.c
    gcc -c hello.c

clean:
    rm hello.o

Step 3: Execute make to generate an executable file.

$ make
gcc -c hello.c
gcc -o hello hello.o
$ ls
hello  hello.c  hello.o  Makefile

In the above Makefile example, there are two targets: “hello” and “hello.o”, and each target specifies the build command. When executing the make command, if the files “hello” and “hello.o” do not exist, the corresponding commands will be executed to generate the targets.

Step 4: Re-run make without updating any files.

$ make
make: 'hello' is up to date.

If the target exists and none of the prerequisites are newer than the target, the corresponding command will not be executed.

Step 5: Update hello.c and run make again.

$ touch hello.c
$ make
gcc -c hello.c
gcc -o hello hello.o

If the target exists and the prerequisites are newer than the target, the corresponding command will be executed again.

Step 6: Clean up the intermediate build files.

A Makefile usually has a phony target “clean” for cleaning up intermediate artifacts during compilation or for performing customized clean-up actions on the source directory.

$ make clean
rm hello.o

Wildcards can be used in rules as well. Make supports three wildcards: “*”, “?”, and “~”. For example:

objects = *.o
print: *.c
    rm *.c

Phony Targets #

Next, let’s introduce phony targets in Makefile. The management capabilities of Makefile are mainly achieved through phony targets.

In the previous Makefile example, we defined a target “clean”, which is actually a phony target, meaning that we don’t generate any files for this target. Since phony targets are not files, Make cannot generate their dependencies or determine whether to execute them.

Usually, we need to explicitly identify a target as a phony target. In Makefile, we can use .PHONY to mark a target as a phony target:

.PHONY: clean
clean:
    rm hello.o

Phony targets can have prerequisites and can also serve as “default targets”. For example:

.PHONY: all
all: lint test build

As phony targets are always executed, their prerequisites are always resolved. This way, we can achieve the purpose of executing all dependencies simultaneously.

Order-only Dependencies #

In the rules introduced above, if any file in the prerequisites changes, the target will be reconstructed. However, sometimes we only want to reconstruct the target when certain files in the prerequisites change. In this case, you can use order-only prerequisites.

The syntax of order-only prerequisites is as follows:

targets : normal-prerequisites | order-only-prerequisites
    command
    ...
    ...

In the above rule, the order-only prerequisites will only be used when constructing the targets for the first time. Even if the order-only prerequisites change later, the targets will not be reconstructed.

The targets will only be reconstructed if any file in the normal-prerequisites changes. Here, the prerequisites after the symbol “|” are the order-only prerequisites.

So far, we have introduced the rules in Makefile. Next, let’s take a look at some core syntax knowledge in Makefile.

Makefile Syntax Overview #

Because Makefile has a lot of syntax, this lecture only introduces the core syntax of Makefile, as well as the syntax used in the IAM project’s Makefile, including commands, variables, conditional statements, and functions. Since Makefile does not have too many complex syntaxes, once you grasp these knowledge points and apply them more in practice, you can write very complex and powerful Makefile files.

Commands #

Makefile supports Linux commands, and the way of calling them is similar to calling commands under the Linux system. By default, make will output the executing commands to the current screen. However, we can use the @ symbol before the command to prevent make from outputting the currently executing command.

Let’s look at an example. Now we have a Makefile like this:

.PHONY: test
test:
    echo "hello world"

Execute the make command:

$ make test
echo "hello world"
hello world

As you can see, make outputs the executed command. In many cases, we don’t need such prompts, because what we want to see is the log generated by the command, not the command being executed. In this case, you can use @ before the command to prevent make from outputting the executed command:

.PHONY: test
test:
    @echo "hello world"

Execute the make command again:

$ make test
hello world

As you can see, make only executes the command without printing the command itself. This makes the make output much clearer.

Here, I recommend adding @ symbol before each command to prevent the printing of the command itself, in order to ensure that your Makefile output is readable and useful.

By default, make checks the return code of each command after it is executed. If the return is successful (return code is 0), make will execute the next instruction; if the return is failed (non-zero return code), make will terminate the current command. In many cases, when a command encounters an error (such as deleting a non-existent file), we may not want to terminate. In this case, we can use the - symbol before the command to let make ignore the command’s error and continue executing the next command, for example:

clean:
    -rm hello.o

Variables #

Variables are probably the most frequently used syntax in Makefile. Makefile supports variable assignment, multiline variables, and environment variables. In addition, Makefile also has some built-in special variables and automatic variables.

Let’s first look at the most basic variable assignment feature.

Similar to other languages, Makefile also supports variables. When using variables, they will be expanded in place, just like shell variables, and then the replaced content will be executed.

Makefile can declare a variable through variable assignment, and a variable needs to be assigned an initial value when declared, such as ROOT_PACKAGE=github.com/marmotedu/iam.

Variables can be referenced using the $( ) or ${ } syntax. My recommendation is to use the $( ) syntax to reference variables, such as $(ROOT_PACKAGE), and I also recommend using consistent variable reference syntax throughout the Makefile.

Variables will be expanded in the same way as bash variables where they are used. For example:

GO=go
build:
    $(GO) build -v .

After expansion, it becomes:

GO=go
build:
    go build -v .

Next, let me introduce the 4 methods of variable assignment in Makefile.

  1. = The most basic assignment method.

For example:

BASE_IMAGE = alpine:3.10

When using = for assignment, pay attention to cases like this:

A = a
B = $(A) b
A = c

The final value of B is c b, not a b. That is, when assigning a variable to another variable, the value on the right side takes the final variable value.

  1. := Directly assigns the value at the current position.

For example:

A = a
B := $(A) b
A = c

The final value of B is a b. Using := assignment can avoid potential inconsistencies caused by = assignment.

  1. ?= If the variable is not assigned, it assigns the value after the equal sign.

For example:

PLATFORMS ?= linux_amd64 linux_arm64
  1. += Adds the value after the equal sign to the preceding variable.

For example:

MAKEFLAGS += --no-print-directory

Makefile also supports multiline variables. You can use the define keyword to set multiline variables, and line breaks are allowed in variables. The definition format is as follows:

define variable_name
variable_content
...
endef

The variable content can contain functions, commands, text, or other variables. For example, we can define a USAGE_OPTIONS variable:

define USAGE_OPTIONS

Options:
  DEBUG        Whether to generate debug symbols. Default is 0.
  BINS         The binaries to build. Default is all of cmd.
  ...
  V            Set to 1 enable verbose build. Default is 0.
endef

Makefile also supports environment variables. In Makefile, there are two types of environment variables, namely, Makefile pre-defined environment variables and custom environment variables.

Among them, custom environment variables can override the pre-defined environment variables in Makefile. By default, the environment variables defined in Makefile are only valid in the current Makefile. If you want to pass them to the lower level (call another Makefile in the Makefile), you need to use the export keyword to declare them.

The following example declares an environment variable, which can be used in the lower level Makefile:

...
export USAGE_OPTIONS
...


In addition, Makefile also supports two built-in variables: special variables and automatic variables.
**Special Variables** are predefined by make and can be directly referenced in the makefile. The list of special variables is as follows:

![](../images/197999b1806b4c4f85dfde7c67f83ec8.jpg)

Makefile also supports **automatic variables**. Automatic variables can improve the efficiency and quality of writing makefiles.

In a pattern rule in the makefile, both the target and the prerequisites are a series of files. So how do we write a command to generate the corresponding target from different prerequisites?

This is where automatic variables come into play. Automatic variables automatically extract a series of files defined in the pattern and continue until all files that match the pattern are exhausted. These automatic variables should only appear in the command of the rule. The automatic variables supported in the makefile are shown in the table below.

![](../images/607d3876f4df49fd82bcbcd37c067cd2.jpg)

Among these automatic variables, `$*` is the most commonly used. `$*` is quite effective for constructing related file names. If the target does not have a pattern defined, then `$*` cannot be deduced. However, if the suffix of the target file is recognized by make, then `$*` will be the part before the suffix. For example, if the target is foo.c, because `.c` is a suffix recognized by make, the value of `$*` will be foo.

### Conditional Statements

Makefile also supports conditional statements. Let's start with an example.

The following example checks whether the variable `ROOT_PACKAGE` is empty. If it is empty, it outputs an error message; otherwise, it prints the variable value:
    
```makefile
ifeq ($(ROOT_PACKAGE),)
$(error the variable ROOT_PACKAGE must be set prior to including golang.mk)
else
$(info the value of ROOT_PACKAGE is $(ROOT_PACKAGE))
endif

The syntax for conditional statements is:

# if ...
<conditional-directive>
<text-if-true>
endif
# if ... else ...
<conditional-directive>
<text-if-true>
else
<text-if-false>
endif

For example, to check if two values are equal:

ifeq condition-expression
...
else
...
endif
  • ifeq indicates the start of a conditional statement and specifies a condition expression. The expression contains two arguments separated by a comma and is enclosed in parentheses.
  • else indicates the case when the conditional expression is false.
  • endif indicates the end of a conditional statement. Each conditional expression should end with endif.
  • $ indicates the conditional keyword, and there are four keywords: ifeq, ifneq, ifdef, ifndef.

To deepen your understanding, let’s look at examples of each of these four keywords.

  1. ifeq: Conditional statement that checks for equality.

For example:

ifeq (<arg1>, <arg2>)
ifeq '<arg1>' '<arg2>'
ifeq "<arg1>" "<arg2>"
ifeq "<arg1>" '<arg2>'
ifeq '<arg1>' "<arg2>"

Compares the values of arg1 and arg2. If they are the same, it evaluates to true. You can also use make functions/variables instead of arg1 or arg2, for example, ifeq ($(origin ROOT_DIR),undefined) or ifeq ($(ROOT_PACKAGE),). The origin function will be explained in a separate discussion on functions.

  1. ifneq: Conditional statement that checks for inequality.

For example:

ifneq (<arg1>, <arg2>)
ifneq '<arg1>' '<arg2>'
ifneq "<arg1>" "<arg2>"
ifneq "<arg1>" '<arg2>'
ifneq '<arg1>' "<arg2>"

Compares the values of arg1 and arg2. If they are different, it evaluates to true.

  1. ifdef: Conditional statement that checks if a variable is defined.
ifdef <variable-name>

If the value is not empty, then the expression evaluates to true; otherwise, it evaluates to false. It can also be the return value of a function.

  1. ifndef: Conditional statement that checks if a variable is not defined.
ifndef <variable-name>

If the value is empty, then the expression evaluates to true; otherwise, it evaluates to false. It can also be the return value of a function.

Functions #

Makefile also supports functions, which include definition syntax and invocation syntax.

Let’s start with user-defined functions. The make interpreter provides a series of functions for the makefile to call. These functions are predefined functions in the makefile. We can define a function using the define keyword. The syntax for defining a function is:

define function-name
function-body
endef

For example, the following is a user-defined function:

define Foo
    @echo "my name is $(0)"
    @echo "param is $(1)"
endef

define is essentially defining a multiline variable that can be used as a function when called, and can only be used as a multiline variable elsewhere. For example:

var := $(call Foo)
new := $(Foo)

User-defined functions are a type of procedural call and do not have any return values. You can use user-defined functions to define a collection of commands and apply them in rules.

Now let’s look at predefined functions. As mentioned earlier, the make compiler also defines many functions. These functions are called predefined functions. The syntax for calling a function is similar to variables:

$(<function> <arguments>)

or

${<function> <arguments>}

<function> is the name of the function, <arguments> are the function parameters, separated by commas. The function parameters can also be variables.

Let’s look at an example:

PLATFORM = linux_amd64
GOOS := $(word 1, $(subst _, ,$(PLATFORM)))

The example above uses two functions: word and subst. The word function has two parameters, 1 and the output of the subst function. The subst function replaces _ with a space in the value of the PLATFORM variable (resulting in the value of PLATFORM being linux amd64). The word function takes the first word from the linux amd64 string. Hence, the value of GOOS becomes linux.

Makefile’s predefined functions can help us achieve many powerful functions. When writing a makefile, if there is a functional requirement, it is recommended to use these functions. If you want to use these functions, you need to know which functions are available and what functionality they provide.

The following are some common functions. Familiarize yourself with them and refer to them as needed in the future.

Including Other Makefiles #

In addition to Makefile rules and syntax, Makefile has many other features, such as including other Makefiles, automatically generating dependencies, file search, and so on. Here I will introduce another key feature used in the IAM project’s Makefile: including other Makefiles.

In a Makefile, we can include other Makefiles using the include keyword, similar to #include in C language. The included file will be inserted at the current position. The syntax for include is include <filename>, as shown below:

include scripts/make-rules/common.mk
include scripts/make-rules/golang.mk

The include statement can also include files using wildcards, such as include scripts/make-rules/*. When searching for Makefile files, the make command will follow the following order:

  1. If the path is absolute or relative, it will include the file directly.
  2. If the make command has the -I or --include-dir parameter, it will look for files in the specified directory.
  3. If the directory <prefix>/include (usually /usr/local/bin or /usr/include) exists, make will also search for files there.

If a file is not found, make will generate a warning message but won’t immediately produce a fatal error. It will continue to load other files. Once the Makefile reading is complete, make will retry the files that were not found or could not be read. If it still fails, make will display a fatal error message. If you want make to ignore files that cannot be read and continue execution, you can use a dash (-) before include, like -include <filename>.

Summary #

In this lesson, I focused on introducing some core syntax knowledge of Makefile rules and Makefile syntax to help you write a high-quality Makefile.

When discussing Makefile rules, we mainly learned about rule syntax, phony targets, and order-only dependencies. Once you grasp these Makefile rules, you will have mastered the most crucial content in Makefile.

When introducing Makefile syntax, I only discussed the core syntax of Makefile and the syntax used in the IAM project’s Makefile, including commands, variables, conditional statements, and functions. You may find these syntaxes a bit dry to learn, but as the saying goes, “If you want to do a good job, you must first sharpen your tools.” I hope you can master the core syntax of Makefile proficiently and lay a solid foundation for writing high-quality Makefiles.

That’s all for today’s content. Feel free to share your thoughts in the comments section below. See you in the next lesson.