01 Workspace and Gopath

01 Workspace and GOPATH #

In this course, there is a lot of Go code, so I recommend you to listen to the audio while reading the article.

Hello, I’m Haolin. Starting from today, I will work with you to explore the entire knowledge system of the Go language.

In the past few years, I have witnessed the rise of Go language together with many enthusiasts.

From the self-bootstrapping of Go 1.5 (using Go language to implement Go language itself) to the fast GC (also known as garbage collector) of Go 1.7, to the comprehensive upgrade of its built-in tools in Go 1.10 released in February 2018, and the foreseeable key features of subsequent versions (such as the go mod command used for program dependency management), all these have made us excited. While Go language is steadily moving towards glory, it has become one of the favorite programming languages for software engineers.

The main purpose of starting this column is to explore the mysteries of Go language together and help you gain more knowledge during learning and practice.

I assume that the readers of this column already have certain computer basics. For example, you should know what an operating system is, how to set environment variables, how to use the command line correctly, and so on.

Of course, if you already have programming experience, especially a little bit of Go language programming experience, that’s even better. After all, what I want to teach you is the very core technologies in Go language.

If you are not familiar enough with the most basic concepts and syntax in Go language, you may need to refer to the Go Language Specification Document during the learning process. You can also take out the introductory knowledge map of the pre-preparation article and study it carefully.

Finally, let me talk about the narration mode of this column. I will always start with a Go language interview question, provide answers to it, tell you why I am concerned about this question, and what knowledge is hidden behind it. Moreover, I will expand on the content accordingly.

Alright, let’s get ready and start together.

The first thing we need to do when learning Go language is to download the corresponding binary package from the Go language official website based on the computer architecture (such as 32-bit or 64-bit) and operating system (such as Windows or Linux).

Afterward, we will unzip the installation package, place it in a certain directory, configure the environment variables, and verify the successful installation by entering go version on the command line.

During this process, we also need to configure three environment variables, namely GOROOT, GOPATH, and GOBIN. Let me briefly introduce them.

GOROOT: the path of the Go language installation root directory, which is the installation path of Go language.
GOPATH: the path of several workspace directories. It is our own defined workspace.
GOBIN: the path of the executable files generated by Go programs.

Among them, the concept behind GOPATH is the most common and important. Now, today’s interview question is: Do you know the significance of setting GOPATH?

Regarding this question, a typical answer is as follows:

You can simply understand GOPATH as the working directory of Go language. Its value is a path to one or multiple directories, where each directory represents a workspace of Go language.

We need to use these workspaces to store Go language source code files, as well as archived files (archive files with the extension “.a”) and executable files generated after installation.

In fact, since all operations (coding, dependency management, building, testing, installation, etc.) of Go language projects are basically centered around GOPATH and workspaces throughout their lifecycle, there are at least three knowledge points behind it, which are:

1. How is the organization of Go language source code?

2. Do you understand the result after the installation of source code (Go language source code can only be used by us or other codes after installation)?

3. Do you understand the process of building and installing Go programs (which is useful in program development and problem-solving, otherwise you may easily go astray)?

Now, let’s focus on these contents.

Knowledge Expansion #

The organization of Go source code —

Like many programming languages, Go organizes its source code into code packages. In the file system, these code packages correspond to directories. Since directories can have subdirectories, code packages can also have sub-packages.

A code package can contain any number of source code files with the “.go” extension, and these source code files need to be declared as belonging to the same code package.

The name of a code package is generally the same as the directory in which the source code files are located. If they are different, the code package name takes precedence during build and installation.

Each code package will have an import path. The import path of a code package is the path that other code uses to import program entities from that package. Before using program entities from a code package, we must first import the corresponding code package. The specific way to do this is to import the import path of the code package. Like this:
```
import "github.com/labstack/echo"
```
In a workspace, the import path of a code package is actually the relative path from the src subdirectory to the actual storage location of the package.

Therefore, the organization of Go source code is based on the environmental variable GOPATH, the workspace, the src directory, and the code package. In general, Go source code files should be located in a code package (directory) under the src directory of a workspace included in the GOPATH environmental variable.
1. Understanding the result after installing the source code —
After understanding how Go source code is organized, it is necessary to know what happens to the source code after it is installed.

Where are the source code files and the resulting files placed? As we all know, source code files are usually placed in the src subdirectory of a workspace.

After installation, if an archive file (file with the “.a” extension) is generated, it will be placed in the pkg subdirectory of the workspace. If an executable file is generated, it may be placed in the bin subdirectory of the workspace.

Let me explain the specific location and rules for storing archive files.

Source code files are organized in the form of code packages, and a code package corresponds to a directory. The resulting archive file generated when installing a code package has the same name as the code package.

The relative directory where the archive file is placed is the direct parent of the import path of the code package. For example, if the import path of an existing code package is:
```
github.com/labstack/echo
```
Then, run the command:
```
go install github.com/labstack/echo
```
The resulting archive file will be placed in the relative directory github.com/labstack, with the file name being echo.a.

By the way, the import path mentioned above has another meaning: the source code files of the code package exist in the code repository echo belonging to the labstack group on the GitHub website.

Going back to the topic, there is another level of directory between the relative directory of the archive file and the pkg directory, called the platform-specific directory. The name of the platform-specific directory is composed of the target operating system for building (also known as the “build”), an underscore, and the code for the target computing architecture.

For example, if the target operating system for building a code package is Linux, and the target computing architecture is 64-bit, then the corresponding platform-specific directory is linux_amd64.

Therefore, the archive file for the code package mentioned above will be placed in the subdirectory pkg/linux_amd64/github.com/labstack of the current workspace.

-(GOPATH and Workspace)

In short, what you need to remember is that the source code files under the src subdirectory of a workspace are usually placed in the corresponding directory under the pkg subdirectory or directly in the bin subdirectory of the workspace after installation.
Understanding the process of building and installing Go programs —

Let’s talk about the process of building and installing Go programs, as well as their similarities and differences.

The go build command is used for building, and the go install command is used for installation. Both building and installing code packages involve compiling, packaging, and other operations. Any files generated during these operations will be saved to a temporary directory.

If a library source code file is being built, the resulting files will only exist in the temporary directory. The main purpose of building in this case is to check and verify.

If a command source code file is being built, the resulting files will be moved to the same directory as the source code file. (I mentioned these two types of source code files in the “Preliminary Article” and will explain them in detail in later articles.)

The installation process first performs building, and then performs linking operations, and moves the resulting files to the designated directory.

Furthermore, if a library source code file is being installed, the resulting files will be moved to a subdirectory of the pkg directory in the workspace where it is located.

If a command source code file is being installed, the resulting files will be moved to the bin directory of the workspace or the directory pointed to by the environment variable GOBIN.

What you need to remember here is the difference between building and installing, as well as where the resulting files will appear after executing the corresponding commands.

Summary #

The concepts and meanings of workspace and GOPATH are things that every Go engineer needs to understand. Although they are relatively simple, it can’t be overstated that they are the core knowledge of Go program development.

However, I still find that some people overlook them during recruitment interviews. Many of the tools provided by Go operate on the basis of GOPATH and the workspace, such as the mentioned go build, go install, and go get. These three commands are also the ones we use most frequently.

Thought Questions #

When it comes to dependency management in Go programs, there are still many questions worth exploring. I have two questions here for you to further contemplate.

In Go language, how does it search for dependency packages in multiple workspaces?
Will conflicts arise if there are code packages with the same import path in multiple workspaces?

These two questions are actually related. The answers are not complicated; you can almost find them by conducting a few experiments. You can also take a look at the source code of the go build package and its sub-packages in the Go standard library. There are also many treasures there to help you deeply understand the construction process of Go programs.

Additional Reading #

Usage and Functionality of Some Optional Flags in the `go build` Command #

By default, when running the go build command, it does not compile the code packages that the target code package depends on. However, if the archived files of the dependent code packages do not exist, or if the source code files have changed, they will be compiled.

If you want to force compile them, you can add the -a flag when executing the command. In this case, not only the target code package will always be compiled, but also the code packages that it depends on will always be compiled, even if they are part of the standard library.

Additionally, if you want to not only compile the dependent code packages but also install their archived files, you can add the -i flag.

So how do you determine which code packages have been compiled? There are two ways.

Run the go build command with the -x flag, which shows the specific operations performed by the go build command. You can also add the -n flag to only view the operations without executing them.
Run the go build command with the -v flag, which shows the names of the code packages compiled by the go build command. This is useful when used in conjunction with the -a flag.

Now let’s talk about the go get command, which is closely related to installing Go source code.

The go get command automatically downloads the target code package from popular public code repositories (like GitHub) and installs them into the corresponding directory in the first workspace included in the GOPATH environment variable. If the GOBIN environment variable exists, only the code packages containing source code files will be installed in the directory pointed to by GOBIN.

The most commonly used flags are as follows:

-u: Download and install code packages regardless of whether they already exist in the workspace.
-d: Only download code packages without installing them.
-fix: After downloading the code package, run a tool to fix the code according to the current Go language version before installing it.
-t: Download code packages required for testing.
-insecure: Allow downloading and installing code packages through insecure network protocols. HTTP is an example of such a protocol.

The go get command provided by the official Go language is quite basic and does not provide dependency management functionality. Currently, there are many third-party tools on GitHub that provide this functionality, such as glide, gb, and the official ones dep and vgo, which mostly use go get internally.

Sometimes, for certain purposes, we may change the repository where the source code is stored or the relative path of the code package. In order to ensure that the remote import paths for the code packages are not affected by such changes, we use custom code package import paths.

The method to customize the remote import paths for code packages is to add an import comment to the right side of the package declaration statement in the library source code files, like this:

package semaphore // import "golang.org/x/sync/semaphore"

The original full import path for this code package is github.com/golang/sync/semaphore. This corresponds to the actual storage location on the network. The source code for this code package is actually located in the semaphore directory of the sync code repository of the golang organization on the GitHub website. After adding the import comment, you can download and install the code package with the following command:

go get golang.org/x/sync/semaphore

However, this requires some support on the server-side backend of the golang.org domain for this command to succeed.

For a complete explanation of custom code package import paths, you can refer to this link.

Well, that’s a brief introduction to the go build command and the go get command. If you want to access more detailed documentation, you can visit the official command documentation page of the Go language or enter commands such as go help build in the command line.

Click here to check out the detailed code accompanying the Go language column article.