16 Code Review How to Conduct Static Code Analysis

16 Code Review How to Conduct Static Code Analysis #

Hello, I’m Kong Lingfei. In the previous lecture, when discussing the specific steps of code development, I mentioned static code inspection. Today, I will explain in detail how to perform static code inspection.

During the process of Go project development, we definitely need to perform static code inspection on Go code. Although the Go command provides go vet and go tool vet, they do not cover all aspects of code inspection, so we need a more powerful static code inspection tool.

In fact, there are many tools like this in the Go ecosystem, some of which are quite excellent. Today, I would like to introduce golangci-lint to you. It is currently the most widely used and popular static code inspection tool, and it is also used in our IAM practical project.

Next, I will introduce golangci-lint to you from three aspects: its advantages, the commands and options provided by golangci-lint, and the configuration of golangci-lint. After you have a basic understanding of these concepts, I will guide you in using golangci-lint for static code inspection, so that you can become familiar with the operation. Based on that, I will also share with you some tips and experiences I have summarized when using golangci-lint.

Why choose golangci-lint for static code analysis? #

Golangci-lint is chosen because it has several advantages that other static code analysis tools do not. In my opinion, its key advantages are as follows:

Speed: Golangci-lint is developed based on gometalinter, but it is on average 5 times faster than gometalinter. This is because golangci-lint can check code in parallel, reuse go build cache, and cache analysis results.
Configurability: It supports YAML format configuration files, making the checks more flexible and controllable.
IDE integration: It can be integrated into various popular IDEs, such as VS Code, GNU Emacs, Sublime Text, Goland, etc.
Linter aggregator: Golangci-lint version 1.41.1 integrates 76 linters, so there is no need to install these 76 linters separately. Golangci-lint also supports custom linters.
Minimal false positives: Golangci-lint adjusts the default settings of the integrated linters to significantly reduce false positives.
Good output: The output includes colors, line numbers, and linter identifiers, making it easy to view and locate issues.

The following image shows an example of golangci-lint’s check results:

You can see that the output of the check results includes the following information:

The source code file, line number, and the line containing the error.
The reason for the issue, which is the violation of the check rule.
The linter that reported the error.

By reviewing the output of golangci-lint, you can accurately locate the errors, quickly understand the reasons for the errors, and facilitate the developers’ fixing process.

In addition to the above advantages, in my opinion, golangci-lint has another significant advantage: it has a fast update cycle and constantly integrates new linters. With such comprehensive linters to protect your code, you will surely have more confidence when delivering your code.

Currently, many companies/projects use golangci-lint as their static code analysis tool, including Google, Facebook, Istio, Red Hat OpenShift, etc.

What commands and options does golangci-lint provide? #

Before using it, you need to install golangci-lint. Installing golangci-lint is also very simple. You just need to execute the following commands to install it.

$ go get github.com/golangci/golangci-lint/cmd/golangci-lint
$ golangci-lint version # Output the version of golangci-lint to verify successful installation
golangci-lint has version v1.39.0 built from (unknown, mod sum: "h1:aAUjdBxARwkGLd5PU0vKuym281f2rFOyqh3GB4nXcq8=") on (unknown)

Please note that to avoid installation failures, it is strongly recommended to install the specified version of golangci-lint from the golangci-lint releases page, for example, v1.41.1.

It is also recommended to regularly update the version of golangci-lint, as the project is actively developed and constantly improved.

After installation, you can start using it. You can use golangci-lint -h to view its usage. The subcommands supported by golangci-lint are shown in the table below:

In addition, golangci-lint also supports some global options. Global options refer to options that are applicable to all subcommands. The global options supported by golangci-lint are as follows:

Next, I will provide a detailed introduction to the core subcommands supported by golangci-lint: run, cache, completion, config, and linters.

The run command #

The run command executes golangci-lint to perform code checks and is the core command of golangci-lint. The run command does not have any subcommands, but it has many options. The specific usage of the run command will be explained in detail when I explain how to perform static code checks.

The cache command #

The cache command is used for cache control and printing cache information. It has two subcommands:

clean is used to clear the cache. When the cache content is considered abnormal, or when the cache takes up too much space, you can use golangci-lint cache clean to clear the cache.
status is used to print the status of the cache, such as the cache directory and cache size, for example:

$ golangci-lint cache status
Dir: /home/colin/.cache/golangci-lint
Size: 773.4KiB

The completion command #

The completion command includes four subcommands: bash, fish, powershell, and zsh, which are used to output the auto-completion scripts for bash, fish, powershell, and zsh respectively.

Here is an example of configuring bash autocompletion:

$ golangci-lint completion bash > ~/.golangci-lint.bash
$ echo "source '$HOME/.golangci-lint.bash'" >> ~/.bashrc
$ source ~/.bashrc

After executing the above commands, when you type the following command, the subcommands will be automatically completed:

$ golangci-lint comp<TAB>

The above command line will be automatically completed as golangci-lint completion.

The config command #

The config command can print the path of the configuration file currently used by golangci-lint, for example:

$ golangci-lint config path
.golangci.yaml

The linters command #

The linters command can print the linters supported by golangci-lint, and categorizes these linters into two types: enabled linters and disabled linters based on the configuration, for example:

$ golangci-lint linters
Enabled by your configuration linters:
...
deadcode: Finds unused code [fast: true, auto-fix: false]
...
Disabled by your configuration linters:
exportloopref: checks for pointers to enclosing loop variables [fast: true, auto-fix: false]
...

I have introduced the commands provided by golangci-lint. Next, let’s take a look at the configuration of golangci-lint.

Configuration of golangci-lint #

Compared with other linters, golangci-lint has a very big advantage in that it is very flexible to use, thanks to its support for custom configuration.

golangci-lint supports two configuration methods: command-line options and configuration files. If bool/string/int options are specified in both command-line options and configuration files, the command-line options will override the options in the configuration files. If it is a slice type option, the configurations from both the command-line and the configuration file will be merged.

golangci-lint run supports many command-line options, which can be viewed by running golangci-lint run -h. Here, I will introduce some of the important options, as shown in the table below:

In addition, we can use the golangci-lint configuration file for configuration. The default configuration file names are .golangci.yaml, .golangci.toml, and .golangci.json. The configuration file can be specified using the -c option. With the configuration file, we can achieve the following functions:

Some options of golangci-lint itself, such as timeout, concurrency, whether to check *_test.go files, etc.
Configure files and folders to be ignored.
Configure which linters to enable and disable.
Configure the output format.
golangci-lint supports many linters, and some linters support certain configuration options that can be configured in the configuration file.
Configure linters to be ignored for files that match specified regular expressions.
Set the severity level of errors, similar to logging; error checking also has severity levels.

For more detailed configuration content, you can refer to the Configuration page. In addition, you can also refer to the golangci-lint configuration in the IAM project: .golangci.yaml. There are some configurations in .golangci.yaml that I recommend you to set, as follows:

run:
  skip-dirs: # specify directories to be ignored
    - util
    - .*~
    - api/swagger/docs
  skip-files: # specify Go source code files that do not need to be checked, support regular expressions; I recommend including: _test.go
    - ".*\\.my\\.go$"
    - _test.go
linters-settings:
  errcheck:
    check-type-assertions: true # I recommend setting this to true; if you really don't need to check, you can write as `num, _ := strconv.Atoi(numStr)`
    check-blank: false
  gci:
    # Put packages that start with `github.com/marmotedu/iam` behind third-party packages
    local-prefixes: github.com/marmotedu/iam
  godox:
    keywords: # I recommend setting these to BUG, FIXME, OPTIMIZE, HACK
      - BUG
      - FIXME
      - OPTIMIZE
      - HACK
  goimports:
    # Set which packages to put behind third-party packages; multiple packages can be set, separated by comma
    local-prefixes: github.com/marmotedu/iam
  gomoddirectives: # Set allowed packages to replace in go.mod
    replace-local: true
    replace-allow-list:
      - github.com/coreos/etcd
      - google.golang.org/grpc
      - github.com/marmotedu/api
      - github.com/marmotedu/component-base
      - github.com/marmotedu/marmotedu-sdk-go
  gomodguard: # Below are packages and versions that can be selectively used as needed; I recommend setting them
    allowed:
      modules:
        - gorm.io/gorm
        - gorm.io/driver/mysql
        - k8s.io/klog
      domains: # List of allowed module domains
        - google.golang.org
        - gopkg.in
        - golang.org
        - github.com
        - go.uber.org
    blocked:
      modules:
        - github.com/pkg/errors:
            recommendations:
              - github.com/marmotedu/errors
            reason: "`github.com/marmotedu/errors` is the log package used by marmotedu projects."
      versions:
        - github.com/MakeNowJust/heredoc:
            version: "> 2.0.9"
            reason: "use the latest version"
      local_replace_directives: false
  lll:
    line-length: 240 # This can be set to 240; 240 is generally sufficient
  importas: # Set the alias for packages as needed
    jwt: github.com/appleboy/gin-jwt/v2         
    metav1: github.com/marmotedu/component-base/pkg/meta/v1

It should be noted that golangci-lint does not recommend using the enable-all: true option. To use as many linters as possible, we can use the following configuration:

linters: 
  disable-all: true  
  enable: # List all <desired linters>
    - typecheck
    - ...

<desired linters> = <all supported linters by golangci-lint> - <linters to be excluded>. We can obtain it by running the following command:

$ ./scripts/print_enable_linters.sh
    - asciicheck
    - bodyclose
    - cyclop
    - deadcode
    - ...

Replace the linters.enable section in the .golangci.yaml configuration file with the output above.

Above, we have introduced some basic knowledge related to golangci-lint. Next, I will show you in detail how to use golangci-lint for static code analysis.

How to use golangci-lint for static code analysis? #

To perform static code analysis on your code, simply execute the golangci-lint run command. Next, I will introduce you to five common usage methods for golangci-lint.

Perform static code analysis on all Go files in the current directory and its subdirectories:

$ golangci-lint run

This command is equivalent to golangci-lint run ./....

Perform static code analysis on specific Go files or Go files in specified directories:

$ golangci-lint run dir1 dir2/... dir3/file1.go

Please note that the above command does not include Go files in subdirectories of dir1. If you want to recursively check a directory, you need to append /... after the directory name, for example: dir2/....

Perform static code analysis based on a specific configuration file:

$ golangci-lint run -c .golangci.yaml ./...

Run a specific linter:

golangci-lint can be run without specifying any configuration file, in which case it runs the linters that are enabled by default. You can use the golangci-lint help linters command to view them.

You can enable a specific linter by passing the -E/--enable option, or disable a specific linter by using the -D/--disable option. The following example only enables the errcheck linter:

$ golangci-lint run --no-config --disable-all -E errcheck ./...

Note that by default, golangci-lint searches for configuration files named .golangci.yaml, .golangci.toml, or .golangci.json starting from the current directory and going up to the root (/) directory. If found, the discovered configuration file is used for the current run. To prevent reading any configuration file, you can use the --no-config option.

Disable specific linters:

If you want to disable certain linters, you can use the -D option.

$ golangci-lint run --no-config -D godot,errcheck

When using golangci-lint for code analysis, there may be many false positives. The so-called false positives are actually issues that we want certain linters of golangci-lint to tolerate. So, how can we reduce false positives as much as possible? golangci-lint provides several ways to achieve this, and I recommend using the following three:

Add the -e flag in the command line, or set the excluded check errors in the issues.exclude section of the configuration file. You can also use issues.exclude-rules to configure which linters are ignored for which files.
Use the run.skip-dirs, run.skip-files, or issues.exclude-rules configuration options to exclude all Go files in specified directories or specific Go files.
Use the //nolint comment in the Go source code files to ignore specific lines of code.

As golangci-lint includes a large number of linters, enabling all linters for a large project may result in many issues being reported. Each project also has different requirements for the granularity of linter checks. Therefore, golangci-lint uses nolint tags to toggle specific checks. The effects of nolint tags differ based on their location. Now I would like to introduce you to several ways to use nolint.

Ignore checks for all linters on a specific line:

var bad_name int //nolint

Ignore checks for specific linters on a specific line, multiple linters can be specified separated by commas ,:

var bad_name int //nolint:golint,unused

Ignore checks for a specific code block:

//nolint
func allIssuesInThisFunctionAreExcluded() *string {
  // ...
}

//nolint:govet
var (
  a int
  b int
)

Ignore checks for specific linters in a specific file:

Add the //nolint comment on the line above the package xx statement.

//nolint:unparam
package pkg
...

When using nolint, there are three things you need to consider:

First, if you enable nolintlint, you need to add a reason after //nolint, like // xxxx.

Second, you should use //nolint instead of // nolint. According to the Go spec, there should be no spaces after the // in comments that need to be read by the program.

Finally, if you want to ignore all linters, you can use //nolint; if you want to ignore specific linters, you can use //nolint:<linter1>,<linter2>.

Tips for using golangci-lint #

I have summarized some tips and tricks while using golangci-lint, and I’m sharing them here for your reference. I hope it can help you make better use of golangci-lint.

Tip 1: When making initial changes, modify by directory.

If it’s your first time using golangci-lint to check your code, there will likely be many errors. To reduce the pressure of modifications, you can check and modify the code by directory. This can effectively reduce the number of failures and alleviate the pressure of modifications.

Of course, if there are too many errors and you can’t fix them all at once, or if you want to slowly fix them later or simply not fix the existing issues, then you can use the --new-from-rev option of golangci-lint to only check the newly added code. For example:

$ golangci-lint run --new-from-rev=HEAD~1

Tip 2: Modify by file to reduce file switching and improve efficiency.

If there are many checking errors that involve multiple files, it is recommended to modify one file first, so you don’t have to switch back and forth between files. You can filter out the failed checks of a specific file using grep, for example:

$ golangci-lint run ./...|grep pkg/storage/redis_cluster.go
pkg/storage/redis_cluster.go:16:2: "github.com/go-redis/redis/v7" imported but not used (typecheck)
pkg/storage/redis_cluster.go:82:28: undeclared name: `redis` (typecheck)
pkg/storage/redis_cluster.go:86:14: undeclared name: `redis` (typecheck)
...

Tip 3: Set linters-setting.lll.line-length to a larger value.

In Go development, for the sake of readability, variables, functions, and constants are often named in a way that may result in lines of code that exceed the default maximum length of 80 set by the lll linter. Here, it is recommended to set linters-setting.lll.line-length to 120/240.

Tip 4: Use as many linters provided by golangci-lint as possible.

golangci-lint integrates many linters, which can be viewed using the following command:

$ golangci-lint linters
Enabled by your configuration linters:
deadcode: Finds unused code [fast: true, auto-fix: false]
...
varcheck: Finds unused global variables and constants [fast: true, auto-fix: false]

Disabled by your configuration linters:
asciicheck: Simple linter to check that your code does not contain non-ASCII identifiers [fast: true, auto-fix: false]
...
wsl: Whitespace Linter - Forces you to use empty lines! [fast: true, auto-fix: false]

These linters are divided into two categories: the ones that are enabled by default and the ones that are disabled by default. Each linter has two attributes:

fast: true/false. If true, it means that the linter can cache type information and supports fast checking. The first run caches this information, so subsequent runs will be very fast.
auto-fix: true/false. If true, it means that the linter supports automatic fixing of discovered errors; if false, it means that automatic fixing is not supported.

If you have configured a golangci-lint configuration file, you can use the command golangci-lint help linters to see which linters are enabled and disabled under the current configuration. golangci-lint also supports custom linter plugins. For more details, you can refer to New linters.

When using golangci-lint, it is recommended to use as many linters as possible. The more linters you use, the more strict the checks will be, which means the code will be more standardized and of higher quality. If time and effort allow, it is recommended to enable all the linters provided by golangci-lint.

Tip 5: Run golangci-lint after every code modification.

Run golangci-lint every time after modifying the code. On one hand, this can help you promptly fix any non-standard areas, and on the other hand, it can reduce the accumulation of errors and relieve the pressure of later modifications.

Tip 6: It is recommended to keep a common golangci-lint configuration file in the root directory.

Store a common golangci-lint configuration file in the root directory ("/"), so you don’t have to configure golangci-lint for each project. When you need to configure golangci-lint individually for a specific project, simply add a project-level golangci-lint configuration file in the project’s root directory.

Summary #

In Go project development, it is necessary to perform static code analysis on the code. Currently, there are many excellent static code analysis tools, but golangci-lint has become the most popular tool due to its fast check speed, configurability, low rate of false positives, and built-in linting capabilities.

golangci-lint is very powerful and supports commands such as run, cache, completion, and linters. The most commonly used command is run, which performs static code analysis in the following ways:

$ golangci-lint run # Perform static code analysis on all Go files in the current directory and its subdirectories
$ golangci-lint run dir1 dir2/... dir3/file1.go # Perform static code analysis on specified Go file(s) or directory(s)
$ golangci-lint run -c .golangci.yaml ./... # Perform static code analysis based on the specified configuration file
$ golangci-lint run --no-config --disable-all -E errcheck ./... # Run the specified errcheck linter
$ golangci-lint run --no-config -D godot,errcheck # Disable the specified godot and errcheck linters

In addition, golangci-lint also supports reducing false positives using annotations such as //nolint , //nolint:golint,unused , etc.

Finally, I shared some of my own experiences and tips when using golangci-lint. For example, when making the first modification, it can be done by directory; modifying by file reduces the number of file switches and improves efficiency; using as many linters provided by golangci-lint as possible. I hope these suggestions will be helpful to you when using golangci-lint.

Exercise #

Execute the command golangci-lint linters to see which linters are supported by golangci-lint and their functions.
Consider how to integrate custom linters into golangci-lint.

If you have any questions, feel free to discuss with me in the comment section. See you in the next lesson.