19 Performance Optimization How to Conduct Code Review and Optimization in Go

19 Performance Optimization - How to Conduct Code Review and Optimization in Go #

In the last lesson, I left you with a small task: when running the go test command, use the -benchmem flag for memory statistics. The answer to this task is quite simple, the command is as follows:

➜ go test -bench=. -benchmem  ./ch18

Running this command will show the memory statistics. This method of checking memory using -benchmem is applicable to all benchmark test cases.

Today we will talk about code inspection and optimization in Go. Let’s start with the explanation of the content.

In project development, ensuring code quality and performance involves not only unit testing and benchmark testing, but also code style checking and performance optimization.

  • Code style checking is a supplement to unit testing. It can check your code from a non-business perspective to see if there is any room for optimization, such as unused variables or dead code.
  • Performance optimization is measured through benchmark testing, so we can determine if the optimization actually improves the program’s performance.

Code Style Checking #

What is Code Style Checking? #

Code style checking, as the name suggests, is a static scanning check of the code you wrote based on Go language standards. This check is independent of your business logic.

For example, if you define a constant that is never used, although it doesn’t affect the code execution, the constant can be deleted. Here’s an example:

ch19/main.go

const name = "飞雪无情"

func main() {

}

In the example, the constant name is actually not used, so you can delete it to save memory. This kind of situation, where a constant is unused, can be detected by code style checking.

Another example is when you call a function that returns an error, but you don’t check the error. In this case, the program can still compile and run. However, the code is not rigorous because we ignore the returned error. Here’s an example:

ch19/main.go

func main() {

   os.Mkdir("tmp",0666)

}

In the example code, the Mkdir function returns an error, but you don’t check the returned error. In this case, even if the directory creation fails, you won’t know because you have ignored the error. If you use code style checking, such potential issues will be detected.

The above two examples help you understand what code style checking is and what it’s used for. In addition to these two cases, code style checking can also detect spelling issues, dead code, code simplification, underscore in naming conventions, redundant code, etc.

golangci-lint #

To perform code checking, the code needs to be scanned to analyze if there are any style issues.

Tip: Static code analysis does not run the code.

There are many tools available for Go language code analysis, such as golint, gofmt, misspell, etc. It would be cumbersome to configure each of them, so we usually don’t use them individually. Instead, we use golangci-lint.

golangci-lint is an integrated tool that incorporates many static code analysis tools, making it easy for us to use. By configuring this tool, we can easily enable the desired code style checks.

To use golangci-lint, you need to install it first. Since golangci-lint is written in Go, we can install it from source code. Open the terminal and enter the following command to install it:

➜ go get github.com/golangci/golangci-lint/cmd/golangci-lint

This command installs version v1.32.2 of golangci-lint. After installation, enter the following command in the terminal to check if it was installed successfully:

➜ golangci-lint version

golangci-lint has version v1.32.2

Tip: On MacOS, you can also install golangci-lint using brew.

Alright, after successful installation of golangci-lint, you can use it for code checking. Let’s take the constant name and Mkdir function from the previous example and demonstrate how to use golangci-lint. Enter the following command in the terminal:

➜ golangci-lint run ch19/

This example checks the code under the ch19 directory. After running it, you will see the following output:

ch19/main.go:5:7: `name` is unused (deadcode)

const name = "飞雪无情"

      ^

ch19/main.go:8:10: Error return value of `os.Mkdir` is not checked (errcheck)

        os.Mkdir("tmp",0666)

The code inspection result shows that the two code style issues mentioned earlier have been detected. Once the issues are detected, you can fix them to make the code more compliant with the style.

golangci-lint Configuration #

The configuration of golangci-lint is quite flexible. For example, you can customize which linters to enable. By default, golangci-lint enables the following linters:

deadcode - Dead code check

errcheck - Check whether errors are used

gosimple - Check if the code can be simplified

govet - Suspicious code check, such as formatting strings and inconsistent types

ineffassign - Check for unused code

staticcheck - Static analysis check

structcheck - Find unused struct fields

typecheck - Type check

unused - Check for unused code

varcheck - Check for unused global variables and constants

> Note: golangci-lint supports more linters. You can use the command `golangci-lint linters` in the terminal to see the descriptions of each linter.

To modify the default enabled linters, you need to configure golangci-lint. Create a file named .golangci.yml in the root directory of your project, which is the configuration file for golangci-lint. It will be automatically used when running code style checks. For example, if I only want to enable the unused check, I can configure it like this:

_.golangci.yml_

linters:
  disable-all: true
  enable:
    - unused

In team collaboration, it is important to have a fixed version of golangci-lint, so that everyone can check the code based on the same standards. Configuring the version used by golangci-lint is also simple, just add the following code to the configuration file:

service:
  golangci-lint-version: 1.32.2 # use the fixed version to not introduce new linters unexpectedly

Additionally, you can configure each enabled linter separately. For example, to set the language for spelling check to US, you can use the following code:

linters-settings:
  misspell:
    locale: US

There are many configuration options for golangci-lint, and you can configure it flexibly. For more information about golangci-lint configuration, you can refer to the [official documentation](https://golangci-lint.run/usage/configuration/). Here is a commonly used configuration:

_.golangci.yml_

linters-settings:
  golint:
    min-confidence: 0
  misspell:
    locale: US
linters:
  disable-all: true
  enable:
    - typecheck
    - goimports
    - misspell
    - govet
    - golint
    - ineffassign
    - gosimple
    - deadcode
    - structcheck
    - unused
    - errcheck
service:
  golangci-lint-version: 1.32.2 # use the fixed version to not introduce new linters unexpectedly

#### Integrating golangci-lint into CI

It is important to integrate code checks into the CI process for better results. This way, when developers submit code, CI will automatically check the code and promptly detect and fix issues.

Whether you are using Jenkins, Gitlab CI, or Github Action, you can run golangci-lint through a Makefile. Now, create a Makefile in the root directory of your project and add the following code:

Makefile

getdeps:

   @mkdir -p ${GOPATH}/bin

   @which golangci-lint 1>/dev/null || (echo "Installing golangci-lint" && go get github.com/golangci/golangci-lint/cmd/[[email protected]](/cdn-cgi/l/email-protection))

lint:

   @echo "Running $@ check"

   @GO111MODULE=on ${GOPATH}/bin/golangci-lint cache clean

   @GO111MODULE=on ${GOPATH}/bin/golangci-lint run --timeout=5m --config ./.golangci.yml

verifiers: getdeps lint

Note: You can search online to learn more about Makefile. It is relatively simple, so I won’t go into detail here.

Now, you can add the following command to your CI, which will automatically install golangci-lint and check your code:

make verifiers

Performance Optimization #

The purpose of performance optimization is to make the program run better and faster, but it is not necessary. So, at the beginning of the program, you don’t have to pursue performance optimization deliberately. Writing correct code is a prerequisite for performance optimization. 19 gold sentences.png

Heap Allocation or Stack #

In the older C language, memory allocation was done manually, and memory deallocation also had to be done manually.

  • The advantage of manual control is that you can request and utilize as much memory as needed.
  • However, the disadvantage is that if you forget to deallocate memory, it can lead to memory leaks.

Therefore, to allow developers to focus better on implementing business code, Go language added garbage collection to automatically reclaim unused memory.

Go language has two memory spaces: stack and heap.

  • Stack memory is automatically allocated and deallocated by the compiler, and developers have no control over it. Stack memory generally stores local variables and parameters in functions. When a function is created, this memory is automatically created; when a function returns, this memory is automatically released.
  • Heap memory has a longer lifespan than stack memory. If the value returned by a function will be used elsewhere, it will be automatically allocated on the heap by the compiler. Unlike stack memory, heap memory cannot be automatically released by the compiler; it can only be released through the garbage collector, so stack memory is more efficient.

Escape Analysis #

Since stack memory is more efficient, it is preferred to use stack memory. So how does Go language determine whether a variable should be allocated on the stack or the heap? This requires escape analysis. Let me explain escape analysis through an example. Here is the code:

ch19/main.go

func newString() *string{

   s:=new(string)

   *s = "飞雪无情"

   return s

}

In this example:

  • Memory is allocated using the new function.
  • It is assigned to the pointer variable s.
  • Finally, it is returned using the return keyword.

Note: The newString function has no real meaning. It is just for demonstration purposes.

Now let’s see if an escape occurs through escape analysis. Run the following command:

➜ go build -gcflags="-m -l" ./ch19/main.go

# command-line-arguments

ch19/main.go:16:8: new(string) escapes to heap

In this command, -m prints the escape analysis information, and -l disables inlining to make escape analysis easier to observe. From the output above, we can see that an escape occurs when a pointer is returned as a function return value, and an escape always occurs in this case.

Variables that escape to the heap cannot be immediately freed, and they can only be released through garbage collection. This adds pressure to garbage collection, so it is recommended to avoid escapes as much as possible and allocate variables on the stack so that resources can be reclaimed when the function returns, improving efficiency.

Below is the optimized version of the newString function that avoids escape:

ch19/main.go

func newString() string{

   s:=new(string)

   *s = "飞雪无情"

   return *s

}

- Again, view the escape analysis of the code above using the following command:

➜ go build -gcflags="-m -l" ./ch19/main.go

command-line-arguments #

ch19/main.go:14:8: new(string) does not escape


From the analysis results, you can see that although a pointer variable `s` is declared, the function does not return a pointer, so there is no escape.

This is an example of a pointer escaping as a function return value. Does this mean that there will be no escape if pointers are not used? Let's look at another example, with the code as follows:

```go
fmt.Println("飞雪无情")

Again, perform escape analysis and you will see the following results:

➜ go build -gcflags="-m -l" ./ch19/main.go

# command-line-arguments

ch19/main.go:13:13: ... argument does not escape

ch19/main.go:13:14: "飞雪无情" escapes to heap

ch19/main.go:17:8: new(string) does not escape

From this result, you can see that the string “飞雪无情” escapes to the heap because it is referenced by an already escaped pointer variable. The corresponding code is as follows:

func (p *pp) printArg(arg interface{}, verb rune) {

   p.arg = arg

   //Omitted irrelevant code

}

Therefore, variables referenced by already escaped pointers will also escape.

In Go, there are 3 special types: slice, map, and chan. Pointers referenced by these three types will also escape. Let’s look at an example:

ch19/main.go

func main() {

   m := map[int]*string{}

   s := "飞雪无情"

   m[0] = &s

}

Perform escape analysis again, and you will see the following results:

➜  gotour go build -gcflags="-m -l" ./ch19/main.go

# command-line-arguments

ch19/main.go:16:2: moved to heap: s

ch19/main.go:15:20: map[int]*string literal does not escape

From this result, you can see that the variable m does not escape, but the variable s, which is referenced by the variable m, escapes to the heap. Therefore, pointers referenced by map, slice, and chan types will always escape.

Escape analysis is a method of determining whether a variable is allocated on the stack or the heap. In actual projects, it is important to avoid escape as much as possible to avoid slowing down the GC and improve efficiency.

Tip: From the perspective of escape analysis, although pointers can reduce memory copying, they can also cause escape. Therefore, you should choose whether to use pointers based on the actual situation.

Optimization Techniques #

Based on the previous sections, I believe you already understand stack and heap memory, and when variables will escape. With this understanding, optimization becomes clearer because it is based on these principles. Here are a few optimization techniques:

The first technique is to avoid escape as much as possible, as stack memory is more efficient and does not require GC. For example, passing small objects by array is more effective than passing by slice.

If escape cannot be avoided and memory is allocated on the heap, it is important to reuse memory for frequent memory allocation operations. One way to achieve this is to use sync.Pool, which is the second technique.

The third technique is to select suitable algorithms to achieve high performance, such as trading space for time.

Pro tip: When optimizing performance, it is important to benchmark and validate the improvements made by your optimizations.

These are the three main techniques I summarized based on Go’s memory management mechanism. By following these techniques, you can achieve the desired optimization effects. In addition, there are other small tips, such as avoiding the use of locks as much as possible, keeping the scope of concurrent locking as small as possible, using StringBuilder for string-to-[]byte conversions, avoiding excessive nesting of defer, and so on.

Finally, I recommend using a built-in Go performance profiling tool called pprof. With this tool, you can analyze CPU usage, memory usage, blocking, and mutex analysis. It is not too difficult to use, and you can search for tutorials on how to use it for more information.

Conclusion #

In this lesson, we mainly covered two topics: code style checking and performance optimization. Code style checking was explained from the perspective of tool usage, while performance optimization was explained based on principles. Understanding the principles allows you to better optimize your code.

I believe the decision to perform performance optimization depends on two factors: business requirements and self-motivation. Therefore, it is important not to actively pursue performance optimization, especially not before ensuring that the code is correct and deploying it. Only after that, based on business needs, can you decide whether to optimize and how much time to spend on optimization. Self-motivation is a reflection of coding ability. Experienced developers, for example, subconsciously avoid escape, reduce memory copying, and design low-latency architectures in high-concurrency scenarios.

As a final assignment, consider incorporating golangci-lint into your own projects. I believe your efforts will pay off.

In the next lesson, I will introduce “Collaborative Development: Why Modular Management Can Improve Development Efficiency”. Make sure to tune in!