09 Synchronization - The sync Package Allows You to Control Concurrency Precisely #

In the last lesson, we left a thinking question: why are channels concurrency safe? It is because channels internally use mutex locks to ensure concurrency safety. In this lesson, I will introduce the use of mutex locks.

In Go language, we not only have convenient and advanced synchronization mechanisms like channels, but also more primitive synchronization mechanisms like sync.Mutex and sync.WaitGroup. Through them, we can more flexibly control data synchronization and concurrent execution of multiple goroutines. Let me explain them one by one.

Resource Contention #

In a goroutine, if the allocated memory is not accessed by other goroutines and is only used within that goroutine, there is no concern of resource contention.

However, if the same memory is accessed by multiple goroutines simultaneously, it can lead to situations where the order of access is unpredictable and the final result cannot be determined. This is called resource contention, and this shared memory can be referred to as a shared resource.

Let’s understand this further with the following example:

ch09/main.go

// Shared resource
var sum = 0
func main() {

  // Spawn 100 goroutines to increment sum by 10
   for i := 0; i < 100; i++ {
      go add(10)
   }

   // Prevent premature termination
   time.Sleep(2 * time.Second)
   fmt.Println("Sum is:",sum)
}

func add(i int) {
   sum += i
}

You might expect the result to be “Sum is 1000”, but when you run the program, the result might match your expectations, or it might be 990 or 980. The core reason for this situation is that the resource sum is not concurrency safe because multiple goroutines can execute sum += i simultaneously, resulting in unpredictable results.

Now that we know the reason, we also have a solution: ensure that only one goroutine executes the sum += i operation at a time. To achieve this, you can use sync.Mutex mutual exclusion lock.

Pro Tip: When using Go’s toolchain commands like go build, go run, and go test, you can add the -race flag to check for race conditions in Go code.

Synchronization Primitives #

sync.Mutex #

Mutex, as the name suggests, refers to a situation where only one goroutine executes a certain piece of code at any given time, and other goroutines have to wait until that goroutine completes execution.

In the example below, I declare a mutex lock called mutex and modify the add function to protect the code snippet sum += i with the lock. This makes the code snippet accessing the shared resource concurrency safe and produces the correct result.

ch09/main.go

var(
   sum int
   mutex sync.Mutex
)

func add(i int) {
   mutex.Lock()
   sum += i
   mutex.Unlock()
}

Pro Tip: The code snippet sum += i protected by the lock is also known as the critical section. In synchronized program design, the critical section refers to a program snippet that accesses shared resources that cannot be accessed simultaneously by multiple goroutines. When a goroutine enters a critical section, other goroutines must wait, ensuring concurrency safety of the critical section.

The use of a mutex lock is very simple. It has two methods, Lock and Unlock, which represent locking and unlocking, respectively. When a goroutine acquires the Mutex lock, other goroutines can only acquire the lock after the Mutex lock is released.

The Lock and Unlock methods of Mutex always appear in pairs, and it is essential to ensure that when Lock acquires the lock, Unlock is executed to release the lock. Therefore, the defer statement is used to release the lock in functions or methods, as shown in the following code:

func add(i int) {
   mutex.Lock()
   defer mutex.Unlock()
   sum += i
}

This ensures that the lock will always be released and not forgotten.

sync.RWMutex #

In the previous section about sync.Mutex, I locked the addition operation for the shared resource sum, ensuring that it is concurrency safe when modifying the sum value. But what if multiple goroutines perform read operations? Let’s see the following code:

ch09/main.go

package main

import (
	"fmt"
	"sync"
	"time"
)

var sum int
var mutex sync.RWMutex
var wg sync.WaitGroup

func main() {
	wg.Add(100)
	for i := 0; i < 100; i++ {
		go add(10)
	}

	wg.Add(10)
	for i := 0; i < 10; i++ {
		go readSum()
	}

	wg.Wait()
	fmt.Println("所有协程执行完毕")
}

func add(i int) {
	mutex.Lock()
	defer mutex.Unlock()
	sum += i
	wg.Done()
}

func readSum() {
	mutex.RLock()
	defer mutex.RUnlock()
	b := sum
	fmt.Println("和为:", b)
	wg.Done()
}

This example starts 10 goroutines that read the value of sum simultaneously. Since the readSum function does not include any locking control, it is not concurrency-safe. If one goroutine is executing the sum += i operation while another goroutine is executing the b = sum operation, it could result in reading an outdated value of num, leading to unpredictable results.

To solve the problem of resource competition mentioned above, you can use a mutual exclusion lock sync.Mutex, as shown in the following code:

func readSum() int {
   mutex.Lock()
   defer mutex.Unlock()
   b := sum
   return b
}

Since both the add and readSum functions use the same sync.Mutex, their operations are mutually exclusive. This means that while one goroutine is performing the sum += i operation, another goroutine attempting to read the value of sum with the b = sum operation will wait until the modification operation is completed.

Now we have solved the problem of resource competition when multiple goroutines are reading and writing simultaneously. However, we encounter another problem - performance. Since locking is required for every read and write operation on the shared resource, the performance is diminished. How can we solve this?

Now let’s analyze this specific scenario of reading and writing. There are several situations:

1. Reading is not allowed when writing, as it may result in reading dirty data (incorrect data).
2. Writing is not allowed when reading, as it may also produce unpredictable results.
3. Reading is allowed when reading, as the data does not change. Therefore, any number of goroutines reading simultaneously is concurrency-safe.

Therefore, the sync.RWMutex read-write lock can be used to optimize this code and improve performance. I will modify the example using the read-write lock to achieve the desired results, as shown below:

var mutex sync.RWMutex

func readSum() int {
   // Obtain read lock only
   mutex.RLock()
   defer mutex.RUnlock()
   b := sum

   return b
}

Compared to the mutex example, there are two changes made to the read-write lock implementation:

1. The lock declaration is changed to a read-write lock sync.RWMutex.
2. The code in the readSum function for reading the data is changed to a read lock using RLock and RUnlock.

This greatly improves performance because multiple goroutines can now read the data simultaneously without waiting for each other.

sync.WaitGroup #

In the example above, you may have noticed the code time.Sleep(2 * time.Second). This is to prevent the main function from returning, as once the main function returns, the program will exit.

Since we do not know when the 100 goroutines executing add and 10 goroutines executing readSum will complete execution, a longer wait time of two seconds is set.

Tip: The return of a function or method means that the current function has completed execution.

So there is a problem. If these 110 goroutines finish execution within two seconds, the main function should return early, but the program has to wait for two seconds before returning, resulting in a performance issue.

If the execution time of these 110 goroutines exceeds two seconds, the program will return early because the set wait time is only two seconds, resulting in some goroutines not completing execution and producing unpredictable results.

Is there a way to solve this problem? In other words, is there a way to monitor the execution of all goroutines and exit the program immediately once they have all finished execution, ensuring that all goroutines complete execution and also promptly exiting to save time and improve performance? Your first thought might be using channels, and you’re right - channels can indeed solve this problem, but it is complex. Go provides a more concise solution for us - it is the sync.WaitGroup.

Before using sync.WaitGroup to refactor the example, let’s refactor the code in the main function into a separate function called run. This will help with better understanding. The modified code is as follows:

package main

import (
	"fmt"
	"sync"
	"time"
)

var sum int
var mutex sync.RWMutex
var wg sync.WaitGroup

func main() {
	wg.Add(100)
	for i := 0; i < 100; i++ {
		go add(10)
	}

	wg.Add(10)
	for i := 0; i < 10; i++ {
		go readSum()
	}

	wg.Wait()
	fmt.Println("All goroutines have finished execution")
}

func add(i int) {
	mutex.Lock()
	defer mutex.Unlock()
	sum += i
	wg.Done()
}

func readSum() {
	mutex.RLock()
	defer mutex.RUnlock()
	b := sum
	fmt.Println("和为:", b)
	wg.Done()
}

func main() {
   run()
}

func run(){
   for i := 0; i < 100; i++ {
      go add(10)
   }

   for i:=0; i<10;i++ {
      go fmt.Println("The sum is:",readSum())
   }

   time.Sleep(2 * time.Second)
}

By moving the logic of the 110 coroutine for reading and writing into the run function, we can simply call the run function in the main function. Now, we can modify the run function using sync.WaitGroup to ensure that it completes properly, as shown below:

ch09/main.go

func run(){

   var wg sync.WaitGroup

   // Since we want to monitor 110 coroutines, we set the counter to 110
   wg.Add(110)
   for i := 0; i < 100; i++ {
      go func() {
         // Decrease the counter by 1
         defer wg.Done()
         add(10)
      }()
   }

   for i:=0; i<10;i++ {
      go func() {
         // Decrease the counter by 1
         defer wg.Done()
         fmt.Println("The sum is:",readSum())
      }()
   }

   // Wait until the counter value is 0
   wg.Wait()
}

The use of sync.WaitGroup is fairly simple and can be divided into three steps:

1. Declare a sync.WaitGroup and use the Add method to set the value of the counter. Set it to the number of coroutines you want to track, which in this case is 110.
2. Call the Done method when each coroutine completes to decrease the counter by 1 and indicate to sync.WaitGroup that the coroutine has finished executing.
3. Finally, call the Wait method to wait until the counter value is 0, which means that all tracked coroutines have finished executing.

Using sync.WaitGroup, we can easily track coroutines. After each coroutine executes, the run function can complete exactly at the same time as the coroutines.

sync.WaitGroup is suitable for scenarios where multiple coroutines coordinate to perform a task together. For example, when downloading a file using 10 coroutines, each coroutine downloads 1/10th of the file. Only when all 10 coroutines have finished downloading, can we consider the download complete. This is what we commonly refer to as multithreaded download, where multiple threads work together to significantly improve efficiency.

Tip: In fact, you can also think of coroutines in Go as threads in common parlance. From a user experience perspective, there isn’t much difference. However, from a technical implementation perspective, it’s important to know that they are different.

sync.Once #

In practice, you may have a requirement to ensure that a piece of code is only executed once, even in high-concurrency scenarios, such as creating a singleton.

To address this scenario, Go provides sync.Once to ensure that code is only executed once, as shown below:

ch09/main.go

func main() {
   doOnce()
}

func doOnce() {
   var once sync.Once
   onceBody := func() {
      fmt.Println("Only once")
   }

   // Used to wait for coroutines to complete
   done := make(chan bool)

   // Start 10 coroutines to execute once.Do(onceBody)
   for i := 0; i < 10; i++ {

go func() {
   // Pass the function to be executed as a parameter to once.Do
   once.Do(onceBody)
   done <- true
}()
}

for i := 0; i < 10; i++ {
<-done
}
}

This is an example provided by Go. Although 10 goroutines are started to execute the onceBody function, because once.Do is used, the function onceBody will only be executed once. In other words, sync.Once also ensures that the onceBody function is only executed once in high concurrency situations.

sync.Once is suitable for scenarios where a singleton object is created, a resource is loaded only once, and only once.

#### sync.Cond

In Go language, sync.WaitGroup is used for scenarios where it is necessary to wait for all goroutines to finish execution.

sync.Cond, on the other hand, can be used to issue commands and allow all goroutines to start execution when a signal is given. The key point is that goroutines are waiting to start and must be awakened by sync.Cond before they can execute.

sync.Cond, from its literal meaning, is a condition variable that has the ability to block and wake up goroutines. Therefore, it can awaken goroutines under certain conditions, but the condition variable is just one of its use cases.

Let's take the example of a race with 10 people to demonstrate the usage of sync.Cond. In this example, there is a referee who must wait for all 10 people to be ready before giving the command for them to start running, as shown below:

```go
// Race with 10 people and 1 referee giving the command
func race() {

cond := sync.NewCond(&sync.Mutex{})
var wg sync.WaitGroup
wg.Add(11)

for i := 0; i < 10; i++ {
   go func(num int) {
      defer wg.Done()
      fmt.Println(num, "is ready")
      cond.L.Lock()
      cond.Wait() // wait for the gunshot
      fmt.Println(num, "starts running...")
      cond.L.Unlock()
   }(i)
}

// wait for all goroutines to enter the wait state
time.Sleep(2 * time.Second)

go func() {
   defer wg.Done()
   fmt.Println("The referee is ready, preparing for the gunshot")
   fmt.Println("The race begins, everyone ready to run")
   cond.Broadcast() // gunshot fired
}()
// to prevent the function from returning prematurely
wg.Wait()
}

The comments in the above example are self-explanatory. Here is a brief description of the steps:

1. Generate a *sync.Cond using the sync.NewCond function for blocking and waking up goroutines.
2. Start 10 goroutines to simulate 10 people. After being ready, each goroutine calls cond.Wait() to block and wait for the gunshot. Note that you need to lock when calling cond.Wait().
3. Use time.Sleep to wait for all people to enter the wait state, so that the referee can call cond.Broadcast() to issue the command.
4. After the referee is ready, he can call cond.Broadcast() to notify everyone to start running.

sync.Cond has three methods, which are:

1. Wait: Block the current goroutine until it is awakened by other goroutines calling Broadcast or Signal methods. It needs to be locked when used, and it uses the lock in sync.Cond, which is the L field.
2. Signal: Awaken one goroutine that has been waiting for the longest time.
3. Broadcast: Awaken all waiting goroutines.

Note: Before calling Signal or Broadcast, make sure that the target goroutine is in the Wait blocking state, otherwise a deadlock may occur.

If you have previously learned Java, you will find that sync.Cond is similar to Java’s wait-notify mechanism. Its three methods, Wait, Signal, and Broadcast, correspond to wait, notify, and notifyAll in Java, respectively.

Summary #

In this lesson, we mainly discussed the use of synchronization primitives in Go, which can provide more flexible control over concurrent execution of multiple goroutines. In terms of usage, Go still recommends using higher-level concurrency control methods such as channels because they are more concise, easier to understand, and use.

However, the synchronization primitives discussed in this lesson are still useful. Synchronization primitives are usually used for more complex concurrency control. If you want more flexible control and performance, you can use them.

This concludes this lesson. There is one more synchronization primitive in the sync package that I haven’t covered, which is sync.Map. The usage of sync.Map is similar to the built-in map type, except that it is concurrency-safe. So the homework for this lesson is to practice using sync.Map.