10 Context - The Essential Multiprocessing Control Tool You Must Grasp #
In the previous lesson, I assigned you a homework, which was to practice using sync.Map. I believe you have already completed it. Now I will explain the methods of sync.Map for you.
- Store: Store a key-value pair.
- Load: Get the value corresponding to the key and check if the key exists.
- LoadOrStore: If the value corresponding to the key exists, return that value; if not, store the corresponding value.
- Delete: Delete a key-value pair.
- Range: Iterate over the sync.Map, similar to using for range.
With the introduction of these methods, I believe you will have a deeper understanding of sync.Map. Now let’s start today’s lesson: how to control concurrency better with Context.
How Goroutines Exit #
After a goroutine is started, in most cases, it needs to wait for the code inside to finish, and then the goroutine will exit on its own. But what if there is a scenario where we need to make the goroutine exit early? In the code below, I implemented a watchdog to monitor the program:
ch10/main.go
func main() {
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
watchDog("【Watchdog 1】")
}()
wg.Wait()
}
func watchDog(name string) {
// Start a for select loop to continuously monitor in the background
for {
select {
default:
fmt.Println(name, "is monitoring...")
}
time.Sleep(1 * time.Second)
}
}
I implemented a watchdog using the watchDog
function, which will continue to run in the background and print the message “Watchdog is monitoring…” every second.
If you need to stop the watchdog from monitoring and exit the program, one approach is to define a global variable, which can be modified by other parts of the code to send a stop signal to the watchdog. Then, in the goroutine, check this variable first and if it is notified to close, stop monitoring and exit the current goroutine.
However, this approach requires locking to ensure concurrency safety in multiple goroutines. Based on this idea, there is an upgraded solution: use select + channel for detection, as shown in the code below:
ch10/main.go
func main() {
var wg sync.WaitGroup
wg.Add(1)
stopCh := make(chan bool) // Used to stop the watchdog
go func() {
defer wg.Done()
watchDog(stopCh, "【Watchdog 1】")
}()
time.Sleep(5 * time.Second) // Let the watchdog monitor for 5 seconds first
stopCh <- true // Send a stop command
wg.Wait()
}
func watchDog(stopCh chan bool, name string) {
// Start a for select loop to continuously monitor in the background
for {
select {
case <-stopCh:
fmt.Println(name, "Stop command received, stopping soon")
return
default:
fmt.Println(name, "is monitoring...")
}
time.Sleep(1 * time.Second)
}
}
This example is a modified version of the watchDog
function using select + channel, which achieves the purpose of making the watchdog stop and the goroutine exit by sending a command through the channel. There have been two major modifications to the example:
- Add a
stopCh
parameter to thewatchDog
function to receive the stop command. - In the
main
function, declarestopCh
for stopping, pass it to thewatchDog
function, and then send the stop command throughstopCh <- true
to make the goroutine exit.
Getting Acquainted with Context #
The above example is a classical use case of select + channel, which also serves as a review of select. The way to make a goroutine exit through select + channel is more elegant. However, what if we want to cancel many goroutines at the same time? What if we want to cancel a goroutine based on a timer? In these situations, the limitations of select + channel become apparent. Even if we define multiple channels to solve the problem, the code logic will become very complex and difficult to maintain.
To solve complex goroutine problems like these, we need a solution that can track goroutines. Only by tracking each goroutine can we better control them, and this solution is Context, which is the focus of this lesson and is provided by the Go standard library.
Now I will rewrite the previous example using Context to achieve the function of stopping the watchdog:
ch10/main.go
func main() {
var wg sync.WaitGroup
wg.Add(1)
ctx, stop := context.WithCancel(context.Background())
go func() {
defer wg.Done()
watchDog(ctx, "【Watchdog 1】")
}()
time.Sleep(5 * time.Second) // Let the watchdog monitor for 5 seconds first
stop() // Send a stop command
wg.Wait()
}
func watchDog(ctx context.Context, name string) {
// Start a for select loop to continuously monitor in the background
for {
select {
case <-ctx.Done():
fmt.Println(name, "Stop command received, stopping soon")
return
default:
fmt.Println(name, "is monitoring...")
}
time.Sleep(1 * time.Second)
}
}
Compared to the select + channel solution, there are mainly four changes in the Context solution.
- The
stopCh
parameter ofwatchDog
is changed toctx
, with the typecontext.Context
. - The original
case <-stopCh
is changed tocase <-ctx.Done()
, used to check if it should stop. - Use the
context.WithCancel(context.Background())
function to generate a cancellable Context, which is used to send the stop command. Here,context.Background()
is used to generate an empty Context, generally used as the root node of the entire Context tree. - The original
stopCh <- true
stop command is changed to the cancel function returned by thecontext.WithCancel
function, which isstop()
.
As you can see, the overall code structure is the same as before, except that channels are replaced with Context. The above example is only one use case of Context, and its capabilities go beyond this. Now let me introduce what Context is.
What is Context #
A task may have many coroutines collaborating to complete it, and a single HTTP request may trigger the launch of many coroutines. These coroutines may also start more child coroutines, and it is impossible to know how many layers of coroutines there are or how many coroutines are in each layer.
If a task is terminated for some reason, and the HTTP request is cancelled, what happens to the coroutines they started? How can they be cancelled? Cancelling these coroutines can save memory, improve performance, and avoid unpredictable bugs.
Context is used to simplify and solve these problems, and it is also concurrency safe. Context is an interface that provides manual, timing, timeout, cancellation, and value passing functionality. It is mainly used to control the collaboration between multiple coroutines, especially the cancellation operation. Once the cancellation command is issued, the coroutines tracked by the Context will receive the cancellation signal so that they can perform cleanup and exit operations.
The Context interface has only four methods, which are described in detail below. You will often use them in development, and you can see them in conjunction with the code below.
type Context interface {
Deadline() (deadline time.Time, ok bool)
Done() <-chan struct{}
Err() error
Value(key interface{}) interface{}
}
- The Deadline method can get the set deadline. The first return value,
deadline
, is the deadline, and when this time is reached, the Context will automatically initiate a cancellation request. The second return value,ok
, represents whether the deadline is set. - The Done method returns a read-only channel of type
struct{}
. In a coroutine, if the chan returned by this method can be read, it means that the Context has initiated a cancellation signal. After receiving this signal through the Done method, cleanup operations can be performed, and then the coroutine can exit and release resources. - The Err method returns the reason for the cancellation, that is, the reason why the Context was cancelled.
- The Value method retrieves the value bound to this Context, which is a key-value pair, so you need a key to get the corresponding value.
Of the four methods of the Context interface, the most commonly used one is the Done method, which returns a read-only channel for receiving cancellation signals. When the Context is cancelled, this read-only channel will be closed, which is equivalent to issuing a cancellation signal.
Context Tree #
We don’t need to implement the Context interface ourselves. Go provides functions that can help us generate different Contexts. Through these functions, a Context tree can be generated so that the Contexts can be associated, and when the parent Context issues a cancellation signal, the child Contexts will also issue cancellation signals, allowing control over the exit of coroutines at different levels.
From a functional perspective, there are four well-implemented Contexts.
- Empty Context: Cannot be cancelled, has no deadline, and is mainly used as the root node of the Context tree.
- Cancellable Context: Used to issue cancellation signals. When cancelled, its child Contexts are also cancelled.
- Timeout Cancellable Context: Includes a timing function.
- Value Context: Used to store a key-value pair.
From the derived tree of Context shown in the figure below, the top is the empty Context, which serves as the root node of the entire Context tree. In Go, you can obtain a root node Context through context.Background()
.
(Derived tree of the four Contexts)
Once you have the root node Context, how do you generate this Context tree? You need to use the four functions provided by Go.
- WithCancel(parent Context): Generates a cancellable Context.
- WithDeadline(parent Context, d time.Time): Generates a timing cancellable Context, with parameter
d
specifying the specific time of timing cancellation. - WithTimeout(parent Context, timeout time.Duration): Generates a timeout cancellable Context, with parameter
timeout
specifying how long until cancellation. - WithValue(parent Context, key, val interface{}): Generates a value-carrying Context with a key-value pair.
Among these four functions that generate Contexts, the first three belong to cancellable Contexts. They are a type of function, while the last one is a value Context used to store a key-value pair.
Cancelling multiple coroutines using Context #
Cancelling multiple coroutines is also relatively simple. Just pass the Context as an argument to the coroutine. Taking the watchdog as an example:
ch10/main.go
wg.Add(3)
go func() {
defer wg.Done()
watchDog(ctx,"【Monitor Dog 2】")
}()
go func() {
defer wg.Done()
watchDog(ctx,"【Monitor Dog 3】")
}()
Two monitoring dogs have been added in the example, which means that two goroutines have been added. In this way, one Context controls three goroutines at the same time. Once the Context sends a cancellation signal, these three goroutines will all be canceled and exit.
In the example above, the Context does not have any child Contexts. What happens when a Context has child Contexts? This is illustrated by the following diagram:
(Context cancellation)
As you can see, when the node Ctx2 is canceled, its child nodes Ctx4 and Ctx5 will also be canceled. If there are further child nodes, they will also be canceled. In other words, all nodes with a root node of Ctx2 will be canceled, while other nodes such as Ctx1, Ctx3, and Ctx6 will not be canceled.
Context Value #
A Context can not only be canceled, but also pass values that can be used by other goroutines. I’ll explain it with the following code:
ch10/main.go
func main() {
wg.Add(4) // Remember to change this to 4, it was originally 3, because we need to start one more goroutine
// Omitted unrelated code
valCtx:=context.WithValue(ctx,"userId",2)
go func() {
defer wg.Done()
getUser(valCtx)
}()
// Omitted unrelated code
}
func getUser(ctx context.Context){
for {
select {
case <-ctx.Done():
fmt.Println("【获取用户】","Goroutine exited")
return
default:
userId:=ctx.Value("userId")
fmt.Println("【获取用户】","User ID:",userId)
time.Sleep(1 * time.Second)
}
}
}
This example is run together with the previous example, so I omitted the repeated code. Here, we store a key-value pair with a userId of 2 using the context.WithValue function. In the getUser function, we can retrieve the corresponding value by using ctx.Value(“userId”), achieving the purpose of passing values.
Principles of Using Context #
Context is a very useful tool that makes it easy to control the cancellation of multiple goroutines. It is also used in the Go standard library, such as in net/http to cancel network requests.
To use Context effectively, it is recommended to follow certain principles:
- Do not put Context in a structure, pass it as a parameter instead.
- When Context is used as a function parameter, it should be the first parameter.
- Use context.Background function to generate the root node Context, i.e., the top-level Context.
- When passing values with Context, only pass the necessary values and avoid passing everything.
- Context is thread-safe and can be safely used in multiple goroutines.
These principles are not enforced by the Go language compiler and require your adherence.
Conclusion #
Context generates a Context tree using the With series of functions, connecting related Contexts, which allows for unified control. Once a command is issued, the associated Contexts will send cancellation signals, and goroutines using these Contexts will receive the cancellation signals and then clean up and exit. When defining functions, if you want the caller to send a cancellation signal to your function, you can add a Context parameter to the function, allowing external callers to control it through Context, such as for a timeout exit requirement when downloading a file.