16 Non-typed Safety - The unsafe Package You Love and Hate #
In the previous lesson, I left a little assignment for you to practice how to use reflection to invoke a method. Now I’ll explain it.
Let’s continue with the person
struct type as an example. I added a method called Print
to it, which prints a piece of text. Here’s the code:
func (p person) Print(prefix string){
fmt.Printf("%s:Name is %s,Age is %d\n",prefix,p.Name,p.Age)
}
Now we can invoke the Print
method using reflection. Here’s the code:
func main() {
p:=person{Name: "飞雪无情",Age: 20}
pv:=reflect.ValueOf(p)
// Reflectively invoke the Print method of person
mPrint:=pv.MethodByName("Print")
args:=[]reflect.Value{reflect.ValueOf("登录")}
mPrint.Call(args)
}
From the example, you can see that to invoke a method via reflection, you first need to find the method using the MethodByName
method. Since the Print
method requires arguments, you need to specify the arguments as a slice of reflect.Value
, which is the args
variable in the example. Finally, you can invoke the Print
method using the Call
method. Remember to pass args
as an argument to the Call
method.
When you run the above code, you will see the following result:
登录:Name is 飞雪无情,Age is 20
From the printed result, you can see that it produces the same result as directly calling the Print
method, which proves that invoking the Print
method via reflection is feasible.
Now let’s delve deeper into the world of Go. This lesson will introduce advanced usage of the unsafe
package that comes with Go.
As the name suggests, unsafe
is not safe. Go defines it as the package name to discourage its usage as much as possible. However, even though it is unsafe, it still has its advantages, which is the ability to bypass Go’s memory safety mechanism and directly read and write memory. Therefore, sometimes for performance reasons, we may still take the risk and use it to manipulate memory.
Pointer type conversion #
To make Go convenient to write, improve efficiency, and reduce complexity, its designers made it a strongly typed static language. Strongly typed means that once a type is defined, it cannot be changed. Static means that type checking is done before runtime. Also for safety considerations, Go does not allow conversion between two pointer types.
We generally use *T
to represent a pointer type that points to a variable of type T
. For safety considerations, two different pointer types cannot be converted to each other, such as *int
cannot be converted to *float64
.
Let’s take a look at the following code:
func main() {
i:= 10
ip:=&i
var fp *float64 = (*float64)(ip)
fmt.Println(fp)
}
This code prompts an error message at compile time: “cannot convert ip (type *int) to type *float64”, indicating that a forced type conversion is not allowed. What if we still need to perform the conversion? We need to use unsafe.Pointer
from the unsafe
package. Let me first explain what unsafe.Pointer
is and then I will explain how to perform the conversion.
unsafe.Pointer #
unsafe.Pointer
is a special kind of pointer that can represent the address of any type, similar to the void*
pointer in C. It is a versatile pointer.
Under normal circumstances, *int
cannot be converted to *float64
, but it can be done by using unsafe.Pointer
as a middleman. In the following example, I convert *int
to *float64
via unsafe.Pointer
and perform a multiplication operation with the new *float64
. You will find that the value of the variable i
is also changed to 30.
ch16/main.go
func main() {
i:= 10
ip:=&i
var fp *float64 = (*float64)(unsafe.Pointer(ip))
*fp = *fp * 3
fmt.Println(i)
}
This example doesn’t have any practical significance, but it demonstrates that using the versatile unsafe.Pointer
, we can perform any conversion between *T
. So what exactly is unsafe.Pointer
? Why can other types of pointers be converted to unsafe.Pointer
? It all depends on the definition of unsafe.Pointer
in the source code, as shown below:
// ArbitraryType is here for the purposes of documentation
// only and is not actually part of the unsafe package.
// It represents the type of an arbitrary Go expression.
type ArbitraryType int
type Pointer *ArbitraryType
According to the official comments in Go, ArbitraryType
can represent any type (the ArbitraryType
here is only for documentation purposes, so don’t pay too much attention to it; just remember that it can represent any type). And unsafe.Pointer
is *ArbitraryType
, which means unsafe.Pointer
is a pointer to any type, a generic pointer that can represent any memory address.
The uintptr
pointer type
#
uintptr
is also a pointer type, and it is large enough to hold the bit pattern of any pointer. It is defined as follows:
// uintptr is an integer type that is large enough
// to hold the bit pattern of any pointer.
type uintptr uintptr
Since unsafe.Pointer
already exists, why is uintptr
necessary? It’s because unsafe.Pointer
does not support arithmetic operations such as the +
operator, but uintptr
does. Through uintptr
, you can perform pointer offset calculations, allowing you to access specific memory and achieve true low-level memory operations.
In the code example below, I demonstrate the usage of uintptr
by modifying fields within a struct
using pointer offsets:
func main() {
p := new(person)
// Name is the first field of person and doesn't require any offset,
// so we can modify it directly through the pointer
pName := (*string)(unsafe.Pointer(p))
*pName = "飞雪无情"
// Age is not the first field of person, so we need to calculate the offset
// in order to correctly locate the memory block of the Age field
pAge := (*int)(unsafe.Pointer(uintptr(unsafe.Pointer(p)) + unsafe.Offsetof(p.Age)))
*pAge = 20
fmt.Println(*p)
}
type person struct {
Name string
Age int
}
This example demonstrates setting field values of a person
struct by directly accessing memory using pointer offsets.
Let me explain the steps:
- We first use the
new
function to declare a pointer variablep
of type*person
. - Next, we convert the
*person
pointer variablep
to a*string
pointer variablepName
usingunsafe.Pointer
. - Since the first field of
person
is of typestring
and is namedName
, thepName
pointer points to theName
field (with an offset of 0). ModifyingpName
essentially means modifying the value of theName
field. - Since
Age
is not the first field ofperson
, we need to perform pointer offset operations to modify it. First, we convert thep
pointer variable to auintptr
usingunsafe.Pointer
, which allows us to perform address calculations. To calculate the offset, we can use theunsafe.Offsetof
function, which returns a uintptr offset. With this offset, we can use the+
operator to obtain the correct memory address of theAge
field, which we then convert to a*int
pointer variablepAge
usingunsafe.Pointer
. - It’s important to note that when performing pointer arithmetic, we need to first convert the pointer to a uintptr before executing the address calculations. After performing the pointer arithmetic, we need to convert the result back to the appropriate pointer type (e.g.,
*int
in the example) usingunsafe.Pointer
. This allows us to perform value assignment or retrieval on the specific memory block. - With the pointer variable
pAge
, which points to theAge
field, we can now modify its value through assignment.
When running the above example, you should see the following result:
{飞雪无情 20}
This example serves to explain pointer arithmetic using uintptr
, which is why the field assignment in the person
struct is written in a more complex manner. In normal coding scenarios, the example code would be written as follows:
func main() {
p := new(person)
p.Name = "飞雪无情"
Both versions of the code will produce the same result. p.Age = 20
fmt.Println(*p)
}
The core of pointer operations lies in the fact that it operates on individual memory addresses. By manipulating memory addresses, you can point to different blocks of memory and perform operations on them without needing to know the name (variable name) of the memory.
Pointer Conversion Rules #
You already know that there are three types of pointers in Go: *T, unsafe.Pointer, and uintptr. Based on the examples explained above, we can summarize the conversion rules for these three pointers:
- Any type of *T can be converted to unsafe.Pointer.
- unsafe.Pointer can also be converted to any type of *T.
- unsafe.Pointer can be converted to uintptr.
- uintptr can also be converted to unsafe.Pointer.
(Pointer conversion diagram)
It can be observed that unsafe.Pointer is mainly used for type conversions of pointers and serves as a bridge for converting between different pointer types. uintptr is mainly used for pointer arithmetic, especially for locating different memory locations using an offset.
unsafe.Sizeof #
The Sizeof function returns the memory size occupied by a type. This size is only related to the type and is independent of the size of the content stored in a variable of that type. For example, a bool type occupies one byte, and int8 also occupies one byte.
Using the Sizeof function, you can check the memory size occupied by any type (such as string, slice, and integer). Here is an example code:
fmt.Println(unsafe.Sizeof(true))
fmt.Println(unsafe.Sizeof(int8(0)))
fmt.Println(unsafe.Sizeof(int16(10)))
fmt.Println(unsafe.Sizeof(int32(10000000)))
fmt.Println(unsafe.Sizeof(int64(10000000000000)))
fmt.Println(unsafe.Sizeof(int(10000000000000000)))
fmt.Println(unsafe.Sizeof(string("飞雪无情")))
fmt.Println(unsafe.Sizeof([]string{"飞雪u无情","张三"}))
For integers, the number of bytes they occupy determines the range of numbers they can store. For example, int8 occupies one byte, which is 8 bits, so it can store numbers ranging from -128 to 127, i.e., -2^(n-1) to 2^(n-1)-1. Here, n represents the number of bits, such as int8 representing 8 bits, int16 representing 16 bits, and so on.
For platform-dependent int types, it depends on whether the platform is 32-bit or 64-bit, and the larger size is chosen. For example, when I tested the above code, I found that the sizes of int and int64 are the same because I am using a 64-bit platform computer.
Tip: The memory size occupied by a struct structure is equal to the sum of memory sizes occupied by its field types.
Summary #
The most commonly used feature of the unsafe package is the Pointer pointer. With it, you can convert between *T, uintptr, and Pointer to fulfill your requirements, such as zero copy of memory or pointer arithmetic using uintptr. These features can improve program efficiency.
Although the features provided by the unsafe package are powerful, they are not safe. They can bypass the checks of the Go compiler and may cause problems due to your mistaken operations. Therefore, I recommend avoiding the use of unsafe as much as possible. However, if it is necessary to improve performance, then it can be used. For example, []byte to string conversion can be achieved using unsafe.Pointer to achieve zero copy. I will explain this in detail in the next lesson.