16 Non Typed Safety the Unsafe Package You Love and Hate

16 Non-typed Safety - The unsafe Package You Love and Hate #

In the previous lesson, I left a little assignment for you to practice how to use reflection to invoke a method. Now I’ll explain it.

Let’s continue with the person struct type as an example. I added a method called Print to it, which prints a piece of text. Here’s the code:

func (p person) Print(prefix string){
   fmt.Printf("%s:Name is %s,Age is %d\n",prefix,p.Name,p.Age)
}

Now we can invoke the Print method using reflection. Here’s the code:

func main() {
   p:=person{Name: "飞雪无情",Age: 20}
   pv:=reflect.ValueOf(p)
   // Reflectively invoke the Print method of person
   mPrint:=pv.MethodByName("Print")
   args:=[]reflect.Value{reflect.ValueOf("登录")}
   mPrint.Call(args)
}

From the example, you can see that to invoke a method via reflection, you first need to find the method using the MethodByName method. Since the Print method requires arguments, you need to specify the arguments as a slice of reflect.Value, which is the args variable in the example. Finally, you can invoke the Print method using the Call method. Remember to pass args as an argument to the Call method.

When you run the above code, you will see the following result:

登录:Name is 飞雪无情,Age is 20

From the printed result, you can see that it produces the same result as directly calling the Print method, which proves that invoking the Print method via reflection is feasible.

Now let’s delve deeper into the world of Go. This lesson will introduce advanced usage of the unsafe package that comes with Go.

As the name suggests, unsafe is not safe. Go defines it as the package name to discourage its usage as much as possible. However, even though it is unsafe, it still has its advantages, which is the ability to bypass Go’s memory safety mechanism and directly read and write memory. Therefore, sometimes for performance reasons, we may still take the risk and use it to manipulate memory.

Pointer type conversion #

To make Go convenient to write, improve efficiency, and reduce complexity, its designers made it a strongly typed static language. Strongly typed means that once a type is defined, it cannot be changed. Static means that type checking is done before runtime. Also for safety considerations, Go does not allow conversion between two pointer types.

We generally use *T to represent a pointer type that points to a variable of type T. For safety considerations, two different pointer types cannot be converted to each other, such as *int cannot be converted to *float64.

Let’s take a look at the following code:

func main() {
   i:= 10
   ip:=&i
   var fp *float64 = (*float64)(ip)
   fmt.Println(fp)
}

This code prompts an error message at compile time: “cannot convert ip (type *int) to type *float64”, indicating that a forced type conversion is not allowed. What if we still need to perform the conversion? We need to use unsafe.Pointer from the unsafe package. Let me first explain what unsafe.Pointer is and then I will explain how to perform the conversion.

unsafe.Pointer #

unsafe.Pointer is a special kind of pointer that can represent the address of any type, similar to the void* pointer in C. It is a versatile pointer.

Under normal circumstances, *int cannot be converted to *float64, but it can be done by using unsafe.Pointer as a middleman. In the following example, I convert *int to *float64 via unsafe.Pointer and perform a multiplication operation with the new *float64. You will find that the value of the variable i is also changed to 30.

ch16/main.go

func main() {
   i:= 10
   ip:=&i
   var fp *float64 = (*float64)(unsafe.Pointer(ip))
   *fp = *fp * 3
   fmt.Println(i)
}

This example doesn’t have any practical significance, but it demonstrates that using the versatile unsafe.Pointer, we can perform any conversion between *T. So what exactly is unsafe.Pointer? Why can other types of pointers be converted to unsafe.Pointer? It all depends on the definition of unsafe.Pointer in the source code, as shown below:

// ArbitraryType is here for the purposes of documentation
// only and is not actually part of the unsafe package.
// It represents the type of an arbitrary Go expression.
type ArbitraryType int

type Pointer *ArbitraryType

According to the official comments in Go, ArbitraryType can represent any type (the ArbitraryType here is only for documentation purposes, so don’t pay too much attention to it; just remember that it can represent any type). And unsafe.Pointer is *ArbitraryType, which means unsafe.Pointer is a pointer to any type, a generic pointer that can represent any memory address.

The uintptr pointer type #

uintptr is also a pointer type, and it is large enough to hold the bit pattern of any pointer. It is defined as follows:

// uintptr is an integer type that is large enough 
// to hold the bit pattern of any pointer.
type uintptr uintptr

Since unsafe.Pointer already exists, why is uintptr necessary? It’s because unsafe.Pointer does not support arithmetic operations such as the + operator, but uintptr does. Through uintptr, you can perform pointer offset calculations, allowing you to access specific memory and achieve true low-level memory operations.

In the code example below, I demonstrate the usage of uintptr by modifying fields within a struct using pointer offsets:

func main() {

   p := new(person)

   // Name is the first field of person and doesn't require any offset,
   // so we can modify it directly through the pointer
   pName := (*string)(unsafe.Pointer(p))
   *pName = "飞雪无情"

   // Age is not the first field of person, so we need to calculate the offset
   // in order to correctly locate the memory block of the Age field
   pAge := (*int)(unsafe.Pointer(uintptr(unsafe.Pointer(p)) + unsafe.Offsetof(p.Age)))
   *pAge = 20

   fmt.Println(*p)
}

type person struct {
   Name string
   Age  int
}

This example demonstrates setting field values of a person struct by directly accessing memory using pointer offsets.

Let me explain the steps:

  1. We first use the new function to declare a pointer variable p of type *person.
  2. Next, we convert the *person pointer variable p to a *string pointer variable pName using unsafe.Pointer.
  3. Since the first field of person is of type string and is named Name, the pName pointer points to the Name field (with an offset of 0). Modifying pName essentially means modifying the value of the Name field.
  4. Since Age is not the first field of person, we need to perform pointer offset operations to modify it. First, we convert the p pointer variable to a uintptr using unsafe.Pointer, which allows us to perform address calculations. To calculate the offset, we can use the unsafe.Offsetof function, which returns a uintptr offset. With this offset, we can use the + operator to obtain the correct memory address of the Age field, which we then convert to a *int pointer variable pAge using unsafe.Pointer.
  5. It’s important to note that when performing pointer arithmetic, we need to first convert the pointer to a uintptr before executing the address calculations. After performing the pointer arithmetic, we need to convert the result back to the appropriate pointer type (e.g., *int in the example) using unsafe.Pointer. This allows us to perform value assignment or retrieval on the specific memory block.
  6. With the pointer variable pAge, which points to the Age field, we can now modify its value through assignment.

When running the above example, you should see the following result:

{飞雪无情 20}

This example serves to explain pointer arithmetic using uintptr, which is why the field assignment in the person struct is written in a more complex manner. In normal coding scenarios, the example code would be written as follows:

func main() {

   p := new(person)

   p.Name = "飞雪无情"

Both versions of the code will produce the same result. p.Age = 20

fmt.Println(*p)

}

The core of pointer operations lies in the fact that it operates on individual memory addresses. By manipulating memory addresses, you can point to different blocks of memory and perform operations on them without needing to know the name (variable name) of the memory.

Pointer Conversion Rules #

You already know that there are three types of pointers in Go: *T, unsafe.Pointer, and uintptr. Based on the examples explained above, we can summarize the conversion rules for these three pointers:

  1. Any type of *T can be converted to unsafe.Pointer.
  2. unsafe.Pointer can also be converted to any type of *T.
  3. unsafe.Pointer can be converted to uintptr.
  4. uintptr can also be converted to unsafe.Pointer.

Figure 1.png

(Pointer conversion diagram)

It can be observed that unsafe.Pointer is mainly used for type conversions of pointers and serves as a bridge for converting between different pointer types. uintptr is mainly used for pointer arithmetic, especially for locating different memory locations using an offset.

unsafe.Sizeof #

The Sizeof function returns the memory size occupied by a type. This size is only related to the type and is independent of the size of the content stored in a variable of that type. For example, a bool type occupies one byte, and int8 also occupies one byte.

Using the Sizeof function, you can check the memory size occupied by any type (such as string, slice, and integer). Here is an example code:

fmt.Println(unsafe.Sizeof(true))

fmt.Println(unsafe.Sizeof(int8(0)))

fmt.Println(unsafe.Sizeof(int16(10)))

fmt.Println(unsafe.Sizeof(int32(10000000)))

fmt.Println(unsafe.Sizeof(int64(10000000000000)))

fmt.Println(unsafe.Sizeof(int(10000000000000000)))

fmt.Println(unsafe.Sizeof(string("飞雪无情")))

fmt.Println(unsafe.Sizeof([]string{"飞雪u无情","张三"}))

For integers, the number of bytes they occupy determines the range of numbers they can store. For example, int8 occupies one byte, which is 8 bits, so it can store numbers ranging from -128 to 127, i.e., -2^(n-1) to 2^(n-1)-1. Here, n represents the number of bits, such as int8 representing 8 bits, int16 representing 16 bits, and so on.

For platform-dependent int types, it depends on whether the platform is 32-bit or 64-bit, and the larger size is chosen. For example, when I tested the above code, I found that the sizes of int and int64 are the same because I am using a 64-bit platform computer.

Tip: The memory size occupied by a struct structure is equal to the sum of memory sizes occupied by its field types.

Summary #

The most commonly used feature of the unsafe package is the Pointer pointer. With it, you can convert between *T, uintptr, and Pointer to fulfill your requirements, such as zero copy of memory or pointer arithmetic using uintptr. These features can improve program efficiency.

Although the features provided by the unsafe package are powerful, they are not safe. They can bypass the checks of the Go compiler and may cause problems due to your mistaken operations. Therefore, I recommend avoiding the use of unsafe as much as possible. However, if it is necessary to improve performance, then it can be used. For example, []byte to string conversion can be achieved using unsafe.Pointer to achieve zero copy. I will explain this in detail in the next lesson.