46 Access to Network Services #
You’re really great, as you have been following me from the very beginning, learning Go language step by step.
In the previous dozens of articles, I have introduced to you bit by bit the core knowledge of Go language and some very basic standard library packages. I believe that you now have the ability to do things independently.
To inspire your interest even more, I plan to write a few articles about Go language’s network programming. However, the topic of network programming has become so vast that it’s impossible to cover it completely even with a couple of dedicated books.
Therefore, what I’m going to discuss here can only be considered an introduction. As long as it can make you feel motivated to try it out, I will be very happy.
Introduction: Socket and IPC #
People often use the Go language to write network programs (of course, this is also one of the things that Go language excels at). When it comes to network programming, we have to mention socket.
Socket, often translated as “套接字” in Chinese, can be considered one of the most essential knowledge in the world of network programming. There is so much to discuss about socket, so here I will only introduce some of the basic knowledge about it in relation to Go language.
The so-called socket is an IPC (Inter-Process Communication) method. IPC is the abbreviation of Inter-Process Communication and can be translated as communication between processes. As the name implies, IPC mainly defines the methods for multiple processes to communicate with each other.
These methods mainly include: system signals, pipes, sockets, file locks, message queues, semaphores, etc. Most mainstream operating systems provide powerful support for IPC, especially socket.
You may already know that Go language also provides some support for IPC.
For example, the os package and os/signal package have APIs for system signals.
For example, the os.Pipe function can create named pipes, while the os/exec package provides support for another type of pipe (anonymous pipe). As for socket, the corresponding entities in Go language are in the net package of its standard library.
It is no exaggeration to say that among the various IPC methods, socket is the most common and flexible one. Unlike other IPC methods, processes that communicate using socket are not limited to the same computer.
In fact, the communicating parties can be located in any corner of the world, as long as they can be connected through the computer’s network interface and the internet, they can use socket.
Operating systems that support socket generally provide a set of APIs. Applications running on these operating systems can communicate with programs in another computer on the internet, other programs on the same computer, or even other threads within the same program using this set of APIs.
For example, in the Linux operating system, the API for creating a socket instance is represented by a system call called socket. This system call is part of the Linux kernel.
The so-called system call can be understood as a special C language function. They serve as a bridge between applications and the operating system kernel, and are the only channel for applications to use operating system functions.
In the syscall package of the Go standard library, there is a function corresponding to this socket system call. The function signatures of these two are basically the same, they both accept three int parameters and return a result that represents a file descriptor.
However, the difference is that the Socket function in the syscall package is platform-independent. At its core, Go language has provided adaptation for each operating system it supports, which allows this function to be effective no matter which platform it is on.
Many entities in Go language’s net package, either directly or indirectly, utilize the syscall.Socket function.
For example, when we call the net.Dial function, we will set values for its two parameters. The first parameter is called network, which determines what type of socket instance will be created by the Go program at the low level and what protocol will be used to communicate with other programs.
Now, let’s take a look at how to correctly call the net.Dial function through a simple question.
Today’s question is: What are the optional values for the first parameter network of the net.Dial function?
The typical answer to this question is as follows.
The net.Dial function takes two parameters, named network and address, both of which are of type string.
There are a total of nine commonly used optional values for the parameter network. These values represent different communication protocols that the socket instance created by the program’s underlying layer can use. They are listed as follows.
"tcp": Represents the TCP protocol, and the version of the underlying IP protocol adapts according to the value of the parameteraddress."tcp4": Represents the TCP protocol based on the fourth version of the IP protocol."tcp6": Represents the TCP protocol based on the sixth version of the IP protocol."udp": Represents the UDP protocol, and the version of the underlying IP protocol adapts according to the value of the parameteraddress."udp4": Represents the UDP protocol based on the fourth version of the IP protocol."udp6": Represents the UDP protocol based on the sixth version of the IP protocol."unix": Represents an internal socket protocol in the Unix domain, with SOCK_STREAM as the socket type."unixgram": Represents an internal socket protocol in the Unix domain, with SOCK_DGRAM as the socket type."unixpacket": Represents an internal socket protocol in the Unix domain, with SOCK_SEQPACKET as the socket type.
Problem Analysis #
In order to better understand the deep meanings of these optional values, we need to understand the three parameters accepted by the syscall.Socket function.
As I mentioned earlier, these three parameters accepted by the function are all of type int. They respectively represent the communication domain, type, and protocol used for the socket instance to be created.
The main options for socket communication domains are the IPv4 domain, IPv6 domain, and Unix domain.
I believe you can guess the meanings of IPv4 domain and IPv6 domain. They correspond to networks based on IP protocol version 4 and IP protocol version 6, respectively.
Currently, most computer networks are based on IP protocol version 4. However, due to the gradual depletion of existing IP addresses, the network world is gradually supporting IP protocol version 6.
Unix domain refers to a communication domain specific to Unix-like operating systems. In a computer with such an operating system, applications can establish socket connections based on this domain.
The above three communication domains can be represented by the constants AF_INET, AF_INET6, and AF_UNIX in the syscall code package.
There are a total of four types of sockets, which are SOCK_DGRAM, SOCK_STREAM, SOCK_SEQPACKET, and SOCK_RAW. The syscall code package also has corresponding constants with the same names. The first two are more commonly used.
“DGRAM” in SOCK_DGRAM stands for datagram, which is a type of socket that has message boundaries but no logical connections and is unreliable. The well-known network communication based on the UDP protocol belongs to this category.
The term “message boundaries” means that the programs in the socket-related operating system kernel (referred to as kernel programs below) send or receive data in units of messages.
You can think of a message as a piece of data with a fixed boundary. The kernel program can automatically recognize and maintain this boundary, and when necessary, split the data into separate messages or concatenate multiple messages into continuous data. In this way, the application only needs to process the messages.
The so-called logical connection means that both parties in the communication must establish a network connection before sending and receiving data. After the connection is established, the two parties can transmit data one-to-one. Obviously, this is not required for network communication based on the UDP protocol.
As long as the application specifies the network address of the other party, the kernel program can immediately send the datagram. This has advantages and disadvantages.
The advantage is that it has a fast sending speed, does not occupy network resources for a long time, and can specify different network addresses for each sending.
Of course, the last advantage is sometimes a disadvantage, because it makes the datagram longer. Other disadvantages include inability to guarantee transmission reliability, inability to achieve data ordering, and one-way transmission of data.
On the other hand, the socket type SOCK_STREAM is the opposite of SOCK_DGRAM. It does not have message boundaries, but has logical connections, can ensure transmission reliability and data ordering, and can also achieve bidirectional data transmission. The well-known network communication based on the TCP protocol belongs to this category.
The form of data transmission in this type of network communication is byte stream, not datagrams. A byte stream is measured in bytes. The kernel program cannot perceive how many messages are contained in a byte stream, and whether these messages are complete. The application program needs to control this on its own.
However, in this type of network communication, one end always faithfully arranges and receives the bytes sent by the other end in the order in which they were sent, and caches them. Therefore, the application program needs to search for message boundaries in the data based on the agreement between the two parties, and split the data according to the boundaries, that’s all.
The third parameter of the syscall.Socket function is used to specify the protocol used by the socket instance.
Usually, as long as the values of the first two parameters are specified, we do not need to determine the value of the third parameter, and generally set it to 0. In this case, the kernel program will choose the most suitable protocol on its own.
For example, when the values of the first two parameters are syscall.AF_INET and syscall.SOCK_DGRAM, the kernel program will choose UDP as the protocol.
For another example, when the values of the first two parameters are syscall.AF_INET6 and syscall.SOCK_STREAM, the kernel program may choose TCP as the protocol.
- (A brief view of the syscall.Socket function)
However, as you can see, when using the high-level APIs in the net package, we don’t even need to specify the values of the first two parameters. We only need to use one of the string literals listed above as the value of the network parameter.
Of course, if you can think of the basic knowledge I mentioned above when using these APIs, it will definitely help you make correct judgments and choices.
Further Knowledge #
Question 1: What does the timeout given when calling the net.DialTimeout function mean?
#
Simply put, the timeout given here represents the maximum time the function will wait for the network connection to be established. This is a relative time and is indicated by the value of the timeout parameter of the function.
The starting time is almost the same as the moment we call the net.DialTimeout function. After that, time is mainly spent on “parsing the values of the network and address parameters” and “creating a socket instance and establishing a network connection”.
Regardless of which step is being executed, if the network connection is not established by the absolute timeout time, the function will return an error value that represents a timeout for the I/O operation.
It is worth noting that when parsing the value of the address, the function will determine the necessary information such as the IP address and port number of the network service and access the DNS service if necessary.
In addition, if multiple IP addresses are resolved, the function will attempt to establish connections in serial or concurrent order. But regardless of the method used to try, the function will always take the connection that is established first as the result.
At the same time, it will set the timeout for each connection attempt according to the remaining time before the timeout, so that they all have enough time to execute.
One more thing. In the net package, there is also a struct type called Dialer. This type has a field named Timeout, which has the same meaning as the above timeout parameter. In fact, the function net.DialTimeout utilizes the value of this type to achieve its functionality.
You should study the net.Dialer type thoroughly, especially the function of each field and its DialContext method.
Summary #
Today we discussed the topic of network programming using the Go language. As an introduction, I first introduced you to some basic knowledge about sockets. Socket, often translated as “套接字” in Chinese, is a type of IPC (Inter-Process Communication) method. IPC defines the methods of communication between multiple processes.
Socket is the most common and flexible IPC method. Unlike other methods, processes communicating with each other using sockets are not limited to the same computer.
As long as the communicating parties can connect through the computer’s network card ports and the network, sockets can be used, no matter where they are located in the world.
Operating systems that support sockets generally provide a set of APIs. The syscall code package in Go language also has corresponding program entities for sockets. One of the most important ones is the syscall.Socket function.
However, these program entities in the syscall package are considered low-level for regular Go programs, and we rarely use them. In general, we use APIs in the net code package and its sub-packages to write network programs.
A very common function in the net package is called Dial. This function is mainly used to connect to network services. It accepts two parameters, and you need to understand how to set the values of these two parameters.
In particular, the network parameter has many optional values, with 9 of them being the most commonly used. These optional values represent the corresponding socket attributes, including communication domain, type, and protocol used. Once you understand these socket attributes, it will help you make the right judgments and choices.
A related function is net.DialTimeout. When calling this function, we need to set a timeout value. You need to understand the meaning of this timeout value.
Through it, we can delve into a lot of implementation details of this function. Additionally, there is a structure type called net.Dialer. This type is actually the underlying implementation of the two functions mentioned earlier, and it is worth studying in depth.
These are the main topics I discussed today, all about accessing network services. You can start from here and enter the world of network programming in Go language.
Thought-provoking question #
Today’s thought-provoking question is also related to timeout settings. After calling functions such as net.Dial, if successful, you will obtain a value of type net.Conn that represents a network connection. My question is: how to correctly set the timeout for read and write operations on a value of type net.Conn?
Click here to view detailed code for the Go language column article.