22 Network Programming - How to Implement Cross-Platform Services through RPC in Go #
In the previous lesson, I explained the specifications and implementation of RESTful APIs and left you with two assignments, which are to delete and update a user. Now I will explain these two assignments to you.
Deleting a user is relatively simple. The API format is the same as getting a user, but the HTTP method is changed to DELETE. The sample code to delete a user is as follows:
ch21/main.go
func main() {
// omitted unchanged code
r.DELETE("/users/:id", deleteUser)
}
func deleteUser(c *gin.Context) {
id := c.Param("id")
i := -1
// SQL query similar to a database
for index, u := range users {
if strings.EqualFold(id, strconv.Itoa(u.ID)) {
i = index
break
}
}
if i >= 0 {
users = append(users[:i], users[i+1:]...)
c.JSON(http.StatusNoContent, "")
} else {
c.JSON(http.StatusNotFound, gin.H{
"message": "User not found",
})
}
}
The logic of this example is to register the DELETE method to achieve the purpose of deleting a user. The logic of deleting a user is to query by ID:
- If the user to be deleted can be found, record the index and exit the loop, then delete the user based on the index.
- If the user to be deleted cannot be found, return 404.
After implementing the logic to delete a user, I believe you already know how to update a user’s name because it is very similar to deleting a user. The implementation code is as follows:
func main() {
// omitted unchanged code
r.PATCH("/users/:id", updateUserName)
}
func updateUserName(c *gin.Context) {
id := c.Param("id")
i := -1
// SQL query similar to a database
for index, u := range users {
if strings.EqualFold(id, strconv.Itoa(u.ID)) {
i = index
break
}
}
if i >= 0 {
users[i].Name = c.DefaultPostForm("name", users[i].Name)
c.JSON(http.StatusOK, users[i])
} else {
c.JSON(http.StatusNotFound, gin.H{
"message": "User not found",
})
}
}
The overall code logic is almost the same as deleting, except that the PATCH method is used here.
What is RPC? #
RPC, also known as Remote Procedure Call, is a way to communicate between different nodes in a distributed system (interprocess communication) and belongs to the C/S (Client/Server) model. RPC is initiated by the client to call methods on the server for communication, and then the server returns the result to the client.
RPC has two core components: communication protocol and serialization. Before HTTP2, custom TCP protocols were generally used for communication. Since the advent of HTTP2, it has also been adopted, for example, gRPC, which is popular.
Serialization and deserialization are the ways to encode and decode transmitted content. Common encoding and decoding methods include JSON, Protobuf, etc.
In most RPC architectures, there are four components: Client, Client Stub, Server, and Server Stub, and communication between the Client and Server is done via sockets. The architecture of RPC is shown in the following figure:
(Image from Google search)
I will summarize the process of RPC invocation for you:
- The client calls the client stub while passing the parameters to the client stub.
- The client stub packs and encodes the parameters, and sends them to the server through a system call.
- The client system sends the information to the server system.
- The server system sends the information to the server stub.
- The server stub decodes the information.
- The server stub calls the actual server program (Server).
- After processing, the server returns the result to the client in the same way.
RPC calls are commonly used in large projects, which are what we now commonly refer to as microservices. It also includes features such as service registration, governance, and monitoring, making it a complete system.
Getting Started with Go RPC #
Since RPC is so popular, Go naturally wouldn’t miss out. In the Go SDK, the net/rpc package has been built-in to help developers implement RPC. In simple terms, the net/rpc package provides the ability to access server object methods over the network.
Now I will demonstrate the use of RPC through an addition operation. The server-side code is as follows: ch22/server/math_service.go
package server
type MathService struct {
}
type Args struct {
A, B int
}
func (m *MathService) Add(args Args, reply *int) error {
*reply = args.A + args.B
return nil
}
In the above code:
MathServiceis defined to represent a remote service object.Argsis a structure used to represent parameters.- The
Addmethod implements the addition functionality, and the result of the addition is returned through thereplypointer variable.
With this defined service object, it can be registered in the exposed service list for use by other clients. In Go language, registering a RPC service object is relatively simple with the RegisterName method, as shown in the following example code:
ch22/server_main.go
package main
import (
"gotour/ch22/server"
"log"
"net"
"net/rpc"
)
func main() {
rpc.RegisterName("MathService", new(server.MathService))
l, e := net.Listen("tcp", ":1234")
if e != nil {
log.Fatal("listen error:", e)
}
rpc.Accept(l)
}
In the above example code, a service object is registered using the RegisterName function, which takes two parameters:
- The service name (“MathService”).
- The specific service object, which is the
MathServicestructure I just defined.
Then, a TCP connection is established using the net.Listen function to listen on port 1234, and finally the rpc.Accept function provides the MathService RPC service on that TCP connection. Now clients can see the MathService service and its Add method.
Every framework has its own rules, and the net/rpc framework provided by Go language is no exception. In order to register an object as an RPC service that can be accessed remotely by clients, the methods of that object (type) must satisfy the following conditions:
- The method’s type must be exported (public).
- The method itself must also be exported.
- The method must have 2 parameters, and the parameter types must be exported or built-in.
- The method must return an error type.
In summary, the format of the method is as follows:
func (t *T) MethodName(argType T1, replyType *T2) error
Both T1 and T2 can be serialized by the encoding/gob package.
- The first parameter,
argType, is provided by the caller (client). - The second parameter,
replyType, is the result returned to the caller and must be a pointer type.
Now that we have a provided RPC service, let’s take a look at how the client calls it. The code for the client is as follows:
ch22/client_main.go
package main
import (
"fmt"
"gotour/ch22/server"
"log"
"net/rpc"
)
func main() {
client, err := rpc.Dial("tcp", "localhost:1234")
if err != nil {
log.Fatal("dialing:", err)
}
args := server.Args{A: 7, B: 8}
var reply int
err = client.Call("MathService.Add", args, &reply)
if err != nil {
log.Fatal("MathService.Add error:", err)
}
fmt.Printf("MathService.Add: %d+%d=%d", args.A, args.B, reply)
}
In the above example code, first a TCP connection is established using the rpc.Dial function. It is important to note that the IP and port here must match the RPC service provided to ensure that an RPC connection can be established.
Once the TCP connection is established, the parameters needed for the remote method must be prepared, which in this example are args and reply. After the parameters are ready, the remote RPC service can be called using the Call method. The Call method has 3 parameters, which serve the following purposes:
- The name of the remote method being called, which is MathService.Add in this case. The part before the dot is the name of the registered service, and the part after the dot is the method of that service.
- The arguments provided by the client to invoke the remote method, in this example, it is args.
- In order to receive the result of the remote method, it must be a pointer, like &reply in the example, so that the client can get the returned result from the server.
Both the server and client code are already written, and now you can run them and test the effectiveness of the RPC calls.
First, run the server code to provide the RPC service. Use the following command:
➜ go run ch22/server_main.go
Then run the client code to test the RPC call. Use the following command:
➜ go run ch22/client_main.go
If you see the result MathService.Add: 7+8=15, then congratulations, you have completed a complete RPC call.
HTTP-based RPC #
In addition to using the TCP protocol for RPC calls, RPC calls can also be made using the HTTP protocol, which is supported by the built-in net/rpc package. Now I will modify the above example code to support HTTP-based calls. The server code is as follows:
ch22/server_main.go
func main() {
rpc.RegisterName("MathService", new(server.MathService))
rpc.HandleHTTP() // added
l, e := net.Listen("tcp", ":1234")
if e != nil {
log.Fatal("listen error:", e)
}
http.Serve(l, nil) // changed to http service
}
The above code shows the modifications made to the server code, which are only two places. I have marked them in the code, making it easy to understand.
There are not many changes to the server code, and even fewer changes to the client code. You only need to modify one place. The modified part of the code is as follows:
ch22/client_main.go
func main() {
client, err := rpc.DialHTTP("tcp", "localhost:1234")
// omitted other unchanged code
}
As you can see from the above code, you only need to change the connection method from Dial to DialHTTP.
Now run the server and client code separately, and you will see the output results, just like when using TCP connections.
In addition, the net/rpc package in Go provides an additional URL for debugging the HTTP-based RPC. After running the server code, enter http://localhost:1234/debug/rpc in the browser and press Enter to see the registered RPC services on the server, as well as the methods of each service, as shown in the following figure:
As shown in the above figure, you can see the registered RPC services, the method signatures, and the number of times each method has been called.
JSON RPC Cross-platform Communication #
The key to implementing cross-language RPC services is to choose a universal encoding that is supported by most languages, such as JSON. In Go, it is very easy to implement a JSON RPC service, just use the net/rpc/jsonrpc package.
Using the above example as an example again, I have modified it to support JSON-based RPC services. The server code is as follows:
ch22/server_main.go
func main() {
rpc.RegisterName("MathService", new(server.MathService))
l, e := net.Listen("tcp", ":1234")
if e != nil {
log.Fatal("listen error:", e)
}
for {
conn, err := l.Accept()
if err != nil {
log.Println("jsonrpc.Serve: accept:", err.Error())
return
}
// JSON RPC
go jsonrpc.ServeConn(conn)
}
}
As you can see from the above code, compared to the gob-encoded RPC service, the JSON RPC service hands over the connection to the jsonrpc.ServeConn function to achieve RPC calls based on JSON.
The client code for JSON RPC is also very small, you only need to modify one place, and the modified part is as follows:
ch22/client_main.go
func main() {
client, err := jsonrpc.Dial("tcp", "localhost:1234")
// omitted other unchanged code
}
As you can see from the above code, you only need to replace the dial method with the one provided by the jsonrpc package.
The above is an example of using Go as a client to call RPC services. Other programming languages are similar, just follow the JSON-RPC specification.
JSON RPC based on HTTP #
Compared to using TCP for RPC calls, using HTTP is definitely more convenient and universal. The built-in jsonrpc in Go does not implement transmission over HTTP, so you need to implement it yourself. Here, I refer to the implementation of gob-encoded HTTP RPC to implement JSON RPC services based on HTTP.
Still using the above example, I modified it to support the HTTP protocol. The RPC server code is as follows:
ch22/server_main.go
func main() {
rpc.RegisterName("MathService", new(server.MathService))
// Register a path for providing JSON-RPC services based on HTTP
http.HandleFunc(rpc.DefaultRPCPath, func(rw http.ResponseWriter, r *http.Request) {
conn, _, err := rw.(http.Hijacker).Hijack()
if err != nil {
log.Print("rpc hijacking ", r.RemoteAddr, ": ", err.Error())
return
}
var connected = "200 Connected to JSON RPC"
io.WriteString(conn, "HTTP/1.0 "+connected+"\n\n")
jsonrpc.ServeConn(conn)
})
l, e := net.Listen("tcp", ":1234")
if e != nil {
log.Fatal("listen error:", e)
}
http.Serve(l, nil)
}
The implementation of the above code is based on the core of the HTTP protocol, which uses http.HandleFunc to register a path to provide JSON-RPC services based on HTTP. In this HTTP service implementation, the connection is hijacked using the Hijack method and then handed over to jsonrpc for processing, thus implementing JSON-RPC services based on the HTTP protocol.
After implementing the server-side code, we will now implement client-side calls. The code for the client is as follows:
func main() {
client, err := DialHTTP("tcp", "localhost:1234")
if err != nil {
log.Fatal("dialing:", err)
}
args := server.Args{A:7,B:8}
var reply int
err = client.Call("MathService.Add", args, &reply)
if err != nil {
log.Fatal("MathService.Add error:", err)
}
fmt.Printf("MathService.Add: %d+%d=%d", args.A, args.B, reply)
}
// DialHTTP connects to an HTTP RPC server at the specified network address
// listening on the default HTTP RPC path.
func DialHTTP(network, address string) (*rpc.Client, error) {
return DialHTTPPath(network, address, rpc.DefaultRPCPath)
}
// DialHTTPPath connects to an HTTP RPC server
// at the specified network address and path.
func DialHTTPPath(network, address, path string) (*rpc.Client, error) {
var err error
conn, err := net.Dial(network, address)
if err != nil {
return nil, err
}
io.WriteString(conn, "GET "+path+" HTTP/1.0\n\n")
// Require successful HTTP response
// before switching to RPC protocol.
resp, err := http.ReadResponse(bufio.NewReader(conn), &http.Request{Method: "GET"})
connected := "200 Connected to JSON RPC"
if err == nil && resp.Status == connected {
return jsonrpc.NewClient(conn), nil
}
if err == nil {
err = errors.New("unexpected HTTP response: " + resp.Status)
}
conn.Close()
return nil, &net.OpError{
Op: "dial-http",
Net: network + " " + address,
Addr: nil,
Err: err,
}
}
The core of the above code is to send an HTTP request to invoke the remote HTTP JSON-RPC service using the established TCP connection, and the HTTP GET method is used here.
By running the server and client separately, you can see the correct result of the HTTP JSON-RPC call.
Summary
In this lesson, we have explained the implementation and invocation of RPC services based on Go's built-in RPC framework. Through this lesson, I believe you have a good understanding of what RPC services are, the differences between RPC services based on TCP and HTTP, and how they are implemented, etc.
However, in actual project development, the built-in RPC framework provided by Go is not used very often. But I still used the built-in framework as an example to explain so that you can better understand the use and implementation principles of RPC. If you can master them well, you can also quickly get started with third-party RPC frameworks.
In actual projects, the more commonly used framework is gRPC developed by Google. It is serialized through Protobuf, uses the HTTP/2 protocol for binary transmission, and supports many programming languages, with high efficiency. You can refer to the official documentation for additional information on using gRPC, and getting started is quite easy.