40 Interfaces and Tools in the io Package - Part 1 #
In the previous articles, we mainly discussed three data types: strings.Builder
, strings.Reader
, and bytes.Buffer
, which are all part of the io
package.
Review of Knowledge #
Do you remember? I asked you before, “What interfaces do they all implement?” Before we continue explaining the interfaces and tools in the io
package, let me answer this question first.
The strings.Builder
type is mainly used for string construction. The pointer type of this type implements the interfaces io.Writer
, io.ByteWriter
, and fmt.Stringer
. In addition, it also implements a package-level private interface in the io
package called io.stringWriter
(renamed to io.StringWriter
since Go 1.12).
The strings.Reader
type is mainly used for reading strings. The pointer type of this type implements several interfaces, including:
io.Reader
io.ReaderAt
io.ByteReader
io.RuneReader
io.Seeker
io.ByteScanner
io.RuneScanner
io.WriterTo
In total, there are 8 interfaces it implements, all of which are in the io
package.
Among them, io.ByteScanner
is an extension interface of io.ByteReader
, and io.RuneScanner
is an extension interface of io.RuneReader
.
The bytes.Buffer
is a data type that combines reading and writing functions, and it is very suitable as a buffer for byte sequences. The pointer type of this type implements even more interfaces.
More specifically, the interfaces related to reading implemented by this pointer type are:
io.Reader
io.ByteReader
io.RuneReader
io.ByteScanner
io.RuneScanner
io.WriterTo
There are 6 interfaces in total. As for the interfaces related to writing, it implements the following:
io.Writer
io.ByteWriter
io.stringWriter
io.ReaderFrom
There are 4 interfaces in total. In addition, it also implements the exported interface fmt.Stringer
.
Introduction: Benefits and advantages of interfaces in the io package #
So, what is the motivation (or purpose) behind implementing so many interfaces in these types?
In simple terms, this is done to enhance interoperability between different program entities. Let’s take some functions in the io
package as examples.
In the io
package, there are several functions used for copying data, which are:
io.Copy
;io.CopyBuffer
;io.CopyN
.
Although these functions have slight differences in their functionalities, they all accept two parameters as their first arguments: dst
, which represents the destination of data and is of type io.Writer
, and src
, which represents the source of data and is of type io.Reader
. The purpose of these functions is to copy data from src
to dst
.
Regardless of the type of the first argument we pass to these functions, as long as this type implements the io.Writer
interface, it can be used.
Similarly, regardless of the actual type of the second argument we pass to them, as long as this type implements the io.Reader
interface, it will work.
Once we meet these two conditions, these functions can almost be executed normally. Of course, the functions will also perform validity checks on the necessary parameter values, and if the checks fail, the execution cannot be successfully completed.
Let’s look at an example code:
src := strings.NewReader(
"CopyN copies n bytes (or until an error) from src to dst. " +
"It returns the number of bytes copied and " +
"the earliest error encountered while copying.")
dst := new(strings.Builder)
written, err := io.CopyN(dst, src, 58)
if err != nil {
fmt.Printf("error: %v\n", err)
} else {
fmt.Printf("Written(%d): %q\n", written, dst.String())
}
I first used strings.NewReader
to create a string reader and assigned it to the variable src
. Then, I new
ed a string builder and assigned it to the variable dst
.
Later, when calling the io.CopyN
function, I passed the values of these two variables as arguments, and set the third parameter value of this function to 58
. In other words, I want to copy the first 58
bytes from src
to dst
.
Although the types of variables src
and dst
are strings.Reader
and strings.Builder
, respectively, when they are passed to the io.CopyN
function, they are already wrapped as values of type io.Reader
and io.Writer
, respectively. The io.CopyN
function doesn’t care about their actual types.
For optimization purposes, the code in the io.CopyN
function will further wrap the parameter values and check if these parameter values implement other interfaces, or even investigate whether the actual type of a wrapped parameter value is a special type.
However, overall, this code is based on the interfaces declared in the parameter list. The author of the io.CopyN
function’s use of interface-oriented programming greatly expands its applicability and use cases.
Looking at it from another perspective, it is because both the strings.Reader
and strings.Builder
types implement multiple interfaces that their values can be used in a wider range of scenarios.
In other words, as a result, there are significantly more functions and data types in various libraries of the Go language that can operate on them.
This is what I want to tell you, the greatest benefit of implementing several interfaces in the strings
package and the bytes
package.
In other words, this is the biggest advantage brought about by programming with interfaces. The approach used by these data types and functions is something worth emulating in the process of programming.
As you can see, most of the types mentioned earlier implement interfaces from the io
package. In fact, the interfaces in the io
package play a very important role in the Go language’s standard library and many third-party libraries. They are very fundamental and important.
Take io.Reader
and io.Writer
, the two most core interfaces, for example. They are extension objects and sources of design for many other interfaces. Moreover, just looking at the Go language’s standard library, there are hundreds of data types that implement them individually, and there are over 400 references to them in code.
Many data types implement the io.Reader
interface because they provide the ability to read data from a source. Similarly, many data types that write data to a source also implement the io.Writer
interface.
In fact, the design intention of many types is to implement one or more of these two core interfaces or their extended interfaces in order to provide richer functionality than simply reading or writing byte sequences, just like the data types mentioned earlier in the strings
package and the bytes
package.
In Go, interface extension is achieved through interface embedding, which is commonly known as interface composition.
As I mentioned when talking about interfaces, Go encourages the use of small interfaces and interface composition to extend program behavior and increase program flexibility. The io
package can be used as a reference standard when applying this technique.
Now, let me ask a question related to interface extension and implementation based on the io.Reader
interface in the io
package. If you have studied this core interface and related data types, the answer to this question should not be difficult.
Our question today is: in the io
package, what are the extension interfaces and implementation types of io.Reader
? What are their respective functions?
The typical answer to this question is as follows. In the io
package, the extension interfaces of io.Reader
are as follows:
-
io.ReadWriter
: This interface is an extension of bothio.Reader
andio.Writer
interfaces. In other words, this interface defines a set of behaviors, which includes only the basic methods for reading byte sequences and writing byte sequences. -
io.ReadCloser
: In addition to the basic methods for reading byte sequences, this interface also has a basic method for closing, which is generally used to close the data input/output channel. This interface is a combination of theio.Reader
interface and theio.Closer
interface. -
io.ReadWriteCloser
: Obviously, this interface is a combination ofio.Reader
,io.Writer
, andio.Closer
interfaces. -
io.ReadSeeker
: The characteristic of this interface is that it has a basic method for seeking the read/write position, which can find a new read/write position based on the given offset. This new read/write position is used to indicate the starting index for the next read or write.Seek
is the only method in theio.Seeker
interface. -
io.ReadWriteSeeker
: Clearly, this interface is another combination of three interfaces:io.Reader
,io.Writer
, andio.Seeker
.
Now let’s talk about the implementation types of the io.Reader
interface in the io
package, which include the following:
-
*io.LimitedReader
: The basic type of this type wraps a value of theio.Reader
type and provides an additional feature of limited reading. The so-called limited reading means that the total amount of data returned by theRead
method of this type is limited, regardless of how many times the method is called. This limit is determined by theN
field of this type, measured in bytes. -
*io.SectionReader
: The basic type of this type can wrap a value of theio.ReaderAt
type and limits itsRead
method to read only a portion of the original data.The starting and ending positions of this data segment need to be specified when it is initialized and cannot be changed later. The behavior of this type is similar to that of a slice because it only exposes the data in its window.
-
*io.teeReader
: This type is a package-private type and is the actual type of the result value of theio.TeeReader
function. This function accepts two parameters,r
andw
, of typeio.Reader
andio.Writer
, respectively. ItsRead
method writes data fromr
tow
using the byte slicep
passed as a method parameter. It can be said that this value is the bridge betweenr
andw
, and the parameterp
is the data carrier on this bridge. -
*io.multiReader
: This type is also a package-private type. Similarly, there is a function calledMultiReader
in theio
package, which accepts multiple values of typeio.Reader
and returns a result value of typeio.multiReader
.When the
Read
method of this result value is called, it sequentially reads data from theio.Reader
value passed as parameters. Therefore, we can also call it a multi-object reader. -
*io.pipe
: This type is a package-private type and is more complex than the previously mentioned types. It not only implements theio.Reader
interface but also implements theio.Writer
interface.
Actually, the io.PipeReader
type and the io.PipeWriter
type have all the pointer methods they have based on it. These methods are just proxies for a certain method owned by the io.pipe
type value.
#
And because the io.Pipe
function returns a pointer value for these two types and uses them as the two ends of the synchronous in-memory pipe it generates, it can be said that the *io.pipe
type is the core implementation of the synchronous in-memory pipe provided by the io
package.
*io.PipeReader
: This type can be seen as a proxy type for theio.pipe
type. It delegates some of the functionality of the latter and implements theio.ReadCloser
interface based on the latter. At the same time, it defines the reading end of the synchronous in-memory pipe.
Note that I have omitted the implementation types in the test source code file here, as well as those implementation types that will not be directly exposed in any form.
Problem Analysis #
The purpose of my question is mainly to assess your familiarity with the io
package. This code package is the foundation of all I/O-related APIs in the Go language standard library, so we must have an understanding of each program entity in it.
However, because the package contains a lot of content, the question here is focused on the io.Reader
interface. Through the io.Reader
interface, we should be able to understand the type tree based on it and know the function of each type.
io.Reader
can be regarded as the core interface of the io
package and even the entire Go language standard library, so we can derive many extended interfaces and implementation types from it.
In the typical answer to this question, I have listed and introduced the relevant data types within the io
package for you.
Each of these types is worth understanding, especially those types that implement the io.Reader
interface. The functionality they implement can vary in detail.
In many cases, we can combine them to meet more complex reading requirements by wrapping the reading function (provided by the Read
method) applied to the original data in multiple layers (such as limited reading and multi-object reading, etc.).
In actual interviews, as long as the candidate can start from one aspect and state the extended interfaces of io.Reader
and their significance, or explain three to five implementation types of this interface, it can be considered as a basic correct answer.
For example, starting from the basic functions of reading, writing, and closing, describing:
io.ReadWriter
;io.ReadCloser
;io.ReadWriteCloser
;
These several interfaces.
Another example is to explain the similarities and differences between the io.LimitedReader
and io.SectionReader
types.
Yet another example is to elaborate on the specific way in which the *io.SectionReader
type implements the io.ReadSeeker
interface, and so on. However, this is only the minimum requirement, and the more comprehensive the candidate’s answer, the better.
I have written some code in the example file demo82.go
to demonstrate some basic usage of the mentioned types for your reference.
Summary #
Today, we have been discussing and organizing the program entities in the io
package, especially those important interfaces and their implementing types.
The interfaces in the io
package play a critical role for both the Go standard library and many third-party libraries. Among them, the most fundamental interfaces are io.Reader
and io.Writer
, which serve as extension objects or design origins for many other interfaces. We will continue discussing the interfaces in the io
package in the next section.
Which interfaces and tools in the io
package have you used? What have you gained and felt? You can leave me a message and we can discuss together. Thank you for listening, and see you next time.
Click here to view the detailed code corresponding to the Go Language column article.