40 Interfaces and Tools in the Io Package Part 1

40 Interfaces and Tools in the io Package - Part 1 #

In the previous articles, we mainly discussed three data types: strings.Builder, strings.Reader, and bytes.Buffer, which are all part of the io package.

Review of Knowledge #

Do you remember? I asked you before, “What interfaces do they all implement?” Before we continue explaining the interfaces and tools in the io package, let me answer this question first.

The strings.Builder type is mainly used for string construction. The pointer type of this type implements the interfaces io.Writer, io.ByteWriter, and fmt.Stringer. In addition, it also implements a package-level private interface in the io package called io.stringWriter (renamed to io.StringWriter since Go 1.12).

The strings.Reader type is mainly used for reading strings. The pointer type of this type implements several interfaces, including:

  1. io.Reader
  2. io.ReaderAt
  3. io.ByteReader
  4. io.RuneReader
  5. io.Seeker
  6. io.ByteScanner
  7. io.RuneScanner
  8. io.WriterTo

In total, there are 8 interfaces it implements, all of which are in the io package.

Among them, io.ByteScanner is an extension interface of io.ByteReader, and io.RuneScanner is an extension interface of io.RuneReader.

The bytes.Buffer is a data type that combines reading and writing functions, and it is very suitable as a buffer for byte sequences. The pointer type of this type implements even more interfaces.

More specifically, the interfaces related to reading implemented by this pointer type are:

  1. io.Reader
  2. io.ByteReader
  3. io.RuneReader
  4. io.ByteScanner
  5. io.RuneScanner
  6. io.WriterTo

There are 6 interfaces in total. As for the interfaces related to writing, it implements the following:

  1. io.Writer
  2. io.ByteWriter
  3. io.stringWriter
  4. io.ReaderFrom

There are 4 interfaces in total. In addition, it also implements the exported interface fmt.Stringer.

Introduction: Benefits and advantages of interfaces in the io package #

So, what is the motivation (or purpose) behind implementing so many interfaces in these types?

In simple terms, this is done to enhance interoperability between different program entities. Let’s take some functions in the io package as examples.

In the io package, there are several functions used for copying data, which are:

  • io.Copy;
  • io.CopyBuffer;
  • io.CopyN.

Although these functions have slight differences in their functionalities, they all accept two parameters as their first arguments: dst, which represents the destination of data and is of type io.Writer, and src, which represents the source of data and is of type io.Reader. The purpose of these functions is to copy data from src to dst.

Regardless of the type of the first argument we pass to these functions, as long as this type implements the io.Writer interface, it can be used.

Similarly, regardless of the actual type of the second argument we pass to them, as long as this type implements the io.Reader interface, it will work.

Once we meet these two conditions, these functions can almost be executed normally. Of course, the functions will also perform validity checks on the necessary parameter values, and if the checks fail, the execution cannot be successfully completed.

Let’s look at an example code:

src := strings.NewReader(
 "CopyN copies n bytes (or until an error) from src to dst. " +
  "It returns the number of bytes copied and " +
  "the earliest error encountered while copying.")
dst := new(strings.Builder)
written, err := io.CopyN(dst, src, 58)
if err != nil {
 fmt.Printf("error: %v\n", err)
} else {
 fmt.Printf("Written(%d): %q\n", written, dst.String())
}

I first used strings.NewReader to create a string reader and assigned it to the variable src. Then, I newed a string builder and assigned it to the variable dst.

Later, when calling the io.CopyN function, I passed the values of these two variables as arguments, and set the third parameter value of this function to 58. In other words, I want to copy the first 58 bytes from src to dst.

Although the types of variables src and dst are strings.Reader and strings.Builder, respectively, when they are passed to the io.CopyN function, they are already wrapped as values of type io.Reader and io.Writer, respectively. The io.CopyN function doesn’t care about their actual types.

For optimization purposes, the code in the io.CopyN function will further wrap the parameter values and check if these parameter values implement other interfaces, or even investigate whether the actual type of a wrapped parameter value is a special type.

However, overall, this code is based on the interfaces declared in the parameter list. The author of the io.CopyN function’s use of interface-oriented programming greatly expands its applicability and use cases.

Looking at it from another perspective, it is because both the strings.Reader and strings.Builder types implement multiple interfaces that their values can be used in a wider range of scenarios. In other words, as a result, there are significantly more functions and data types in various libraries of the Go language that can operate on them.

This is what I want to tell you, the greatest benefit of implementing several interfaces in the strings package and the bytes package.

In other words, this is the biggest advantage brought about by programming with interfaces. The approach used by these data types and functions is something worth emulating in the process of programming.

As you can see, most of the types mentioned earlier implement interfaces from the io package. In fact, the interfaces in the io package play a very important role in the Go language’s standard library and many third-party libraries. They are very fundamental and important.

Take io.Reader and io.Writer, the two most core interfaces, for example. They are extension objects and sources of design for many other interfaces. Moreover, just looking at the Go language’s standard library, there are hundreds of data types that implement them individually, and there are over 400 references to them in code.

Many data types implement the io.Reader interface because they provide the ability to read data from a source. Similarly, many data types that write data to a source also implement the io.Writer interface.

In fact, the design intention of many types is to implement one or more of these two core interfaces or their extended interfaces in order to provide richer functionality than simply reading or writing byte sequences, just like the data types mentioned earlier in the strings package and the bytes package.

In Go, interface extension is achieved through interface embedding, which is commonly known as interface composition.

As I mentioned when talking about interfaces, Go encourages the use of small interfaces and interface composition to extend program behavior and increase program flexibility. The io package can be used as a reference standard when applying this technique.

Now, let me ask a question related to interface extension and implementation based on the io.Reader interface in the io package. If you have studied this core interface and related data types, the answer to this question should not be difficult.

Our question today is: in the io package, what are the extension interfaces and implementation types of io.Reader? What are their respective functions?

The typical answer to this question is as follows. In the io package, the extension interfaces of io.Reader are as follows:

  1. io.ReadWriter: This interface is an extension of both io.Reader and io.Writer interfaces. In other words, this interface defines a set of behaviors, which includes only the basic methods for reading byte sequences and writing byte sequences.

  2. io.ReadCloser: In addition to the basic methods for reading byte sequences, this interface also has a basic method for closing, which is generally used to close the data input/output channel. This interface is a combination of the io.Reader interface and the io.Closer interface.

  3. io.ReadWriteCloser: Obviously, this interface is a combination of io.Reader, io.Writer, and io.Closer interfaces.

  4. io.ReadSeeker: The characteristic of this interface is that it has a basic method for seeking the read/write position, which can find a new read/write position based on the given offset. This new read/write position is used to indicate the starting index for the next read or write. Seek is the only method in the io.Seeker interface.

  5. io.ReadWriteSeeker: Clearly, this interface is another combination of three interfaces: io.Reader, io.Writer, and io.Seeker.

Now let’s talk about the implementation types of the io.Reader interface in the io package, which include the following:

  1. *io.LimitedReader: The basic type of this type wraps a value of the io.Reader type and provides an additional feature of limited reading. The so-called limited reading means that the total amount of data returned by the Read method of this type is limited, regardless of how many times the method is called. This limit is determined by the N field of this type, measured in bytes.

  2. *io.SectionReader: The basic type of this type can wrap a value of the io.ReaderAt type and limits its Read method to read only a portion of the original data.

    The starting and ending positions of this data segment need to be specified when it is initialized and cannot be changed later. The behavior of this type is similar to that of a slice because it only exposes the data in its window.

  3. *io.teeReader: This type is a package-private type and is the actual type of the result value of the io.TeeReader function. This function accepts two parameters, r and w, of type io.Reader and io.Writer, respectively. Its Read method writes data from r to w using the byte slice p passed as a method parameter. It can be said that this value is the bridge between r and w, and the parameter p is the data carrier on this bridge.

  4. *io.multiReader: This type is also a package-private type. Similarly, there is a function called MultiReader in the io package, which accepts multiple values of type io.Reader and returns a result value of type io.multiReader.

    When the Read method of this result value is called, it sequentially reads data from the io.Reader value passed as parameters. Therefore, we can also call it a multi-object reader.

  5. *io.pipe: This type is a package-private type and is more complex than the previously mentioned types. It not only implements the io.Reader interface but also implements the io.Writer interface.

Actually, the io.PipeReader type and the io.PipeWriter type have all the pointer methods they have based on it. These methods are just proxies for a certain method owned by the io.pipe type value. #

And because the io.Pipe function returns a pointer value for these two types and uses them as the two ends of the synchronous in-memory pipe it generates, it can be said that the *io.pipe type is the core implementation of the synchronous in-memory pipe provided by the io package.

  1. *io.PipeReader: This type can be seen as a proxy type for the io.pipe type. It delegates some of the functionality of the latter and implements the io.ReadCloser interface based on the latter. At the same time, it defines the reading end of the synchronous in-memory pipe.

Note that I have omitted the implementation types in the test source code file here, as well as those implementation types that will not be directly exposed in any form.

Problem Analysis #

The purpose of my question is mainly to assess your familiarity with the io package. This code package is the foundation of all I/O-related APIs in the Go language standard library, so we must have an understanding of each program entity in it.

However, because the package contains a lot of content, the question here is focused on the io.Reader interface. Through the io.Reader interface, we should be able to understand the type tree based on it and know the function of each type.

io.Reader can be regarded as the core interface of the io package and even the entire Go language standard library, so we can derive many extended interfaces and implementation types from it.

In the typical answer to this question, I have listed and introduced the relevant data types within the io package for you.

Each of these types is worth understanding, especially those types that implement the io.Reader interface. The functionality they implement can vary in detail.

In many cases, we can combine them to meet more complex reading requirements by wrapping the reading function (provided by the Read method) applied to the original data in multiple layers (such as limited reading and multi-object reading, etc.).

In actual interviews, as long as the candidate can start from one aspect and state the extended interfaces of io.Reader and their significance, or explain three to five implementation types of this interface, it can be considered as a basic correct answer.

For example, starting from the basic functions of reading, writing, and closing, describing:

  • io.ReadWriter;
  • io.ReadCloser;
  • io.ReadWriteCloser;

These several interfaces.

Another example is to explain the similarities and differences between the io.LimitedReader and io.SectionReader types.

Yet another example is to elaborate on the specific way in which the *io.SectionReader type implements the io.ReadSeeker interface, and so on. However, this is only the minimum requirement, and the more comprehensive the candidate’s answer, the better.

I have written some code in the example file demo82.go to demonstrate some basic usage of the mentioned types for your reference.

Summary #

Today, we have been discussing and organizing the program entities in the io package, especially those important interfaces and their implementing types.

The interfaces in the io package play a critical role for both the Go standard library and many third-party libraries. Among them, the most fundamental interfaces are io.Reader and io.Writer, which serve as extension objects or design origins for many other interfaces. We will continue discussing the interfaces in the io package in the next section.

Which interfaces and tools in the io package have you used? What have you gained and felt? You can leave me a message and we can discuss together. Thank you for listening, and see you next time.

Click here to view the detailed code corresponding to the Go Language column article.