44 Using APIs in the os Package - Part 1 #

Today we will talk about the APIs in the os code package. This package allows us to have the ability to manipulate computer operating systems.

Introduction: API in the os package #

The APIs provided by this code package are all platform-independent. So, what does it mean to be platform-independent?

It means that these APIs are based on (or abstracted from) the operating system to provide high-level support for using the functionality of the operating system, but they do not depend on specific operating systems.

Whether it’s Linux, macOS, Windows, FreeBSD, OpenBSD, Plan9, the os package can provide a unified interface. This allows us to manipulate different operating systems in the same way and get similar results.

The APIs in the os package mainly help us use the file system, permission system, environment variables, system processes, and system signals in the operating system.

Among them, the APIs for manipulating the file system are the most extensive. With these APIs, we can not only create and delete files and directories, but also get various information about them, modify their contents, change their access permissions, and so on.

Speaking of which, we have to mention a very commonly used data type: os.File.

From the literal meaning, the os.File type represents a file in the operating system. But in reality, it can represent much more than that. Perhaps you already know that for Unix-like operating systems (including Linux, macOS, FreeBSD, etc.), everything can be seen as a file.

In addition to text files, binary files, compressed files, and directories, there are symbolic links, various physical devices (including built-in or external block-oriented or character-oriented devices), named pipes, and sockets (also known as sockets), and so on.

Therefore, it can be said that there are too many things we can manipulate using the os.File type. However, in order to focus on the os.File itself and make the content of this article more general, here we mainly apply the os.File type to regular files.

The following question starts with the most basic content represented by the os.File type. The question for today is: Which interfaces in the io package are implemented by the os.File type?

The typical answer to this question is as follows.

The os.File type has pointer methods, so it doesn’t implement any interfaces itself except for the empty interface. However, its pointer type implements many interfaces in the io package.

First of all, the *os.File type implements the three core simple interfaces in the io package: io.Reader, io.Writer, and io.Closer.

Secondly, this type also implements three other simple interfaces, namely: io.ReaderAt, io.Seeker, and io.WriterAt.

Because the *os.File type implements these simple interfaces, it also incidentally implements 7 out of 9 of the extended interfaces in the io package.

However, since it does not implement the simple interfaces io.ByteReader and io.RuneReader, it does not implement the corresponding extended interfaces io.ByteScanner and io.RuneScanner.

In short, the value of the os.File type and its pointer type can not only read and write the contents of a file in various ways but also seek and set the starting index position for the next read or write. Additionally, the file can be closed at any time.

However, they cannot specifically read the next byte or the next Unicode character in the file, nor can they perform any read-back operations.

However, the functionality of reading the next byte or character can also be achieved through other means, such as calling its Read method and passing appropriate parameter values.

Problem Analysis #

This question is indirectly asking “How can the os.File type operate on files?” In my previous typical response, I provided a brief answer.

Before going into further details, let’s first see how we can obtain a pointer value of type os.File (referred to as a File value).

In the os package, there are several functions available: Create, NewFile, Open, and OpenFile.

The os.Create function creates a new file based on the given path. It returns a File value and an error value. We can perform read or write operations on the corresponding file using the returned File value.

Moreover, files created using this function can be read and written by all users in the operating system.

In other words, once a file is created in this way, any user logged into the operating system can read its contents at any time or write content to it.

Note that if a file already exists at the path given to os.Create, the function will first clear the existing file’s content and then return it as the first result value.

Additionally, os.Create may return a non-nil error value.

For example, if a certain parent directory on the given path does not exist, the function will return an error value of type *os.PathError to indicate “file or directory does not exist”.

Now let’s look at the os.NewFile function. This function needs to be called with a uintptr value representing the file descriptor and a string value representing the file name.

If the given file descriptor is not valid, the function will return nil; otherwise, it will return a File value representing the corresponding file.

Note that the name of this function can be misleading—it does not create a new file, but rather creates a File value that wraps an already existing file based on its descriptor.

For example, we can obtain a File value that wraps the standard error output like this:

file3 := os.NewFile(uintptr(syscall.Stderr), "/dev/stderr")

And then, we can write some content to the standard error output using this File value:

if file3 != nil {
 defer file3.Close()
 file3.WriteString(
  "The Go language program writes the contents into stderr.\n")
}

The os.Open function opens a file and returns a File value that wraps the file. However, this function can only open files in read-only mode. In other words, we can only read content from the File value returned by this function, but cannot write to it.

If any write methods of this File value are called, an error value indicating a “bad file descriptor” will be returned. In fact, the read-only mode we mentioned earlier is applied to the file descriptor held by the File value.

A file descriptor is represented by a small non-negative integer. It is generally returned by I/O-related system calls and serves as an identifier for a file.

From the operating system’s perspective, this file descriptor is needed for any I/O operations on a file. However, some data types in Go hide this descriptor from us, so we don’t need to constantly focus on and distinguish it (like the os.File type).

In fact, when we call the os.Create function mentioned earlier, or the os.Open function, or the os.OpenFile function to be mentioned later, they all perform the same system call and obtain a file descriptor upon success. This file descriptor will be stored in the File value returned by them.

The os.File type has a pointer method called Fd. When called, it returns a uintptr value representing the file descriptor currently held by the File value.

However, except for the NewFile function in the os package, there is not much else to do with it if you are only working with regular files or directories.

Finally, let’s discuss the os.OpenFile function. This function provides underlying support for both the os.Create and os.Open functions and is the most flexible one.

This function has three parameters named name, flag, and perm. name represents the file path, and flag specifies the modes to apply to the file descriptor, including the read-only mode we mentioned earlier.

In Go, the read-only mode is represented by the os.O_RDONLY constant, which is of type int. Of course, besides the read-only mode, there are several other modes available, which we will discuss in detail later.

The perm parameter of the os.OpenFile function represents the permission mode, and its type is os.FileMode, which is a redefined type based on uint32.

To distinguish them, we can say that the mode indicated by flag is the operation mode, while the mode indicated by perm is the permission mode. The operation mode determines how the file is operated, while the permission mode controls file access permissions. We will discuss more details about the permission mode later.

- Several ways to obtain a pointer value of type os.File

To summarize, using a value of type os.File, we can not only read from, write to, and close a file, but also set the starting index position for the next read or write operation.

In addition to these functions mentioned, the os package also provides the Create function for creating a new file, the NewFile function for wrapping an existing file, and the Open and OpenFile functions for opening existing files.

Summary #

Today we talked about the os code package and its program entities. Firstly, we discussed the significance of the os package and its main uses. The APIs included in the code package are high-level abstractions of various operating system functions, allowing us to manipulate different operating systems in a unified way and achieve similar results.

In this code package, the API for manipulating the file system is the most comprehensive, with the most representative data type being os.File. The os.File type can not only represent files in the operating system, but also many other things. Especially in Unix-like operating systems, it can almost represent everything that can be manipulated, whether it’s software or hardware.

In the next article, I will continue to explain the content of the APIs in the os package. If you have any questions about this knowledge, feel free to leave me a message. Thank you for listening, and see you in the next issue.

Click here to view the detailed code associated with the Go language column articles.