Extra Machine Learning is Just That Many Things #

Hello, I’m Fang Yuan.

Through the previous learning, we know that PyTorch was developed as a tool for implementing machine learning or deep learning. Therefore, when learning PyTorch, it is inevitable to come across some relevant concepts and terms in machine learning.

After conducting research and launching the column, I have received a lot of feedback and messages, hoping that I can introduce some basic knowledge of machine learning in the column.

In today’s extra content, let’s take a look at what machine learning is, how it is classified, and what common terms there are. After supplementing these basic knowledge, I will also talk to you about the essence of model training, which you can consider as a midterm summary as the column is more than halfway through.

Alright, let’s officially start today’s learning.

Artificial Intelligence, Machine Learning, and Deep Learning #

When it comes to the terms Artificial Intelligence, Machine Learning, and Deep Learning, although we are familiar with them, the relationship between the three is always unclear.

In fact, the relationship between these three is one of inclusion. Artificial Intelligence encompasses Machine Learning, and Machine Learning in turn encompasses Deep Learning.

The concept of Artificial Intelligence has actually been around for a long time, but due to limitations in technological capabilities, it rarely entered people’s field of vision. When you search for the concept of Artificial Intelligence on the Internet, you may find that each search result is explained with lengthy text. In essence, the essence of Artificial Intelligence is that people want computers to think like humans to help solve repetitive and laborious tasks.

The applications of Artificial Intelligence mainly include the following:

• Expert Systems
• Natural Language Processing
• Computer Vision
• Speech Systems
• Robotics
• Others

Among them, Natural Language Processing, Computer Vision, and Speech Systems are currently hot fields. This can be seen from job postings, such as computer vision engineers, natural language processing engineers, etc. The problems in these fields can essentially be solved using traditional machine learning, but they are still limited by technological capabilities and have been in a bottleneck.

In the past decade, with the development of Deep Learning, people have made significant progress in research in these three fields. More and more Artificial Intelligence products have been implemented, making our lives more convenient and efficient. PyTorch, which we talk about in this column, is also active in these fields.

Machine Learning (Deep Learning) #

Deep Learning originated from artificial neural networks in machine learning, so in terms of mechanisms, machine learning and deep learning are exactly the same. Next, let’s take a look at the classification and workflow of machine learning and deep learning (referred to as machine learning in the following text).

As mentioned earlier, the goal of machine learning is to enable machines to think like humans. So let’s think about how humans solve problems based on what? Obviously, when we think about a problem, we usually make judgments based on past experiences.

Similar to the process of humans summarizing and generalizing experiences, the main goal of machine learning is to transform the process of humans summarizing experiences into a process of computer summarization and generalization.

Among them, the historical experiences accumulated by humans are reflected in machine learning as a large amount of data.

For example, in the process of image classification, we provide the computer with a large number of images. Summarizing and generalizing this process is the training process of machine learning, which is the process of the computer processing images and “learning” their underlying features. The ultimate “rules” are reflected in the models in machine learning, which are the ultimate output of our machine learning tasks.

Therefore, machine learning is a method that uses data to train models and then uses models for prediction.

Supervised Learning vs Unsupervised Learning #

Just now we talked about training models in machine learning. The models in machine learning can basically be divided into two categories: supervised learning and unsupervised learning. Of course, based on these two major classifications, there are many small sub-classifications, but we won’t discuss them for now.

Let’s first clarify what supervised learning and unsupervised learning are.

The most obvious difference between supervised learning and unsupervised learning lies in whether real labels are used during training. To help you understand quickly, let me explain using an example of face recognition.

First, let’s look at supervised learning. Suppose we want to train a face recognition model to automatically recognize whether a face is person A or person B. In the training process, the model needs to be shown a large number of photos labeled as A and photos labeled as B, so that the model can learn who is A and who is B. Only after this training, when we make predictions, the model can correctly determine whether the face in the image is A or B.

Now let’s talk about unsupervised learning. Our phone’s photo album has a function that automatically gathers photos of a particular person, but the phone actually doesn’t know who the photos are. What is the training principle behind this? During training, a bunch of photos are given to the model, but the model doesn’t know the true labels of these photos. Instead, the model explores the latent features in these photos on its own.

Most of the deep learning applications we can experience belong to supervised learning, such as face recognition, image classification, gesture recognition, portrait segmentation, sentiment analysis, etc. The recently popular GAN belongs to unsupervised learning.

Explanation of Common Nouns #

In this column, we have encountered many professional terms, and here we will summarize and explain what they mean. In order to avoid making this part feel like a textbook, I have decided to use an example to connect these nouns.

As we mentioned at the beginning, the essence of machine learning is to make machines think like humans . Therefore, I will use the process of learning this column to explain some of the terms in machine learning.

Training Set and Validation Set #

The data used during training is called the training set. The data used to evaluate the model is called the evaluation set, validation set, or test set.

Through the learning of this column, you will master the knowledge of PyTorch from scratch. At the end of the column, we have also set a final test to help you measure your learning achievements.

So, the content of this column is equivalent to the training set , and the test questions are the validation set (or test set). The training set is used to train the model, while the validation set is used to evaluate the model.

During model training, it is important to note that the training set and validation set must come from different data of the same problem. Just like learning PyTorch in this column, but having Python test questions later, it obviously cannot reflect your real learning achievements.

Epoch and Step #

Training the entire dataset once is called an epoch , which means going through the learning of this column once is called an epoch.

However, due to hardware limitations, all data is not loaded at once during training. Instead, a part of the data is loaded and trained at a time. It’s similar to updating a part of the content every Monday, Wednesday, and Friday, and you correspondingly learn a part of the content. The concept corresponding to this “each time” is called step . The amount of data loaded each time is called batch_size .

The Essence of Model Training #

Just now we clarified a lot of key terms in machine learning through an example. In fact, up until now in this column, we have covered most of the content related to model training using PyTorch. Congratulations on sticking with it this far.

When I first started learning about machine learning, I was always overwhelmed by its algorithms. There was a stage where I was constantly confused and didn’t know how to use some algorithms even if I understood them.

Fortunately, after persistently studying for a while, I gradually discovered that machine learning is actually just a few things, which can be considered as the essence behind all the variations. Let’s now review several important aspects of machine learning and even deep learning development together.

First, let’s take a look at the steps involved in machine learning development. I have mentioned them before in this column, so you can review them if you don’t remember.

1. Data processing: This mainly includes data cleaning, data preprocessing, and data augmentation. In short, it is about constructing the training and validation sets for the model to use.
2. Model training: Determining the network structure, selecting the loss function, and setting the optimization method.
3. Model evaluation: Using various evaluation metrics to assess the performance of the model.

Now you can think about it, almost no project development can be separate from these three steps. Whether it’s a deep model in deep learning or a shallow model in machine learning, they all essentially rely on these three steps.

Next, let’s focus on the model training part. Despite the countless variations among different models, they still cannot do without the following steps:

1. Model structure design: For example, regression algorithms in machine learning, various network structures such as VGG, ResNet, SENet in deep learning, or your own custom model structure.
2. Defining the loss function: The loss function measures the gap between the current model’s predictions and the true labels.
3. Selecting the optimization method: Paired with the loss function, it updates the parameters of the model.

Now think about it again, isn’t it true that almost all model training can’t do without these three steps? In fact, these six points I mentioned above all come from the content we discussed earlier. Learning the previous content is like learning how to manufacture car components. Assembling these components is equivalent to completing the entire production of a car, and this step is what we will continue to study later.

Among these points, the model structure is the most variable. There is no shortcut other than reading papers, looking at related blogs, and expanding our knowledge. Moreover, we should not underestimate the significance of the loss function. Different loss functions have different emphases. If your model training is bottlenecked and difficult to improve, or if it cannot solve existing problems, consider adjusting the loss function.

In my view, the essence of model training is to determine the network structure, set the loss function, and select the optimization method. Next, we will learn how to assemble the various aspects we have learned so far to complete a complete model training.

Feel free to engage in discussion with me in the comments section. Let’s keep up the good work together.