Practice Sample to Run Hot Issue Q&A 4th Phase #

Hello, I am Sun Pengfei. Today we will review columns 7 and 8 of this section and see what issues to pay attention to when running the practice sample. In addition, I will address some doubts raised by students and discuss the impact of file order on I/O, as well as some methods and suggestions for learning Linux.

In column 7, the sample uses the systrace tool to obtain the trace of method execution by using bytecode processing framework for function instrumentation. This sample is quite complete and can also be used in your daily work.

Although this sample is very simple to use, its internal implementation is relatively complex. Its implementation involves Gradle Transform, Task implementation, incremental processing, ASM bytecode processing, use of mapping files, and the use of systrace tools.

For Gradle, we should be familiar with it as it is the build tool for the Android platform. For regular use, most of the time we only need to pay attention to some parameter configurations of the Android Gradle Plugin to achieve many functions. The official documentation provides very detailed parameter settings instructions. For operations that require intrusion into the packaging process, we need to implement our own Task or Transform code to complete tasks such as handling Class and JAR files, and processing resources.

The difficulty of learning Gradle comes more from the encapsulation and extension of Gradle by the Android Gradle Plugin, which Google has not provided complete documentation for, and there are some interface changes with each version. For learning this part of the content, I mainly read the Gradle tool code implemented by others and the Android Gradle Plugin code.

Regarding the implementation of this sample, let’s discuss some possible questions.

The Gradle plugin for this sample is published to the local Maven repository, so if you haven’t executed the publication and want to compile it directly, you need to publish the plugin to the local Maven before you can compile it successfully.

Another possible issue is that if you want to use the sample in other projects, you need to transplant the “e.systrace.TraceTag” class in the SampleApp to your own project, otherwise it will cause compilation errors.

Regarding bytecode processing, the sample mainly uses the ASM framework for processing. There are many bytecode processing frameworks on the market. Common ones include the ASM and Javassist frameworks. You can use the keywords “Java bytecode manipulation” to search for other frameworks on Google. Using bytecode processing frameworks requires a deep understanding of bytecode. I need to remind you that the bytecode here is not Dalvik bytecode but Java bytecode. For learning bytecode, you can refer to the official documentation and the “Java Virtual Machine Specification”. Both of them provide detailed descriptions of bytecode execution rules and instructions. You can also use the javap command to see the decompiled bytecode corresponding to the source code, which will have good results for learning. Bytecode processing is a very subtle operation. A slight mistake can lead to compilation errors, execution errors, or crashes. There are also many things to note, such as the impact of Try Catch Blocks on the operand stack, and the impact of inserted code on the local variable table and operand stack. Another way to implement AOP is to manipulate Dex files to inject Dalvik bytecode. You can use the dexer library or the AOP features provided in Facebook’s Redex to achieve this.

Now let’s take a look at the eighth article in this column. From the comments, I noticed that some students have questions about the impact of data reordering on I/O performance and are not clear about the optimization principles. Actually, the principle behind this optimization is quite simple. During the process of reading files, the system reads more file content than expected and caches it in the Page Cache. So, the next time a read request comes, some pages are directly read from the Page Cache instead of fetching data from the disk, which speeds up the reading process. There is a detailed description of this principle in the “Principle” section of the article “Analysis of Alipay App Optimization”, so I won’t repeat it here. If you are interested in “readahead,” I can provide you with some materials for further study.

The source code of the “readahead” mechanism can be found in the readahead.c file. It’s worth noting that the readahead mechanism may vary in different system versions, so most of the materials I provide below are based on the Linux 2.6.x kernel. For later versions, there may have been modifications to the readahead mechanism, so please take note.

For a detailed explanation of the readahead mechanism, you can refer to the following three documents: “Linux readahead: less tricks for more”, “Sequential File Prefetching In Linux”, and “Linux kernel’s file readahead”.

From the content of the previous articles in this column, many optimizations are based on Linux mechanisms. If you are not familiar with Linux mechanisms and optimizations, it’s not easy to come up with these solutions. For example, the small file system mentioned in the column runs in user mode, but it still relies on the functions provided by the existing file system at the lower level. Therefore, we need to deeply understand the implementation of Linux VFS, ext4, their advantages and disadvantages, and their principles. Only then can we discover why managing a large number of small files using the existing file system has performance limitations and how to address these performance shortcomings.

As an Android developer, how should we learn about Linux? I don’t recommend starting directly with reading books on system source code analysis. Instead, I suggest starting with understanding the concepts of operating systems. I recommend two books related to operating systems: “Computer Systems: A Programmer’s Perspective” and “Computer Systems: An Integrated Approach to Architecture and Operating Systems”. The implementation of Linux systems actually has some differences in detail compared to traditional operating system concepts. So I also recommend a book that analyzes the Linux operating system: “The Linux Programming Interface”. In this book, Linux mechanisms are analyzed in great detail using source code.

For students engaged in Android development, it is indeed necessary to have a deep understanding of Linux system-related knowledge, because many features in Android depend on the underlying foundation systems. For example, as I mentioned earlier, the “readahead” mechanism can not only be applied to resource loading in Android but also be extended to resource loading in Flutter. If we encounter a system that is not based on the Linux kernel in the future, such as Fuchsia OS, we can apply the knowledge we have already mastered to the existing operating system because memory management, file systems, signal mechanisms, process scheduling, system calls, interrupt mechanisms, drivers, and other content are all common. Having a global perspective of learning can help us get up to speed quickly when migrating to a new system. When it comes to operating system content, my learning path is to become familiar with the system mechanisms first and then get acquainted with various interfaces provided by the system in each direction, such as I/O operations, process creation, signal interrupt handling, thread usage, epoll, communication mechanisms, etc. One can follow the content of the book “Advanced Programming in the UNIX Environment” to gradually accomplish this step, and after becoming familiar with it, one can learn modules of personal interest at their own pace. At this point, you can find a book on source code analysis to further enhance your understanding. For example, if you want to understand the implementation of the fork mechanism, the scheduling and execution of read and write operations in kernel mode for I/O operations, etc., you need to dig deeper with purpose.

This learning path is the experience I have gained from my own learning process. I’m just a beginner in operating systems, so I welcome and encourage you to share your experiences and questions in the comments. Let’s learn and improve together.

Finally, I would like to give away three “Geek Calendars” to the top three students who left the most likes on the user story “Column Learning is Difficult? Maybe You Haven’t Found the Right Method”. Congratulations to @坚持远方, @蜗牛, and @JIA. Thank you all for participating.