34 Testing Non-Functional Testing #
Hello, I am Ishikawa.
In the previous lesson, we learned about functional testing. Today, let’s take a look at performance, security, and accessibility testing in non-functional testing. For a web application, the complexity of performance testing is not lower than that of backend or end-to-end testing. The main reason for the high complexity of frontend performance testing is that there are many factors that affect web application performance, and many of them are not fully controllable by the application itself. So today, let’s focus on performance testing and explore the metrics, factors affecting performance, and ways to optimize performance.
Performance Testing #
Before any testing, we need to have a clear understanding of the objectives to be achieved, and performance testing is no exception. Therefore, first of all, let’s take a look at the performance indicators for web applications.
Performance Indicators #
Previously, we mentioned that when doing memory management and optimization in JS virtual machine development, we focus on the smoothness and responsiveness of the browser and the application. In fact, for JS developers, we can also refer to similar RAIL indicators. RAIL stands for Response, Animation, Idle, and Load. Now let’s take a closer look at what each of them represents.
- Regarding Response, the principle is that our application should process a response within 50ms and provide a visible response within 50-100ms.
- Regarding Animation, as mentioned before, when rendering, our frame budget is 60 frames per second, which means each frame should be rendered within 16.6ms.
- Regarding Idle, whether it’s in the loading mode or in the garbage collection lecture, we have mentioned that we should maximize the utilization of idle time. However, here’s a point to note: this utilization also needs to have a limit. If the tasks that need to be processed during this time exceed 50ms, it will affect our first point mentioned, which is the response within 100ms.
- Regarding Load, for page loading, our application should complete the initial loading within 5 seconds, and subsequent page loading within 2 seconds.
Factors Affecting Performance #
After discussing the testing indicators, let’s take a look at the factors that affect web performance. As mentioned earlier, the factors that affect performance are complex. From a broad perspective, they include but are not limited to external factors such as network environment, resource loading, and browser rendering, as well as internal factors such as memory management and garbage collection.
Faced with so many factors, if we rely entirely on manual testing, it would consume a lot of time and effort. Moreover, converting the results of these tests into the indicators mentioned above would be a cumbersome process. To solve this problem, most browsers provide developer tools to assist us in performing these tests. Taking Chrome as an example, we can use tools like Lighthouse and Performance Insights.
Performance Testing Tools #
First, let’s take a look at Lighthouse. Under the Lighthouse tab in the developer tools, we can select the option for Performance Audit. From the audit results, we can see the FCP, TTI, and other indicators that we mentioned before, regarding rendering and loading. At the same time, we can see some performance diagnosis suggestions, such as reducing excessive DOM nodes and using efficient caching strategies for static resources. Additionally, we can verify optimized items, such as compressed JS and CSS files.
However, the results mentioned above basically give us individual numbers, without a visual sense of the time distribution during the loading process. By using the Performance Insights tab, we can have a more intuitive display of the time-consuming process of page loading based on actual usage scenarios. This allows us to directly analyze, judge, and optimize some performance bottlenecks.
Troubleshooting Performance Bottlenecks #
Earlier, we were able to analyze external factors through Lighthouse and Performance Insights. However, for issues related to memory management and garbage collection, we have no way of knowing. So, while we can see the symptoms, we cannot analyze the underlying causes. In this case, we need more in-depth analysis tools for memory management. Chrome also provides corresponding analysis and troubleshooting tools in the Developer Tools.
First, we can enter the memory management window through More Tools -> Task Manager. Here, we can see the memory usage displayed in the Memory column. If we right-click and select JavaScript Memory, we can also load the memory usage information for JavaScript. What is the difference between the Memory and JavaScript Memory columns?
The difference here is that the Memory column contains the DOM elements generated by our page, while the JavaScript Memory column contains the number of objects generated by JavaScript and the number of currently active objects enclosed in parentheses. If the numbers here are too high, further investigation is required.
During the troubleshooting process, we can analyze from the dimensions of space and time.
First, from the perspective of time, we can perform performance analysis using the Performance tab in the developer tools. Here, we can check the memory option, start a recording, and perform garbage collection by clicking the trashcan icon at the beginning and end to avoid interference from outside the test period.
In the overview window of the results, the heap graph represents the JS heap, and below the overview are the counters. Here, you can see the detailed memory usage in subdivisions such as JS heap, nodes, and GPU memory. If, during the recording, we see that the number of JS heap or nodes at the end is higher than at the beginning, it means that there may be memory leaks during the process.
Additionally, through the Memory tab, we can analyze performance bottlenecks from the perspective of space. For browsers, DOM nodes are only garbage collected when they are no longer referenced in the DOM tree or JavaScript code. When a node is removed from the DOM tree but is still referenced by some JavaScript, it is called a “detached node”. Detached nodes are a common cause of memory leaks.
Heap snapshots are a method to identify detached nodes. As the name suggests, heap snapshots show us the memory distribution between JS objects and DOM nodes at the time the snapshot was taken. In the Memory tab, we can see the option for JS heap snapshots, which helps us understand the distribution of JS objects and related DOM nodes on the page.
After obtaining a snapshot, we can search for detached nodes in the object type filter using the keyword “detached”. This is where we may see red and yellow nodes. The nodes we need to pay attention to are the yellow nodes because they are directly referenced nodes that need to be garbage collected in a timely manner. When they are collected, the below indirectly referenced red nodes will also be deleted.
In addition, we can also trace the memory allocation of objects over time using the timeline of JavaScript memory allocation to identify memory leak points.
Security Testing #
After discussing performance testing, let’s take a look at security testing. In network security testing, it mainly includes penetration testing and vulnerability scanning.
Penetration testing can be done in a black box or white box manner. The testing personnel acting as intruders test the provided test environment and the developed application. In the case of black box testing, the intruder assumes no access to the code and tries to penetrate the system through attempts.
Vulnerability scanning is usually done in a white box manner. For example, many cloud service providers support code hosting, and on top of that, they generally have additional testing tools for vulnerability scanning. During this scanning process, the program responsible for automatic testing can access the code in the code repository and scan it to find vulnerabilities.
Whether it is penetration testing or vulnerability scanning, these tests are performed by specialized testing personnel or tools provided by cloud platforms in software engineering. Therefore, we will not go into detail about them in the context of development testing.
Now, let’s look at what preliminary tests we can perform from a development perspective. Previously, we also discussed security issues and solutions that may be encountered in front-end development. When we talked about HTTP, we also learned about HTTPS, which consists of Transport Layer Security (TLS) and Secure Sockets Layer (SSL). So, how can we test secure connections from the front-end?
Here, we can also use the developer tools provided by the browser. Taking Chrome as an example, the security certificate of the main source, the security of the connection, and the security of related resources that can be seen in the Security tab can help us conduct preliminary checks for security issues.
Accessibility Testing #
After talking about security, let’s take a look at accessibility testing.
Accessibility is an important feature on many Western websites. It allows people with disabilities to browse through the pages we develop. This includes people with color blindness, visual impairment, and other disabilities. They should be able to access our developed pages without any barriers. When developing accessibility features, we generally need to consider two issues: whether users can navigate the pages using a keyboard or screen reader, and whether the page elements are correctly marked for screen readers.
Here, let me provide some data. When we talk about visually impaired individuals, some may think that the proportion of this group is low. However, in reality, there are more than 17 million visually impaired people in China, with over 8 million of them completely blind. Another frequently overlooked group is color blind or color vision deficient individuals. There are over 60 million people with color vision deficiency and over 20 million people with color blindness in China. Added together, this astonishingly amounts to 80 million people. In other words, in China alone, there could be hundreds of millions of people who cannot access relevant content on our pages due to the lack of accessibility features.
Developing applications that are friendly to this group of people is not only an ethical responsibility for large companies, but also, from a utilitarian perspective, this group of people represents potential customers who could purchase our services or products. Furthermore, the higher the traffic a website receives, the larger this group becomes. If we do not pay attention to this group of users, it will not only affect our reputation, but also our business. Therefore, whether from the perspective of social development and progress or from the market point of view, we should strive to provide relevant features and testing for these users as much as possible.
The main challenge faced by color blind or color vision deficient individuals is the inability to distinguish colors with low contrast. So how can we select user-friendly interface colors for this group of users? We can refer to the relevant guidance provided by WCAG (Web Content Accessibility Guidelines), which is a set of accessibility guidelines. There are a total of three levels: Level A, which represents the basic requirements for disabled users to access and use web content; Level AA, which indicates that the content is overall accessible and eliminates significant barriers to accessing the content; and Level AAA, which provides further improvements and enhancements to web accessibility for certain disabled users.
Taking Chrome Developer Tools as an example again, in the CSS Overview Tab, we can see font and background colors that have contrast issues. For example, in the example below, when the grayish text has a similar color to the background, it does not meet the AA standard. However, the white text on a blue background meets the AA standard, although it falls slightly short of the AAA standard.
In addition to the CSS Overview Tab, in Chrome Developer Tools, we can also perform an audit on the website by selecting accessibility under the Lighthouse Tab. Through this audit, we can see the passed results, issues that require manual review, and test cases that are not applicable in this test.
Summary #
In this lecture, we have learned how to test non-functional aspects such as performance, security, and accessibility in development. Through the three lectures on testing, we have also realized the importance of testing in the development process. From a functional testing standpoint, we understand that testing is not just a task handed over to testers after development, but rather the starting point of “starting with the end in mind” driven by testing. From the perspective of non-functional testing, we can see that front-end applications developed with JavaScript have many easily overlooked issues in terms of performance, security, and accessibility, and these issues cannot be effectively addressed solely through manual testing. In this case, we need to use tools to help us improve testing efficiency.
Thought Questions #
Previously, we talked about using ArrayBuffer and SharedArrayBuffer in JavaScript multi-threaded development. So, do you know how to analyze, troubleshoot, and solve performance problems in multi-threaded development using tools provided by browsers or independent developer tools?
Feel free to share your experiences, exchange learning insights, or raise questions in the comments section. If you find it valuable, you are also welcome to share today’s content with more friends. See you in the next class!