Answer Compilation Plus Meal Compilation Thoughts Answer Collection #
Today, let’s continue analyzing the answers to the after-class questions in the “Additional” section of this course. These questions cover several important knowledge points related to Java 8 basic knowledge, application problem location, and analysis.
Now, let’s analyze them one by one.
Additional 1 | Get Familiar with Important Java 8 Knowledge Points in This Course (Part 1) #
Question: For the example of parallel consumption of numbers 1 to 100 in the parallel stream part, what do you think will happen if you replace forEach with forEachOrdered?
Answer: forEachOrdered will cause the parallelStream to lose part of its parallel processing ability, mainly because the iteration logic of forEach cannot be parallelized (it needs to be iterated in order and cannot be parallelized).
Let’s compare the following three approaches:
// Simulate that consuming a message takes 1 second
private static void consume(int i) {
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.print(i);
}
// Simulate that filtering data takes 1 second
private static boolean filter(int i) {
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
return i % 2 == 0;
}
@Test
public void test() {
System.setProperty("java.util.concurrent.ForkJoinPool.common.parallelism", String.valueOf(10));
StopWatch stopWatch = new StopWatch();
stopWatch.start("stream");
stream();
stopWatch.stop();
stopWatch.start("parallelStream");
parallelStream();
stopWatch.stop();
stopWatch.start("parallelStreamForEachOrdered");
parallelStreamForEachOrdered();
stopWatch.stop();
System.out.println(stopWatch.prettyPrint());
}
// Filter and forEach in serial
private void stream() {
IntStream.rangeClosed(1, 10)
.filter(ForEachOrderedTest::filter)
.forEach(ForEachOrderedTest::consume);
}
// Filter and forEach in parallel
private void parallelStream() {
IntStream.rangeClosed(1, 10).parallel()
.filter(ForEachOrderedTest::filter)
.forEach(ForEachOrderedTest::consume);
}
// Filter in parallel and forEach in serial
private void parallelStreamForEachOrdered() {
IntStream.rangeClosed(1, 10).parallel()
.filter(ForEachOrderedTest::filter)
.forEachOrdered(ForEachOrderedTest::consume);
}
The output is:
---------------------------------------------
ns % Task name
---------------------------------------------
15119607359 065% stream
2011398298 009% parallelStream
6033800802 026% parallelStreamForEachOrdered
From the output, we can see the following:
- In the
streammethod, both filtering and iteration are executed serially with a total time of 10 seconds + 5 seconds = 15 seconds. - In the
parallelStreammethod, both filtering and iteration are executed in parallel with a total time of 1 second + 1 second = 2 seconds. - In the
parallelStreamForEachOrderedmethod, filtering is executed in parallel and iteration is executed serially with a total time of 1 second + 5 seconds = 6 seconds.
Additional 2 | Get Familiar with Important Java 8 Knowledge Points in This Course (Part 2) #
Question 1: Using Stream, we can easily perform various operations on a List. Is there any way to observe the data changes throughout the entire process? For example, how can we observe the original data after performing filter+map operations?
Answer: There are two main ways to observe the data changes during various Stream operations on a List:
First, use the peek method. For example, in the following code, we performed two filters on numbers 1 to 10, finding numbers greater than 5 and finding even numbers. We used the peek method to save the original data before the two filtering operations:
List<Integer> firstPeek = new ArrayList<>();
List<Integer> secondPeek = new ArrayList<>();
List<Integer> result = IntStream.rangeClosed(1, 10)
.boxed()
.peek(i -> firstPeek.add(i))
.filter(i -> i > 5)
.peek(i -> secondPeek.add(i))
.filter(i -> i % 2 == 0)
.collect(Collectors.toList());
System.out.println("firstPeek: " + firstPeek);
System.out.println("secondPeek: " + secondPeek);
System.out.println("result: " + result);
The output shows that the numbers before the first filtering were 1 to 10, after the first filtering they became 6 to 10, and finally the output is 6, 8, and 10:
firstPeek: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
secondPeek: [6, 7, 8, 9, 10]
result: [6, 8, 10]
Second, use the Stream debugging feature in IDEs such as IntelliJ IDEA. See the following example, which is similar to the debugging feature in the image below:
Question 2: The Collectors class provides many ready-made collectors. Is there a way to implement a custom collector? For example, implement a MostPopularCollector to get the most frequently appearing elements in a List, satisfying the following two test cases:
assertThat(Stream.of(1, 1, 2, 2, 2, 3, 4, 5, 5).collect(new MostPopularCollector<>()).get(), is(2));
assertThat(Stream.of('a', 'b', 'c', 'c', 'c', 'd').collect(new MostPopularCollector<>()).get(), is('c'));
Answer: Let me explain my implementation ideas and methods: Use a HashMap to store the occurrence of each element, and finally, find the element with the highest occurrence in the map when collecting:
public class MostPopularCollector<T> implements Collector<T, Map<T, Integer>, Optional<T>> {
// Use a HashMap to store intermediate data
@Override
public Supplier<Map<T, Integer>> supplier() {
return HashMap::new;
}
// Accumulate the data by adding the value each time
@Override
public BiConsumer<Map<T, Integer>, T> accumulator() {
return (acc, elem) -> acc.merge(elem, 1, (old, value) -> old + value);
}
// Combine multiple maps by merging their values
@Override
public BinaryOperator<Map<T, Integer>> combiner() {
return (a, b) -> Stream.concat(a.entrySet().stream(), b.entrySet().stream())
.collect(Collectors.groupingBy(Map.Entry::getKey, summingInt(Map.Entry::getValue)));
}
// Find the key with the maximum value in the map
@Override
public Function<Map<T, Integer>, Optional<T>> finisher() {
return (acc) -> acc.entrySet().stream()
.reduce(BinaryOperator.maxBy(Map.Entry.comparingByValue()))
.map(Map.Entry::getKey);
}
@Override
public Set<Characteristics> characteristics() {
return Collections.emptySet();
}
}
Extra 3 | Locating application issues, troubleshooting skills are important #
Question: If you open an app and find that the homepage is blank, how do you determine whether it is a client compatibility issue or a server issue? If it is a server issue, how do you further narrow down the location? Do you have any analytical ideas?
Answer: First, we need to distinguish between client and server errors. We can start by checking if it is a server problem from the client’s perspective, which means checking the server’s response by capturing packets (usually, the client goes through testing before being released and cannot be changed at any time, so the server is more likely to have errors). Because a client program can correspond to hundreds of server interfaces, starting from the client (the root cause of the request) to troubleshoot the problem can find the direction more easily.
If the server does not return the correct output, then we need to continue troubleshooting the server interface or the upper-level load balancing, using the following methods:
Check the logs of the load balancer (such as Nginx).
Check the server logs.
Check the server monitoring.
If the server returns the correct output, then it may be due to a bug in the client or external configurations. Use the following methods to troubleshoot:
Check the client’s error reporting (usually, the client will connect to an exception service provided by SAAS).
Start the client locally for debugging.
Extra 4 | Analyzing and locating Java problems, make good use of these tools (Part 1) #
Question 1: There is also a jmap tool in the JDK, and we will use the jmap -dump command to generate a heap dump. So what is the difference between this command and jmap -dump:live? Can you design an experiment to prove their difference?
Answer: The jmap -dump command dumps all objects in the heap, while jmap -dump:live dumps only live objects in the heap. Because jmap -dump:live triggers a Full GC.
Please write a program to test:
```java
@SpringBootApplication
@Slf4j
public class JMapApplication implements CommandLineRunner {
//-Xmx512m -Xms512m
public static void main(String[] args) {
SpringApplication.run(JMapApplication.class, args);
}
@Override
public void run(String... args) throws Exception {
while (true) {
//Simulate generation of string, this string will lose reference and can be GC'd after each loop
String payload = IntStream.rangeClosed(1, 1000000)
.mapToObj(__ -> "a")
.collect(Collectors.joining("")) + UUID.randomUUID().toString();
log.debug(payload);
TimeUnit.MILLISECONDS.sleep(1);
}
}
}
Then, generate two dumps using jmap, one without live objects and one with live objects:
jmap -dump:format=b,file=nolive.hprof 57323
jmap -dump:live,format=b,file=live.hprof 5732
You can see that the dump without live objects, nolive.hprof, contains 164MB of char[] (which can be considered as strings):
The dump with live objects, live.hprof, only contains 1.3MB of char[], indicating that these strings in the program’s loop have been GC’d:
Question 2: Have you ever wondered how the client authenticates with MySQL? Can you use Wireshark to observe and analyze this process with reference to the MySQL documentation?
Answer: Generally, the authentication (handshake) process consists of three steps.
First, the server sends a handshake message to the client:
Wireshark has already parsed the message fields, and you can compare with the official documentation for the protocol format. The first byte of the HandshakeV10 message is the message version 0a, as indicated by the red box in the image. The preceding four bytes are the MySQL message header, with the first three bytes representing the message length (hexadecimal 4a = 74 bytes) and the last byte representing the message sequence number.
Then, the client responds with the HandshakeResponse41 message, containing the username and password for login:
As can be seen, the username is of type string[NUL], indicating that the string is terminated by 00. You can refer to this page for information on the field types in the MySQL protocol.
Finally, the server responds with an OK message, indicating successful handshake:
By analyzing the client and MySQL authentication process using Wireshark, you can easily understand the process. Without the use of Wireshark, you would have to analyze the content byte by byte according to the protocol documentation.
In fact, communication protocols defined by various CS systems are not complicated in themselves. You can continue following the approach I mentioned earlier, combining packet capture with documentation analysis, to analyze the MySQL query protocol.
Bonus 5 | Analyzing and Locating Java Issues: Make Good Use of These Tools (Part 2) #
Question: Arthas also has a powerful hot-fixing feature. For example, when encountering high CPU usage, we can identify that the admin user executes MD5 many times, consuming a large amount of CPU resources. At this point, can we directly apply hot-fixes on the server by decompiling the code with the jad command, modifying the code directly with a text editor (like Vim), using the sc command to find the classloader of the code, and using the redefine command to hot update the code? Can you try to use this process to directly fix the program (comment out the relevant code in the doTask method)?
Answer: The official documentation of Arthas provides detailed operation steps for implementing the jad->sc->redefine process. It is worth noting that:
-
The
redefinecommand conflicts with other commands likejad/watch/trace/monitor/tt. After executingredefine, if the above-mentioned commands are executed again, the bytecode ofredefinewill be reset. This is because the JDK’s redefine and retransform mechanisms are different, and using both mechanisms to update bytecode will only reflect the changes made by the last modification. -
redefinedoes not allow adding or deleting fields/methods, and methods that are already running will not take effect immediately. You need to wait for the next run for the changes to take effect.
These are the answers to the questions from the 5 bonus articles in our course. If there are any parts that you still feel unclear about or have any questions about the topics behind them, feel free to leave a comment and share today’s content with your friends or colleagues for further discussion.