09 Case Analysis- Application Scenarios of Pooling Objects #

In our everyday coding, we often need to store certain objects, primarily considering the cost of object creation. Examples of such objects include thread resources, database connection resources, or TCP connections. The initialization of these objects usually takes a long time, and if we frequently request and destroy them, it will consume a large amount of system resources, causing unnecessary performance loss.

Furthermore, these objects have a notable feature: they can be reused through lightweight resetting. In this case, we can use a virtual “pool” to store these resources. When needed, we can quickly retrieve one from the pool.

In Java, the pooling technique is widely used, with common examples being database connection pools and thread pools. In this lesson, we will focus on connection pooling, while thread pools will be covered in Lesson 12.

Commons Pool 2.0: a Common Pooling Package #

Let’s first take a look at the commonly used pooling package in Java, Commons Pool 2.0, to understand the general structure of an object pool. With this set of APIs, we can easily implement object pool management according to our business requirements.

The core class of an object pool is GenericObjectPool, which allows us to quickly create an object pool by providing a pool configuration and an object factory.

public GenericObjectPool(final PooledObjectFactory<T> factory, final GenericObjectPoolConfig<T> config)

Jedis, a popular Redis client, utilizes Commons Pool to manage connection pools, making it a best practice. The following code block shows the main code for creating objects using a factory in Jedis.

Now let’s introduce the process of object generation. As shown in the following diagram, when an object is acquired, it first attempts to retrieve one from the object pool. If there are no idle objects in the pool, a new one will be generated using the factory class.

So where are the objects stored? This responsibility lies with a structure called LinkedBlockingDeque, which is a bidirectional queue.

Next, let’s take a look at the main properties of GenericObjectPoolConfig:

private int maxTotal = DEFAULT_MAX_TOTAL;
private int maxIdle = DEFAULT_MAX_IDLE;
private int minIdle = DEFAULT_MIN_IDLE;
private boolean lifo = DEFAULT_LIFO;
private boolean fairness = DEFAULT_FAIRNESS;
private long maxWaitMillis = DEFAULT_MAX_WAIT_MILLIS;
private long minEvictableIdleTimeMillis = DEFAULT_MIN_EVICTABLE_IDLE_TIME_MILLIS;
private long evictorShutdownTimeoutMillis = DEFAULT_EVICTOR_SHUTDOWN_TIMEOUT_MILLIS;
private long softMinEvictableIdleTimeMillis = DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME_MILLIS;
private int numTestsPerEvictionRun = DEFAULT_NUM_TESTS_PER_EVICTION_RUN;
private EvictionPolicy<T> evictionPolicy = null; // Only 2.6.0 applications set this
private String evictionPolicyClassName = DEFAULT_EVICTION_POLICY_CLASS_NAME;
private boolean testOnCreate = DEFAULT_TEST_ON_CREATE;

private boolean testOnBorrow = DEFAULT_TEST_ON_BORROW;
private boolean testOnReturn = DEFAULT_TEST_ON_RETURN;
private boolean testWhileIdle = DEFAULT_TEST_WHILE_IDLE;
private long timeBetweenEvictionRunsMillis = DEFAULT_TIME_BETWEEN_EVICTION_RUNS_MILLIS;
private boolean blockWhenExhausted = DEFAULT_BLOCK_WHEN_EXHAUSTED;

There are many parameters. To understand the meaning of these parameters, let’s first take a look at the lifecycle of an object in the entire pool. As shown in the following figure, there are two main operations in the pool: the business thread and the monitoring thread.

When the object pool is initialized, three main parameters need to be specified:

• maxTotal: The maximum number of objects managed in the object pool
• maxIdle: The maximum number of idle objects
• minIdle: The minimum number of idle objects

Both maxTotal and the business thread are related. When the business thread wants to obtain an object, it first checks if there are any idle objects. If there are, it returns one; otherwise, it enters the creation logic. At this time, if the number of objects in the pool has reached the maximum value, the creation fails and an empty object is returned.

When an object is being acquired, there is a very important parameter, which is the maximum waiting time (maxWaitMillis). This parameter has a significant impact on the performance of the application. The default value of this parameter is -1, which means it will never time out until an idle object is available.

As shown in the figure below, if the object creation is slow or the usage is very busy, the business thread will continue to be blocked (blockWhenExhausted is set to true), which will cause normal services to fail to run.

Generally, interviewers will ask: how big should you set the timeout parameter?

I usually set the maximum waiting time to the maximum delay that the interface can tolerate. For example, if a normal service response time is around 10ms, experiencing a delay of 1 second will feel laggy. In this case, setting this parameter to 500~1000ms is acceptable. After the timeout, a NoSuchElementException will be thrown, and the request will fail quickly without affecting other business threads. This Fail Fast idea is widely used in internet applications.

Parameters with evcit are mainly used to handle object eviction. In addition to being expensive during initialization and destruction, pooled objects also consume system resources during runtime. For example, a connection pool occupies multiple connections, while a thread pool increases scheduling overhead, etc. When the business has a burst of traffic, it can request more objects than normal and put them in the pool. And when these objects are no longer used, we need to clean them up.

Objects that exceed the specified value of the minEvictableIdleTimeMillis parameter will be forcibly reclaimed. The default value is 30 minutes. The softMinEvictableIdleTimeMillis parameter is similar, but it only takes effect when the current number of objects is greater than minIdle, so the former is more forceful.

There are also 4 test parameters: testOnCreate, testOnBorrow, testOnReturn, testWhileIdle, which specify whether to perform validity checks on pooled objects during creation, acquisition, return, and idle detection.

Enabling these checks can ensure the validity of resources, but it consumes performance, so the default value is false. In production environments, it is recommended to only set testWhileIdle to true, and adjust the idle detection interval (timeBetweenEvictionRunsMillis), such as 1 minute, to ensure resource availability while maintaining efficiency.

Jedis JMH Benchmark #

How big is the performance difference between using a connection pool and not using a connection pool? The following is a simple JMH benchmark example (see repository), which performs a simple set operation to set a random value for a redis key.

@Fork(2)
@State(Scope.Benchmark)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 5, time = 1) 
@BenchmarkMode(Mode.Throughput) 
public class JedisPoolVSJedisBenchmark { 
    JedisPool pool = new JedisPool("localhost", 6379);  
    @Benchmark 
    public void testPool() { 
        Jedis jedis = pool.getResource(); 
        jedis.set("a", UUID.randomUUID().toString()); 
        jedis.close(); 
    }  
    @Benchmark 
    public void testJedis() { 
        Jedis jedis = new Jedis("localhost", 6379); 
        jedis.set("a", UUID.randomUUID().toString()); 
        jedis.close(); 
    } 
...

Use meta-chart to draw the test results, as shown in the following figure. It can be seen that using connection pooling has 5 times the throughput compared to not using connection pooling!

Database Connection Pool HikariCP #

HikariCP comes from the Japanese word “光” (meaning light) and is the default database connection pool in Spring Boot. Databases are components that we often use in our work, and there are many client connection pools designed specifically for databases. Their design principles are similar to what we mentioned at the beginning of this lesson, and they can effectively reduce resource consumption for creating and destroying database connections.

However, not all connection pools perform the same. The following figure is a benchmark test chart from HikariCP’s official documentation, which shows its excellent performance. The official JMH test code can be found on Github, and I have also copied a copy to the repository.

The typical interview question would be: Why is HikariCP fast? There are three main reasons:

• It uses FastList instead of ArrayList, and by initializing default values, it reduces the overhead of boundary checks.
• It optimizes and simplifies bytecode by using Javassist, reducing the performance loss of dynamic proxies. For example, it uses invokestatic instructions instead of invokevirtual instructions.
• It implements a lock-free ConcurrentBag to reduce lock contention in concurrent scenarios.

HikariCP’s optimizations for performance are very valuable for us to learn from. We will analyze several optimization scenarios in detail in the next 16 lessons.

Database connection pools also face the issue of maximum size (maximumPoolSize) and minimum size (minimumIdle). There is also a very common interview question: What is the optimal size for the connection pool?

Many students think that the bigger the connection pool size, the better. Some even set this value to more than 1000, which is a misunderstanding. Based on experience, only 20 to 50 database connections are needed. The specific size should be adjusted based on the specific business requirements, but setting it to an extremely large value is definitely not appropriate.

HikariCP does not recommend setting the minimumIdle value and it will be automatically set to the same size as maximumPoolSize. If there is a large amount of idle connection resources on your database server, you can also remove the dynamic adjustment functionality of the connection pool. In addition, according to the database query and transaction type, it is possible to configure multiple database connection pools in an application. This optimization technique is not well known, so let me briefly describe it.

There are usually two types of business scenarios: one requires quick response time to return data to the user as soon as possible, and the other can be executed in the background slowly with a longer processing time and lower requirement for timeliness. If these two types of business scenarios share the same database connection pool, resource contention is likely to occur, which in turn affects the response speed of the interface. Although microservices can solve this problem, most services do not have this condition. In such cases, the connection pool can be split.

As shown in the diagram, in the same business scenario, we have split the connection pool into two based on the attributes of the business to deal with this situation.

HikariCP also mentions another point that in the JDBC4 protocol, the connection’s validity can be detected through Connection.isValid(). This way, we don’t need to set a bunch of test parameters, and HikariCP does not provide such parameters.

Result Cache Pool #

At this point, you may find that the pool and cache have many similarities.

A common point between them is that they both store objects in a relatively high-speed area after processing. I tend to see cache as data objects and objects in the pool as executable objects. Data in the cache has a cache hit rate issue, while objects in the pool are generally equivalent.

Consider the following scenarios: JSP provides dynamic functionality for web pages, which can be compiled into class files after execution to speed up execution; or some media platforms may convert popular articles into static HTML pages at regular intervals, and can handle high concurrent requests solely through the load balancing of Nginx (dynamic and static separation).

In these cases, it is difficult to say whether this optimization is for the cache or for object pooling. In essence, they both store the result of a certain step of execution so that it doesn’t need to start from scratch the next time it is accessed. I usually refer to this technology as result cache pool, which is a synthesis of various optimization techniques.

Summary #

Let me briefly summarize the key points discussed in this section:

We started with the most commonly used pooling package in Java, Commons Pool 2.0, and introduced some implementation details and explained the application of some important parameters. Jedis is a wrapper based on Commons Pool 2.0. Through JMH testing, we found that object pooling can improve performance by nearly 5 times. Next, we introduced the fast speed database connection pool HikariCP. It further improves performance through coding techniques on top of pooling technology. HikariCP is one of the libraries I focus on, and I recommend you to add it to your task list as well.

Overall, when you encounter the following scenarios, you can consider using pooling to increase system performance:

• Creating or destroying objects requires a lot of system resources;
• Creating or destroying objects is time-consuming and requires complex operations and long waiting time;
• Objects can be reused through state reset after creation.

Pooling objects is just the first step of optimization. To achieve optimal performance, you need to adjust some key parameters of the pool. With a reasonable pool size and proper timeout, the pool can provide greater value. Similar to the cache hit rate, monitoring the pool is also very important.

In the following image, you can see that the number of database connections in the connection pool remains high for a long time without being released, while the number of waiting threads sharply increases. This can help us quickly locate database transaction problems.

Similar scenarios can often be encountered in everyday coding. For example, with HTTP connection pool, both OkHttp and HttpClient provide the concept of connection pooling. You can analyze and focus on connection size and timeout in a similar way. In the underlying middleware, such as RPC, connection pooling technology is usually used to accelerate resource acquisition, such as Dubbo connection pool, Feign switching to HttpClient implementation, and other technologies.

You will find that the design of pooling at different resource layers is similar. For example, thread pools provide a two-layer buffer for tasks through queues and provide various rejection strategies. We will discuss thread pools in lesson 12. You can also apply the features of thread pools to connection pooling technology to alleviate request overflow and create some overflow strategies. We often do this in reality. So how exactly can this be done? What are the practices? I leave this part for you to think about. Feel free to leave a comment below to share and discuss with everyone, and I will provide comments on your thoughts one by one.

Regardless of how objects are handled, the goal is to keep objects concise and improve their reusability. Therefore, in the next lesson, I will explain in detail the reuse of large objects and their considerations.