16 Service Stress and System Slow Responses - How to Break Through Gateway Traffic Control #
Due to the differences in service granularity and varying requirements for data package wrapping, we introduced the BFF (Backend For Frontend) layer in the previous chapters. The calling party can directly call the BFF layer, which will then distribute the requests to different microservices for data assembly. Since many sub-services require user authentication, permission verification, and traffic control, do we really need to duplicate the logic of user authentication in each sub-service? This chapter will introduce you to the gateway, which handles these common requirements at the gateway layer.
Why Introduce a Gateway #
Without a gateway, service invocation faces several immediate problems:
- Each service requires independent authentication, leading to unnecessary redundancy.
- The client directly interfaces with the service, so any changes in the backend service require corresponding changes in the frontend, leading to a lack of independence.
- Exposing the backend services directly to the outside world poses a challenge to ensuring the security of the services.
- Some common operations, such as logging, need to be implemented in each sub-service, resulting in unnecessary duplicate work.
The calling structure of the existing system is shown in the following graph:
Calls are directly initiated by the frontend, and the calls between services can be coordinated by the service registry center. However, making calls from the frontend is not as simple, especially when the backend services appear as multiple instances. Each sub-service has its own service name, port number, etc., and with the repetition of common elements (such as authorization, logging, service control, etc.) in each sub-module, unnecessary costs are incurred. At this time, we urgently need a gateway to wrap all the sub-services, provide services uniformly to the outside, and handle common functionalities at the gateway layer, greatly improving the maintainability and robustness of the services.
With the introduction of a gateway, the structure of the request changes as shown in the following graph:
The changes are clearly visible: the gateway layer performs unified request routing, freeing the frontend from making choices; the backend services are hidden and only the gateway’s address is visible externally, greatly improving security; some shared operations are directly implemented at the gateway layer, relieving the specific service implementation from handling these operations and allowing it to focus more on business logic.
This article will guide you in introducing the spring-cloud-gateway component into your project. Some students may ask, why not use Zuul? The answer is that due to some reasons regarding component development, Zuul has entered the maintenance period. To ensure the integrity of the component, the Spring official team developed Gateway to replace Zuul for implementing gateway functionality.
Adding Gateway Service #
When introducing the jar, please note that Spring Cloud Gateway is based on Netty and WebFlux development, so there is no need for related Web Server dependencies such as Tomcat. WebFlux conflicts with spring-boot-starter-web, so it needs to be excluded; otherwise, it will not start.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
<version>0.2.2.RELEASE</version>
</dependency>
The startup class is no different from a normal business module. Preliminary configuration is done in the application.yml configuration file.
server:
port: 10091
management:
endpoints:
web:
exposure:
include: '*'
#nacos config
spring:
application:
name: gateway-service
cloud:
nacos:
discovery:
register-enabled: true
server-addr: 127.0.0.1:8848
# config:
# server-addr: 127.0.0.1:8848
gateway:
discovery:
locator:
enabled: false
lowerCaseServiceId: true
filters:
- StripPrefix=1
routes:
- id: member-service
uri: lb://member-service
predicates:
- Path= /member/**
filters:
- StripPrefix=1
Card Sub-service #
- id: card-service
uri: lb://card-service
predicates:
- Path=/card/** filters:
- StripPrefix=1
Resource Sub-service #
- id: resource-service
uri: lb://resource-service
predicates:
- Path=/resources/** filters:
- StripPrefix=1
Charging Sub-service #
- id: charging-service
uri: lb://charging-service
predicates:
- Path=/charging/** filters:
- StripPrefix=1
Finance Sub-service #
- id: finance-service
uri: lb://finance-service
predicates:
- Path=/finance/** filters:
- StripPrefix=1
The routes configuration item specifies the specific service routing rules, with each service configured as an array. id is used to differentiate between services, while uri corresponds to the direct call service. lb indicates accessing the service in a load-balanced manner, followed by the service name in Nacos. predicates are used to match incoming requests, eliminating the need to access services in the form of services.
With this, the simple routing function of the Gateway gateway service is complete. The frontend can directly access the gateway to call the corresponding services, without worrying about the service names or ports of the sub-services.
Implementing Circuit Breaker and Downgrade #
In the previous service invocation section, we implemented service degradation using Hystrix. Can we configure a unified configuration at the gateway layer? The answer is yes. Next, we will introduce Hystrix into the Gateway module to configure services. When the service times out or exceeds the specified configuration, it will quickly return a prepared exception method, failing fast to achieve service circuit breaking.
Add the following dependencies to the project:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>
Set the circuit breaker timeout in the configuration file:
# Timeout time configuration, default time is 1000ms
hystrix:
command:
default:
execution:
isolation:
thread:
timeoutInMilliseconds: 2000
Implement the fallback response class, which needs to be configured at the location where the sub-service call fails.
@RestController
@RequestMapping("error")
@Slf4j
public class FallbackController {
@RequestMapping("/fallback")
public CommonResult<String> fallback() {
CommonResult<String> errorResult = new CommonResult<>("Invoke failed.");
log.error("Invoke service failed...");
return errorResult;
}
}
Card Sub-service #
- id: card-service
uri: lb://card-service
predicates:
- Path=/card/** filters:
- StripPrefix=1
Configure quick circuit breaker fallback #
- name: Hystrix
args:
name: fallbackcmd
fallbackUri: forward:/error/fallback
If the service is temporarily unavailable but can return normally after retrying, you can ensure the availability of the service by setting the number of retries.
#card子服务
- id: card-service
uri: lb://card-service
predicates:
- Path=/card/**
filters:
- StripPrefix=1
- name: Hystrix
args:
name: fallbackcmd
fallbackUri: forward:/error/fallback
- name: Retry
args:
# Retry 3 times, including the first attempt, the correct execution should be 4 attempts
retries: 3
statuses:
- OK
methods:
- GET
- POST
# Exception configuration consistent with the exceptions thrown in the code
exceptions:
- com.mall.parking.common.exception.BusinessException
How to test? You can add exception throwing to the code to test whether the request is retried 3 times. When calling from the frontend, through the gateway to access this service call, you can find that the number of times it is called is 4.
/* The exception is thrown here to test whether the spring-cloud-gateway retry mechanism works correctly
* if (StringUtils.isEmpty("")) {
throw new BusinessException("test retry function");
}*/
### Implementing Service Rate Limits
Why do we need rate limiting? When the pressure of service calls suddenly increases, it can have a great impact on the system. It is necessary to take some rate limiting measures to ensure the availability of the system.
Common rate limiting algorithms include token bucket and leaky bucket. The Gateway component internally provides default Redis+Lua implementation for rate limiting, and you can specify rate limiting based on IP, user, or URI through customization. Let's take a look.
The RedisRateLimiter provided by Spring Cloud Gateway by default uses Redis and Lua to implement the core logic of determining if a token has been acquired. It accomplishes rate limiting based on the token bucket algorithm by calling the META-INF/scripts/request*rate*limiter.lua script. Let's see how to use this feature to achieve our goals.
Introduce support for reactive stream-based Redis:
<!--基于 reactive stream 的redis -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
Configure rate limiting based on IP. For example, when exchanging coupons in a mall, there are only a fixed number of mall coupons available in a fixed time period to cope with a sudden large number of requests. It is easy to encounter a peak trading case, resulting in the service freezing and becoming unavailable.
- name: RequestRateLimiter
args:
redis-rate-limiter.replenishRate: 3 # Allow users to process how many requests per second
redis-rate-limiter.burstCapacity: 5 # The capacity of the token bucket, which allows the maximum number of requests to be completed in one second
key-resolver: "#{@remoteAddrKeyResolver}" # SPEL expression to get the corresponding bean
The key resolver in the previous text is used to define the rate limiting rules. In this case, rate limiting is based on IP. Write a corresponding implementation class to implement this interface:
public class AddrKeyResolver implements KeyResolver {
@Override
public Mono<String> resolve(ServerWebExchange exchange) {
return Mono.just(exchange.getRequest().getRemoteAddress().getAddress().getHostAddress());
}
}
Define it as a @Bean in the startup class:
@Bean
public AddrKeyResolver addrKeyResolver() {
return new AddrKeyResolver();
}
With this, the configuration is complete. Next, let's verify if the configuration takes effect.
#### **Test if rate limiting takes effect**
Earlier, we used the Postman component for many interface tests. In fact, it can provide simulated concurrency testing capabilities (if you want to perform real concurrency testing, it is recommended to use the Apache JMeter tool). Many users have not yet discovered this feature. Let's use Postman together to launch simulated concurrency testing. The steps are as follows.
**1. Create the test script directory**
**2. Put the test requests into the directory**
**3. Run the script**
**4. Open a terminal, enter the Redis corresponding database, and enter the monitor command to monitor the execution of Redis commands.**
Click the "Run" button in the previous image to view the execution of Redis commands. Check the PostMan console to see that 3 requests have been ignored.
With this, rate limiting using the native rate limiting component can be used normally. Rate limiting based on IP is simple, but often there are more personalized requirements. At this time, customization is required to achieve advanced functionality.
### Implement Cross-Origin Resource Sharing (CORS) Support
The popular system deployment architecture these days is to deploy the front and backend separately, which gives rise to another issue: cross-origin requests. It is necessary to support cross-origin requests at the gateway level, otherwise requests cannot be routed to the correct processing node. Here are two ways to achieve this, one is through code implementation, and the other is through configuration file. It is recommended to use the configuration approach.
#### **Code Implementation**
@Configuration
public class CORSConfiguration {
@Bean
public CorsWebFilter corsWebFilter() {
CorsConfiguration config = new CorsConfiguration();
config.setAllowCredentials(Boolean.TRUE);
//config.addAllowedMethod("*");
config.addAllowedOrigin("*");
config.addAllowedHeader("*");
config.addExposedHeader("setToken");
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource(new PathPatternParser());
source.registerCorsConfiguration("/**", config);
return new CorsWebFilter(source);
}
}
#### **Configuration File Configuration**
spring:
cloud:
gateway:
discovery:
# Cross-origin resource sharing (CORS)
globalcors:
corsConfigurations:
'[/**]':
allowedHeaders: "*"
allowedOrigins: "*"
# To ensure the security of requests, only GET or POST requests are supported in the project, and all other requests are blocked to avoid unnecessary problems
allowedMethods:
- POST
With this, common issues in gateway such as route configuration, circuit breaker failure, request rate limiting, and cross-origin requests have been addressed to some extent. With more in-depth usage, there are still many advanced features waiting for you to develop and use. Here's a question for you: Besides Spring Cloud Gateway, do you know of any other middleware that can implement gateway functionality? You can do some research.