15 Addressing Common Issues in Services with Complex Handling Methods via API Gateway #
Due to the differences in service granularity and the varying requirements for data packaging, we introduced the BFF layer in the previous chapters. The calling client can directly call the BFF layer, which then distributes the request to different microservices for data assembly. Since many sub-services require user authentication, permission verification, and traffic control, do we really need to duplicate the logic for user authentication in each sub-service? This chapter will bring you closer to the gateway and handle these common requirements at the gateway level.
Why Introduce a Gateway #
Without a gateway, service calls face several direct challenges:
- Each service requires independent authentication, increasing unnecessary duplication.
- The frontend client interfaces directly with the services. Once the backend services change, the frontend also needs to change, causing a lack of independence.
- Exposing the backend services directly poses a challenge to the security of the services.
- Certain common operations, such as logging, need to be implemented in each sub-service, causing unnecessary redundant work.
The existing system call structure is shown in the following figure:
Calls are initiated directly by the frontend, and calls between services can be allocated by the service registry. However, calling from the frontend is not that simple, especially when the backend services appear in the form of multiple instances. Because each sub-service has its own service name, port number, etc., coupled with the duplication of certain common elements (such as authentication, logging, service control, etc.) in each sub-module, unnecessary costs are incurred. At this time, a gateway is urgently needed to wrap all the sub-services, provide services to the outside world uniformly, and handle common features at the gateway level, which greatly improves the maintainability and robustness of the services.
After introducing the gateway, the structure of the request call evolves as shown in the following figure:
The changes are obvious: the gateway layer performs unified request routing, liberating the choice of frontend calling; the backend services are hidden, and only the gateway address is visible externally, greatly enhancing the security; some common operations are directly implemented by the gateway layer, and the specific service implementations no longer carry out this work, enabling more focus on business implementation.
This article will guide you to introduce the spring-cloud-gateway component into your project. Some of you may ask, why not use Zuul? The answer is that due to the development of certain components, Zuul has entered a maintenance period. To ensure the integrity of the component, the Spring official team developed Gateway to replace Zuul for implementing gateway functionality.
Establishing the Gateway Service #
When introducing the JAR, note that Spring Cloud Gateway is developed based on Netty and WebFlux, so there is no need for related web server dependencies such as Tomcat. WebFlux conflicts with spring-boot-starter-web, so these two items need to be excluded, otherwise the application cannot start.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
<version>0.2.2.RELEASE</version>
</dependency>
The startup class is the same as a normal business module, and preliminary configuration is done in the application.yml configuration file.
server:
port: 10091
management:
endpoints:
web:
exposure:
include: '*'
#nacos config
spring:
application:
name: gateway-service
cloud:
nacos:
discovery:
register-enabled: true
server-addr: 127.0.0.1:8848
gateway:
discovery:
locator:
enabled: false
lowerCaseServiceId: true
filters:
- StripPrefix=1
routes:
- id: member-service
uri: lb://member-service
predicates:
- Path= /member/**
filters:
- StripPrefix=1
# Card sub-service
- id: card-service
uri: lb://card-service
predicates:
- Path=/card/**
filters:
- StripPrefix=1
# Resource sub-service
- id: resource-service
uri: lb://resource-service
predicates:
- Path=/resources/**
filters:
- StripPrefix=1
# Charging sub-service
- id: charging-service
uri: lb://charging-service
predicates:
- Path=/charging/**
filters:
- StripPrefix=1
# Finance sub-service
- id: finance-service
uri: lb://finance-service
predicates:
- Path=/finance/**
filters:
- StripPrefix=1
The routes configuration is for specifying the routing rules for specific services. Each service is configured as an array item. The id is used to differentiate between services, and the uri corresponds to the direct service call. lb indicates load balancing to access the service, and the service name in Nacos is specified after lb. predicates are used to match requests, and there is no need to access the service in the form of services.
With this, the simple routing function of the Gateway gateway service is completed. The frontend can directly access the gateway to call the corresponding service without having to worry about the service name or service port of the sub-service.
Circuit Breaker and Fallback #
In the chapter about service invocation, we implemented fallback for services using Hystrix. Can we make a unified configuration at the gateway level? The answer is yes. Next, let’s introduce Hystrix into the Gateway module to configure circuit breaker for services. When a service times out or exceeds the specified configuration, we will quickly return a prepared exceptional method, achieving rapid failure and implementing circuit breaker for the service.
Add the necessary dependencies:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>
Set the circuit breaker timeout in the configuration file:
# Timeout time config, default time is 1000ms
hystrix:
command:
default:
execution:
isolation:
thread:
timeoutInMilliseconds: 2000
Write the fallback response class. This class needs to be configured at the failure call location of the sub-service.
@RestController
@RequestMapping("error")
@Slf4j
public class FallbackController {
@RequestMapping("/fallback")
public CommonResult<String> fallback() {
CommonResult<String> errorResult = new CommonResult<>("Invoke failed.");
log.error("Invoke service failed...");
return errorResult;
}
}
# Card sub-service
- id: card-service
uri: lb://card-service
predicates:
- Path=/card/**
filters:
- StripPrefix=1
# Configure fast failure fallback
- name: Hystrix
args:
name: fallbackcmd
fallbackUri: forward:/error/fallback
If the service is temporarily unavailable but can return to normal after retrying, you can ensure the availability of the service by setting the retry count.
# Card sub-service
- id: card-service
uri: lb://card-service
predicates:
- Path=/card/**
filters:
- StripPrefix=1
- name: Hystrix
args:
name: fallbackcmd
fallbackUri: forward:/error/fallback
- name: Retry
args:
# Retry 3 times, with the initial access, there should be 4 requests in total
retries: 3
statuses:
- OK
methods:
- GET
- POST
# Exception configuration, consistent with the exceptions thrown in the code
exceptions:
- com.mall.parking.common.exception.BusinessException
How to test it? You can add an exception throw in the code to test whether the request is retried 3 times. When calling from the frontend, the number of calls will be 4 when accessing this service through the gateway.
/* The exception is thrown here to test whether the spring-cloud-gateway retry mechanism is working properly
* if (StringUtils.isEmpty("")) {
throw new BusinessException("test retry function");
}*/
Service Throttling #
Why do we need throttling? When the pressure of service calls suddenly increases, it has a great impact on the system. It is necessary to take some throttling measures to ensure the availability of the system.
Common throttling algorithms include token bucket and leaky bucket. The Gateway component internally provides Redis + Lua to implement throttling by default. You can specify whether to throttle based on IP, user, or URI. Let’s take a look.
The core logic of the RedisRateLimiter provided by Spring Cloud Gateway is to judge whether a token is obtained. It implements throttling based on the token bucket algorithm by calling the request_rate_limiter.lua script in META-INF/scripts. Let’s see how to achieve our goal with this feature.
Introduce the support of reactive stream Redis:
<!-- Reactive stream Redis -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
Configure throttling based on IP. For example, when exchanging coupons in a shopping mall, there are only a fixed number of coupons available within a fixed period of time to deal with a sudden increase in requests. It is easy to encounter high peak transactions, leading to service unavailability.
- name: RequestRateLimiter
args:
redis-rate-limiter.replenishRate: 3 # Allow users to process how many requests per second
redis-rate-limiter.burstCapacity: 5 # Capacity of the token bucket, the maximum number of requests that can be completed in one second
key-resolver: "#{@remoteAddrKeyResolver}" # Use SPEL expression to obtain the corresponding bean
The KeyResolver configuration item mentioned above is used to define the rules for throttling. For example, in this case, throttling is based on IP. Write an implementation class to implement this interface:
public class AddrKeyResolver implements KeyResolver {
@Override
public Mono<String> resolve(ServerWebExchange exchange) {
return Mono.just(exchange.getRequest().getRemoteAddress().getAddress().getHostAddress());
}
}
Define it as a bean in the startup class:
@Bean
public AddrKeyResolver addrKeyResolver() {
return new AddrKeyResolver();
}
At this point, the configuration is complete. Let’s verify if the configuration is effective.
Evaluating Throttling #
In the early stages, we used the Postman component for a lot of interface testing work. In fact, it can provide concurrency testing capabilities. Many users have not yet discovered this feature. Here, let’s use Postman together to initiate concurrent testing. The operation steps are as follows.
1. Establish the test script directory
2. Put the test requests into the directory
3. Run the script
4. Open the terminal, enter the corresponding Redis database, and enter the monitor command to monitor the execution of Redis commands. Click the “Run” button in the above figure to view the execution of Redis commands. By checking the Postman console, it can be seen that 3 requests have been ignored.
At this point, the native throttling component can be used normally. Throttling based on IP is simple. There are often more personalized requirements. At this time, customization is needed to implement advanced functions.
Cross-Origin Support #
The popular system deployment architecture nowadays is to deploy the front-end and back-end independently, which directly brings another problem-cross-origin requests. It is necessary to support cross-origin requests at the gateway level, otherwise requests cannot be routed to the correct processing node. Here are two ways to achieve this, one is through code writing, and the other is through configuration files. It is recommended to use the configuration method to complete it.
Code Approach #
@Configuration
public class CORSConfiguration {
@Bean
public CorsWebFilter corsWebFilter() {
CorsConfiguration config = new CorsConfiguration();
config.setAllowCredentials(Boolean.TRUE);
//config.addAllowedMethod("*");
config.addAllowedOrigin("*");
config.addAllowedHeader("*");
config.addExposedHeader("setToken");
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource(new PathPatternParser());
source.registerCorsConfiguration("/**", config);
return new CorsWebFilter(source);
}
}
Configuration File Approach #
spring:
cloud:
gateway:
discovery:
# Cross-origin
globalcors:
corsConfigurations:
'[/**]':
allowedHeaders: "*"
allowedOrigins: "*"
# To ensure the security of requests, only GET or POST requests are supported in the project, and other requests are blocked to avoid unnecessary problems
allowedMethods:
- POST
This concludes the common shared issues in gateway routing configuration, circuit breaker failure, request throttling, and cross-origin request support. With deeper utilization, there will be more advanced features waiting for everyone to develop and use.
Take a thinking question:
- Apart from Spring Cloud Gateway, do you know other middleware that can implement gateway functions? You can do some research.