12 Architectural Example Based on OAuth 2 #
In the previous lectures, we have learned about the usage of OAuth 2.0 in open environments. However, OAuth 2.0 can be applied to any situation that requires authorization/authentication, including microservices.
Today, I have invited my friend Yang Bo to share a reference architecture for microservices based on OAuth 2.0/JWT. Yang Bo, who has previously served as the Director of Development in the Framework Department of Ctrip and the Director of Development in the Infrastructure Department of Paipaidai, has extensive practical experience in microservices and OAuth 2.0.
During his time at Ctrip, he was responsible for the development of Ctrip’s API gateway product, including its integration with Ctrip’s token service. During his time at Paipaidai, he was responsible for the development and operation of Paipaidai’s token service. The token services of these two companies are similar to OAuth 2.0 but are simpler.
Now, let’s begin with the content that Yang Bo has prepared for us.
Hello, I am Yang Bo.
The evolution from monolithic to microservices architecture is a major trend in the current digital transformation of enterprises. OAuth 2.0 is the industry’s standard authorization protocol, which consists of several processes for token issuance and management for different scenarios. JWT is a lightweight, self-contained token that can be used to securely transfer user information between microservices.
Based on my current understanding, although many companies have partially or fully transitioned to microservices architecture, they generally have custom-made authorization/authentication mechanisms, such as Ctrip and Paipaidai’s token services. The reason for this is that the standard OAuth 2.0 protocol is relatively complex and has a high entry threshold. Custom development can temporarily solve the problems of enterprises, but it lacks generality and may have many potential security risks.
So, how should we integrate the industry-standard OAuth 2.0/JWT with microservices? Is there a practical reference architecture that we can follow?
I believe the architecture provided by Thijs does have practicality and reference value, but the naming of some microservice layers in his architecture, such as BFF and Facade layers, does not conform to the mainstream microservices architecture. Additionally, his architecture appears to be hand-drawn, making it less clear and easy to understand. For this reason, I want to improve Thijs’s architecture and provide additional processes for different scenarios in today’s lecture.
For the sake of understanding, in the following discussion, I will assume the existence of a new retail company called ACME, which has completed its digital transformation, and the microservices e-commerce platform is the core infrastructure supporting its business operations.
In terms of business architecture, ACME has nearly a thousand offline stores, which are integrated with the e-commerce platform through POS systems. The company also has a number of logistics fulfillment centers, and the order picking system also needs to be integrated with the e-commerce platform. In addition, the company has many delivery drivers who interact with the e-commerce platform through an app. Of course, ACME also has several e-commerce websites that serve as the main source of traffic for the e-commerce platform.
Although the technology platform that supports ACME’s business operations is complex, its core can be described using a simplified microservices architecture diagram:
As shown, this microservices architecture runs on a Kubernetes cluster. Of course, this architecture does not necessarily require a Kubernetes environment and can also be used in a traditional data center. Additionally, the overall authentication and authorization architecture is implemented based on OAuth 2.0/JWT.
Next, I will analyze each layer of this microservices architecture according to its hierarchy and discuss the relevant processes for application authentication/authorization and service invocation. This way, you can not only understand how a typical microservices architecture is layered but also learn how OAuth 2.0/JWT can be integrated with microservices.
Microservice Layered Architecture #
ACME Company’s microservice architecture can be roughly divided into layers including Nginx reverse proxy layer, web application layer, gateway layer, BEF layer, and domain service layer, as well as an IDP service. Overall, this is a popular layered approach for microservice architecture, with each layer having a single and clear responsibility.
Next, let’s take a closer look at the main functionalities of each layer.
Nginx Reverse Proxy Layer #
Firstly, the Nginx cluster serves as the entry point for the entire platform. Nginx is a 7-layer HTTP reverse proxy that is responsible for reverse routing. It routes external traffic to different backends based on HOST host headers or PATH, directing it to web applications or directly to the gateway, for example.
Within the Kubernetes ecosystem, Nginx works in concert with an Ingress Controller (referred to as Nginx Ingress), which enables the configuration of routing rules with Ingress Rules.
Web Application Layer #
This layer mainly consists of web applications, including HTML, CSS, JS, and other resources.
Web service layer typically adopts a traditional Web MVC + template engine approach to handle server-side rendering. Alternatively, it can use a single-page SPA mode. This layer is primarily managed by the company’s frontend team, which typically utilizes Node.js technology stack, but can also utilize the Spring MVC technology stack. The specific implementation depends on which technology the frontend team is more proficient in. When this layer requires backend data, it can retrieve data by invoking backend services through the gateway.
Gateway Layer #
This layer serves as the entry point for microservice traffic. The main responsibility of the gateway is reverse routing, routing incoming frontend requests to target microservices in the backend based on HOST host headers, PATH, or query parameters (e.g., IDP/BFF in the diagram).
Additionally, the gateway assumes two crucial security responsibilities:
Firstly, it verifies and converts tokens. It validates the OAuth 2.0 access token passed from the frontend by calling the IDP and converts it into a JWT token containing user and permission information, which is then passed to backend microservices.
Secondly, it performs permission checks. The gateway’s routing table can be associated with OAuth 2.0 scopes. Therefore, based on the scope specified in the request token, the gateway can determine if the request has the necessary permission to call the backend service.
I will further explain security-related scenarios and processes in the next section.
Furthermore, the gateway is also responsible for centralized rate limiting, log monitoring, and supporting CORS functionality.
For the technology selection of the gateway layer, popular API gateway products like Netflix’s open-source Zuul and Spring Cloud Gateway can be considered.
IDP Service #
IDP stands for Identity Provider and is mainly responsible for handling the OAuth 2.0 authorization protocol, issuing and managing OAuth 2.0 and JWT tokens, as well as user authentication. The IDP uses the backend’s Login-Service for user authentication.
For the technology selection of the IDP, popular options like Spring Security OAuth or the RedHat open-source KeyCloak can be considered. Spring Security OAuth is a development framework for OAuth 2.0, suitable for enterprise customization. On the other hand, KeyCloak is an out-of-the-box OAuth 2.0/OIDC product.
BFF Layer #
BFF stands for Backend for Frontend and mainly implements the aggregation functionality of backend domain services (with a somewhat similar concept to database joins). It also provides more user-friendly APIs and data formats for different frontend experiences (PC/Mobile/Open Platform, etc.).
The BFF layer can include some business logic and may even have its own database storage. Typically, a BFF needs to call two or more domain services and in certain cases, it may call other BFFs as well (although it is generally not recommended as it can result in complex and hard-to-understand call relationships).
If a BFF needs to obtain user-specific or OAuth 2.0 scope-related information, it can directly retrieve it from the passed JWT token.
BFF services can be developed using Node.js or frameworks like Java/Spring.
Domain Service Layer #
The domain service layer is the foundation of the entire microservice architecture. These services contain business logic, typically have their own independent database storage, and can call external services as needed.
Following the principles of microservice layering, domain services are prohibited from calling other domain services and are not allowed to call BFF services in reverse. This is done to maintain the single responsibility and bounded context of microservices, avoiding complex domain dependencies. Domain services are developed, tested, and released as independent units. In the e-commerce domain, common domain services include user service, product service, order service, and payment service.
Similar to BFF, if a domain service needs to obtain user-specific or OAuth 2.0 scope-related information, it can directly retrieve it from the passed JWT token.
As can be seen, both domain services and BFF services are stateless. They do not store user state themselves but retrieve user information from passed JWT data. Therefore, in the overall architecture, microservices are stateless and can be horizontally scaled as needed, with the state either residing on the user side (browser or mobile app) or in a centralized database.
How to integrate OAuth 2.0/JWT with microservices? #
Above is the layered architecture of ACME Company’s microservices. This layered architecture is suitable for most internet business system scenarios. Therefore, if you are an enterprise architect and need to design a microservices architecture, you can refer to it for design. Next, I will demonstrate several typical application authentication scenarios and the corresponding service invocation processes to help you understand how OAuth 2.0/JWT integrates with microservices.
Scenario 1: First-party Web Application + Resource Owner Credentials Grant #
In this scenario, users access ACME Company’s own e-commerce website, which is developed using Spring MVC. Considering that this is a first-party scenario (i.e., a website application developed by the company itself), we can choose OAuth 2.0’s Resource Owner Password Credentials Grant or the more secure Authorization Code Grant. Since there is no third-party involved here, we will choose the relatively simple Resource Owner Credentials Grant.
Below is an example of an authentication and authorization process. Note that this only highlights the key steps, and there are many areas that need improvement and optimization in actual production. In addition, for simplicity of description, it is assumed that the process is successful.
In the above diagram, the user corresponds to the resource owner in OAuth 2.0, and ACME IDP corresponds to the authorization server in OAuth 2.0. In addition, the backend microservices (including BFF and basic domain services) in the previous architecture diagram correspond to protected resources in OAuth 2.0.
The process is explained as follows:
- The user accesses ACME Company’s e-commerce website through a browser and clicks on the login link.
- The web application returns the login page (this login page can be customized by the website itself).
- The web application forwards the username and password to the IDP’s token endpoint through the gateway (POST /oauth2/token, grant_type=password).
- The IDP authenticates the user through the login service.
- The IDP authentication is successful and returns a valid access token (and optionally a refresh token).
- The web application receives the access token, creates a user session, saves the OAuth 2.0 token in the session, and returns the login success to the user.
- The user’s browser records the session cookie and the login is successful.
Next, let’s look at the service invocation process after authentication and authorization. Similarly, this only highlights the key steps and assumes a successful process.
- The web application calls the backend API (querying the user’s purchase history) through the gateway and includes the OAuth 2.0 token in the HTTP header (from the user session).
- The gateway intercepts the OAuth 2.0 token and verifies it with the IDP.
- The IDP verifies the token and then queries the user and scope information based on the token to build a JWT token and returns it.
- The gateway receives the JWT token, checks if the scope has permission to call the API, and forwards the request to the backend API for invocation.
- The backend BFF (or domain service) retrieves the user information from the JWT and queries the purchase history based on the user ID and returns it.
- The web application receives the user’s purchase history data, caches it in the session if needed, and returns it to the user.
Note that this service invocation process can also be applied in other scenarios, such as the upcoming “First-party Mobile Application + Authorization Code Grant” and “Third-party Web Application + Authorization Code Grant” scenarios. Basically, as long as you understand the principles of this process, you can flexibly apply it to actual scenarios. IDP authenticates users through the Login Service.
The app intercepts the authorization code returned by the browser and forwards it to the IDP token endpoint (POST /oauth2/token, grant_type=authorization-code) along with the PKCE code verifier.
IDP verifies the PKCE and authorization code. If the verification is successful, IDP returns a valid access token.
Afterwards, if the app needs to interact with the backend, it can directly call the backend microservices through the gateway, including the OAuth 2.0 access token in the HTTP header. The subsequent service call process is similar to the “first-party application + resource owner credentials” scenario.
Scenario 3: Third-Party Web Application + Authorization Code Mode #
The third scenario involves a third-party partner developing a web application to access ACME company’s e-commerce open platform API. This is typically a third-party web application scenario that uses the OAuth 2.0 authorization code grant type.
Next, let’s take a look at the authentication and authorization flow for this scenario. Again, these are the key steps, assuming a successful flow.
The web application backend sends an authorization code request (GET /authorize) to ACME company’s IDP service.
The user is redirected to ACME company’s IDP unified login page.
IDP authenticates the user through the Login Service.
The web application obtains the authorization code and makes a request to the IDP token endpoint (POST /oauth2/token, grant_type=authorization-code).
IDP verifies the authorization code and, if successful, returns a valid OAuth 2.0 token (and optionally a refresh token).
The web application creates a user session, stores the OAuth 2.0 token in the session, and returns a successful login response to the user.
The user’s browser stores the session cookie and is logged in successfully.
Afterwards, if the third-party web application needs to interact with the ACME e-commerce platform, it can directly call the microservices through the gateway, including the OAuth 2.0 access token in the HTTP header. The subsequent service call process is similar to the “first-party application + resource owner credentials” scenario mentioned earlier.
Additional Notes #
In addition to the three main scenarios and flows mentioned above, I would like to share six additional points that should be considered for enterprise-level OAuth 2.0 applications.
The first point is that the IDP API should support the conversion between OAuth 2.0 access token and JWT token. The integrated architecture mentioned earlier uses a combination of OAuth 2.0 access token and JWT token, requiring the implementation of an API to convert between the two. This conversion API is not part of the OAuth 2.0 standard and may not be supported by all IDP products (such as Spring Security OAuth), so it may require customization or extension.
The second point is about Single-Page Application (SPA) scenarios. For SPA scenarios, the implicit grant is a simple approach, but it is considered less secure in OAuth 2.0 and is generally not recommended. For pure SPA applications, the recommended approach is:
- If the browser supports Web Crypto for PKCE, consider using a similar flow to the “first-party mobile application” scenario with authorization code grant + PKCE extension.
- Otherwise, consider a hybrid mode for SPA + traditional web, where the frontend resides in the client’s browser but the login/authentication is handled by the backend web site, using either the “first-party web application” scenario with resource owner credentials or the “third-party web application” scenario with authorization code grant.
The third point is about Single Sign-On (SSO) scenarios. For simplicity, the flow described above does not consider SSO scenarios. If web SSO support is required, all application scenarios must be logged in through a centralized browser + IDP login page, and the IDP must support sessions for maintaining the login state. If the IDP is deployed in a clustered manner, sticky sessions or centralized sessions should also be considered.
In this way, once a user logs in to one web application, subsequent logins to other web applications can be automatically completed as long as the session on the IDP is still valid, achieving the effect of single sign-on.
Of course, to support SSO, the IDP’s session cookie needs to be set at the root domain of the web applications, which means that the root domains of different web applications must be the same to avoid cross-domain issues.
The fourth point is about the deployment of the IDP and gateway. In the previous architecture diagrams, although the IDP is hidden behind the gateway, it can also be exposed directly to the outside world through Nginx without going through the gateway. Alternatively, the login and authorization pages of the IDP can be exposed directly through Nginx, while the API interfaces go through the gateway.
The fifth point is about refresh tokens. To simplify the description, the integration method for refresh tokens was not detailed in the flow described above. Depending on the requirements of the scenario, refresh tokens can be enabled to extend the user’s login time. The specific integration method needs to consider security requirements.
The sixth point is about web sessions. To simplify the description, the flow described above assumes the use of web sessions, i.e., server-side sessions. But in actual scenarios, web sessions are not the only choice. It is also possible to use simple client-side sessions, also known as stateless sessions, by storing the OAuth 2.0 access token in a client-side browser cookie.
Summary #
Alright, that’s the main content for today. Today, I shared with you how to integrate industry-standard OAuth 2.0/JWT with microservices. Here are four key points to remember.
First, the mainstream microservices architecture can generally be divided into 5 layers: Nginx traffic access layer -> Web application layer -> API gateway layer -> BFF aggregation layer -> Domain service layer. This architecture can reside in a cloud-native Kubernetes environment or in a traditional data center.
Second, the API gateway is the entry point for microservice calls, and it plays an important role in security authentication and authorization. The main security operations include: 1) verifying the OAuth 2.0 access token through the IDP and obtaining a JWT token with user and permission information, and 2) performing authorization for API calls based on OAuth 2.0 scopes.
Third, in a microservices architecture, a centralized IDP service is usually needed, which acts as an Authentication & Authorization as a Service role, responsible for token issuance/validation/management and user authentication.
Fourth, in the architecture proposed in today’s lecture, the security mechanism at the Web application layer (before the gateway) is mainly based on the OAuth 2.0 access token, which is a transparent token or referred to as a reference token. The security mechanism at the microservice layer (after the gateway) is mainly based on the JWT token, which is an opaque self-contained token. The gateway layer implements the conversion between the two types of tokens. This is a hybrid mode of OAuth 2.0 access token + JWT token.
The reason for this design is that the Web layer is closer to the user side. If JWT tokens were used, user information would be exposed, which poses certain security risks. Therefore, OAuth 2.0 access tokens, which are meaningless random strings, are used. On the other hand, after the gateway, the security risks are relatively lower, and many services require user information, so it is more appropriate to use JWT tokens which contain user information.
Of course, if there are no special security considerations within an enterprise’s intranet, it is also possible to directly pass completely transparent user information (for example, using the JSON format).
Thought-provoking Questions #
In addition to the hybrid mode of OAuth 2.0 access token + JWT token that we discussed today, it is also possible to use OAuth 2.0 access token throughout the entire process, or to use JWT token throughout the entire process. Compared to the hybrid mode, what are the advantages and disadvantages of using OAuth 2.0 access token exclusively or JWT token exclusively, in your opinion?
Can you share your understanding of the authentication and authorization mechanisms based on traditional web applications? And compared to the authentication and authorization mechanisms of modern microservices that we discussed today, can you talk about the fundamental differences and similarities between them?
You are welcome to share your thoughts in the comments section, and feel free to share today’s content with other friends so that we can exchange ideas together.