流程可控 #

流程可控是持续交付平台的重要特征之一。它确保了整个流水线中的每个阶段都可以被有效地管理和控制。通过流程可控，开发团队可以定义和优化每个阶段的工作流程，并确保各个阶段之间的协调和协作。

动静分离配置化 #

动静分离配置化是指将应用程序的静态资源（如HTML、CSS和JavaScript文件）与动态资源（如接口和数据）进行分离，并通过配置实现灵活的部署和扩展。这种分离可以提高应用程序的性能和可维护性，并且使得团队可以更加灵活地管理和调整系统的不同部分。

快速接入 #

快速接入是指持续交付平台能够提供方便快捷的接入方式，以便开发团队可以快速部署和测试他们的应用程序。这种特性可以减少开发和测试的时间，提高团队的工作效率。

内建质量门禁 #

持续交付平台应该具备内建的质量门禁机制，以确保应用程序的可靠性和稳定性。通过内建质量门禁，团队可以在发布之前进行自动化测试和代码审查，并确保代码的质量达到预定标准，以减少潜在的问题和故障。

数据采集聚合 #

数据采集聚合是指持续交付平台能够收集和汇总应用程序和流水线的相关数据。这些数据可以用于监控和分析应用程序的性能、稳定性和质量。通过数据采集聚合，团队可以及时发现并解决潜在的问题，以确保应用程序的顺利运行。

以上就是现代流水线必备的十大特性的下半部分了。下次我们将继续介绍另外五个特性，敬请期待。

Feature Six: Controllable Workflow #

In the previous session, I mentioned that a pipeline is a platform that covers the entire end-to-end software delivery process. In other words, the main purpose of a pipeline is to improve the efficiency of the software delivery process and visualize its status.

During offline discussions, I found that many students have some misconceptions about this concept. They believe that they need to build a large and comprehensive pipeline to complete the entire software delivery process.

In fact, a pipeline is meant to cover the end-to-end workflow, but this does not mean that there must be a single pipeline that runs through the entire process from code submission to software release. In practice, multiple pipelines often cover different stages, such as development stage pipeline, integration stage pipeline, and deployment stage pipeline. These pipelines together cover the entire software delivery process.

This reflects the controllability of the pipeline workflow. Pipelines can exist to meet the business goals of different stages, and each pipeline implements different functionalities. To achieve this, pipelines need to support various triggering methods, such as scheduled triggering, manual triggering, and event triggering. Among them, event triggering is a very important capability of continuous integration.

Taking Gitlab as an example, you can add a webhook in the code repository, and the URL of the webhook is the API that triggers the pipeline task. This API can be automatically registered through Gitlab’s API.

It should be noted that to achieve automatic registration of webhooks, the account accessing Gitlab must have Master-level permissions for the corresponding code repository, otherwise it will not be added successfully.

After registering the webhook, when the code repository captures the corresponding event, such as a code push action, it will automatically call the webhook and pass the basic information of this code submission (such as branch, committer, etc.) to the registered address.

After receiving the API request, the pipeline platform can filter the requests based on certain rules, particularly the trigger branch information. When the rule conditions are met, the pipeline task is executed, and the result is written back to the code repository through Gitlab’s API. In this way, each commit history will be associated with an execution record of the pipeline, which can be used to assist code merge reviews.

I have drawn a flowchart to illustrate the process I just described, as well as the interface information called.

In addition to supporting various triggering methods, pipelines also need to support manual approval. This means that the flow from one stage to another can be automatic, where the next stage is automatically executed after the previous stage is completed, or it can be manually executed, and human confirmation is required before continuing. The human confirmation here needs to be combined with permission control.

In fact, manual approval scenarios are very common in the software delivery process. If it is a self-built workflow engine, implementing manual approval is not difficult. However, if you are implementing this process based on Jenkins, although Jenkins provides the input method to achieve manual approval, I still recommend that you implement it by extending the code yourself. For example, abstract the execution process of each atomic operation into three stages: before(), execute(), and after(). You can write the logic for manual approval in the before() or after() method.

In this way, for all atoms, you can default to executing the base class methods to obtain the capability of manual approval. Whether to enable manual approval can be implemented through parameters in the atom configuration. You don’t need to manually inject the input method into each atom, making the execution process of the pipeline more clear.

Here is a design implementation of an abstract atom class, as shown in the code below:

abstract class AbstractAtom extends AtomExecution {
  def atomExecution() {
    this.beforeAtomExecution()
    
    // Pre-processing steps for the atom, you can write common execution logic, such as manual approval, here
    echo('AtomBefore')
    before()
    
    // Core logic of the atom
    echo('AtomExecution')
    execute()
    
    // Post-processing steps for the atom, you can write common execution logic, such as manual approval, here
    echo('AtomAfter')
    after()
    
    this.afterAtomExecution()
  }
}

Feature 7: Configuration-driven dynamic separation of actions #

The flexibility of the pipeline is not only reflected in the ability to orchestrate and control the process, but also in the need for continuous iteration of each atomic function. So, how can we dynamically configure the atoms without changing the code?

This is where the design method of dynamic separation of actions comes into play. So, what is dynamic separation of actions?

In fact, dynamic separation of actions is a configuration-driven implementation approach. This means that the content that needs to be frequently adjusted or customized by the user is saved in a static configuration file. Then, the system reads the configuration data through an interface during the loading process and dynamically generates the interactive interface visible to the user.

You may find it a bit abstract, so let me give you an example. Take a look at the screenshot below.

If I want to add a new function or provide a new user configuration parameter for a particular atom, the traditional approach is to add a section of HTML code to the front-end page. In this case, every change to the atomic function requires collaboration with the front-end, and the independence of the atom is lost, becoming tightly coupled to the page.

Furthermore, the front-end page has incorporated so much business logic that if someday compatibility with different versions of the atom is required, the front-end page would also need to maintain two sets. If this were the case for just one or two applications, it would be manageable, but if there are hundreds of applications, it would be unimaginable.

So, how exactly should we do it? The most important thing is to define a standardized atomic data structure.

For example, on the left side of the above diagram, I have provided a reference structure for you. For all atoms, the functionality they expose is defined through this standardized approach. When loading the atom, the front-end retrieves the atom’s data structure through the API provided by the back-end, and renders it into different control types based on the specified parameter types.

Furthermore, the attributes of the control can also be flexibly adjusted, such as the default value of the control, whether the control is required, and whether there are character input restrictions, and so on. So, when you want to add a new parameter, you only need to modify the atomic configuration without modifying the front-end code. Separating structure definition from specific implementation significantly simplifies the flexibility of atomic upgrades.

Whether in the design of atomic structures or in areas such as front-end and back-end interactions, defining a universal data structure is the best practice for designing standardized systems.

For the pipeline platform, configuration-driven approaches are used in many places, not just for atoms. For example, the fields and chart types reflected in system reports are designed to meet the diverse needs of users. Moreover, by incorporating configuration into version control, you can quickly query the change history of atomic configurations, achieving the goal of traceability for all changes.

Feature 8: Quick Integration #

As mentioned before, many of the capabilities of a pipeline platform are not provided by the platform itself, but rather come from vertical business platforms. Therefore, when building a pipeline platform, the ability to quickly integrate external platform capabilities becomes a problem that must be solved.

The classic solution is to provide a plugin mechanism to integrate platform capabilities. For example, the Jenkins platform has established a very powerful plugin ecosystem using this approach. However, if each platform integration requires the enterprise to implement plugins themselves, the willingness of enterprises to integrate with the platform will be greatly reduced.

In fact, the cost of integration directly affects the expansion of platform capabilities, and the number of capabilities supported by the pipeline platform is the core competitiveness of the platform.

So, is there a more lightweight way to integrate with platforms? Let me provide you with a solution: automated generation of atomic code associated with the platform.

In the seventh feature, we have already defined the data structure of the atomic component through a standardized descriptive language. Can the implementation code of the atomic component also be automatically generated? In fact, there are two types of external platform integration in most cases.

• The platform provider provides a locally executed tool, similar to the way SonarQube’s Scanner works, to achieve the corresponding functionality by calling this tool locally.
• Through interface calls, integration between platforms is achieved, and the implementation process of the calls can be either synchronous or asynchronous.

Since there are commonalities in platform integration, we can plan a solution.

Firstly, the pipeline platform needs to define a standardized integration approach. Taking the interface call type as an example, the integrating platform needs to provide a task calling interface, a status query interface, and a result reporting interface.

• Task Calling Interface: Used to trigger tasks in the pipeline, generally defined and implemented by the integrating platform. For more mature platforms, these types of interfaces are generally readily available. The interface call parameters can be directly converted into atomic parameters, and some platform configuration information (such as interface addresses, protocol, etc.) can be defined in the atomic data structure.
• Status Query Interface: Used to query the execution status of tasks in the pipeline and obtain the progress of task execution. This interface is also defined and implemented by the integrating platform, and the returned content generally includes task status and execution logs, etc.
• Data Reporting Interface: Used for tasks to report the execution results to the pipeline platform for storage. This interface is defined by the pipeline platform and provides a set of standard data interfaces to the integrating platform. The integrating platform must report data according to this standard interface to simplify the data reporting process.

By simplifying platform integration into several standardized steps, the implementation cost of platform integration can be greatly reduced. According to our experience, integration with a platform can be completed within a few days.

Feature 9: Built-in Quality Gates #

In Lesson 14, I introduced the concept of built-in quality and the related implementation steps. Do you remember the two principles of built-in quality?

• The earlier a problem is discovered, the lower the cost of fixing it.
• Quality is the responsibility of each individual, not just the quality team.

There is no doubt that the best battlefield for built-in quality is the continuous delivery pipeline, and the specific manifestation is the quality gates. By injecting quality checks at various stages of the continuous delivery pipeline, built-in quality can truly be put into practice.

In general, the pipeline platform should have the ability of quality gates, and we should even consider it as a primary capability of the pipeline platform. On the pipeline platform, we need to complete quality rule formulation, collection and inspection of gate data, and complete closed-loop reporting of gate results. Most of the quality gates come from vertical business platforms, such as the UI automation testing platform, which can provide metrics such as automation test pass rate. Only by reporting the data used for quality gates to the pipeline platform can the inspection function be activated.

So, how should the functionality of quality gates be designed?

Working backwards, the first step is to set up the gate inspection function. This function is also a common capability of the pipeline, so it is similar to the manual review function and can also be placed in the after() step of atomic execution, or be a separate step called qualityGates().

This step is executed every time an atomic action is performed. In this step, the current pipeline checks whether the gate inspection function has been enabled and whether the current atomic action provides gate inspection capability. If gate rules have been configured and the current atomic action falls within the inspection scope, it waits for the execution result to return, extracts the data, and triggers the inspection process. You can refer to the example code below.

def qualityGates() {
  // Retrieve quality gate configuration and its activation status
  boolean isRun = qualityGateAction.fetchQualityGateConfig(host, token, pipelineId, oneScope)
  // If gate inspection is activated, wait for the result to return, maximum waiting time is 30 minutes
  if (isRun) {
    syncHandler.doSyncOperation(
      30,
      'MINUTES',
      {
        // Wait for the execution result to return, quality gate function must be executed synchronously
        return httpUtil.doGetToExternalResult(host, externalMap.get(oneScope), token)
      })
    // Extract the returned data
    qualityGateAction.fetchExecutionResult(host, token, externalMap.get(oneScope), buildId)
    // Verify the quality gate
    qualityGateAction.verify(oneScope)
  }
}

Now that we have addressed the issue of how to conduct inspections, let’s take a step back and see how the rules for quality gates should be defined.

In a company, the definition and management of quality rules are generally handled by the QA team. Therefore, we need to provide them with a unified entry point to facilitate rule configuration and adjustment of specific values.

For quality gates, there can be various types of checks.

• In terms of comparison, the results can be compared as greater than, equal to, less than, contains, or does not contain.
• In terms of comparison results, there can be failure values and warning values. A failure value means that if this condition is met, the pipeline execution is terminated directly. A warning value means that if this condition is met, a warning flag is given, but the pipeline execution is not terminated.

These conditions often need to be adapted based on the rules defined by the QA team.

A quality rule can be composed of a set of sub-rules. For example, unit test pass rate is 100%, line coverage is greater than 50%, no critical blocking code issues, etc.

So, you see, in order to define a flexible quality gate, some effort needs to be put into system design. In our previous practice, we used the adapter and strategy pattern to meet the flexible expansion of rules.

The strategy pattern is one of the commonly used patterns among the 23 design patterns. If you are not familiar with it, I recommend you to read this reference article. If you want to delve deeper into design patterns, GeekTime has a corresponding column, or you can also purchase the classic book “Design Patterns”. In fact, the key is to develop based on interfaces rather than procedures, and to implement different inspection strategies through different implementation classes.

Feature 10: Data Aggregation and Collection #

As the carrier of the software delivery process, the visualization of the pipeline means that you can see the execution status of each step on the pipeline. What does this mean?

Before the systems are integrated, if you want to see the execution result of the tests, you have to go to the testing system to check; if you want to see the execution status of database changes, you have to go to the database management platform to check. The problem here is that there is no unified place to view all the status information of this release, and this is also the problem that the visualization of the pipeline aims to solve.

After the platform’s capabilities are integrated into the pipeline in atomic form, the pipeline needs to have the ability to obtain the result data related to this execution. That’s why it is necessary to require subsystems to implement data reporting interfaces when integrating with the platform. As for the granularity of the reported data, there is actually no fixed rule. The principle is to satisfy the user’s requirement for viewing the most basic result data.

Taking unit testing as an example, the data that needs to be collected includes two aspects: one is the execution result of the unit tests, such as the total number of test cases, how many were executed, and the number of successes and failures. In addition, at least the coverage information needs to be collected, including coverage indicators for various dimensions. However, the data at the granularity of each specific file’s coverage is relatively large, and it can be presented by generating a report instead of being reported in detail and saved to the pipeline backend.

Before establishing an independent data measurement platform within the organization, the pipeline platform assumes the display function of the entire process data. However, after all, the goal of the pipeline is to display objective data results, not to analyze and explore the data. Therefore, when the organization begins to build a data measurement platform, the pipeline can also serve as one of the data sources to meet the measurement requirements for various engineering capabilities of the measurement platform.

Summary #

So far, I have provided a complete introduction to the ten essential features of a modern pipeline. In fact, the functional features of a pipeline go beyond these ten. With the development of cloud computing and cloud-native applications, cloud-native pipelines have become a topic of increasing discussion. I will share some of my thoughts on this in future courses.

It can be said that a good continuous delivery pipeline platform is the pinnacle of an enterprise’s DevOps capability. It is not surprising that more and more companies are making efforts in this field and even treating it as a core competency to be exported externally, becoming a part of the commercial operations of the enterprise. Mastering these ten features and implementing them in the construction of pipeline platforms is the necessary path for the construction of enterprise DevOps platforms.

Just as the famous American actress Lily Tomlin once said:

The road to success is always under construction.

The road to success in enterprise continuous delivery is also not smooth sailing. The endless pursuit is the guide for us to move forward. I hope you can continue to think and practice, and continuously improve on the path of pipeline construction.

Thinking Questions #

What are the shortcomings, areas for improvement, or “anti-human” designs of the pipeline platforms you are currently using? After reading the content of these two lectures, do you have any new thoughts and suggestions for improvement?

Feel free to write down your thoughts and answers in the comments section. Let’s discuss together and learn together. If you find this article helpful, please also feel free to share it with your friends.