04Exploring Jenkins CD Practices #

After understanding the system configuration and plugins of Jenkins, this section will formally learn the basic CD practice using Jenkins.

Before we begin, it should be noted that the deployment methods and scripts introduced in this article are relatively basic. The aim is to introduce the process and methods of using and configuring Jenkins tasks (jobs). In subsequent articles, deployment methods will be optimized and improved. Project configuration needs to be done according to your own actual situation for different scenarios and requirements. Of course, if you are already familiar with Jenkins usage or the process of continuous delivery, you can skip this chapter.

Creating Jenkins projects #

Jenkins includes various types of project configurations, such as freestyle, maven, pipeline, etc. Each type of project has its own advantages, disadvantages, and specific usage scenarios. The key is to choose the appropriate project type based on your actual situation. The following briefly introduces the usage of different types of projects.

Creating a Freestyle Project #

Freestyle projects are projects with a free-form configuration. They offer strong configurability, allowing various methods to achieve the desired goals without being confined to specific plugins for each step.

To create a Freestyle project in Jenkins, follow these steps:

1. Click on the “New” menu on the Jenkins main dashboard.
2. Enter a name for your task in the redirected page (e.g., test-freestyle-project).
3. Select “Freestyle project” and click “OK” at the bottom of the page.

After clicking “OK,” you will be redirected to the job configuration page, shown in the following image:

This job consists of several configuration steps (highlighted with red boxes in the image):

• General: This section is used to configure global settings for the job build. Options include:

  • “Run the build inside Docker containers”: Determines whether the build will be performed inside Docker containers.
  • “Discard old builds”: Configures the number of build histories to retain and the time to keep them.
  • “Restrict where this project can be run”: Specifies the Jenkins node to run this project on.
  • “This project is parameterized”: Configures the project to use parameters for the build.

• Source Code Management: This step is used to configure the source code repository for the build. It involves specifying the repository address and authentication credentials. The supported repository types are CVS, Git, SVN, etc.

• Build Triggers: This step determines when the job should be triggered. Options include remote build using an “Authentication Token,” triggering the build after a specific project is built, and configuring scheduled builds using cron-like syntax.

• Build Environment: This step sets up the environment for the build. It can include configuring credentials, specifying specific configuration files for compiling the code, sending necessary files to the server before building, or executing commands.

• Build: This step is where the actual build operations take place, using different plugins depending on the requirements. For example, code compilation, command execution, or container image building.

• Post-build Actions: This step is the final step in the Jenkins job configuration. It performs any necessary post-build tasks, such as deploying application services, generating test reports, performing code quality analysis, triggering other jobs upon successful completion, or sending emails.

When using newly installed Jenkins plugins, relevant options may appear within one or more of these steps.

Compared to other job types (such as Maven or Pipeline), each job type has its own specific configuration steps. The availability of these steps may also differ based on the installed plugins. However, the overall configuration steps are generally the same.

Because Jenkins offers a wide range of configuration options, and these options may vary depending on the installed plugins, let’s start with a simple example.

In the “Build” step, select “Execute shell” from the “Add build step” options. This option allows executing shell commands on the target host. Please note that if no special configuration is made in the “General” step to restrict the running node for the project, the job will run on the Jenkins master node (i.e., the Jenkins server).

Here is an example:

The execution result will be as follows:

The output shows the hostname of the Jenkins server.

If the checkbox for “Restrict where this project can be run” is selected in the “General” step, and the “Label Expression” input box is set to the label of the slave node added in the previous section (jenkins-slave1), the job will be executed on that slave node, as shown below:

The result indicates that the Jenkins job was successfully executed on the slave node.

Although the above examples successfully execute Jenkins jobs, our usage of Jenkins is not limited to simply running shell commands on the Jenkins server. From an operations perspective, the most common usage is to use Jenkins to pull application code, compile and test it, and finally deploy it to specified servers.

For instance, let’s configure Jenkins to pull code from the test-helloworld repository created in the previous section when we installed GitLab. We will compile the code using the Maven build tool.

Code Compilation #

Configure the job using the freestyle type as follows, and use the git service in the Source Code Management step.

In the above image, the “Credentials” is the credential created for authenticating with the Gitlab repository. Click on the Jenkins option to select the Username with password in the “Type” dropdown, and enter the username, password, and user to distinguish this credential in the input box below. Once entered, save it. Then, you can see the newly created credential in the dropdown box under the “Credentials” parameter. The details about Jenkins credentials will be discussed in detail in the next chapter.

In the dropdown list of “Add build step”, select “Invoke top level Maven target” and configure Maven build parameters.

In this build step, Maven plugin is used for the build, where:

The “Maven Version” parameter is a dropdown. If the Maven tool’s environment variable is configured in the Jenkins global configuration, it will be shown in the dropdown list. For example, the one being shown here is the global name set in the Jenkins global configuration.

The “Goals” here should be the Maven command to compile the code. Available commands are clean, package, install, and deploy. These commands can be used for code compilation, testing, packaging, etc.

The “POM” is the pom.xml file used for code compilation. This file defines basic information about the project, describes how the project is built, and declares project dependencies. You can write the absolute path of this file or the relative path. In the example configured above, it is written as ${WORKSPACE} (an environment variable provided by Jenkins) followed by the path to the pom.xml file in the project workspace. So you can directly write pom.xml here without any path.

The “JVM Options” configures the parameters used during Maven code compilation. In the above example, the parameter is configured to skip the execution of JUnit test cases in src/test/java.

The “Configuration Files” here is used to set the configuration file used when compiling the code using Maven.

If you find it cumbersome to configure the compilation parameters using the Maven plugin dialog, you can also compile the code using a command. In the “Add build step” section, use “Exec shell” and enter mvn clean install. It is simpler. I will not demonstrate it here. You can try it yourself if interested.

Once configured, save it and click “build now” in the project main menu to start the build. As shown below:

As shown above, a war package is generated in the target directory after the build is completed. Now, let’s configure the job to deploy this war package to a specified server.

Adding Host #

The project code has been compiled successfully, and now it’s time to deploy the project (in actual work, you may use Sonar for code analysis, but since this “helloworld” project is quite simple, we’ll skip the code quality analysis step for now. The use of Sonar will be covered in later chapters). Deploying the project to a local server is relatively easy - you can copy the WAR package to the specified directory using shell commands and start it with Tomcat, or directly start it with the java -jar command. Below, we’ll explain how to deploy the project to a remote server.

To deploy to a remote server, first, you need to add a machine that can be remotely connected to the Jenkins system. Follow the steps below:

Click “Manage Jenkins” -> “Configure System”, find the “Publish over SSH” parameter, and click “Add” to add a machine.

Explanation

The “Publish over SSH” parameter is used to configure the information of the server to remotely connect to, including the server’s IP address, username, password, port, etc. The IP address of the server used in this test is “192.168.176.160”.

The “Name” is a custom hostname, which will be used to connect to the specified server in the Jenkins project configuration.

The “Remote Directory” is used to define the path where files will be copied to on the remote server, which is the “/data” directory in this case. Therefore, make sure that the directory exists on the server with the IP address “192.168.176.160”. Otherwise, when copying files using this plugin, it will prompt that the path does not exist.

It is recommended to set the “Timeout” option to a larger value, with the default being 30 seconds. This means that if the connection exceeds 30 seconds, it will be disconnected. If your project’s deployment time exceeds 30 seconds, you need to change this value accordingly (by default, it usually doesn’t exceed 30 seconds).

After the configuration is done, there is a “Test Configuration” button below, which can be used to test whether the connection is successful.

If you are concerned that the remote server’s password is frequently changed, you can also configure the connection to this host through the ssh-key method by setting the “Path to key” or “Key” parameters mentioned above. The “Path to key” parameter is used to configure the path of the key file (private key). The “Key” parameter can be used to copy the private key to this box. If the user starting the Jenkins service already has passwordless authorization to the target host, you can also skip configuring these two parameters.

After the configuration is done, save and exit.

Code Deployment #

After adding the host, you can deploy the compiled package to the server. Here, we take deploying the service to Tomcat as an example:

Modify the job configuration, click “Send build artifacts over SSH” in the drop-down list of “Post-build Actions”, and configure as follows:

Explanation:

The Name under SSH Server is the remote host where the files will be sent to or commands will be executed. You have configured the name of the remote target server in the previous steps (in this example, it is docker-test).

In the Transfers section, the Source files field should be filled with the .jar or .war package to be copied from the local machine (usually .jar for microservices). The format is similar to the find command in Linux, for example, **/xx.war, which means searching for the .war package in all directories under the current workspace.

The Remote Prefix is used to remove the prefix of the found war package path. After the code is compiled, the default behavior is to generate the target directory under the project directory, which holds the compiled artifacts such as .jar and .war packages. It is important to note that the path here is relative to the path where the current project is located.

The Remote directory is the directory where the files will be copied to. In this example, it is set to /. It is important to note that the directory specified here is not the / directory in the Linux file system. It refers to the / directory under the directory specified by the “Remote Directory” option in the “Configure System” section of Jenkins. You can see in the screenshot of the “Add Host” section that I set the “Remote Directory” to /data when adding the server, so the “Remote Directory” here corresponds to the /data/ directory on the remote host. This directory needs to be created before the build, otherwise, an error will be prompted for a non-existent directory during file transfer.

The exec command is used to execute shell commands on the remote server. The commands are executed line by line, and if one command fails, it does not affect the execution of the next command. However, if these commands have dependencies (e.g., the next command depends on the previous command), it will cause the deployment to fail. For convenience in testing and learning, the commands are written line by line here. You can also write the commands into a script, and then deploy the service by executing the script.

By completing the configuration, you have set up an automatic deployment of the Tomcat service job. Let’s take a look at the execution result in Jenkins.

Then, you can access the Tomcat service IP address, port, and URI (in this example, it is test1) in the browser. As shown in the image:

When configuring the deployment of a WAR package for a Jenkins project, in addition to using the “Publish Over SSH” plugin, you can also directly enter shell commands or shell scripts in the “Exec shell” option box to deploy the Tomcat service. This method is simpler, more flexible, and easier to manage compared to the “Publish Over SSH” plugin in terms of customization. I will not demonstrate how to write shell scripts here. Those interested can try it themselves.

Creating a Maven-based Project #

After going through the previous section, I believe you should have a better understanding of Jenkins. Now let’s create a Maven-based project to deepen the understanding further.

As the name suggests, a Maven-based project is suitable for Java projects that use the Maven tool for code compilation. When configuring a Maven-based project, the configuration steps are slightly different from a freestyle project.

For example, I have created a job named “test-maven-project” with the default configuration steps as follows:

Explanation:

In a Maven-based project, the “Build” step is split into two steps: “Pre Steps” and “Build”. The “Build” step is specifically used to configure the parameters for compiling the code using the Maven tool. The “Post Steps” step is similar to the “Pre Steps” step, and the “Build Settings” step adds the email notification feature.

Code Compilation #

Compared to the project configuration of the freestyle type, the configuration of the “Source Code Management” step remains unchanged. You only need to configure the Maven parameters in the “Build” step, as shown below:

Explanation:

In the Maven type project configuration, you only need to configure the two items listed in the image. For the “Root Pom” setting, use the current (${WORKSPACE}) path (Jenkins work path) as the base path. Writing pom.xml represents the pom.xml file in the root directory of the project.

In the dropdown menu of the “Advanced” button, there are some parameter configurations for Maven build, such as the use of configuration file definitions. In practice, you can use the default configuration (the configuration file under the current Maven deployment program). Of course, if there are special changes to the configuration file, you can define it in the configuration yourself.

After saving the configuration, you can directly build it. It’s that simple.

Code Deployment #

The method of code deployment is the same as the method of deploying projects using the freestyle method. You can use either the publish over ssh plugin or the Exec shell method. The only difference is that in a Maven project, the parameter option name for the Publish over SSH plugin is changed to Send files execute commands over ssh, but the configuration parameters remain the same.

I won’t repeat the deployment process mentioned above. Other deployment methods will be discussed in future chapters.

Creating a Pipeline Project #

Creating and using a pipeline project (including the multi-branch pipeline project) requires some basic knowledge of Jenkins pipelines. Due to the amount of content involved and the use of some plugins, the content related to pipelines will be explained in detail in the following chapters.

Deploying a Spring Boot Project to Containers #

The core of this Jenkins course series is to use Jenkins for continuous delivery and deployment of microservice architecture projects. According to the concept of microservices, if containers are used as the infrastructure, it can achieve quick delivery and deployment of business applications. Therefore, I will briefly introduce how to deploy services to containers. Of course, this requires you to have a basic understanding of Docker.

First, create a Maven-type job in the Jenkins interface, as shown below:

Select “Build a Maven project” and click “OK” at the bottom.

After creating the project, you need to configure the source code repository, branch, and pom.xml path. I have already explained how to set them, so I won’t demonstrate here. Let’s directly look at the post-build actions.

In the “Add Post-build Steps” drop-down menu, select “Send files or execute commands over ssh” and configure as follows:

Explanation:

The explanations of each parameter of the plugin have already been mentioned in the above example and will not be repeated here.

In the deployment command above, besides creating the specified directory on the server in advance, you also need to place a Dockerfile on the target server (/data/base/nop directory) in advance, which is used to build the Maven-compiled jar file into an image.

The Dockerfile is as follows:

FROM fabric8/java-alpine-openjdk8-jre

COPY fw-base-nop.jar /deployments/
CMD java -jar /deployments/fw-base-nop.jar

The code in the command is as follows:

docker stop base-nop
docker rm base-nop

docker rmi base-nop
cd /data/fw-base-nop
docker build -t  base-nop .
docker run --network host -e TZ="Asia/Shanghai" -d -v /etc/localtime:/etc/localtime:ro -m 2G --cpu-shares 512 -v /data/logs/fw-base-nop:/data/logs/fw-base-nop --name base-nop base-nop

The command stops/deletes the container, stops/deletes the image, and finally starts the container.

Execution result:

This completes the process of deploying a service to a container. The configuration is relatively simple. The purpose of introducing this example is mainly to understand the process. In future chapters, this example will be optimized and used to achieve continuous delivery and deployment of services in various ways.

That’s the end of the basic practice of using Jenkins in this chapter. This chapter covered basic content and introduced the workflow of using different types of Jenkins projects. In the advanced chapters, we will optimize the code in the example mentioned above and use multiple different methods to achieve continuous delivery and deployment of services.