07 Integration of Jenkins With Ansible Key Points

07Integration of Jenkins with Ansible Key Points #

In the previous chapters, we used the built-in plugins in Jenkins to deploy projects to virtual machines and containers. Although we were able to deploy application code, it is difficult to meet the requirements for more complex scenarios using only the built-in plugins. You may encounter the following problems:

Deploying code on a large number of hosts and performing operations such as replacing configuration files is complex and difficult to achieve with the built-in plugins.
As the number of projects continues to increase, the difficulty of adding/modifying project configurations and maintaining project configurations also increases.

In order to address the above-mentioned issues and other factors that may affect work efficiency, we need to use the automation operations tool Ansible that we introduced earlier. However, before refactoring the scripts for deploying services in the previous chapters using Ansible, let’s first learn the basic knowledge related to Ansible that will be used in the following chapters.

This section mainly covers the following topics:

Ansible inventory
Ansible variables
Ansible conditionals
Ansible loops
Ansible templates

Ansible Inventory #

Inventory refers to the list of managed hosts. In large-scale configuration management work, Ansible can work on multiple systems in the infrastructure at the same time by selecting the list of hosts specified in Ansible inventory.

By default, this list is saved in the /etc/ansible/hosts file on the local server. It is a static INI format file. Of course, we can also specify this file temporarily using the -i parameter when running the ansible or ansible-playbook command. If the -i parameter specifies a directory, ansible will specify multiple inventory files under that directory when executed.

Hosts and Groups #

There are multiple formats that can be used to define an inventory file. However, in this guide, we will focus on using the INI format in the /etc/ansible/hosts file, as shown below:

mail.example.com

[webservers]
foo.example.com
bar.example.com

[dbservers]
one.example.com
two.example.com
bar.example.com

Explanation:

mail.example.com is a host that is not assigned to any group. Hosts that are not assigned to any group are typically listed at the beginning of the inventory file.
The group names, enclosed in square brackets [], are used to categorize the hosts and facilitate distinct management of different hosts.
A host can belong to multiple groups. For example, the bar.example.com host belongs to both the webservers and dbservers groups. In this case, variables assigned to this host from both groups can be used, and the precedence of these variables will be discussed in later sections.
The hosts listed here can be specified as an IP address or as a Fully Qualified Domain Name (FQDN) as long as the Ansible host can communicate with them. More detailed configurations will be discussed later on.

Now, let’s see how to list the inventory of hosts within a specific group:

ansible <group_name>|host --list-hosts

Default Groups #

Ansible has two default groups: all and ungrouped.

all includes every host.
ungrouped includes hosts that only belong to the all group and do not belong to any other group.

The following syntax is equivalent to matching all machines in the inventory file:

all
*

Port #

If the SSH port of a host is not the standard port 22, you can specify the port number after the hostname using a colon as a separator.

For example, if the port number is not set to the default, you can configure it as follows:

badwolf.example.com:5309

Aliases #

As mentioned above, the host can be in the form of FQDN or IP address. If you have some static IP addresses and want to set some aliases without modifying the system’s host file, or if you want to connect via a tunnel, you can set them in the following way:

test_ansible ansible_ssh_port=5555 ansible_ssh_host=192.168.177.43

Note:

In the above example, using the alias test_ansible will result in an SSH connection to the host 192.168.177.43 on port 5555 when running Ansible. This is achieved through the variable setting feature of the inventory file.

Matching #

If you want to add a large number of hosts with similar formats, you don’t have to list each hostname: when viewing a group of similar hostnames or IPs, you can abbreviate them as follows:

[webservers]
www[01:50].example.com

Note:

The range in square brackets “[]” [01:50] can also be written as [1:50], and they have the same effect, which means that hosts from www1 to www50 are specified, and the webservers group has a total of 50 hosts.

If you are not familiar with the ini format, you can also use the yaml format, like this:

webservers:
  hosts:
    www[01:50].example.com:

You can also define an abbreviated pattern for alphabetical ranges:

[databases]
db-[a:f].example.com

This represents a total of 6 hosts in the databases group, ranging from db-a to db-f.

The following syntax represents one or more groups. Multiple groups are separated by colons to represent an “or” relationship. This means that a host can belong to multiple groups at the same time:

webservers
webservers:dbservers

You can also exclude a specific group. In the following example, all machines that execute commands must belong to the webservers group, but not the dbservers group:

webservers:!dbservers

You can also specify the intersection of two groups. In the following example, machines that execute commands need to belong to both the webservers and nosql groups at the same time:

webservers:&nosql

You can also combine more complex conditions:

webservers:dbservers:&nosql:!phoenix

In the above example, machines belonging to the webservers and dbservers groups, belong to the nosql group, and do not belong to the phoenix group will execute the command.

You can also use variables. If you want to specify the group through parameters, you need to pass the parameters through the -e option, but this usage is not common:

webservers:!{{excluded}}:&{{required}}

You don’t have to strictly define groups, individual hostnames, and IPs. Wildcards are supported for groups:

*.example.com
*.com

Ansible also supports a combination of wildcards and groups:

one*.com:dbservers

In advanced syntax, you can select corresponding numbered servers or a part of the servers in a group:

webservers[0]

webservers[0-25]

Note:

webservers[0] means matching the first host in the webservers1 group.
webservers1[0:25] means matching the first to 26th hosts in the webservers1 group.

Most people use regular expressions when matching, just prefix with ~:

~(web|db).*\.example\.com

You can also add exclusion conditions using the --limit flag, which is supported by both /usr/bin/ansible and /usr/bin/ansible-playbook:

ansible-playbook site.yml --limit datacenter2

If you want to read hosts from a file, the filename should be prefixed with @.

For example, the following example lists the hosts specified in site.yaml that exist in list.txt:

ansible-playbook site.yml --limit @list.txt --list-hosts

Ansible Variables #

When writing shell scripts, you will inevitably use variables to some extent. Similarly, when executing tasks using playbooks, variables are also used. Using variables can make your playbooks more flexible, reduce code redundancy, and at the same time, Ansible uses variables to help handle differences between systems.

Before using variables, let’s take a look at how to define valid variables: variable names should consist of letters, numbers, and underscores. Variable names should always start with a letter, and Ansible’s built-in keywords cannot be used as variable names.

There are several ways to define and use variables, and here are three commonly used ones:

Defining and Using Variables in Playbooks #

Defining and using variables in playbooks or play is one of the most common ways to utilize variables in Ansible. This can be done using the vars and vars_files keywords, and they are written in the format key:value. Here’s an example:

- hosts: web
  vars:
      app_path: /usr/share/nginx/html
  tasks:
  - name: copy file
    copy:
      src: /root/aa.html
      dest: "{{ app_path }}"

In the above example, the variable app_path is defined using the vars keyword and then referenced in the dest parameter. You can define multiple variables as well:

vars:
  app_path: /usr/share/nginx/html
  db_config_path: /usr/share/nginx/config

You can also use YAML syntax to set variables:

vars:
- app_path: /usr/share/nginx/html
- db_config_path: /usr/share/nginx/config

You can use an “attribute” style of defining variables as well:

vars:
  nginx:
    app_path: /usr/share/nginx/html
    db_config_path: /usr/share/nginx/config
tasks:
- name: copy file
  copy:
    src: /root/aa.html
    dest: "{{ nginx.app_path }}"
- name: copy file
  copy:
    src: /root/db.config
  dest: "{{ nginx['db_config_path'] }}"

When referencing variables:

If the separator between key and value is a colon, and the value starts with a variable, it needs to be enclosed in double quotation marks. If the variable doesn’t start at the beginning of the value, quotes are not necessary.
If the separator between key and value is an equal sign, quotes are not necessary regardless of whether the value starts with a variable.

For example, using the following syntax will result in an error:

- hosts: app_servers
  vars:
      app_path: {{ base_path }}/22

To avoid the error, you can use this syntax:

- hosts: app_servers
  vars:
       app_path: "{{ base_path }}/22"

vars_files

You can also define variables in file format using the vars_files keyword. For example:

---
- hosts: all
  vars_files:
  - /vars/external_vars.yml
  tasks:
  - name: test
    command: /bin/echo "{{ somevar }}"

The vars_files content should be in the format of key:value pairs:

---
somevar: somevalue
password: magic
ip: ['ip1','ip2']

include_vars

In playbooks, you can use the include_vars parameter to dynamically reference variables from a file.

Example:

$ cat test_var.yaml 
---
name: test_CI
version: v1

$ cat test_vars.yaml
---
- hosts: ansible
  gather_facts: no
  tasks:
  - include_vars:
      file: /root/test_var.yaml
  - debug:
      msg: "{{version}}"

# Execution result
$ ansible-playbook test_vars.yaml 
......
TASK [debug] ***************************************************************************************
ok: [192.168.177.43] => {
    "msg": "v1"
}
......

You can also assign all variables from the file to a variable name:

$ cat test_vars.yaml
---
- hosts: ansible
  gather_facts: no
  tasks:
  - include_vars:
      file: /root/test_var.yaml
      name: test_include_vars
  - debug:
      msg: "{{test_include_vars}}------>{{ test_include_vars.version }}"

Execution result:

TASK [debug] ***************************************************************************************
ok: [192.168.177.43] => {
    "msg": "{u'version': u'v1', u'name': u'test_CI'}------>v1"
}

Register Variables #

Another primary usage of variables is to run commands and register the output of those commands as register variables. This is also a commonly used way to use variables.

You can save the values of tasks executed in Ansible into variables, which can then be referenced by other tasks.

Here is a simple syntax example (showing the execution result of a task using the debug module):

$ cat template-debug.yaml

- hosts: ansible
  tasks:
  - name: exec command
    shell: hostname
    register: exec_result

  - name: Show debug info
    debug:
      msg: "{{ exec_result }}"

Execution result:

$ ansible-playbook template-debug.yaml

......

TASK [Show debug info] *****************************************************************************
ok: [192.168.177.43] => {
    "msg": {
        "changed": true, 
        "cmd": "hostname", 
        "delta": "0:00:00.115138", 
        "end": "2020-02-26 15:51:54.825096", 
        "failed": false, 
        "rc": 0, 
        "start": "2020-02-26 15:51:54.709958", 
        "stderr": "", 
        "stderr_lines": [], 
        "stdout": "ansible", 
        "stdout_lines": [
            "ansible"
        ]
    }
}

PLAY RECAP *****************************************************************************************
192.168.177.43             : ok=3    changed=1    unreachable=0    failed=0

In this example:

The register parameter assigns the execution result to the variable exec_result, which is then referenced by the debug module.

Based on the result, it can be seen that the execution result of this task returns a dictionary. The value of the variable is a dictionary, and the value of the stdout key in the dictionary is the actual execution result of the command. The keys such as changed, cmd, stderr, and stdout_lines are additional keys for the execution result of the task. Therefore, specific values can be obtained using dict[‘key’]. For example:

- name: Show debug info
  debug:
    msg: "{{ exec_result.stdout }}"

Execution result:

TASK [Show debug info] *****************************************************************************
ok: [192.168.177.43] => {
    "msg": "ansible"
}

The rc represents the return code of the task execution, where 0 indicates success.

If the value of stdout is a list (or already a list), it can be used in a task loop.

- name: Loop through a list
  hosts: all
  tasks:
    - name: List all files and directories in a directory
      command: ls /home
      register: home_dirs

    - name: Add a symbolic link
      file:
        path: /mnt/bkspool/{{ item }}
        src: /home/{{ item }}
        state: link
      loop: "{{ home_dirs.stdout_lines }}"

In the above example, the files in the /home directory are listed and stored in the home_dirs variable. The list is then looped through for further tasks.

Defining Variables in Command Line #

In addition to the variable definition methods mentioned above, you can also pass variables while executing the playbook using the -e parameter. For example:

$ cat release.yml
---
- hosts: ansible
  gather_facts: no
  tasks:
  - name: test
    debug:
      msg: "version: {{ version }}, other: {{ other_variable }}"

# Execution result
$ ansible-playbook release.yml --extra-vars "version=1.23.45 other_variable=foo"

.....
TASK [test] ****************************************************************************************
ok: [192.168.177.43] => {
    "msg": "version: 1.23.45, other: foo"
}

Or using the JSON method:

$ ansible-playbook release.yml --extra-vars '{"version":"1.23.45","other_variable":"foo"}'

$ ansible-playbook arcade.yml --extra-vars '{"pacman":"mrs","ghosts":["inky","pinky","clyde","sue"]}'

Using JSON file:

ansible-playbook release.yml --extra-vars "@some_file.json"

This is a simple introduction to the usage of variables.

Ansible Conditional Statements #

In most programming languages, conditional expressions are commonly used with if/else statements. In Ansible, conditional expressions are mainly used in template templates. In this section, we will focus on the commonly used conditional statement “when” in playbooks.

when #

First, let’s demonstrate how to use “when” with a simple example:

Loop through a list and print the number when num is greater than 5:

---
- hosts: ansible
  gather_facts: false
  tasks:
  - name: tes
    debug: 
      msg: " {{ item }}"
    with_items: [0,2,4,6,8,10]
    when: item > 5

Explanation:

with_items is used to loop through the list.
The result of the execution is as follows:

$ ansible-playbook test-when.yaml 
.....
ok: [192.168.177.43] => (item=6) => {
    "msg": " 6"
}
ok: [192.168.177.43] => (item=8) => {
    "msg": " 8"
}
ok: [192.168.177.43] => (item=10) => {
    "msg": " 10"
}
......

In the above example, the “>” comparison operator is used, which in Ansible can be expressed as follows:

==: equals
!=: not equals
>: greater than
<: less than
>=: greater than or equal to
<=: less than or equal to

In addition to these comparison operators, Ansible also supports logical operators:

and: and
or: or
not: not, also known as negation
(): grouping, can group a series of conditions together

In addition to standalone usage, the “when” statement is often used in combination with registered variables (register). For example:

You can pass the result of a task to the variable specified by register and use the “when” statement for conditional checks, as shown in the following example:

tasks:
  - command: /bin/false
    register: result
    ignore_errors: True

  - command: /bin/something
    when: result is failed

  - command: /bin/something_else
    when: result is succeeded

  - command: /bin/still/something_else
    when: result is skipped

Explanation:

The ignore_errors parameter specifies that if the command fails (e.g., command does not exist), ignore the error.
The three commands following it check the value of the result variable and determine whether to execute the current task based on the result.

The above example only demonstrates the syntax of using “when” in combination with register. Let’s now demonstrate with actual examples:

Example 1:

You can use the stdout value to access the string output of the register variable.

---
- hosts: ansible
  gather_facts: false
  tasks:
  - shell: cat /etc/motd
    register: motd_contents

  - shell: echo "motd contains the word hi"
    when: motd_contents.stdout.find('hi') != -1

Explanation:

As mentioned in the previous section, the register variable is a dictionary with multiple keys. In this example, we use the stdout key, which contains the actual output of the executed command. The find method is then used to search for the specified value, and the “when” statement is used for conditional checks.

Example 2:

List the string contents of the register variable.

For example, check if the register variable containing the string is empty:

- name: check
  hosts: all
  tasks:
      - name: list contents of directory
        command: ls mydir
        register: contents

      - name: check_wether_empty
        debug:
          msg: "Directory is empty"
        when: contents.stdout == ""

Or check if a branch exists:

- command: chdir=/path/to/project  git branch
  register: git_branches

- command: /path/to/project/git-clean.sh
  when: "('release-v2'  in git_branches.stdout)"

“When” can also be used in conjunction with other variables set within the playbook, as shown in the following example:

vars:
  epic: true

tasks:
    - shell: echo "This certainly is epic!"
      when: epic

tasks:
    - shell: echo "This certainly isn't epic!"
      when: not epic

tasks:
    - shell: echo "I've got '{{ foo }}'"
      when: foo is defined

    - fail: msg="Bailing out. this play requires 'bar'"
      when: bar is undefined

Here, defined and undefined are used to check whether the variable is defined.

The exists keyword is similar to the Linux command “test -e” for checking if a path or file exists on the Ansible control node.

For example, to check if the homed directory of the Ansible host exists, you can use the following:

---
- hosts: ansible
  gather_facts: no
  vars:
    path: /homed
  tasks:
  - debug:
      msg: "file exists"
    when: path is exists

Similarly, you can use is not exists to check if a directory does not exist. This comparison is straightforward and will not be demonstrated here.

What if you want to check if a remote host directory exists? You can use the result of a command as a condition. For example:

- name: Check if directory exists
  shell: ls {{ dest_dict }}
  register: dict_exist
  ignore_errors: true

- name: Create relevant directory
  file: dest="{{ item }}" state=directory  mode=755
  with_items:
    - "{{ dest_dict }}"
  when: dict_exist is failure

defined/undefind/none

defined and undefined are used to check if a variable is defined, while none is used to check if a variable value is empty. If the value is empty, it returns true.

For example:

---
- hosts: ansible
  gather_facts: no
  vars:
    name: "web"
    version: ""
  tasks:
  - debug:
      msg: "Variable is defined"
    when: name is defined
  - debug:
      msg: "Variable is not defined"
    when: version1 is undefined
  - debug:
      msg: "Variable is defined but empty"
    when: version is none

This is just a brief explanation. Feel free to try it out yourself if you are interested. This concludes the explanation about conditional statements.

Ansible Loops #

In daily work, you may need to copy multiple files to remote hosts, execute multiple start service commands in sequence, or iterate through the execution result of a task, etc. In Ansible, you can use loops to accomplish these tasks.

loop #

A simple loop, such as creating multiple users:

The usual way to write it is like this:

- name: add user testuser1
  user:
    name: "testuser1"
    state: present
    groups: "wheel"

- name: add user testuser2
  user:
    name: "testuser2"
    state: present
    groups: "wheel"

But if you use a loop, it can also be done like this:

- name: add several users
  user:
    name: "{{ item }}"
    state: present
    groups: "wheel"
  loop:
     - testuser1
     - testuser2

If you have defined a list in a YAML file or vars, it can be like this:

loop: "{{ somelist }}"

To make it more interesting, if the type of items being iterated is not a simple string list, for example, a dictionary, it can be like this:

- name: add several users
  user:
    name: "{{ item.name }}"
    state: present
    groups: "{{ item.groups }}"
  loop:
    - { name: 'testuser1', groups: 'wheel' }
    - { name: 'testuser2', groups: 'root' }

register with loop #

As mentioned earlier, when register is used together with loop, the data structure placed in the variable will contain a results attribute, which is a list of execution results from the module.

For example:

- shell: "echo {{ item }}"
  loop:
    - "one"
    - "two"
  register: res

Afterwards, the execution results will be stored in the variable:

- shell: echo "{{ item }}"
  loop:
    - one
    - two
  register: res
  changed_when: res.stdout != "one"

You can also loop through the results of a task execution using loop:

- name: list dirs
  shell: ls /dir
  register: list_dict

- name: test loop
  debug:
    msg: "{{ item }}"
  loop: list_dict.stdout_lines

with_x and loop #

With the release of Ansible 2.5, it is recommended to use the loop keyword instead of the with_X (which is not a specific keyword, but rather a collection of keywords) style of loops.

In many cases, using loop with filters provides a better syntax than using query for more complex use cases involving lookup.

The following examples will demonstrate the conversion of many common with_ style loops to loop and filters.

with_list

With with_list, each element nested in the main list is treated as a whole and stored in the item variable. It does not flatten nested lists, but instead iterates over each item in the (outermost) list.

- name: with_list
  debug:
    msg: "{{ item }}"
  with_list:
    - one
    - two
    - [b, c]

- name: with_list -> loop
  debug:
    msg: "{{ item }}"
  loop:
    - one
    - two

- name: with_list -> loop
  vars:
    testlist:
    - a
    - [b, c]
    - d
  tasks:
  - debug:
      msg: "{{ item }}"
    loop: "{{ testlist }}"

with_items

With with_items, nested lists are flattened. If there are smaller lists within the main list, they are “expanded” and the elements inside the lists are looped over and output.

The following example copies the four txt files (a.txt, b.txt, c.txt, d.txt) and the shell directory to the destination folder.

- name: copy file
  copy: 
    src: /root/{{ item }} 
    dest: /root/test/{{ item }}
  with_items:
    - a.txt
    - b.txt
    - [c.txt, d.txt]
    - shell

  Or it can be written like this:
  
- name: copy file
  copy: 
    src: /root/{{ item }} 
    dest: /root/test/{{ item }}
  with_items: [a.txt, b.txt, shell]

with_items can also be used to loop over dictionary variables:

- name: add users
  user:
    name: {{ item }}
    state: present
    groups: wheel
  with_items:
  - testuser1
  - testuser2

- name: add users
  user:
    name: {{ item.name }}
    state: present
    groups: {{ item.groups }}
  with_items:
  - { name: testuser1, groups: test1 }
  - { name: testuser2, groups: test2 }

The above examples can be replaced using loop and the flatten filter:

- name: with_items
  debug:
    msg: "{{ item }}"
  with_items: "{{ items }}"

- name: with_items -> loop
  debug:
    msg: "{{ item }}"
  loop: "{{ items|flatten(levels=1) }}"

- name: with_item_list
  vars:
    testlist:
    - a
    - [b, c]
    - d
  tasks:
  - debug:
      msg: "{{ item }}"
    loop: "{{ testlist | flatten(levels=1) }}"

The flatten filter can replace with_flattened and will flatten all levels of nested lists when dealing with multi-layered nested lists.

with_indexed_items

When iterating over a list, with_indexed_items adds an index to each element.

- name: with_indexed_items
  debug:
    msg: "{{ item.0 }} - {{ item.1 }}"
  with_indexed_items: "{{ items }}"

The following example prints the index and corresponding value of each element in the list:

$ cat test_with_index.yaml 
---
- hosts: ansible
  tasks:
  - name: with_indexed_items
    debug:
      msg: "{{ item.0 }} - {{ item.1 }}"
    with_indexed_items:
    - ['a', 'b', 'c', 'd']

# Output
ok: [192.168.177.43] => (item=[0, u'a']) => {
    "msg": "0 - a"
}
ok: [192.168.177.43] => (item=[1, u'b']) => {
    "msg": "1 - b"
}
ok: [192.168.177.43] => (item=[2, u'c']) => {
    "msg": "2 - c"
}
ok: [192.168.177.43] => (item=[3, u'd']) => {
    "msg": "3 - d"
}

with_indexed_items can be replaced with the loop keyword, the flatten filter (with arguments), and the loop_control keyword. When dealing with multi-layered nested lists, only the first layer of the list will be flattened:

- name: with_indexed_items -> loop
  debug:
    msg: "{{ index }} - {{ item }}"
  loop: "{{ items|flatten(levels=1) }}"
  loop_control:
    index_var: index

Note:

The keyword “loop_control” can be used to control the behavior of loops, such as obtaining the index of elements during the loop.
“index_var” is an option for loop_control that specifies a variable where the loop_control will store the index value. When the loop reaches this index value, the corresponding value can be obtained based on this index.

with_flattened

The effect of “with_flattened” is the same as using “with_items”.

Replace “with_flattened” with “loop” and “flatten” filters.

- name: with_flattened
  debug:
    msg: "{{ item }}"
  with_flattened: "{{ items }}"

- name: with_flattened -> loop
  debug:
    msg: "{{ item }}"
  loop: "{{ items|flatten }}"

---
- name: test_with_flattened
  vars:
    testlist:
    - a
    - [b,c]
    - d
  tasks:
  - debug:
      msg: "{{item}}"
    loop: "{{testlist | flatten}}"

with_together

This keyword combines multiple lists. If the lengths of the lists are different, the longest list length is used for alignment. When the elements in the short list are not enough, they are aligned with the none value in the long list. Similar to the zip function in Python.

Example:

---
- hosts: ansible
  tasks:
  - name: with_together
    debug:
      msg: "{{ item.0 }} - {{ item.1 }}"
    with_together:
    - ['a','b','c','d']
    - [1,2,3,4,5,6,7]

Execution result:

ok: [192.168.177.43] => (item=[u'a', 1]) => {
    "msg": "a - 1"
}
ok: [192.168.177.43] => (item=[u'b', 2]) => {
    "msg": "b - 2"
}
ok: [192.168.177.43] => (item=[u'c', 3]) => {
    "msg": "c - 3"
}
ok: [192.168.177.43] => (item=[u'd', 4]) => {
    "msg": "d - 4"
}
ok: [192.168.177.43] => (item=[None, 5]) => {
    "msg": " - 5"
}
ok: [192.168.177.43] => (item=[None, 6]) => {
    "msg": " - 6"
}
ok: [192.168.177.43] => (item=[None, 7]) => {
    "msg": " - 7"
}

Replace “with_together” with “loop” and “zip” filter.

- name: with_together -> loop
  debug:
    msg: "{{ item.0 }} - {{ item.1 }}"
  loop: "{{ list_one|zip(list_two)|list }}"

zip combines the lists based on the length of the shortest list.

With_dict

Loop through a dictionary, for example:

users:
  alice:
    name: Alice Appleworth
    telephone: 123-456-7890
  bob:
    name: Bob Bananarama
    telephone: 987-654-3210

tasks:
  - name: Print records
    debug: msg="User {{ item.key }} is {{ item.value.name }} ({{ item.value.telephone }})"
    with_dict: "{{ users }}"

with_dict can be replaced by loop and either dictsort or dict2items filter.

- name: with_dict
  debug:
    msg: "{{ item.key }} - {{ item.value }}"
  with_dict: "{{ dictionary }}"

- name: with_dict -> loop (option 1)
  debug:
    msg: "{{ item.key }} - {{ item.value }}"
  loop: "{{ dictionary|dict2items }}"

- name: with_dict -> loop (option 2)
  debug:
    msg: "{{ item.0 }} - {{ item.1 }}"
  loop: "{{ dictionary|dictsort }}"

- name: with_dict -> loop (option 2)
  debug:
    msg: "{{ item.0 }} - {{ item.1 }}"
  loop: "{{  lookup('dict',dict_name) }}"

with_random_choice

“with_random_choice” can be replaced by the “random” filter without using “loop”.

- name: with_random_choice
  debug:
    msg: "{{ item }}"
  with_random_choice: "{{ my_list }}"

- name: with_random_choice -> loop (No loop is needed here)
  debug:
    msg: "{{ my_list|random }}"
  tags: random

That’s the content about loops. The examples listed above are commonly used in work. There are many ways to implement a requirement, so choose the most appropriate one.

Ansible Template #

When writing playbooks with Ansible, it is common to replace the contents of configuration files. The template module is the best tool for this task as it uses Jinja2 templates to enable dynamic expressions and access to variables. Templates are referenced using files (roles/templates/xxx.j2) and allow for concatenation and replacement of configuration file contents.

Jinja2 is a template engine based on Python. Compared to the original Jinja2 templates in Python, Ansible allows for more concise and efficient deployment code by using filters, plugins, and variable testing.

The usage of the template module is similar to that of the copy module. In this section, we will mainly discuss the usage of templates and common basic syntax in Jinja2.

Syntax for using the template module #

As mentioned before, the usage of the template module is similar to the copy module, so the syntax is also quite similar:

ansible <group_name> -m template -a "src=<src_path>.j2 dest=<dest_path> option=..."

The option parameter represents other arguments, with some commonly used ones listed below:

backup: Creates a backup of the original file before overwriting it, with the backup file including a timestamp.
follow: Specifies whether to follow symlinks in the destination machine’s file system.
force: Specifies whether to force execution.
group: Sets the group ownership of the file/directory.
mode: Sets the file permissions. The mode is actually an octal number (e.g., 0644). Omitting the leading zero may lead to unexpected results. Starting from version 1.8, it is also possible to specify the mode in symbolic notation (e.g., u+rwx or u=rw,g=r,o=r).
newline_sequence (2.4+): Specifies the newline character used in the template file.
owner: Sets the user ownership of the file/directory.
src: Specifies the file location of the Jinja2 formatted template (required).
trim_blocks: If set to True, removes the first newline character after a block.
unsafe_writes: Specifies whether to operate in an unsafe manner, which may result in data corruption.
validate: Specifies whether to validate the destination path before copying.
dest: Specifies the destination address where the Jinja2 formatted template file should be copied (required). …

Here are some commonly used examples:

# After filling in the parameters for the nginx.conf.j2 file, copy it to the nginx configuration directory on the remote node
$ ansible web -m template -a "src=/root/nginx.conf.j2 dest=/usr/share/nginx/conf/nginx.conf owner=nginx group=nginx mode=0644"

# After filling in the parameters for the sshd_config.j2 file, copy it to the ssh configuration directory on the remote node
$ ansible web -m template -a "src=/etc/ssh/sshd_config.j2 dest=/etc/ssh/sshd_config owner=root group=root mode=0600 backup=yes"

## Using Variables in Jinja2

After defining variables, you can use them in playbooks using the Jinja2 template system. Variables need to be enclosed in double curly braces `{{ }}`. In addition to variables, Jinja2 supports other expressions:

- `{{ }}`: used to load variables, but can also be used to load other types of Python data such as lists, strings, expressions, or Ansible filters
- `{% %}`: used to load control statements like loops (e.g., for loop) and conditionals (e.g., if statement)
- `{# #}`: used to load comments, which will not be included in the rendered file

Below is a simple Jinja2 template:

My amp goes to {{ max_amp_value }}


This expression demonstrates the most basic form of variable substitution.

The same syntax can be used in playbooks. For example:

```yaml
template: src=foo.cfg.j2 dest={{ remote_install_path }}/foo.cfg

$ cat foo.cfg.j2
{% set test_list=['test','test1','test2'] %}
{% for i in list %}
    {{i}}
{% endfor %}

Inside the template, you can automatically access all variables within the host scope.

Jinja2 Syntax #

Jinja2’s syntax consists of variables and statements, as shown below:

Variables: Used to output data
```
{{ my_variable }}
```
{{ some_dudes_name | capitalize }}
Statements: Used to create conditions and loops, etc.
```
# Simple if else elif endif
```
{% if my_conditional %} … {% elif my_conditionalN %} … {% else %} {% endif %}

For loop #

{% for item in all_items %} {{ item }} {% endfor %}

{# COMMENT #} represents a comment

From the second example of variables, it can be seen that Jinja2 supports using Unix-like pipeline operators with filters, which can also be called filters. Ansible has many built-in filters available.

Here is an example to further explain the syntax:

$ cat /root/template-test.yaml 
---
- hosts: ansible
  gather_facts: no
  tasks:
  - template:
      src: /tmp/ansible/test.j2
      dest: /opt/test
    vars:
      test_list: ['test','test1','test2']
      test_list1: ['one','two','three']
      test_dict: {'key1':'value1'}
      test_dict1: {'key1':{'key2':'value2'}}

$ cat /tmp/ansible/test.j2 

{% for i in test_list  %}
    {{i}}
{% endfor %}

{% for i in test_list1 %} 
    {% if i=='one' %}
value1 --> {{i}}
    {% elif loop.index == 2 %}
value2 -->{{i}}
    {% else %}
value3 -->{{i}}
    {% endif %}
{% endfor %}

{% for key,value in test_dict.iteritems() %}
-------------->{{value}}
{% endfor %}

{{ test_dict1['key1']['key2'] }}

$ ansible-playbook template-test.yaml
or execute with an ad-hoc command
$ ansible ansible -m template -a "src=test.j2 dest=/opt/test"

Execution result:

$ cat /opt/test

    test
    test1
    test2

    value1 --> one
    
    value2 -->two
    
    value3 -->three
    
--------------->value1

value2

In the above example, for simplicity, all the variables are placed in the playbook’s YAML file. However, these variables can also be placed in the template file, like this:

{% set test_list=['test','test1','test2'] %}
{% for i in test_list  %}
    {{i}}
{% endfor %}

The effect is the same as the previous example.

Remove line breaks #

Upon observing the above results, it is noticed that there is an automatic line break after each result. If you do not want line breaks, you can add a “-” before the closing control symbol “%}”, as shown below:

{% for i in test_list1 -%}
{{ i }}
{%- endfor %}

String Concatenation #

In jinja2, the tilde symbol “~” is used as a string concatenation operator. It converts all operands into strings and concatenates them.

{% for key,val in {'test1':'v1','num':18}.items() %}
{{ key ~ ':' ~ val }}
{% endfor %}

Escaping #

When operating on configuration files, it is inevitable to encounter special characters such as spaces, “#”, single or double quotation marks. If you want to preserve these special characters, you need to escape them.

For simple string processing, you can use single quotation marks “’” for escaping. For example:

$ cat /tmp/ansible/test.j2 

{% for i in [8,1,5,3,9] %}
        '{{ i }}'
        {{ '{# i #}' }}
        {{ '{{ i }}' }}
{% endfor %}

$ cat template-test.yaml
---
- hosts: ansible 
  gather_facts: no
  tasks:
  - name: copy
    template:
      src: /tmp/ansible/test.j2
      dest: /opt/test

The execution result is as follows:

    '8'
    {# i #}
    {{ i }}
    '1'
    {# i #}
    {{ i }}
    '5'
    {# i #}
    {{ i }}
    ......

Jinja2 can use the “{% raw %}” block to keep everything between “{% raw %}” and “{% endraw %}” from being parsed by Jinja2, such as:

# cat test.j2

{% raw %}
    {% for i in [8,1,5,3,9] %}

    {% {{ i }} %}
    {# {{ i }} #}
    {% {{ i }} %}
  {% endfor %}
{% endraw %}

The execution result is:

  {% for i in [8,1,5,3,9] %}

    {% {{ i }} %}
    {# {{ i }} #}
    {% {{ i }} %}
  {% endfor %}

That’s all for the content related to Ansible. The above content introduces some knowledge points that should be frequently used when using Ansible. They are enough for us to write scripts for continuous deployment. In the next section, we will see how to use Ansible to write playbook scripts for continuous deployment operations.

07Integration of Jenkins with Ansible Key Points #

Ansible Inventory #

Hosts and Groups #

Default Groups #

Port #

Aliases #

Matching #

Ansible Variables #

Defining and Using Variables in Playbooks #

Register Variables #

Defining Variables in Command Line #

Ansible Conditional Statements #

when #

Ansible Loops #

loop #

register with loop #

with_x and loop #

Ansible Template #

Syntax for using the template module #

Jinja2 Syntax #

For loop #

Remove line breaks #

String Concatenation #

Escaping #