03 Project Deployment How to Quickly Deploy the Iam System

03 Project Deployment How to Quickly Deploy the IAM System #

Hello, I’m Kong Lingfei.

In the previous lesson, we installed and configured a basic Go development environment together. In this lesson, I will teach you how to quickly deploy an IAM system based on it.

Because we are going to develop an enterprise-level Go project through an IAM project, it is important to have a good understanding of the IAM project. The most direct and effective way to understand an IAM project is to deploy and use it.

This not only helps you understand the relationship between the components in the IAM system and deepens your understanding of the IAM system, but also helps you troubleshoot, especially with deployment-related issues. In addition, deploying the IAM system will prepare you with an experimental environment for your future learning, which helps improve your learning efficiency.

So, today we will spend this lesson specifically on how to deploy and use the IAM system. And because the IAM system is an enterprise-level project with a certain level of complexity, it is recommended that you strictly follow the steps I mention, otherwise the installation may fail.

Overall, I divided the deployment process into 2 major steps.

  1. Installation and configuration of the database: We need to install and configure MariaDB, Redis, and MongoDB.
  2. Installation and configuration of IAM services: We need to install and configure iam-apiserver, iam-authz-server, iam-pump, iamctl, and man page.

Of course, if you really don’t want to go through the hassle of installation, I will also provide a one-click deployment method at the end of this lesson. However, I still hope that you can follow the detailed steps I mentioned today.

Without further ado, let’s get started!

Download IAM Project Code #

Because the installation script of IAM is located in the IAM code repository, and the binary files required for installation also need to be built through IAM code, we need to download the IAM code before installation:

$ mkdir -p $WORKSPACE/golang/src/github.com/marmotedu
$ cd $WORKSPACE/golang/src/github.com/marmotedu
$ git clone --depth=1 https://github.com/marmotedu/iam
$ go work use ./iam

In the above commands, the directories marmotedu and marmotedu/iam store the code of this practical project. During the learning process, you will need to frequently access these two directories. In order to facilitate access, we can append the following two environment variables and two aliases to the $HOME/.bashrc file:

$ tee -a $HOME/.bashrc << 'EOF'
# Alias for quick access
export GOSRC="$WORKSPACE/golang/src"
export IAM_ROOT="$GOSRC/github.com/marmotedu/iam"
alias mm="cd $GOSRC/github.com/marmotedu"
alias i="cd $GOSRC/github.com/marmotedu/iam"
EOF
$ bash

Afterwards, you can access the directory $GOSRC/github.com/marmotedu by executing the alias command mm, and access the directory $GOSRC/github.com/marmotedu/iam by executing the alias command i.

Here, I also recommend that you make good use of aliases to configure commonly used operations for convenience in future operations.

Before installing and configuring, you need to execute the following command to export the password of the going user. Here, let’s assume the password is iam59!z$:

export LINUX_PASSWORD='iam59!z$'

Installing and Configuring Databases #

Since the IAM system uses MariaDB, Redis, and MongoDB databases to store data, the IAM service will attempt to connect to these databases when it starts. To avoid connection failures during startup, let’s first install the required databases.

Installing and Configuring MariaDB #

IAM stores the definition information of REST resources in a relational database, and I have chosen MariaDB as the relational database. Why MariaDB and not MySQL? One reason is that MariaDB is the fastest-growing branch of MySQL. Compared to MySQL, MariaDB has added many new features, and it is fully compatible with MySQL, including APIs and command lines. The other reason is that MariaDB is open source and has fast iteration speed.

First, we can install and configure MariaDB with the following commands, and set the root password to iam59!z$:

$ cd $IAM_ROOT
$ ./scripts/install/mariadb.sh iam::mariadb::install

Then, we can test if MariaDB is successfully installed with the following command:

$ mysql -h127.0.0.1 -uroot -p'iam59!z$'
MariaDB [(none)]>

Installing and Configuring Redis #

In the IAM system, since the iam-authz-server pulls and caches user key/policy information from iam-apiserver, the same set of key/policy data will exist in both services, which may cause data inconsistency. Data inconsistency can lead to problems. For example, if we create a pair of keys through iam-apiserver, but these keys have not been cached by iam-authz-server, accessing iam-authz-server with these keys will fail.

To ensure data consistency, we can use Redis’ publish-subscribe (pub/sub) feature for message notification. In addition, iam-authz-server also caches authorization audit logs in Redis, so we need to install the Redis key-value database as well. We can install and configure Redis with the following commands, and set the initial password for Redis to iam59!z$:

$ cd $IAM_ROOT
$ ./scripts/install/redis.sh iam::redis::install

Here, we need to pay attention that the iam::redis::install function in the scripts/install/redis.sh script configures Redis by running it as a daemon, modifying the password of Redis to iam59!z$, etc. For detailed configuration, please refer to the iam::redis::install function.

After the installation is complete, we can test if Redis is successfully installed with the following command:

$ redis-cli -h 127.0.0.1 -p 6379 -a 'iam59!z$' # Connect to Redis, -h specifies the host, -p specifies the listening port, -a specifies the login password

Installing and Configuring MongoDB #

Since iam-pump will process the data generated by iam-authz-server and store it in MongoDB, we also need to install the MongoDB database. The installation process consists of two steps: first, installing MongoDB, and then creating a MongoDB account.

Step 1: Installing MongoDB #

First, we can install MongoDB through the following four steps.

  1. Configure MongoDB YUM repository and install MongoDB.

CentOS 8.x does not come with the YUM repository for installing MongoDB, so we need to configure the YUM repository first before installing:

$ sudo tee /etc/yum.repos.d/mongodb-org-5.0.repo<<'EOF'
[mongodb-org-5.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/5.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-5.0.asc
EOF
 
$ sudo yum install -y mongodb-org
  1. Disable SELinux.

During the installation process, SELinux may prevent MongoDB from accessing /sys/fs/cgroup, so we also need to disable SELinux:

$ sudo setenforce 0
$ sudo sed -i 's/^SELINUX=.*$/SELINUX=disabled/' /etc/selinux/config # Disable SELINUX permanently
  1. Enable external network access and login authentication. After installing MongoDB, by default, external access permission and login verification are not enabled. In order to facilitate usage, I recommend enabling these features. Execute the following commands to enable them:
$ sudo sed -i '/bindIp/{s/127.0.0.1/0.0.0.0/}' /etc/mongod.conf
$ sudo sed -i '/^#security/a\security:\n  authorization: enabled' /etc/mongod.conf
  1. Start MongoDB.

After configuring MongoDB, we can start it. The specific commands are as follows:

$ sudo systemctl start mongod
$ sudo systemctl enable mongod # Set to start on boot
$ sudo systemctl status mongod # Check the running status of mongod. If the output contains "active (running)", it means mongod has been successfully started

After installing MongoDB, we can log in to the MongoDB Shell using the mongo command. If there are no errors, it means MongoDB has been successfully installed.

$ mongosh --quiet "mongodb://127.0.0.1:27017"
test>

Step 2: Create a MongoDB Account #

After installing MongoDB, there are no user accounts by default. In order to facilitate the use of IAM services, we need to create an administrator account first. By logging in to MongoDB with the administrator account, we can perform operations such as creating ordinary users and databases.

  1. Create an administrator account.

First, we use the use admin command to switch to the admin database. Then, we use the db.auth("username","password") command to verify the user’s login permissions. If the return value is 1, it means the verification is successful; if the return value is 0, it means the verification has failed. The specific commands are as follows:

$ mongosh --quiet "mongodb://127.0.0.1:27017"
test> use admin
switched to db admin
admin> db.createUser({user:"root",pwd:"iam59!z$",roles:["root"]})
{ ok: 1 }
admin> db.auth("root", "iam59!z$")
{ ok: 1 }

In addition, if you want to delete a user, you can use the db.dropUser("username") command.

db.createUser uses the following 3 parameters:

  • user: the username
  • pwd: the user password
  • roles: used to set the user’s permissions, such as read, read-write, write, etc.

Because the admin user has MongoDB’s Root permissions, the security will be compromised due to excessive permissions. In order to improve security, we also need to create an ordinary IAM user to connect and operate MongoDB.

  1. Create the IAM user using the following command:
$ mongosh --quiet mongodb://root:'iam59!z$'@127.0.0.1:27017/iam_analytics?authSource=admin # Connect to MongoDB using the administrator account
iam_analytics> db.createUser({user:"iam",pwd:"iam59!z$",roles:["dbOwner"]})
{ ok: 1 }
iam_analytics> db.auth("iam", "iam59!z$")
{ ok: 1 }

After creating the iam ordinary user, we can log in to MongoDB using the iam user:

$ mongosh --quiet mongodb://iam:'iam59!z$'@127.0.0.1:27017/iam_analytics?authSource=iam_analytics

With this, we have successfully installed the necessary databases MariaDB, Redis, and MongoDB for the IAM system.

Installation and Configuration of IAM System #

To complete the installation of the IAM system, we also need to install and configure iam-apiserver, iam-authz-server, iam-pump, and iamctl. The functions of these components were discussed in detail in Lesson 1. If you don’t remember, you can go back and take a look.

Note: I will maintain and update the IAM project on a regular basis. Feel free to Star & Contributing.

Preparation #

Before starting the installation, we need to do some preparation work, which mainly consists of 5 steps.

  1. Initialize the MariaDB database and create the iam database.
  2. Configure the scripts/install/environment.sh file.
  3. Create the required directories.
  4. Create the CA root certificate and key.
  5. Configure the hosts file.

Step 1: Initialize the MariaDB database and create the iam database.

After installing the MariaDB database, we need to create the database, tables, and stored procedures required by the IAM system in the MariaDB database. The SQL statements for creating them are stored in the configs/iam.sql file in the IAM code repository. The specific steps are as follows.

  1. Log in to the database and create the iam user.
$ cd $IAM_ROOT
$ mysql -h127.0.0.1 -P3306 -uroot -p'iam59!z$' # Connect to MariaDB, -h specifies the host, -P specifies the listening port, -u specifies the login user, -p specifies the login password
MariaDB [(none)]> grant all on iam.* TO [[email protected]](/cdn-cgi/l/email-protection) identified by 'iam59!z$';
Query OK, 0 rows affected (0.000 sec)
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.000 sec)
  1. Log in to MariaDB with the iam user and execute the iam.sql file to create the iam database.
$ mysql -h127.0.0.1 -P3306 -uiam -p'iam59!z$'
MariaDB [(none)]> source configs/iam.sql;
MariaDB [iam]> show databases;
+--------------------+
| Database           |
+--------------------+
| iam                |
| information_schema |
+--------------------+
2 rows in set (0.000 sec)

The above commands will create the iam database and the following database resources.

  • Tables: user is the user table used to store user information; secret is the secret table used to store secret information; policy is the policy table used to store authorization policy information; policy_audit is the policy history table, deleted policies will be archived to this table.
  • Admin User: In the user table, we need to create an admin user with the username admin and password Admin@2021.
  • Stored Procedures: When a user is deleted, the associated secret and policy information will be automatically deleted.

Step 2: Configure the scripts/install/environment.sh file.

The installation and configuration of IAM components are configured through the environment variable file scripts/install/environment.sh, so we need to configure the scripts/install/environment.sh file first. Here, you can use the default values to speed up your installation process.

Step 3: Create the required directories.

When installing and running the IAM system, we need to store the configuration, binary files, and data files in the specified directories. So we need to create these directories first. The creation steps are as follows.

$ cd $IAM_ROOT
$ source scripts/install/environment.sh
$ sudo mkdir -p ${IAM_DATA_DIR}/{iam-apiserver,iam-authz-server,iam-pump} # Create Systemd WorkingDirectory directory
$ sudo mkdir -p ${IAM_INSTALL_DIR}/bin # Create IAM system installation directory
$ sudo mkdir -p ${IAM_CONFIG_DIR}/cert # Create IAM system configuration file storage directory
$ sudo mkdir -p ${IAM_LOG_DIR} # Create IAM log file storage directory

Step 4: Creating the CA Root Certificate and Key.

To ensure security, the IAM system components need to use x509 certificates to encrypt and authenticate communications. Therefore, we need to create the CA certificate first. The CA root certificate is shared by all components, so we only need to create one CA certificate, and all subsequent certificates will be signed by it.

We can use CloudFlare’s PKI toolset cfssl to create all the certificates.

  1. Install the cfssl toolset.

We can directly install the pre-compiled binary files of cfssl. The cfssl toolset includes many tools. Here, we need to install cfssl, cfssljson, and cfssl-certinfo, which have the following functions.

  • cfssl: Certificate signing tool.
  • cfssljson: Converts the certificate generated by cfssl (in JSON format) into a file-based certificate.
  • cfssl-certinfo: Displays the information of the certificate.

The installation method for these two tools is as follows.

$ cd $IAM_ROOT
$ ./scripts/install/install.shiam::install::install_cfssl
  1. Create the configuration file.

The CA configuration file is used to configure the usage scenarios (profile) and specific parameters (usage, expiration time, server authentication, client authentication, encryption, etc.) of the root certificate. It can be used to specify specific scenarios when signing other certificates.

$ cd $IAM_ROOT
$ tee ca-config.json << EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "iam": {
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ],
        "expiry": "876000h"
      }
    }
  }
}
EOF

In the above JSON configuration, some fields are explained as follows:

  • signing: Indicates that the certificate can be used to sign other certificates (the created ca.pem certificate has CA=TRUE).
  • server auth: Indicates that the client can use this certificate to verify the certificate provided by the server.
  • client auth: Indicates that the server can use this certificate to verify the certificate provided by the client.
  • expiry: 876000h, the certificate validity period is set to 100 years.
  1. Create the certificate signing request (CSR) file.

We create a JSON configuration file to generate the CSR for the CA certificate.

$ cd $IAM_ROOT
$ tee ca-csr.json << EOF
{
  "CN": "iam-ca",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "marmotedu",
      "OU": "iam"
    }
  ],
  "ca": {
    "expiry": "876000h"
  }
}
EOF

In the above JSON configuration, some fields are explained as follows.

  • C: Country.
  • ST: State.
  • L: Locality (L) or City.
  • CN: Common Name. iam-apiserver extracts this field from the certificate as the username for the request (User Name). Browsers use this field to verify the legitimacy of a website.
  • O: Organization. iam-apiserver extracts this field from the certificate as the group that the request user belongs to (Group).
  • OU: Company division (or Organization Unit - OU).

In addition, there are two points we need to pay attention to:

  • The combinations of CN, C, ST, L, O, and OU in different certificate CSR files must be different, otherwise the “PEER’S CERTIFICATE HAS AN INVALID SIGNATURE” error may occur.
  • When creating CSR files for subsequent certificates, CN and OU should be different (C, ST, L, O are the same), in order to differentiate them.
  1. Create CA certificate and private key

First, we use the cfssl gencert command to create it:

$ cd $IAM_ROOT
$ source scripts/install/environment.sh
$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca
$ ls ca*
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem
$ sudo mv ca* ${IAM_CONFIG_DIR}/cert # Need to copy the certificate file to the specified folder (distribute the certificate) for easy reference by various components

The above command will create the files necessary for running the CA: ca-key.pem (private key), ca.pem (certificate), and ca.csr (certificate signing request) for cross-signing or re-signing.

After creating them, we can use the cfssl certinfo command to view the cert and csr information:

$ cfssl certinfo -cert ${IAM_CONFIG_DIR}/cert/ca.pem # View cert (certificate information)
$ cfssl certinfo -csr ${IAM_CONFIG_DIR}/cert/ca.csr # View CSR (certificate signing request) information

Step 5: Configure hosts

IAM accesses the API interface through domain names. Since these domain names have not been registered and cannot be resolved on the Internet, we need to configure the hosts. The specific steps are as follows:

$ sudo tee -a /etc/hosts <<EOF
127.0.0.1 iam.api.marmotedu.com
127.0.0.1 iam.authz.marmotedu.com
EOF

Install and configure iam-apiserver #

After completing the preparation work, we can now install the various components of the IAM system. First, we install the iam-apiserver service in 3 steps.

Step 1: Create iam-apiserver certificate and private key

Other services access iam-apiserver using the HTTPS protocol for security, so we need to create the iam-apiserver certificate and private key first.

  1. Create certificate signing request:
$ cd $IAM_ROOT
$ source scripts/install/environment.sh
$ tee iam-apiserver-csr.json <<EOF
{
  "CN": "iam-apiserver",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "marmotedu",
      "OU": "iam-apiserver"
    }
  ],
  "hosts": [
    "127.0.0.1",
    "localhost",
"iam.api.marmotedu.com"
]
}
EOF

The `hosts` field in the code is used to specify the list of IPs and domains that are authorized to use the certificate. The above `hosts` list specifies the IP and domain of the `iam-apiserver` service.

1. Generating the certificate and private key:

```bash
$ cfssl gencert -ca=${IAM_CONFIG_DIR}/cert/ca.pem \
  -ca-key=${IAM_CONFIG_DIR}/cert/ca-key.pem \
  -config=${IAM_CONFIG_DIR}/cert/ca-config.json \
  -profile=iam iam-apiserver-csr.json | cfssljson -bare iam-apiserver
$ sudo mv iam-apiserver*pem ${IAM_CONFIG_DIR}/cert

Move the generated certificate and private key files to the configuration directory.

Step 2: Install and run iam-apiserver.

iam-apiserver is the core component of the iam system and needs to be installed first.

  1. Install the iam-apiserver executable:
$ cd $IAM_ROOT
$ source scripts/install/environment.sh
$ make build BINS=iam-apiserver
$ sudo cp _output/platforms/linux/amd64/iam-apiserver ${IAM_INSTALL_DIR}/bin
  1. Generate and install the configuration file for iam-apiserver (iam-apiserver.yaml):
$ ./scripts/genconfig.sh scripts/install/environment.sh configs/iam-apiserver.yaml > iam-apiserver.yaml
$ sudo mv iam-apiserver.yaml ${IAM_CONFIG_DIR}
  1. Create and install the iam-apiserver systemd unit file:
$ ./scripts/genconfig.sh scripts/install/environment.sh init/iam-apiserver.service > iam-apiserver.service
$ sudo mv iam-apiserver.service /etc/systemd/system/
  1. Start the iam-apiserver service:
$ sudo systemctl daemon-reload
$ sudo systemctl enable iam-apiserver
$ sudo systemctl restart iam-apiserver
$ systemctl status iam-apiserver # Check the status of iam-apiserver, if the output contains the phrase active (running), iam-apiserver has been successfully started

Step 3: Test whether iam-apiserver is successfully installed.

Testing iam-apiserver mainly involves testing the CURD operations of the RESTful resources: user CURD, key CURD, and policy CURD.

First, we need to obtain the Token for accessing iam-apiserver. Use the following API request:

$ curl -s -XPOST -H'Content-Type: application/json' -d'{"username":"admin","password":"Admin@2021"}' http://127.0.0.1:8080/login | jq -r .token
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA

In the code below, replace <Token> with the Token obtained from the above request in the -H 'Authorization: Bearer <Token>' authentication header.

User CURD

Create a user, list users, get detailed information of a user, modify a user, delete a single user, delete multiple users using the following methods:

# Create user
$ curl -s -XPOST -H'Content-Type: application/json' -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' -d'{"password":"User@2021","metadata":{"name":"colin"},"nickname":"colin","email":"[[email protected]](/cdn-cgi/l/email-protection)","phone":"1812884xxxx"}' http://127.0.0.1:8080/v1/users

# List users

Please let me know if you need any further assistance. $ curl -s -XPOST -H’Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA’ ‘http://127.0.0.1:8080/v1/secrets’ -d’{“name”:“secret0”,“value”:“123456”}’

# 列出所有密钥
$ curl -s -XGET -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' 'http://127.0.0.1:8080/v1/secrets'

# 获取 secret0 密钥的详细信息
$ curl -s -XGET -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' 'http://127.0.0.1:8080/v1/secrets/secret0'

# 修改 secret0 密钥
$ curl -s -XPUT -H'Content-Type: application/json' -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' -d'{"name":"secret0","value":"654321"}' http://127.0.0.1:8080/v1/secrets/secret0

# 删除 secret0 密钥
$ curl -s -XDELETE -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' http://127.0.0.1:8080/v1/secrets/secret0

# 创建策略
$ curl -s -XPOST -H'Content-Type: application/json' -H'Authorization: Bearer <token>' -d'{"metadata":{"name":"policy0"},"permissions":["read","write"]}' http://127.0.0.1:8080/v1/policies

# 列出所有策略
$ curl -s -XGET -H'Authorization: Bearer <token>' http://127.0.0.1:8080/v1/policies

# 获取 policy0 策略的详细信息
$ curl -s -XGET -H'Authorization: Bearer <token>' http://127.0.0.1:8080/v1/policies/policy0

# 修改 policy0 策略
$ curl -s -XPUT -H'Content-Type: application/json' -H'Authorization: Bearer <token>' -d'{"metadata":{"name":"policy0"},"permissions":["read","write","execute"]}' http://127.0.0.1:8080/v1/policies/policy0

# 删除 policy0 策略
$ curl -s -XDELETE -H'Authorization: Bearer <token>' http://127.0.0.1:8080/v1/policies/policy0

注意： <token> 是一个有效的身份验证令牌，需要替换为实际的令牌值。

Create Policy # 

$ curl -s -XPOST -H'Content-Type: application/json' -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' -d'{"metadata":{"name":"policy0"},"policy":{"description":"One policy to rule them all.","subjects":["users:<peter|ken>","users:maria","groups:admins"],"actions":["delete","<create|update>"],"effect":"allow","resources":["resources:articles:<.*>","resources:printer"],"conditions":{"remoteIPAddress":{"type":"CIDRCondition","options":{"cidr":"192.168.0.1/16"}}}}}' http://127.0.0.1:8080/v1/policies

List all policies # 

$ curl -s -XGET -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' http://127.0.0.1:8080/v1/policies

Get detailed information of policy0 # 

$ curl -s -XGET -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' http://127.0.0.1:8080/v1/policies/policy0

Update policy0 # 

$ curl -s -XPUT -H'Content-Type: application/json' -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' -d'{"metadata":{"name":"policy0"},"policy":{"description":"One policy to rule them all(modified).","subjects":["users:<peter|ken>","users:maria","groups:admins"],"actions":["delete","<create|update>"],"effect":"allow","resources":["resources:articles:<.*>","resources:printer"],"conditions":{"remoteIPAddress":{"type":"CIDRCondition","options":{"cidr":"192.168.0.1/16"}}}}}' http://127.0.0.1:8080/v1/policies/policy0

Delete policy0 # 

$ curl -s -XDELETE -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXBpLm1hcm1vdGVkdS5jb20iLCJleHAiOjE2MTc5MjI4OTQsImlkZW50aXR5IjoiYWRtaW4iLCJpc3MiOiJpYW0tYXBpc2VydmVyIiwib3JpZ19pYXQiOjE2MTc4MzY0OTQsInN1YiI6ImFkbWluIn0.9qztVJseQ9XwqOFVUHNOtG96-KUovndz0SSr_QBsxAA' http://127.0.0.1:8080/v1/policies/policy0

Install iamctl #

Above, we installed the API service of the iam system. However, to access the iam service, we also need to install the iamctl client tool. Specifically, we can complete the installation and configuration of iamctl in 3 steps.

Step 1, create iamctl certificate and private key.

iamctl uses the HTTPS protocol to communicate securely with the iam-apiserver. The iam-apiserver authenticates and authorizes the certificate included in the iamctl request. Since iamctl is used for accessing and managing the iam system, we create the admin certificate with the highest permission.

  1. Create a certificate signing request.

The certificate created below will only be used as the client certificate by iamctl, so the hosts field is empty. The code is as follows:

$ cd $IAM_ROOT
$ source scripts/install/environment.sh
$ cat > admin-csr.json <<EOF
{
  "CN": "admin",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [

{
  "C": "CN",
  "ST": "Beijing",
  "L": "Beijing",
  "O": "marmotedu",
  "OU": "iamctl"
}
],
"hosts": []
}
EOF
  1. Generate the certificate and private key:
$ cfssl gencert \
  -ca=${IAM_CONFIG_DIR}/cert/ca.pem \
  -ca-key=${IAM_CONFIG_DIR}/cert/ca-key.pem \
  -config=${IAM_CONFIG_DIR}/cert/ca-config.json \
  -profile=iam admin-csr.json | cfssljson -bare admin
$ mkdir -p $(dirname ${CONFIG_USER_CLIENT_CERTIFICATE}) $(dirname ${CONFIG_USER_CLIENT_KEY}) # Create the directory to store the client certificates
$ mv admin.pem ${CONFIG_USER_CLIENT_CERTIFICATE} # Install the client certificate for TLS
$ mv admin-key.pem ${CONFIG_USER_CLIENT_KEY} # Install the client private key file for TLS

Step 2, install iamctl.

iamctl is the client tool for the IAM system, and its installation location is different from that of iam-apiserver, iam-authz-server, and iam-pump. In order to be able to run the iamctl command directly in the shell, we need to install iamctl under $HOME/bin and store iamctl’s configuration in the default loading directory: $HOME/.iam. It is done in two steps.

  1. Install the iamctl executable:
$ cd $IAM_ROOT
$ source scripts/install/environment.sh
$ make build BINS=iamctl
$ cp _output/platforms/linux/amd64/iamctl $HOME/bin
  1. Generate and install iamctl’s configuration file (iamctl.yaml):
$ ./scripts/genconfig.sh scripts/install/environment.sh configs/iamctl.yaml> iamctl.yaml
$ mkdir -p $HOME/.iam
$ mv iamctl.yaml $HOME/.iam

Since iamctl is a client tool that may run on multiple machines, we can simplify the complexity of deploying the iamctl tool by encrypting the CA file content related to CA authentication in the config configuration file using base64 and placing it in the config configuration file. The specific idea is to replace the configuration items client-certificate, client-key, and certificate-authority in the config file with the configuration items client-certificate-data, client-key-data, and certificate-authority-data, respectively. The values of these configuration items can be obtained by using base64 encryption on the CA file.

For example, if the value of certificate-authority is /etc/iam/cert/ca.pem, then the value of certificate-authority-data is cat "/etc/iam/cert/ca.pem" | base64 | tr -d '\r\n', and the values of other -data variables are similar. This way, when we deploy the iamctl tool again, we only need to copy iamctl and the configuration file, but not the CA file.

Step 3, test whether iamctl is successfully installed.

Running iamctl user list will list the pre-created admin user as shown below:

Install and configure iam-authz-server #

Next, we need to install another core component: iam-authz-server can be installed in the following three steps.

Step 1, create iam-authz-server certificates and private keys.

  1. Create a certificate signing request:
$ cd $IAM_ROOT
$ source scripts/install/environment.sh
$ tee iam-authz-server-csr.json <<EOF
{
  "CN": "iam-authz-server",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "marmotedu",
      "OU": "iamctl"
    }
  ],
  "hosts": []
}
EOF

$ cat authz_test_policy.json
{
  "id": "test-policy",
  "resource": ["/*"],
  "action": ["*"],
  "effect": "allow",
  "subject": {
    "type": "user",
    "condition": {
      "username": ["admin"]
    }
  }
}

$ curl -X POST -H"X-API-Token:$token" -H"Content-Type: application/json" -d @authz_test_policy.json http://127.0.0.1:8080/api/v1/authz/policies
  1. 验证授权策略是否生效
$ curl -XGET -H"X-API-Token:$token" http://127.0.0.1:8080/api/v1/authz/policies/test-policy

应该返回以下信息：

{"id":"test-policy","resource":["/*"],"action":["*"],"effect":"allow","subject":{"type":"user","condition":{"username":["admin"]}}}

这样，上面创建的授权策略就生效了。

$ curl -s -XPOST -H"Content-Type: application/json" -H"Authorization: Bearer $token" -d'{"metadata":{"name":"authztest"},"policy":{"description":"One policy to rule them all.","subjects":["users:<peter|ken>","users:maria","groups:admins"],"actions":["delete","<create|update>"],"effect":"allow","resources":["resources:articles:<.*>","resources:printer"],"conditions":{"remoteIPAddress":{"type":"CIDRCondition","options":{"cidr":"192.168.0.1/16"}}}}}' http://127.0.0.1:8080/v1/policies
  1. Create a secret key and extract the secretID and secretKey from the command’s output.
$ curl -s -XPOST -H"Content-Type: application/json" -H"Authorization: Bearer $token" -d'{"metadata":{"name":"authztest"},"expires":0,"description":"admin secret"}' http://127.0.0.1:8080/v1/secrets
{"metadata":{"id":23,"name":"authztest","createdAt":"2021-04-08T07:24:50.071671422+08:00","updatedAt":"2021-04-08T07:24:50.071671422+08:00"},"username":"admin","secretID":"ZuxvXNfG08BdEMqkTaP41L2DLArlE6Jpqoox","secretKey":"7Sfa5EfAPIwcTLGCfSvqLf0zZGCjF3l8","expires":0,"description":"admin secret"}
  1. Generate a token to access the iam-authz-server.

iamctl provides the jwt sign command to issue a token based on secretID and secretKey for your convenience.

$ iamctl jwt sign ZuxvXNfG08BdEMqkTaP41L2DLArlE6Jpqoox 7Sfa5EfAPIwcTLGCfSvqLf0zZGCjF3l8 # Replace $secretID $secretKey with the key pair created in the previous step
eyJhbGciOiJIUzI1NiIsImtpZCI6Ilp1eHZYTmZHMDhCZEVNcWtUYVA0MUwyRExBcmxFNkpwcW9veCIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXV0aHoubWFybW90ZWR1LmNvbSIsImV4cCI6MTYxNzg0NTE5NSwiaWF0IjoxNjE3ODM3OTk1LCJpc3MiOiJpYW1jdGwiLCJuYmYiOjE2MTc4Mzc5OTV9.za9yLM7lHVabPAlVQLCqXEaf8sTU6sodAsMXnmpXjMQ

If you have repetitive operations in your development process, you can also integrate these operations as subcommands of iamctl to make it more convenient.

  1. Test if resource authorization is granted.

We can complete resource authorization by requesting /v1/authz:

$ curl -s -XPOST -H'Content-Type: application/json' -H'Authorization: Bearer eyJhbGciOiJIUzI1NiIsImtpZCI6Ilp1eHZYTmZHMDhCZEVNcWtUYVA0MUwyRExBcmxFNkpwcW9veCIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXV0aHoubWFybW90ZWR1LmNvbSIsImV4cCI6MTYxNzg0NTE5NSwiaWF0IjoxNjE3ODM3OTk1LCJpc3MiOiJpYW1jdGwiLCJuYmYiOjE2MTc4Mzc5OTV9.za9yLM7lHVabPAlVQLCqXEaf8sTU6sodAsMXnmpXjMQ' -d'{"subject":"users:maria","action":"delete","resource":"resources:articles:ladon-introduction","context":{"remoteIPAddress":"192.168.0.5"}}' http://127.0.0.1:9090/v1/authz
{"allowed":true}

If the authorization is granted, it will return {"allowed":true}.

Installing and Configuring iam-pump #

The steps for installing iam-pump are similar to installing iam-apiserver and iam-authz-server. The specific steps are as follows.

Step 1: Install the iam-pump executable.

$ cd $IAM_ROOT
$ source scripts/install/environment.sh
$ make build BINS=iam-pump
$ sudo cp _output/platforms/linux/amd64/iam-pump ${IAM_INSTALL_DIR}/bin

Step 2: Generate and install the iam-pump configuration file (iam-pump.yaml).

$ ./scripts/genconfig.sh scripts/install/environment.sh configs/iam-pump.yaml > iam-pump.yaml
$ sudo mv iam-pump.yaml ${IAM_CONFIG_DIR}

Step 3: Create and install the iam-pump systemd unit file.

$ ./scripts/genconfig.sh scripts/install/environment.sh init/iam-pump.service > iam-pump.service
$ sudo mv iam-pump.service /etc/systemd/system/

Step 4: Start the iam-pump service.

$ sudo systemctl daemon-reload
$ sudo systemctl enable iam-pump
$ sudo systemctl restart iam-pump
$ systemctl status iam-pump # Check the status of iam-pump, if the output contains active (running), it means iam-pump has been successfully started.

Step 5: Test if iam-pump is successfully installed.

$ curl http://127.0.0.1:7070/healthz
{"status": "ok"}

After these 5 steps, if it returns {"status": "ok"}, it means the iam-pump service is healthy.

Installing man files #

The IAM system generates man1 files for each component by calling the functions of the github.com/cpuguy83/go-md2man/v2/md2man and github.com/spf13/cobra packages. This process consists of 3 steps.

Step 1: Generating man1 files for each component.

$ cd $IAM_ROOT
$ ./scripts/update-generated-docs.sh

Step 2: Installing the generated man1 files.

$ sudo cp docs/man/man1/* /usr/share/man/man1/

Step 3: Verifying the successful installation of man1 files.

$ man iam-apiserver

After executing the man iam-apiserver command, the man document interface will pop up as shown in the following image:

At this point, all components of the IAM system have been successfully installed. You can use the iamctl version command to view the client and server versions, as shown in the following example:

$ iamctl version -o yaml
clientVersion:
  buildDate: "2021-04-08T01:56:20Z"
  compiler: gc
  gitCommit: 1d682b0317396347b568a3ef366c1c54b3b0186b
  gitTreeState: dirty
  gitVersion: v0.6.1-5-g1d682b0
  goVersion: go1.16.2
  platform: linux/amd64
serverVersion:
  buildDate: "2021-04-07T22:30:53Z"
  compiler: gc
  gitCommit: bde163964b8c004ebb20ca4abd8a2ac0cd1f71ad
  gitTreeState: dirty
  gitVersion: bde1639
  goVersion: go1.16.2
  platform: linux/amd64

Summary #

In this lecture, I guided you step by step in installing the IAM application. By completing the installation, I hope it has deepened your understanding of the IAM application and prepared your environment for the upcoming practical exercises. In order to present the installation process more clearly, I have summarized the entire installation steps into a mind map, which you can take a look at.

Additionally, I would like to remind you that the passwords set for all the components we discussed today are iam59!z$. Make sure to remember it.

After-class Exercise #

Please try to call the API interface provided by iam-apiserver to create a user: xuezhang, and create policy and secret resources under this user. Finally, call the /v1/authz interface provided by iam-authz-server for resource authorization. If you have any interesting discoveries, remember to share them.

Looking forward to seeing your attempts in the comments section. See you in the next lecture!

Easter Egg: One-Click Installation #

If you have completed Lesson 02, you can directly execute the following script to complete the installation of the IAM system:

$ export LINUX_PASSWORD='iam59!z$' # Important: Export the password of the "going" user here
$ version=latest && curl https://marmotedu-1254073058.cos.ap-beijing.myqcloud.com/iam-release/${version}/iam.tar.gz | tar -xz -C /tmp/
$ cd /tmp/iam/ && ./scripts/install/install.sh iam::install::install

In addition, you can also refer to the IAM Deployment Guide tutorial for installation. This installation manual enables you to deploy the entire IAM system, including practical environments and IAM services, with just one click after creating normal users.