08 Application Configuration Management #

Key Points of this Section

1. Creation and usage of ConfigMaps and Secret resources
2. Implementation and principles of Pod identity authentication
3. Configuration and usage of container resources, security, pre-validation, and more.

These are divided into the following eight areas:

Source of Requirements #

Background Issues #

First, let’s take a look at the source of requirements together. I believe many of you have had the experience of starting a container using a container image. To start this container, there are many accompanying issues that need to be resolved:

• First, there are some configurable parameters. It is not feasible to include these parameters in the container image because they may need to be changed. Rebuilding the image every time a configuration needs to be changed is not acceptable;
• Second, there is the storage and use of sensitive information. For example, the application may need to use passwords or tokens;
• Third, the container needs to access the cluster itself. For example, if I want to access the kube-apiserver, there is an issue of authentication;
• Fourth, the resource requirements of the container after it is running on a node;
• Fifth, since containers share the kernel on a node, there is a need for security control;
• Finally, let’s talk about the preconditions verification before starting a container. For example, before starting a container, I may need to check if the DNS service is working properly. Or I may need to check if the network is reachable. These are actually some pre-validation steps.

Pod Configuration Management #

In Kubernetes, how does it handle these configuration management tasks? As shown in the figure below:

• Configurable parameters are handled using ConfigMap;
• Sensitive information is stored in Secret;
• ServiceAccount is used for authentication;
• Resources are used for resource configuration;
• Security control is achieved through SecurityContext;
• Pre-validation is done using InitContainers. These are additional fields added in the spec to handle these configuration management tasks.

ConfigMap #

Introduction to ConfigMap #

Let’s start by introducing the first part, which is ConfigMap. We will discuss what ConfigMap is used for and the benefits it provides. ConfigMap is mainly used to manage various configurable information, such as application configuration files, environment variables, or command-line parameters.

The benefit of using ConfigMap is that it decouples the variable configurations from container images, ensuring the portability of containers. Take a look at the screenshot of the configuration file on the right-hand side of the following image.

This is the definition of ConfigMap itself, which includes two parts: ConfigMap metadata, where we focus on the name and namespace information. Next, in the data section, you can see that it manages two configuration files. The structure is as follows: as the name suggests, ConfigMap contains the word “Map”, where Map refers to the key:value pair. The key is the file name, and the value is the content of the file.

ConfigMap Creation #

After understanding the introduction, let’s take a closer look at how to create ConfigMap. We recommend using the kubectl command to create ConfigMap. It has two main parameters: one is to specify the name, and the other is DATA. DATA can be specified by a file, a directory, or directly by specifying key-value pairs. Let’s take a look at the example below.

When specifying a file, the file name corresponds to the key in the Map, and the file content corresponds to the value in the Map. When specifying key-value pairs, you directly map the key:value pairs.

Using ConfigMap #

After creating ConfigMap, how should we use it?

As shown in the diagram above, ConfigMap is mainly used in pods:

• The first method is through environment variables. By using the valueFrom field and ConfigMapKeyRef, you can specify the name of the ConfigMap and the key in the ConfigMap.data. In this way, when the busybox container starts, executing env will show an environment variable called SPECIAL*LEVEL*KEY.
• The second method is through command-line parameters. The command-line parameters are directly obtained from the first line of environment variables and used in the cmd field.
• The last method is by mounting the ConfigMap directly to a directory in the container using volumes. In the example above, the special-config ConfigMap content is mounted to the /etc/config directory in the container. This is another way of using ConfigMap.

Points to Note about ConfigMap #

Let’s summarize the usage of ConfigMap and highlight some key points. A total of five points are listed:

1. File size of ConfigMap: Although there is no file size limitation for ConfigMap files, there is a size limitation for writing data in ETCD, which is currently limited to within 1MB.
2. The second point to note is that when a pod refers to ConfigMap, it must be in the same Namespace as the ConfigMap. As you can see earlier, the ConfigMap.metadata contains the namespace field.
3. The third point is that if the ConfigMap referred to by a pod does not exist, the pod cannot be created successfully. This means that before creating the pod, the ConfigMap that needs to be referenced must be created first.
4. The fourth point is about using the envFrom method. When importing all the information in the ConfigMap as environment variables, if there are some invalid keys in the ConfigMap, such as keys containing numbers in their names, these environment variables will not be injected into the container and will be ignored. However, the pod itself can still be created. This is different from the third point, as it is a way of importing the entire ConfigMap into environment variables while the ConfigMap file exists.
5. The final point is: what kind of pod can use ConfigMap? Only pods created through the K8s API can use ConfigMap. For example, a pod created using the kubectl command line can certainly use ConfigMap. However, other pods created in different ways, such as static pods created by the kubelet through manifests, cannot use ConfigMap.

Secret #

Introduction to Secret #

Now let’s talk about Secret. Secret is a resource object primarily used to store sensitive information such as passwords and tokens. The sensitive information is stored in base-64 encoding. Let’s take a look at the definition of Secret data in the following image.

As for the metadata, there are mainly two fields: name and namespace. Next is the type field, which is a very important field that indicates the type of the Secret. There are various types of Secret, and the following are the commonly used four types:

• The first type is Opaque, which is a regular Secret file.
• The second type is service-account-token, which is used for service account authentication.
• The third type is dockerconfigjson, which is used for pulling images from private repositories.
• The fourth type is bootstrap.token, which is used for node cluster authentication.

Next is the data field, which stores the Secret data in key-value format.

Creating a Secret #

Next, let’s take a look at how to create a Secret.

As shown in the image above, there are two ways to create a Secret:

• System-generated: For example, Kubernetes creates a Secret for the default user (default ServiceAccount) of each namespace.
• User-created: Manually create the Secret using command-line tools such as kubectl. In comparison to ConfigMap, the Secret creation command includes an additional type parameter. The data field also follows a similar pattern, allowing you to specify files and key-value pairs. If the type is not specified, it defaults to the Opaque type.

In the image above, there are two examples. The first example creates a Secret for pulling images from a private repository by specifying the file /root/.docker/config.json. The type is specified as dockerconfigjson. In the second example, we specify key-value pairs without specifying the type, so it defaults to Opaque. The key-value pairs are in the format of key:value, where the value content is encoded in base64. Creating a Secret works this way.

Using a Secret #

After creating a Secret, let’s see how to use it. It is mainly used by pods, usually mounted to a specified directory inside the container through volumes. The business process within the container can then read the Secret from the directory for use. Additionally, when accessing a private image repository, it can be achieved by referencing the Secret.

First, let’s take a look at the way to mount it to a user-specified directory:

• The first way is shown on the left side of the image above, where the user directly specifies to mount mysecret to the /etc/foo directory in the container.
• The second way is shown on the right side of the image above, where it is system-generated. The serviceaccount-secret is automatically mounted to the /var/run/secrets/kubernetes.io/serviceaccount directory in the container. It generates two files: ca.crt and token. These are two certificate files that contain authentication information.

Using a Private Image Repository #

Now let’s see how to use a Secret to access a private image repository. Firstly, the information of the private image repository is stored in the Secret (refer to the previous section on creating a Secret), and then with the configuration of the two methods shown in the image below, you can pull images from the private repository:

• The first method, as shown on the left side of the image below, is to directly configure it within the pod using the imagePullSecrets field.
• The second method is automatic injection. Users pre-configure imagePullSecrets in the service account used by the pod, and the system will automatically inject this imagePullSecrets when the pod is created.

Important Points to Note when Using a Secret #

Finally, let’s take a look at a few important points to note when using a Secret. The following are three points:

1. The first point is the file size limit of the Secret. Just like ConfigMap, it is also 1MB.
2. The second point is that although Secret uses base64 encoding, it is not much different from plaintext. Therefore, if there is sensitive information that needs to be stored using Secret, it should be carefully considered. In other words, you should consider who will access your cluster and who will use your Secret. If someone can access the cluster, they can obtain the Secret. If there is a high requirement for sensitive information in Secret and a strong need for encryption, it is recommended to use Kubernetes and the open-source vault to develop a solution for encryption and access authorization of sensitive information.
3. The third point is about the best practices for reading a Secret. It is not recommended to use list/watch. If you use list/watch operations, all Secrets in the namespace will be retrieved, revealing more information. It is recommended to use the GET method to only retrieve the Secret that you need.

ServiceAccount #

Introduction to ServiceAccount #

Next, let’s talk about ServiceAccount. ServiceAccount is primarily used to solve the authentication problem of pods within a cluster, and the authentication information is stored in Secrets.

First, let’s take a look at the screenshot on the left. At the bottom, within the red box, there is a field called Secret, which specifies which Secret is used by the ServiceAccount. This is automatically added by Kubernetes for the ServiceAccount. Now, let’s take a look at the screenshot on the right. It corresponds to the data within the Secret and contains two sections: ca.crt and token. ca.crt is used for server validation, and token is used for pod authentication. Both are encoded in base64. Additionally, in the metadata section, there is information related to the ServiceAccount (which ServiceAccount uses this secret). Lastly, let’s take note of the type, which is service-account-token.

Example: Application within a Pod accessing its corresponding Kubernetes cluster #

After introducing ServiceAccount and its corresponding secret, let’s take a look at how a pod uses ServiceAccount or the secret to access its corresponding Kubernetes cluster.

When a pod is created, it first mounts the secret into a fixed directory within the container. This is achieved by Kubernetes functionality. It is required to mount the ca.crt and token files into a fixed directory.

How does a pod utilize this file when accessing the cluster? Let’s take a look at the code snippet below:

When we implement pod access to the Kubernetes cluster in Go, we generally call the InClusterConfig method to generate some client information for accessing the service. Take a look at the Config, and it contains two parts of information:

• First, there is tlsClientConfig, which is primarily used for server-side ca.crt validation;
• Second, there is the Bearer Token, which is the pod’s authentication. On the server-side, the pod is authenticated using the token.

Let’s go back to the left side of the above image. After authentication, there will be two parts of pod identity information: Group and User. Authentication is performed on these two parts of information. Then, we can use the RBAC functionality to perform authorization management on the pod.

If RBAC is not configured, by default, the pod has GET permissions for resources, meaning it can retrieve data from its corresponding Kubernetes cluster. If additional permissions are required, RBAC needs to be configured manually. We will discuss RBAC in detail in future courses, so stay tuned.

Resource #

Container Resource Configuration #

Below is an introduction to Resource, which is a container resource configuration management.

Currently, there are three supported types: CPU, memory, and temporary storage. If users feel that these three types are not sufficient and have their own resources, such as GPUs or other resources, they can define them themselves. However, when configuring, the specified quantity must be an integer. Currently, resource configuration is mainly divided into two types: request and limit. One represents the required quantity, while the other represents the resource limit. CPU, memory, and temporary storage are all declared in the Resource field under the container.

For example, the resource requirements of the Wordpress container include both request and limit. They respectively declare the required resources and the resource limits.

Pod Quality of Service (QoS) Configuration #

Based on the CPU requirements for container memory resources, we categorize the QoS of pods into Guaranteed, Burstable, and BestEffort.

• Guaranteed: Each container in the pod must have a declaration of memory and CPU request and limit, and the request and limit must be the same. This is the Guaranteed QoS level.
• Burstable: At least one container has a declaration of memory and CPU request.
• BestEffort: Any pod that is not Guaranteed or Burstable falls into the BestEffort category.

So, what is this QoS about? After configuring the resources, when the pod container runs on the node, if there is insufficient memory quota on the node, the kubelet will evict some low-priority or low QoS (such as BestEffort and Burstable) pods. They are evicted in the order of removing BestEffort first, then Burstable pods.

SecurityContext #

Introduction to SecurityContext #

SecurityContext is mainly used to restrict the behavior of containers, ensuring the safety of the system and other containers. This capability is not inherent to Kubernetes or container runtimes themselves, but rather, Kubernetes and runtimes enforce SecurityContext by passing user configurations to the kernel, which then enforces the SecurityContext through its mechanisms. Therefore, the content here may be relatively simple or abstract.

SecurityContext can be applied at three levels:

• The first level is container-level, only affecting individual containers.
• The second level is pod-level, affecting all containers within a pod.
• The third level is cluster-level, specifically the Pod Security Policy (PSP), which affects all pods within a cluster.

There are currently seven settings for permissions and access control (this number may change in the future):

1. The first setting controls file access permissions through user IDs and group IDs.
2. The second setting is SELinux, which controls user or process access to files through policy configurations.
3. The third setting is privileged containers.
4. The fourth setting is Capabilities, which allows specific processes to be configured with privileged capabilities.
5. The fifth setting is AppArmor, which controls access control permissions for executable files through configuration files, such as read and write permissions for certain ports.
6. The sixth setting is control over system calls.
7. The seventh setting is a restriction on whether child processes can gain more privileges than their parent.

In the end, all these settings are enforced by the kernel to control permissions.

The above diagram is an example of configuring SecurityContext at the pod and container levels. If you have more specific requirements regarding these topics, you can search for more in-depth resources based on this information to further your learning.

InitContainer #

Introduction to InitContainer #

Next, let’s take a look at InitContainers. First, let’s discuss the differences between InitContainers and regular containers, which can be summarized in the following three points:

1. InitContainers are started before regular containers, and regular containers are only started after all InitContainers have successfully executed.
2. InitContainers are started and executed in the order defined, one after another, whereas regular containers are started concurrently.
3. InitContainers exit upon successful execution, while regular containers may continue running indefinitely. They could be long-running or could be restarted upon failure. This is another difference between InitContainers and regular containers.

Based on the above three points, let’s consider one use case for InitContainers. They primarily serve regular containers. For example, they can perform initialization tasks before the regular containers start, such as preparing configuration files that might change. They can also perform preconditions checks, such as verifying network connectivity.

The above screenshot shows the configuration of an InitContainer for the flannel component. The InitContainer is responsible for preparing network configuration files before starting the regular container kube-flannel.

Conclusion #

• ConfigMap and Secret: First introduced the creation method and usage scenarios of ConfigMap and Secret, and then classified and organized common usage points of ConfigMap and Secret. Finally, the use and configuration of private repository images are introduced.
• Pod Identity Authentication: First introduced the association between ServiceAccount and Secret, then analyzed the Pod identity authentication process and implementation details from the source code perspective, and introduced Pod’s permission management (i.e., RBAC configuration management).
• Container Resources and Security: First introduced the configuration of common container resource types (CPU/Memory), followed by a detailed introduction to Pod service quality classification. Briefly explained the valid hierarchy of SecurityContext and its permission configuration options.
• InitContainer: First introduced the differences between InitContainer and regular containers, as well as the purposes of InitContainer. Then, based on practical use cases, the uses of InitContainer were explained.

Alright, that’s all for today. Thank you, everyone.