32 Practical Exercises With Kubernetes 3

32 Practical Exercises with Kubernetes 3 #

Hello, I’m Chrono.

Today, we will be concluding our “Advanced” course. Compared to the previous “Beginner” and “Intermediate” courses, this section contains more knowledge points and is more challenging. If you can study this section thoroughly, I believe you will have a deeper understanding of Kubernetes.

In today’s lesson, we will review and summarize the previous knowledge, extracting the key points and learning objectives from the articles. You can also test your understanding by trying to explain the related operational details fluently without referring back to the course.

After the review, we will proceed with the final practical exercise. First, we will continue improving the WordPress website that has been the theme throughout the course by changing MariaDB to a StatefulSet and adding NFS persistent storage. Then, we will install the Dashboard in the Kubernetes cluster and practice using Ingress and namespaces together.

Recap 1: API Objects #

The “Advanced Guide” can be divided into three parts. The first part discusses API objects such as PersistentVolume and StatefulSet.

(Lesson 24) PersistentVolume, abbreviated as PV, is an abstraction of persistent storage in Kubernetes. It represents storage devices such as LocalDisk, NFS, and Ceph, and, like CPU and memory, is a common resource of the cluster.

Due to the significant differences between different storage devices, StorageClass was introduced to better describe the characteristics of PV. Its role is to classify storage devices, making it easier for us to select PV objects.

PVs are generally created by system administrators. If we want to use PVs, we need to apply for them using PersistentVolumeClaim (PVC), clearly specifying parameters such as capacity and access mode. Kubernetes will then find the most suitable PV to allocate for our use.

(Lesson 25) Creating PVs manually is time-consuming, troublesome, and prone to errors, which led to the concept of “dynamic storage volumes.” It requires binding a Provisioner object in StorageClass to automatically create PVs that meet the requirements specified by PVC, replacing manual work.

With PVs and PVCs, we can use “persistentVolumeClaim” in Pods to reference PVC and create volumes that are available for containers. Then, in the container, we can mount it to a specific path using “volumeMounts,” allowing the container to read and write PVs, achieving persistent storage of data.

(Lesson 26) An important application area for persistent storage is to save application state data. To manage stateful applications, we need to use a new object called StatefulSet, which can be considered as a special case of managing stateless application objects, like Deployment.

The YAML description of StatefulSet objects is very similar to Deployments. Only the “spec” field has an additional “serviceName” field. However, the way they deploy applications is significantly different.

Pods created by Deployments have random names, while StatefulSet orders and creates Pods in sequence, ensuring that applications have a specific startup order. This allows for the establishment of relationships such as master-slave or primary-replica.

When using Service to create services for StatefulSets, it also creates a separate domain name for each Pod, which is sequentially numbered. This ensures that Pods have stable network identifiers, and external users can use this domain name to accurately access a specific Pod.

StatefulSet also uses the “volumeClaimTemplates” field to define persistent storage, which is essentially a PVC. Each Pod can use this template to generate its own PVC to apply for PV, achieving independent binding of storage volumes and Pods.

Through the key capabilities of startup order, stable domain names, and storage templates, StatefulSet can handle stateful applications such as Redis and MySQL efficiently.

Summary Review 2: Application Management #

The second part of the “Advanced Series” discusses application management, including rolling updates, resource quotas, and health checks.

([Lecture 27]) After deploying an application in Kubernetes, we still need to perform continuous operational management on it, including version updates and rollbacks.

Version updates are straightforward. You just need to write a new YAML (Deployment, DaemonSet, StatefulSet) and apply it using kubectl apply. Kubernetes adopts a “rolling update” strategy, which actually consists of two synchronized actions: scaling up and scaling down. This ensures that there are always available Pods during the update process, allowing for smooth service provision.

The update history of an application can be viewed using the kubectl rollout history command. If something unexpected occurs, the kubectl rollout undo command can be used to rollback. These two commands serve as a safety net for our update process, allowing us to proceed confidently. In case of failure, use the “S/L big trick.”

([Lecture 28]) In order to ensure the stable operation of containers within Pods, we can utilize resource quotas and health probes.

Resource quotas can limit the amount of CPU and memory resources that containers can request, preventing over or under-allocation, and maintaining a reasonable level that benefits Kubernetes scheduling.

Health probes are built-in application monitoring tools in Kubernetes, with three types: Startup, Liveness, and Readiness. They probe the startup, alive, and ready statuses of containers. Probing can be done through exec, tcpSocket, or httpGet. By combining and utilizing these probes, container states can be checked flexibly. If Kubernetes detects unavailability, it will restart the container, keeping the overall health of the application.

Key Review Part 3: Cluster Management #

Part 3 of the “Advanced Series” talks about cluster management, including the concepts of namespaces, system monitoring, and network communication.

([Lecture 29]) Although there are many compute resources in a Kubernetes cluster, they are still limited. In addition to assigning resource quotas to pods, we also need to assign resource quotas to the cluster. The method is to use namespaces to divide the overall resource pool into smaller units and allocate them to different users as needed.

Resource quotas for namespaces are managed using “ResourceQuota”. In addition to basic CPU and memory, it can also limit storage capacity and the number of various API objects. This allows for efficient utilization of cluster resources and prevents users from encroaching on each other.

([Lecture 30]) System monitoring is another important aspect of cluster management. Kubernetes provides two tools for this: Metrics Server and Prometheus:

Metrics Server is specifically used to collect core resource metrics in Kubernetes. The cluster’s status can be viewed using kubectl top. It is also a prerequisite for HorizontalPodAutoscaler, which enables horizontal scaling.
Prometheus, the second CNCF graduated project after Kubernetes, is the “de facto standard” in the cloud-native monitoring field. Once deployed in the cluster, various metrics can be visualized using Grafana. It can also be integrated with automatic alerting and other features.

([Lecture 31]) For the underlying infrastructure of the network, Kubernetes defines a flat network model called “IP-per-pod”, which requires compliance with the CNI standard. Common network plugins include Flannel, Calico, and Cilium. Flannel uses an overlay mode with lower performance, while Calico uses a route mode with higher performance.

Now that we have gone through all the key points of the “Advanced Series,” do you understand and grasp them all?

Build a WordPress website #

Next, let’s continue optimizing the WordPress website based on the [Lesson 22]. The key here is to make MariaDB implement data persistence.

The overall architecture of the website hasn’t changed much. Nginx and WordPress remain the same, only MariaDB needs to be modified:

Because MariaDB has changed from Deployment to StatefulSet, we need to modify the YAML file. Add the fields “serviceName” and “volumeClaimTemplates” to define the network identifier and NFS dynamic storage volume. Then, mount them to the data directory “/var/lib/mysql” in the container using “volumeMounts”.

The modified YAML looks like this:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: maria-sts
  name: maria-sts

spec:
  # headless svc
  serviceName: maria-svc

  # pvc
  volumeClaimTemplates:
  - metadata:
      name: maria-100m-pvc
    spec:
      storageClassName: nfs-client
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 100Mi

  replicas: 1
  selector:
    matchLabels:
      app: maria-sts

  template:
    metadata:
      labels:
        app: maria-sts
    spec:
      containers:
      - image: mariadb:10
        name: mariadb
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 3306

        envFrom:
        - prefix: 'MARIADB_'
          configMapRef:
            name: maria-cm

        volumeMounts:
        - name: maria-100m-pvc
          mountPath: /var/lib/mysql

After modifying MariaDB, we need to make a minor modification to WordPress.

Remember? Each Pod managed by StatefulSet has its own domain name, so we need to change the environment variables of WordPress to the new name of MariaDB, which is “maria-sts-0.maria-svc”:

apiVersion: v1
kind: ConfigMap
metadata:
  name: wp-cm

data:
  HOST: 'maria-sts-0.maria-svc'  #注意这里
  USER: 'wp'
  PASSWORD: '123'
  NAME: 'db'

After modifying these two YAML files, we can create the MariaDB, WordPress, Ingress, and other objects one by one.

Just like before, you can access the WordPress website through the NodePort “30088” or the Ingress Controller’s “wp.test” domain:

Did the persistence storage of StatefulSet work?

You can delete all these objects and recreate them, then enter the website to see if the original data still exists. Or, you can check the storage directory of NFS directly to see some database files generated by MariaDB:

Both methods can prove that our MariaDB, after being deployed using StatefulSet, has saved the data on the disk and will not be lost due to object destruction.

By now, you have completed the first mini practice. Give yourself a pat on the back. Let’s move on to the second practice, which is installing the Kubernetes Dashboard in the Kubernetes cluster.

Deploy Dashboard #

In the hands-on practice session of the “Beginner’s Guide” (Lesson 15), I briefly introduced the graphical management interface of Kubernetes called Dashboard. I’m not sure if you still remember it. At that time, Dashboard was directly built into minikube, so there was no need to install it. Just one command to start it, and you could easily manage the Kubernetes cluster in the browser, which was very convenient.

Now, after deploying an actual multi-node cluster using kubeadm, can we also use Dashboard? Next, I will guide you through the process of installing Dashboard from scratch.

Firstly, you should go to the project website of Dashboard (https://github.com/kubernetes/dashboard) to take a look at its documentation and understand its basic information.

The installation process is quite simple, only requiring a YAML file, which you can directly download using the following command:

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.0/aio/deploy/recommended.yaml

This YAML file contains many objects, and although the file is quite large, you should be able to understand most of it by now. Here are a few key points:

All objects belong to the “kubernetes-dashboard” namespace.
The Dashboard is deployed as a Deployment object with port number 8443.
The container has a Liveness probe enabled, using HTTPS to check its live status.
The Service object uses port 443, which maps to the Dashboard’s 8443 port.

You can easily deploy the Dashboard using the kubectl apply command:

kubectl apply -f dashboard.yaml

Deploying Ingress/Ingress Controller #

However, to add a bit of difficulty to our hands-on practice, we can set up an Ingress entry in the beginning, and access it using reverse proxy.

Since the Dashboard uses encrypted HTTPS protocol by default and refuses plaintext HTTP access, we need to generate a certificate first and make Ingress use the HTTPS protocol as well.

For simplicity, I will directly use the command line tool “openssl” in Linux to generate a self-signed certificate (if you have the means, you can also consider applying for a free certificate from a CA website):

openssl req -x509 -days 365 -out k8s.test.crt -keyout k8s.test.key \
  -newkey rsa:2048 -nodes -sha256 \
    -subj '/CN=k8s.test' -extensions EXT -config <( \
       printf "[dn]\nCN=k8s.test\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:k8s.test\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")

The openssl command is quite lengthy, so let me explain it briefly: it generates a certificate in X509 format that is valid for 365 days, with a 2048-bit RSA private key and SHA256 as the digest algorithm, for the website “k8s.test”.

After running the command line, two files will be generated: one is the certificate “k8s.test.crt”, and the other is the private key “k8s.test.key”. We need to store these two files in Kubernetes for Ingress to use.

Since these two files are confidential information, we will store them using a Secret. You can still use the command kubectl create secret to automatically create YAML, but the type is not “generic”, it is “tls”. Additionally, you need to use -n to specify the namespace and use --cert and --key to specify the files:

export out="--dry-run=client -o yaml"
kubectl create secret tls dash-tls -n kubernetes-dashboard --cert=k8s.test.crt --key=k8s.test.key $out > cert.yml

The resulting YAML will look something like this:

apiVersion: v1
kind: Secret
metadata:
  name: dash-tls
  namespace: kubernetes-dashboard
type: kubernetes.io/tls

data:
  tls.crt: LS0tLS1CRUdJTiBDRVJU...
  tls.key: LS0tLS1CRUdJTiBQUklW...

After creating this Secret object, you can use kubectl describe to check its status:

Next, we will write the Ingress Class and Ingress objects, and to keep the namespace organized, we will also place them in the “kubernetes-dashboard” namespace.

The Ingress Class object is very simple. Its name is “dash-ink” and it specifies that the controller is the Nginx official Ingress Controller we used before:

apiVersion: networking.k8s.io/v1
kind: IngressClass

metadata:
  name: dash-ink
  namespace: kubernetes-dashboard
spec:
  controller: nginx.org/ingress-controller

The Ingress object can be automatically generated using the kubectl create command. If you have forgotten a bit, you can refer back to Lesson 21:

kubectl create ing dash-ing --rule="k8s.test/=kubernetes-dashboard:443" --class=dash-ink -n kubernetes-dashboard $out

But this time, since it is using the HTTPS protocol, we need to add a few more things in the Ingress. One is the “annotations” field, which specifies that the backend target is an HTTPS service, and the other is the “tls” field, which specifies the domain name and certificate, i.e., the Secret we created earlier:

apiVersion: networking.k8s.io/v1
kind: Ingress

metadata:
  name: dash-ing
  namespace: kubernetes-dashboard
  annotations:
    nginx.org/ssl-services: "kubernetes-dashboard"

spec:
  ingressClassName: dash-ink

  tls:
    - hosts:
      - k8s.test
      secretName: dash-tls

  rules:
  - host: k8s.test
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard
            port:
              number: 443

The last object is the Ingress Controller itself. We will modify an existing template, but remember to change the Ingress Class in the “args” to our own “dash-ink”:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dash-kic-dep
  namespace: nginx-ingress

spec:
  ...
        args:
          - -ingress-class=dash-ink

To enable external access to the Ingress Controller, we also need to define a Service for it. The type is “NodePort”, and the port is specified as “30443”:

apiVersion: v1
kind: Service
metadata:
  name: dash-kic-svc
  namespace: nginx-ingress

spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
    nodePort: 30443

  selector:
    app: dash-kic-dep
  type: NodePort

After creating the above Secret, Ingress Class, Ingress, Ingress Controller, and Service, let’s confirm their running status:

Since these objects are quite a few, located in different namespaces, and have complex associations, I have created a simple diagram for your reference:

Accessing the Dashboard #

By now, the deployment of the Dashboard is almost complete. In order to access it, we need to create a user that can log in to the Dashboard.

There is a simple example on the Dashboard’s website (https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md) that we can use directly:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

This YAML creates an admin account for the Dashboard named “admin-user”, using Kubernetes’ RBAC mechanism, which won’t be further explained.

This account cannot be logged in using a simple “username + password” method; it requires a token, which can be obtained by using kubectl get secret and kubectl describe secret:

kubectl get secret -n kubernetes-dashboard
kubectl describe secrets -n kubernetes-dashboard admin-user-token-xxxx

The token is a long string. Copy it and keep it safe. Then, add the test domain “k8s.test” to the hosts file (/etc/hosts). After that, we can access the Dashboard by entering the URL “https://k8s.test:30443” in a browser:

The following two screenshots show the state of the “kube-system” namespace in the cluster. Since we previously installed Metrics Server, the Dashboard can also display CPU and memory usage in a graphical way, giving it a bit of a Prometheus + Grafana feel:

Summary #

Alright, today we reviewed the main points from the “Advanced Tutorial” together. The following mind map is a comprehensive summary of these key points, and you can study it carefully:

Today we had two practical projects. First, we transformed the backend storage service MariaDB into a StatefulSet for WordPress and mounted an NFS network disk. This achieved a relatively complete website with basic functionality.

Then, we installed Dashboard in Kubernetes, mainly deployed in the namespace “kubernetes-dashboard”. The installation of Dashboard itself is straightforward, but we also set up a reverse proxy for it earlier, added a security certificate, and further practiced the usage of Ingress.

However, these two projects have not fully covered the content of the “Advanced Tutorial”. You can continue to improve them, such as adding health checks, resource quotas, automatic scaling, etc. Get hands-on to solidify what you have learned.

Homework #

Today’s homework time, I want to leave it to you to practice the two practical exercises in this lesson. If you encounter any problems, feel free to ask in the comment section.