14 Serverless Kubernetes Container Service Introduction

14 Serverless Kubernetes Container Service Introduction #

Introduction: Serverless Kubernetes is a serverless service based on containers and Kubernetes. It provides a simple, easy-to-use, highly elastic, cost-effective, and pay-as-you-go Kubernetes container service. It eliminates the need for node management and maintenance, as well as capacity planning, allowing users to focus more on application development rather than infrastructure management. We can abbreviate Serverless Kubernetes as ASK.

Serverless Container #

Let’s start with Serverless. We are familiar with the core values of the Serverless concept, which include eliminating the need to manage underlying infrastructure and not having to worry about upgrading and maintaining the underlying operating system. Serverless allows us to focus more on application development, resulting in shorter time to market. At the same time, Serverless architecture is naturally scalable. When the number of business users or resource consumption increases, we only need to create more application resources. This scalability cannot be achieved by purchasing machines. Serverless applications are usually created on demand, so users do not have to pay for idle resources, reducing overall computing costs.

The above mentioned core values are common to both the Serverless concept and Serverless containers. However, the difference between Serverless containers and other Serverless forms is that it is based on container delivery.

Based on containers means universality and standardization. We can “build once and run anywhere” with containers. Containers are not limited by language and libraries. Any application can be containerized and started using container deployment methods. Based on container standardization, the open-source community has built a rich cloud-native ecosystem centered around Kubernetes. This greatly enriches the peripheral application frameworks and tools for Serverless containers, such as conveniently deploying Helm Chart packages. Based on container and Kubernetes standardization, we can easily migrate applications between different environments (online and offline) and even between different cloud providers without worrying about vendor lock-in. These are the core values of Serverless containers.

(Serverless container product landscape)

Major cloud providers have launched their own Serverless container services. The above image is the Serverless container product landscape compiled by Gartner. Alibaba Cloud has Serverless Kubernetes (ASK) and Elastic Container Instance (ECI); AWS has Fargate, which includes EKS on Fargate and ECS on Fargate; Azure has ACI. Gartner also predicts that by 2023, 70% of AI applications will run in containers and Serverless form.

ASK/ACK on ECI Container Service #

Next, let’s introduce Alibaba Cloud’s Serverless container product family: ECI, ACK on ECI, and Serverless Kubernetes.

1. ECI #

ECI stands for “Elastic Container Instance.” It is the underlying infrastructure of Serverless containers and allows container images to be launched. ECI makes containers as first-class citizens in the cloud, just like ECS. The underlying runtime environment of ECI is based on secure container technology, which provides strong isolation and ensures runtime security. ECI supports CPU specifications from 0.25c to 64c and also supports GPUs. It is billed on a per-second basis and created on demand. Like ECS, ECI also supports Spot instances, which can save up to 90% of costs in some scenarios. The startup time for ECI instances is currently around 10 seconds, after which the container image is pulled. We also provide image snapshot functionality, which reads the image from the snapshot every time the container starts, eliminating the need for remote image pulling. It is worth noting that ECI and ECS share the same elastic computing resource pool. This means that ECI’s elastic supply can be fully guaranteed to enjoy the scalability benefits of the elastic computing resource pool.

ECI can only create a single container instance and does not have orchestration capabilities, such as scaling applications with multiple replicas or allowing SLB and Ingress to access pod traffic. Therefore, we need to use ECI in the Kubernetes orchestration system. We provide two ways to use ECI in Kubernetes: ACK on ECI and ASK.

In the integration with the Kubernetes orchestration system, we manage each ECI container instance in the form of Pods, where each Pod corresponds to an ECI instance. ECI Pods are mutually isolated, and the startup time of an ECI Pod is approximately 10s. Since ECI Pods are managed within a Kubernetes cluster, they are fully integrated into the Kubernetes ecosystem, as demonstrated by the following:

Convenient management of ECI Pods using Kubectl, allowing the use of standard Kubernetes API to manipulate resources.
Connection of SLB and ECI Pods through Service and Ingress.
Container orchestration using Deployment/Statefulset and dynamic scaling using HPA (Horizontal Pod Autoscaler).
Monitoring of ECI Pods using Proms (Prometheus).
Running Istio for traffic management, Spark/Presto for data computation, and Kubeflow for machine learning.
Deployment of various Helm Charts.

These are the values of using Kubernetes to manage container instances.

It is important to note that ECI Pods in Kubernetes are serverless containers, so they do not support certain functionalities compared to regular Pods (such as DaemonSet), and they do not support privileges like Privilege Escalation and HostPort. Apart from these limitations, ECI Pods possess the same capabilities as regular Pods, such as the ability to mount cloud disks, NAS, and OSS data volumes.

2. ACK on ECI #

Next, let’s discuss the usage of ECI in an ACK Kubernetes cluster. This approach is suitable when users already have an ACK cluster with several ECS nodes. In this case, ECI can be utilized for running short-run applications to address the issue of insufficient resources in the main cluster. Alternatively, ECI can be used for rapid scaling of applications as it is more efficient than ECS node scaling.

In ACK on ECI, ECS and ECI Pods can intercommunicate, ECI Pods can access CoreDNS within the cluster, and they can also access ClusterIP Services.

3. Serverless Kubernetes #

Unlike ACK on ECI, the ASK (Alibaba Serverless Kubernetes) cluster does not have any ECS nodes. This is the most significant difference between ASK Serverless Kubernetes and traditional Kubernetes clusters. Therefore, in ASK clusters, there is no need to manage any nodes. It provides a completely nodeless operational environment, making Kubernetes more accessible and eliminating the complexity of managing underlying nodes and issues like Notready nodes. In ASK clusters, users can focus solely on their applications without concerning themselves with infrastructure management.

The elastic capabilities of ASK are superior to those of regular Kubernetes clusters, allowing for the creation of 500 Pods in the Running state within 30 seconds. By default, ECI Pods in the cluster are billed based on pay-as-you-go pricing, but they also support using Spot instances and reserved instance coupons to reduce costs. In terms of compatibility, ASK does not have any physical nodes, so functionalities related to nodes, such as DaemonSet, are not supported. However, Deployment/Statefulset/Job/Service/Ingress/CRD and other features are seamlessly supported.

In ASK, the default Ingress is implemented using SLB (Server Load Balancer) for Layer 7 forwarding, eliminating the need to deploy Nginx Ingress and simplifying maintenance. Meanwhile, based on the SLB 7-layer, we have implemented the Knative Serving capability. Knative Controller is managed by ASK, so users do not need to bear the cost of the Controller.

Similar to ACK, ASK is well integrated with cloud products such as Arms/SLS, making it convenient to monitor Pods and collect Pod logs into SLS.

This is the overall architecture of ASK. The core part is ASK-Scheduler, which is responsible for watching Pod changes and creating corresponding ECI instances. It also synchronizes ECI instance status with Pods. There are no actual ECS nodes registered with the ApiServer in the cluster. This Nodeless architecture decouples the Kubernetes orchestration layer from the ECI resource layer, allowing Kubernetes to completely eliminate the elasticity and capacity limitations caused by the underlying node scale, becoming a cloud-oriented Nodeless Kubernetes elastic architecture.

Typical Features of ASK #

Let’s introduce several typical features of ASK:

1. GPU instances #

The first one is GPU instances. Using GPU container instances in a Serverless cluster is a very simple process. There is no need to install GPU drivers. You just need to specify the GPU Pod specifications and the number of GPU cards required by the container. After that, you can deploy it with one click. This greatly improves the efficiency of development and testing in machine learning scenarios.

2. Spot Preemptible instances #

The second one is Spot Preemptible instances. Preemptible instances are on-demand instances that can reduce computing costs in data computing and other scenarios. After the preemptible instance is created successfully, it has a one-hour protection period. The market price of preemptible instances fluctuates with supply and demand. We support two Spot strategies: one is based on market bidding, and the other is to specify a price upper limit. We just need to add the corresponding annotations to the Pods, and the usage is very simple.

3. Elastic workload #

The third important feature is elastic workload. Elastic workload enables the deployment of multiple replicas of Deployment on different units, such as ECS, ECI, and ECI-Spot. Through this hybrid scheduling mode, the computing cost of the workload can be reduced. In this example, the Deployment has 6 replicas, including 2 normal ECI Pods and the remaining replicas are ECI-Spot instances.

Use Cases of ASK #

We have already introduced the basic product and features of Serverless Kubernetes. So, in which use cases is ASK suitable for?

1. Managed hosting of maintenance-free applications #

The biggest advantage of Serverless clusters is that they solve the operational and maintenance problems of underlying node resources. Therefore, they are very suitable for managed hosting of applications, allowing users to focus on application development itself. Applications in traditional K8s clusters can be seamlessly deployed in Serverless clusters, including various Helm Charts. At the same time, by combining reserved instance coupons, the long-term computing cost of Pods can be reduced.

2. Elastic Container Instance resource pool #

The second use case is the advantage of ACK on ECI. We can choose to add ECI as an elastic resource pool to the existing Kubernetes cluster. When the application load peaks, it can dynamically and flexibly scale through ECI. Compared with scaling ECS nodes, this is more efficient. This approach is more suitable for business scenarios with obvious peaks and troughs, such as e-commerce or online education. Users do not need to manage a large node resource pool and can reduce the overall computing cost through the elastic capability of ECI.

3. Big data computing #

The third use case is big data computing. Many users use Serverless clusters or ACK on ECI for data computing or machine learning purposes, such as using Spark/Presto/AI. By using ECI, resource planning and resource shortages can be easily addressed.

4. CI/CD continuous integration #

The fourth use case is CI/CD continuous integration. We can connect Jenkins and Gitlab-Runner to the ASK cluster, create CI/CD build tasks on-demand, and deploy them directly to the ASK testing environment for verification. This way, we don’t need to maintain a fixed resource pool for Job-like tasks. By creating resources on-demand, costs can be greatly reduced. Moreover, if combined with Spot instances, costs can be further reduced.

These are the typical use cases of Serverless Kubernetes clusters. There are also quick start links, product documentation, and usage examples for everyone’s learning and communication:

Console: https://cs.console.aliyun.com/ask
Product documentation: https://www.alibabacloud.com/help/doc-detail/86366.htm
Examples: https://github.com/AliyunContainerService/serverless-k8s-examples