27 Ecosystem: What are some common Rust libraries available for use? #

Hello, I’m Tyr Chen.

The capabilities of a programming language are determined 40% by the design of the language itself and 60% by the ecosystem built around it.

[Previously], we compared Golang and Rust, and in my opinion, Golang is a language with equally prominent advantages and disadvantages. Some of Golang’s shortcomings are very serious. However, with Google’s strong support, and riding on the trends of microservices and cloud-native, Golang has built a very grand ecosystem. Basically, if you want to do microservices, Golang’s comprehensive third-party libraries can meet almost all your needs.

Therefore, an ecosystem can compensate for the disadvantages of a language, the capabilities a programming language presents externally is a collection of the language plus its ecosystem.

For example, due to the lack of macro programming support, Golang has to introduce a large amount of scaffolding code when developing many projects. Writing these scaffolding codes by oneself is time-consuming and laborious, but there will be many excellent frameworks in the community to help you generate them.

A typical example is kubebuilder, which directly reduces the barrier to developing Kubernetes operators. Without such tools, developing Kubernetes with Golang is not easier than with Python. On the other hand, thanks to the outstanding and practical ecosystem in data science and machine learning, Python is able to dominate in these two fields, unbeaten and unparalleled.

So, what does the Rust ecosystem look like? What can we do with Rust? Why do I say that the Rust ecosystem is already good and has great potential and sustainability? Let’s talk about this topic.

Today’s content mainly enriches your understanding of the Rust ecosystem, making it easier for you to quickly find suitable libraries and tools when doing different projects. Of course, I can’t list all the important crates, but if the content in this article does not cover your needs, you can also search on crates.io yourself.

Basic Libraries #

First, let’s introduce some libraries that might be used in various applications.

Let’s briefly mention them in order of importance for your convenience: serialization and deserialization tool serde, networking and high-performance I/O library tokio, error handling libraries thiserror and anyhow, command-line processing library clap and others, asynchronous handling libraries futures and async-trait, crossbeam which offers concurrency-related data structures and algorithms, and parser writing libraries nom and others. -

serde #

Every developer who switches to Rust from other languages is amazed by the powerful capabilities of serde and its surrounding libraries. Simply by adding the #[derive(Serialize, Deserialize)] macro on a data structure, it can be serialized and deserialized to and from most formats: JSON/YAML/TOML/MsgPack/CSV/Bincode, and so on.

You can also write support for your formats for serde, such as using DynamoDB, you can use serde_dynamo:

#[derive(Serialize, Deserialize)]
pub struct User {
    id: String,
    name: String,
    age: u8,
};

// Get documents from DynamoDB
let input = ScanInput {
    table_name: "users".to_string(),
    ..ScanInput::default()
};
let result = client.scan(input).await?;

if let Some(items) = result.items {
    // Just a sentence, and you get a list of Users
    let users: Vec<User> = serde_dynamo::from_items(items)?;
    println!("Got {} users", users.len());
}

If you have used ORM in other languages, you can think of serde as an enhanced, universal ORM. It can serialize any serializable data structure to any format, or deserialize from any format.

What data structures are not “serializable”? Simply put, any data structure whose state cannot be easily reconstructed, such as a TcpStream, a file descriptor, a Mutex, are not serializable, while a HashMap> is serializable.

tokio #

If you want to deal with high-performance networking in Rust, you cannot ignore tokio and its surrounding libraries.

Tokio’s place in Rust is similar to Golang’s runtime for handling concurrency. However, Golang developers have no choice but to use the runtime, whereas Rust developers can choose not to use any runtime, or selectively use tokio/async-std/smol when necessary.

Among all these runtimes, the most commonly used and widely used is tokio, with libraries such as tonic/axum/tokio-uring/tokio-rustls/tokio-stream/tokio-util for networking and asynchronous IO, as well as bytes/tracing/prost/mio/slab and others. We briefly read about bytes when discussing [how to read Rust code], and encountered many of the libraries mentioned here during the writing of the KV server.

thiserror/anyhow #

Two libraries for error handling, thiserror/anyhow, are recommended to master as they are currently the most mainstream error handling tools in the Rust ecosystem.

If you’re still not very familiar with their use, you can review the [error handling] lesson, and see how we used thiserror and anyhow in the KV server.

clap/structopt/dialoguer/indicatif #

clap and structopt remain the main choices for Rust command-line processing, where clap 3 has integrated structopt. So, once it releases a stable version, users can switch from structopt with confidence.

If you want an interactive command line, dialoguer is a good choice. If you want to provide friendly progress bars in the command line, try indicatif.

futures/async-trait #

Although we have not formally learned about futures, we have used the futures library and async-trait library in many contexts.

The future trait from the futures library has been adopted by the standard library and made asynchronous handling part of the language through the async/await keywords. However, the futures library still contains many other important traits and data structures, such as Stream/Sink which we have used before. The futures library also comes with a simple executor that can replace tokio during testing.

The async-trait library, as the name suggests, is designed to solve the problem of Rust not yet supporting async fn in traits.

crossbeam #

crossbeam is an excellent library under Rust that deals with concurrency and concurrency-related data structures. When you need to write your scheduler, you can consider using deque; when you need a better-performing MPMC channel, you can use channel; when you need an epoch-based GC, you can use epoch.

nom/pest/combine #

All three are excellent parser libraries that can be used to write efficient parsers.

Under Rust, when you need to handle certain file formats, you can first consider serde, and then these libraries; if you need to handle syntax, they are the best choices. I personally favor nom, followed by combine. They are parser combinator libraries, and pest is a PEG library where you can define the grammar with an EBNF-like structure and then access the generated code.

Web and Web Service Development #

Although Rust is much younger relative to many other languages, the fierce competition in the Rust web development tooling is no less than Golang/Python and other more mature languages.

From the perspective of web protocol support, Rust has hyper for processing http1/http2, quinn/quiche for QUIC/http3, tonic for gRPC, and tungstenite/tokio-tungstenite for websocket.

From the perspective of protocol serialization/deserialization, Rust has avro-rs for apache avro, capnp for Cap’n Proto, prost for protobuf, flatbuffers for google flatbuffers, thrift for apache thrift, and serde_json for the familiar JSON.

In general, if you offer REST/GraphQL API, JSON is the preferred serialization tool; if you offer binary protocols, without special circumstances (such as games which tend towards flatbuffers), protobuf is recommended.

From the web framework perspective, there’s actix-web, which has the top performance in the universe; there’s rocket, which is easy to use and soon will support asynchronous programming, greatly improving performance; there’s also the promising newcomer axum recently released by the tokio community.

In our get hands dirty project, where we implemented thumbor with Rust, we used axum. If you like Django-like all-encompassing web frameworks, you can try rocket version 0.5 or above. If you care a lot about web performance, consider actix-web.

From the database support perspective, Rust supports almost all mainstream databases, including but not limited to MySQL, Postgres, Redis, RocksDB, Cassandra, MongoDB, ScyllaDB, CouchDB, etc. If you enjoy using ORM, diesel or sea-orm are available. If you enjoy direct but safe SQL queries, use sqlx.

From the template engine perspective, Rust has askama that supports jinja syntax, tera similar to jinja2, and comrak for markdown processing.

From the web frontend perspective, Rust offers yew and seed for front-end development, and MoonZoon which leans more towards full-stack development. Among them, yew is a bit more mature, easier for those familiar with react/elm to get started with.

From the perspective of web testing, Rust has headless_chrome that rivals puppeteer, and thirtyfour and fantoccini which parallel selenium.

From the cloud platform deployment perspective, Rust supports aws with rusoto and aws-sdk-rust, azure with azure-sdk-for-rust. Currently, Google Cloud, Alibaba Cloud, and Tencent Cloud do not have official SDK support.

In the static website generation field, Rust has zola competing with hugo and mdbook akin to gitbook. They are all very mature products and safe to use.

Client-side Development #

The client-side mentioned here specifically refers to GUI client development. CLI has been mentioned [previously] and will not be elaborated here.

On the areweguiyet.com page, we see numerous GUI libraries. Personally, I think promising cross-platform solutions are tauri, druid, iced and sixtyfps.

Tauri is an alternative to Electron. If you are tired of Electron’s large size and greedy memory usage but still like to use the web tech stack to build client GUIs, then Tauri is worth trying. It uses the webview built into the operating system, plus the extremely restrained memory usage of Rust itself, its performance and memory usage can far surpass Electron.

The remaining three provide native GUI, with sixtyfps being a very good choice for embedded systems with good support. However, note that its license is GPLv3, so be cautious when using it for commercial products (it has a commercial license).

If you hope to create richer, more outstanding GUIs, you can use skia-safe and tiny-skia. The former is the Rust binding of Google’s Skia graphics engine, and the latter is a subset compatible with Skia. Skia is currently the underlying graphics engine of Flutter, which is very popular in cross-platform GUI, allowing you to perform any complex layer processing.

Of course, you can draw the UI with Flutter and build the logic layer with Rust. Rust can output C FFI, which dart can wrap for use by Flutter.

Cloud Native Development #

Cloud-native has always been dominated by Golang. If you look at Kubernetes ecosystem operators in use, they are almost uniformly written in Golang.

However, Rust is gradually emerging in this field. This is thanks to the previously mentioned serde and the great efforts of the kube-rs project dealing with Kubernetes API, as well as Rust’s strong macro programming capabilities, which make interacting with Kubernetes incredibly easy.

For instance, to build a CRD:

use kube::{CustomResource, CustomResourceExt};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};

// Book as a new Custom resource
#[derive(CustomResource, Debug, Clone, Serialize, Deserialize, JsonSchema)]
#[kube(group = "k8s.tyr.app", version = "v1", kind = "Book", namespaced)]
pub struct BookSpec {
    pub title: String,
    pub authors: Option<Vec<String>>,
}

fn main() {
    let book = Book::new(
        "rust-programming",
        BookSpec {
            title: "Rust programming".into(),
            authors: Some(vec!["Tyr Chen".into()]),
        },
    );
    println!("{}", serde_yaml::to_string(&Book::crd()).unwrap());
    println!("{}", serde_yaml::to_string(&book).unwrap());
}

With just 20 lines of code, a CRD is created. Is it not clean and smooth, and effortless to write?

❯ cargo run | kubectl apply -f -
    Finished dev [unoptimized + debuginfo] target(s) in 0.14s
     Running `/Users/tchen/.target/debug/k8s-controller`
customresourcedefinition.apiextensions.k8s.io/books.k8s.tyr.app configured
book.k8s.tyr.app/rust-programming created

❯ kubectl get crds
NAME                CREATED AT
books.k8s.tyr.app   2021-10-20T01:44:57Z

❯ kubectl get book
NAME               AGE
rust-programming   5m22s

If you’ve used Golang’s kubebuilder to do similar things, don’t you find that the process of generating a large amount of scaffolding code and numerous YAML files suddenly less appealing?

While Rust is still a novice in the field of cloud-native, this novice has a strong capability for dimensional reduction. The same functionality, Rust can be implemented with about 1/4 to 1/10 of Golang’s code, thanks to the power of Rust’s macro programming.

In addition to foundational libraries like kube, Rust also has emerging krator and krustlet. Krator can help you build Kubernetes operators better. Although operators don’t emphasize efficiency, less code to achieve more functionality and lower memory usage means I am very optimistic about more Kubernetes operators being developed with Rust in the future.

Krustlet, as the name suggests, is intended to replace the kubelet. Krustlet uses wasmtime as the data platform (dataplane) runtime instead of traditional containerd. This means that you can handle tasks previously only possible with containers using more efficient, more streamlined WebAssembly.

Currently, WebAssembly’s use in the cloud-native field, particularly in serverless/FAAS, is still in its early stages, and the ecosystem is not fully mature, but it is definitely a differentiating factor compared to bulky containers.

Another major direction of cloud-native is serverless. In this field, since Amazon open-sourced the high-performance micro VM firecracker developed with Rust, Rust has taken a leading position in the serverless/FAAS arena.

WebAssembly Development #

If Web development and cloud-native are fields where Rust excels, then WebAssembly can be said to be one of Rust’s main battlefields.

Rust contains wasm32-unknown-unknown as a compile target by default. If you haven’t added it, you can add it with rustup, then specify the target when compiling to get wasm:

$ rustup target add wasm32-unknown-unknown
$ cargo build --target wasm32-unknown-unknown --release

You can use wasm-pack and wasm-bindgen to