08 Ownership: How Borrowing Works in Value? #
Hello, I’m Chen Tian.
In the last lecture, we learned about the basic rules of ownership in Rust, where a value has a single owner.
When we perform variable assignments, pass arguments, and return functions, if the involved data structure does not implement the Copy trait, the ownership of the value will default to being transferred using Move semantics, and the variable that loses ownership will no longer be able to access the original data; if the data structure has implemented the Copy trait, Copy semantics will be used, automatically copying the value, and the original variable can continue to access it.
Although, single ownership solves the problems caused by arbitrary sharing of values in other languages, it has also caused some inconvenience. We mentioned in the last lecture: What if you do not want the ownership of the value to be transferred and you cannot use Copy semantics because the Copy trait is not implemented? You can “borrow” data, which is Borrow semantics we will continue to introduce in this lecture.
Borrow Semantics #
As the name suggests, Borrow semantics allow the ownership of a value to be used by other contexts without transferring ownership. It’s like staying in a hotel or renting a house, where the guests/tenants only have temporary rights to use the room, but not its ownership. Additionally, Borrow semantics are realized through reference syntax (& or &mut).
At this point, you might be a bit confused; why introduce a new concept of “borrowing,” but then write “reference” syntax?
In fact, in Rust, “borrowing” and “reference” are the same concept, it’s just that the meaning of reference in other languages is different from Rust, so Rust introduced the new concept of “borrowing” to distinguish them.
In other languages, a reference is a kind of alias. You can simply understand it as Lu Xun is to Zhou Shuren, multiple references have indistinct access rights to a value, essentially sharing ownership; while in Rust, all references are merely borrowing “temporary usage rights,” and it does not violate the constraint of a single ownership of a value.
Therefore, by default, Rust’s borrows are read-only, just like staying in a hotel, you must leave the room intact when checking out. But in some cases, we also need mutable borrowing, like renting a house, where you can do necessary decorations on the house, which will be explained in detail later.
So, if we want to avoid Copy or Move, we can use borrowing, or say referencing.
Read-only Borrow/Reference #
Essentially, a reference is a controlled pointer that points to a specific type. When learning other languages, you will notice that there are two ways to pass arguments to functions: pass-by-value and pass-by-reference.
Taking Java as an example, passing an integer to a function is pass-by-value, consistent with Rust’s Copy semantics; whereas passing an object, or any data structure on the heap, Java will automatically pass the reference implicitly. As previously mentioned, Java’s reference is an alias of the object, which also causes references to the same memory block to be everywhere as the program executes, necessitating reliance on GC for memory recovery.
But Rust has no concept of pass-by-reference. All parameter passing in Rust is pass-by-value, whether it is Copy or Move. So in Rust, you must explicitly pass a reference of certain data to another function.
Rust’s references implement the Copy trait, so according to Copy semantics, this reference will be copied and handed over to the called function. For this function, it does not own the data itself; the data is temporarily lent to it for use, and the ownership is still with the original owner.
In Rust, references are first-class citizens, equal in status to other data types.
Still using the code 2 from the last lecture with two errors to demonstrate.
fn main() {
let data = vec![1, 2, 3, 4];
let data1 = data;
println!("sum of data1: {}", sum(data1));
println!("data1: {:?}", data1); // error1
println!("sum of data: {}", sum(data)); // error2
}
fn sum(data: Vec<u32>) -> u32 {
data.iter().fold(0, |acc, x| acc + x)
}
We slightly alter code 2 by adding a reference to let it compile and view the addresses of the value and reference (code 3):
fn main() {
let data = vec![1, 2, 3, 4];
let data1 = &data;
// What's the address of the value? What about the address of the reference?
println!(
"addr of value: {:p}({:p}), addr of data {:p}, data1: {:p}",
&data, data1, &&data, &data1
);
println!("sum of data1: {}", sum(data1));
// What is the address of the data on the heap?
println!(
"addr of items: [{:p}, {:p}, {:p}, {:p}]",
&data[0], &data[1], &data[2], &data[3]
);
}
fn sum(data: &Vec<u32>) -> u32 {
// Will the address of the value change? Will the address of the reference change?
println!("addr of value: {:p}, addr of ref: {:p}", data, &data);
data.iter().fold(0, |acc, x| acc + x)
}
Before running this block of code, you can think about it first: whether the address corresponding to the value of data remains unchanged, and if the address of the reference data1, after being passed to the sum() function, still points to the same address.
Okay, if you have an idea, you can run the code to verify if you are correct. We will then analyze the following diagram:
data1, &data, and data1' passed into sum() all point to data itself, this value’s address is fixed. However, their reference addresses are different, which confirms what we discussed when introducing the Copy trait, that read-only references implement the Copy trait, which means assignment and passing references will produce new shallow copies.
Although there are many read-only references pointing to data, the data on the heap still only has one owner, data, so any number of references to a value do not affect the uniqueness of ownership.
But we immediately found a new problem: once data leaves the scope and is released, if there are still references pointing to data, wouldn’t that cause the memory safety problem we extremely want to avoid, like using freed memory (use after free)? What to do?
The Lifetime of Borrowing and Its Constraints #
So, we need constraints on references to values, which is: borrowing cannot outlive the value’s lifetime.
This constraint is intuitive and easy to understand. In the code above, the sum() function is on the next level of the call stack from the main() function, after it ends, the main() function will continue to execute, so the lifespan of data defined in the main() function is longer than the reference to data in sum(), so there is no problem.
But what about code like this (scenario 1)?
fn main() {
let r = local_ref();
println!("r: {:p}", r);
}
fn local_ref<'a>() -> &'a i32 {
let a = 42;
&a
}
Obviously, the variable r in the longer-lived main() function is referencing the local variable in the shorter-lived local_ref() function, which violates the constraints regarding references, so Rust does not allow such code to compile.
Then, what if we use a reference from stack memory in heap memory?
Based on past development experience, you might blurt out: No! Because the lifetime of heap memory is clearly longer and more flexible than stack memory, doing so is unsafe.
We write a piece of code to test it, storing a reference to a local variable in a mutable array. From the basics, we know that mutable arrays are placed on the heap, and there is only a fat pointer on the stack pointing to it, so this is a classic example of storing a reference to stack memory on the heap (scenario 2):
fn main() {
let mut data: Vec<&u32> = Vec::new();
let v = 42;
data.push(&v);
println!("data: {:?}", data);
}
Surprisingly it compiles! What’s going on? Let’s change it up and see if it still compiles (scenario 3), and again it doesn’t!
fn main() {
let mut data: Vec<&u32> = Vec::new();
push_local_ref(&mut data);
println!("data: {:?}", data);
}
fn push_local_ref(data: &mut Vec<&u32>) {
let v = 42;
data.push(&v);
}
By this point, are you a bit confused? These three scenarios are seemingly complicated, but if you grasp a core element, “in one scope, at the same time, a value can only have one owner,” you will find, in fact, it is simple.
Heap memory’s lifetime does not have the flexibility of being arbitrarily long or short, because the life and death of heap memory are firmly bound to the stack owner. And the lifetime of stack memory is also related to the lifetime of the stack, so we only need to focus on the lifespan of the call stack.
Now can you easily judge why the code in scenarios 1 and 3 can’t compile, because they reference values with shorter lifetimes, while the code in scenario 2 can compile despite referencing stack memory in heap memory, but the lifespans are the same?
Okay, that covers the default situation of Rust’s read-only borrowing, where the borrower cannot modify the borrowed value, simply analogized as staying in a hotel, only having the right to use.
But as mentioned earlier, there are some situations where we also need mutable borrowing, wanting to modify the content of the value during the borrowing process, like renting a house, needing to do necessary decorations.
Mutable Borrow/Reference #
Before introducing mutable borrowing, because a value has only one owner at any given moment, if you want to modify this value, you can only do so through the sole owner. But allowing borrowing to change the value itself brings new problems.
Let’s first look at the first situation, multiple mutable references coexisting:
fn main() {
let mut data = vec![1, 2, 3];
for item in data.iter_mut() {
data.push(*item + 1);
}
}
This code, while iterating over the mutable array data, is also adding new data to data. This is a very dangerous move as it breaks the loop invariant potentially leading to an infinite loop and even system crashes. Thus, having multiple mutable references in the same scope is unsafe.
As the Rust compiler prevents this situation, the above code will fail to compile. We can use Python to experience the potential infinite loop that multiple mutable references might cause:
if __name__ == "__main__":
data = [1, 2]
for item in data:
data.append(item + 1)
print(item)
# unreachable code
print(data)
Multiple mutable references in the same context are unsafe. What about having one mutable reference and several read-only references at the same time? Let’s look at another piece of code:
fn main() {
let mut data = vec![1, 2, 3];
let data1 = vec![&data[0]];
println!("data[0]: {:p}", &data[0]);
for i in 0..100 {
data.push(i);
}
println!("data[0]: {:p}", &data[0]);
println!("boxed: {:p}", &data1);
}
In this code, the immutable array data1 references an element from the mutable array data, which is a read-only reference. Subsequently, we add 100 elements to data, accessing the mutable reference of data when calling data.push().
In this code, with data’s read-only reference and mutable reference coexisting, there seems to be no impact since the element referenced by data1 has not changed at all.
If you reflect carefully, you will notice the potential unsafe operation here: if we keep adding elements, and the reserved space in heap memory is not enough, a new large enough memory block will be reallocated, the previous values will be copied over, then the old memory will be released. This will render the reference in data1, &data[0], invalid, leading to memory safety problems.
Rust’s Limitations #
Both issues—multiple mutable references coexisting, mutable references and read-only references coexisting—can be avoided by solutions such as GC for the second issue, but GC can do nothing about the first issue.
To ensure memory safety, Rust imposes strict constraints on the use of mutable references:
- Only one active mutable reference is allowed in a scope. Active means the mutable reference that is actually used to modify data. If it is defined but not used or is used as a read-only reference, it is not considered active.
- In a scope, an active mutable reference (write) and a read-only reference (read) are mutually exclusive and cannot coexist.
Does this constraint feel familiar? Yes, it is similar to the rules of concurrent data access (such as RwLock). You can make the comparison for learning.
From the constraints on mutable references, we can see that Rust not only solves memory safety issues that can be addressed by GC but also tackles problems that GC can’t solve. When writing code, the Rust compiler acts like your good mentor and friend, constantly urging you to adopt best practices to write safe code.
After learning today’s content, if you look back at the first-principles diagram [starting words] displayed, isn’t your understanding deeper?
In fact, if we strip away the numerous rules of ownership, dig into the underlying concepts, clarify how values are stored in the heap or stack, how values are accessed in memory, then start from these concepts, either extend their denotation or limit their use, fundamentally seek solutions, this is the best way to handle complex problems, and also Rust’s design philosophy.
Summary #
Today we learned Borrow semantics, clarified the principles of read-only and mutable references, combined with the Move/Copy semantics studied in the previous lecture, the Rust compiler will check to ensure that the code does not violate these series of rules:
- A value has only one owner at any moment. Once the owner leaves the scope, the value is discarded. Assignment or passing arguments will result in the value moving, transferring ownership, and once ownership is transferred, previous variables cannot access it.
- If a value implements the Copy trait, then assignment or argument passing will use Copy semantics, and the corresponding value will be copied creating a new value.
- A value can have multiple read-only references.
- A value can have only one active mutable reference. Mutable references (write) and read-only references (read) are mutually exclusive, just like the mutual exclusion of data read and write in concurrency.
- A reference’s lifetime cannot outlive the value’s lifetime.
You can also look at this diagram for a quick review:
But there are always some special cases, such as DAG, where we want to bypass the limitation of “a value can only have one owner,” what to do? In the next lecture, we will continue to learn…
Thinking Questions #
-
In the last lecture, when we talked about the Copy trait, we mentioned that mutable references do not implement the Copy trait. Combine the content of this lecture, think why?
-
How can the following code be modified to make it compile, to avoid having read-only references and mutable references at the same time?
fn main() {
let mut arr = vec![1, 2, 3];
// cache the last item
let last = arr.last();
arr.push(4);
// consume previously stored last item
println!(“last: {:?}”, last);
}
Feel free to share your thoughts in the comments. Today you have completed Rust’s eighth punch of learning! If you feel you have gained something, you are also welcome to share it with friends around you and invite them to join the discussion.
References #
Some students asked in the comments how mutable references can cause heap memory to be reallocated. Let’s look at an example. First, I allocate a Vec with a capacity of 1, then put 32 elements in it. At this point, it will reallocate, and when we print the addresses of &v[0] before and after the reallocation, we will see a change.
Therefore, if we hold on to the old address of &v[0], we will read from freed memory. This is why in the text, I say why mutable references and read-only references cannot coexist in the same scope ([