25 Local Storage and Database Usage and Optimization

25 Local Storage and Database Usage and Optimization #

Hello, I am Chen Hang.

In the previous article, I took you through the process of learning about network programming in Flutter, which includes establishing a communication connection with a web server, data exchange, and parsing structured communication information.

Among them, there are three basic methods for establishing a communication connection in Flutter, including HttpClient, http, and dio. Considering that HttpClient and http do not support complex network request behaviors, I focused on introducing how to use dio to implement advanced operations such as resource access, interface data request and submission, file upload and download, and network interception.

As for parsing information, since Flutter does not support reflection, I only provided a manual way to parse JSON: convert JSON to a dictionary and then assign values to the properties of a custom class.

Because we have a network, our app has a channel for exchanging information with the outside world and therefore has the ability to update data. However, the data exchanged is usually stored in memory, and once the application finishes running, the memory is released, and this data disappears.

Therefore, we need to save these updated data in a certain form through a certain medium so that the data can be read from the storage medium the next time the application runs, achieving data persistence.

There are many scenarios where data persistence is required. For example, a user’s account login information needs to be saved for identity verification with the web service each time; or downloaded images need to be cached to avoid reloading them every time and wasting user data.

Since Flutter only takes control of the rendering layer, when it comes to storage and other underlying operating system behaviors, it still needs to rely on the native Android and iOS development. Similarly, Flutter provides three methods for data persistence, which are file storage, SharedPreferences, and databases. Next, I will explain these three methods to you in detail.

Files #

A file is a collection of ordered information stored at a specific path on some medium (such as a disk) and has a file name. From its definition, in order to achieve data persistence using files, we first need to determine one thing: where should the data be placed? This means defining the storage path for the file.

Flutter provides two directories for file storage, namely the temporary directory and the documents directory:

  • The temporary directory is a directory that can be cleared by the operating system at any time. It is usually used to store some temporary cache data that is not important. This directory corresponds to the value returned by NSTemporaryDirectory on iOS and getCacheDir on Android.
  • The documents directory, on the other hand, is a directory that is only cleared when the application is deleted. It is usually used to store important data files generated by the application. On iOS, this directory corresponds to NSDocumentDirectory, and on Android, it corresponds to the AppData directory.

Next, I will demonstrate how to perform file read and write operations in Flutter through an example.

In the code below, I have defined three functions: the function to create the file directory, the function to write to the file, and the function to read from the file. It is worth noting that file read and write operations are time-consuming operations, so these operations need to be performed in an asynchronous environment. Additionally, to prevent exceptions during file reading, we need to wrap the code in a try-catch block:

// Create the file directory
Future<File> get _localFile async {
  final directory = await getApplicationDocumentsDirectory();
  final path = directory.path;
  return File('$path/content.txt');
}

// Write the string to the file
Future<File> writeContent(String content) async {
  final file = await _localFile;
  return file.writeAsString(content);
}

// Read the string from the file
Future<String> readContent() async {
  try {
    final file = await _localFile;
    String contents = await file.readAsString();
    return contents;
  } catch (e) {
    return "";
  }
}

With the file read and write functions, we can now perform read and write operations on the content.txt file in our code. In the code below, we write a string to the file and then read it back after a while:

writeContent("Hello World!");
...
readContent().then((value)=>print(value));

In addition to reading and writing strings, Flutter also provides the ability to read and write binary streams, which supports reading and writing binary files such as images and compressed files. These topics are not the focus of this demo, but if you want to delve deeper, you can refer to the official documentation.

SharedPreferences #

Files are suitable for persisting large amounts of ordered data. If we only need to cache a small amount of key-value pairs (such as recording whether a user has read a notice or simple counting), we can use SharedPreferences.

SharedPreferences provides persistent storage for simple key-value pair data through native platform-specific mechanisms, using NSUserDefaults on iOS and SharedPreferences on Android.

Next, I will demonstrate how to read and write data using SharedPreferences in Flutter through an example. In the code below, we persist a counter to SharedPreferences and provide methods for reading and incrementing its value.

Please note that the setter (setInt) method synchronously updates the key-value pair in memory and then saves the data to disk, so we don’t need to call any update methods to flush the cache. Similarly, since file read and write operations are time-consuming, we must wrap these operations in an asynchronous manner:

// Read the value of 'counter' from SharedPreferences
Future<int> _loadCounter() async {
  SharedPreferences prefs = await SharedPreferences.getInstance();
  int counter = (prefs.getInt('counter') ?? 0);
  return counter;
}

// Increment the value of 'counter' and write it to SharedPreferences
Future<void> _incrementCounter() async {
  SharedPreferences prefs = await SharedPreferences.getInstance();
  int counter = (prefs.getInt('counter') ?? 0) + 1;
  prefs.setInt('counter', counter);
}

After wrapping the counter read and write methods, we can update and persist the counter data anytime in our code. In the code below, we first read and print the counter data, then increment it, and finally read and print it again:

// Read and print the counter data
_loadCounter().then((value) => print("before: $value"));

// Increment the counter data, then read and print it again
_incrementCounter().then((_) {
  _loadCounter().then((value) => print("after: $value"));
});

As you can see, the usage of SharedPreferences is extremely simple and convenient. However, it is important to note that key-value pairs can only store basic data types such as int, double, bool, and string.

Database #

Although SharedPreferences is convenient to use, it is only suitable for scenarios where a small amount of data needs to be persisted. It cannot be used to store a large amount of data, such as file contents (file paths are allowed).

If we need to persist a large amount of formatted data that will be updated frequently, and in order to consider further scalability, we usually choose a SQLite database to handle such scenarios. Compared to files and SharedPreferences, databases can provide faster and more flexible solutions for data reading and writing.

Next, I will introduce to you the usage of a database using an example.

Let’s take the Student class mentioned in the previous article as an example:

class Student {
  String id;
  String name;
  int score;

  // Constructor
  Student({this.id, this.name, this.score});

  // Factory method that converts a JSON dictionary to a class object
  factory Student.fromJson(Map<String, dynamic> parsedJson) {
    return Student(
      id: parsedJson['id'],
      name: parsedJson['name'],
      score: parsedJson['score'],
    );
  }
}

The JSON class has a factory method that can convert a JSON dictionary to a class object. We can also provide an instance method to convert a class object back to a JSON dictionary. Because the actual data stored in the database is not the entity class object itself, but a dictionary composed of strings, integers, and other basic types, we can use these two methods to achieve database reading and writing. In addition, we also define three Student objects that will be inserted into the database later:

class Student {
  // ...

  // Convert a class object to a JSON dictionary for easy insertion into the database
  Map<String, dynamic> toJson() {
    return {'id': id, 'name': name, 'score': score};
  }
}

var student1 = Student(id: '123', name: '张三', score: 90);
var student2 = Student(id: '456', name: '李四', score: 80);
var student3 = Student(id: '789', name: '王五', score: 85);

With the entity class as the object to be stored in the database, the next step is to create the database. In the following code, we use the openDatabase function to specify the path for the database storage, and through the database table initialization statement, create a students table to store Student objects:

final Future<Database> database = openDatabase(
  join(await getDatabasesPath(), 'students_database.db'),
  onCreate: (db, version) => db.execute(
      "CREATE TABLE students(id TEXT PRIMARY KEY, name TEXT, score INTEGER)"),
  onUpgrade: (db, oldVersion, newVersion) {
    // do something for migration
  },
  version: 1,
);

The above code is a general template for creating a database, and there are three things to note:

  1. When setting the database storage path, we use the join method to concatenate two parts of the path. The join method will use the path separator of the operating system, so we don’t need to worry about whether the path separator should be “/” or “".
  2. When creating the database, we pass in a version 1, and there is also a version inside the onCreate method. These two versions are identical.
  3. The database is created only once, which means that the onCreate method will only be executed once during the lifecycle of the application, from installation to uninstallation. If we want to modify the storage fields of the database during the version upgrade process, how should we handle it? SQLite provides the onUpgrade method for this purpose. We can determine the upgrade strategy based on the oldVersion and newVersion passed in to this method. The former represents the database version on the user’s phone, while the latter represents the current version of the database. For example, our application has three versions: 1.0, 1.1, and 1.2. In 1.1, we upgraded the database version to 2. Considering that the user’s upgrade sequence is not always continuous, and they may directly upgrade from 1.0 to 1.2, we can compare the current database version with the database version on the user’s phone in the onUpgrade function to formulate a database upgrade plan.

After creating the database, we can insert the three previously created Student objects into the database. Database insertion requires calling the insert method. In the following code, we convert the Student objects to JSON and, after specifying the insertion conflict strategy (if the same object is inserted twice, the latter replaces the former) and the target database table, complete the insertion of Student objects:

Future<void> insertStudent(Student std) async {
  final Database db = await database;
  await db.insert(
    'students',
    std.toJson(),
    conflictAlgorithm: ConflictAlgorithm.replace, // Insert conflict resolution strategy: replace the old one with the new one
  );
}

// Insert the 3 Student objects
await insertStudent(student1);
await insertStudent(student2);
await insertStudent(student3);

After inserting the data, we can then retrieve them by calling the query method. It is worth noting that when writing, we insert one by one in order, but when reading, we use batch reading (of course, you can also specify query rules to read specific objects). The retrieved data is an array of JSON dictionaries, so we need to convert it back to an array of Student objects. Finally, don’t forget to release the database resources:

Future<List<Student>> students() async {
  final Database db = await database;
  final List<Map<String, dynamic>> maps = await db.query('students');
  return List.generate(maps.length, (i) => Student.fromJson(maps[i]));
}

// Retrieve the collection of Student objects inserted in the database
students().then((list) => list.forEach((s) => print(s.name)));
// Release the database resources
final Database db = await database;
db.close();

As you can see, when dealing with a large number of formatted data models, databases provide a faster and more flexible solution for persistence.

In addition to basic database read and write operations, SQLite also provides advanced features such as update, delete, and transactions, which are no different from native SQLite or MySQL on Android and iOS. Therefore, there is no need to elaborate on these features here. You can refer to the API documentation of the sqflite plugin or consult the SQLite tutorial for specific usage methods.

Summary #

Alright, that’s all for today’s sharing. Let’s briefly review the content we have learned today.

Firstly, I introduced you to file, which is the most common way of persisting data. Flutter provides two types of directories, namely, temporary directory and document directory. We can use these directories to persist data by writing strings or binary streams based on our actual needs.

Next, I explained SharedPreferences to you through a small example. SharedPreferences is a storage solution suitable for persisting small key-value pairs.

Lastly, we learned about databases together. I introduced you to the methods of creating, writing, and reading from a database, focusing on how to persist an object into the database. As you can see, although using databases requires more initial preparation work, it provides stronger adaptability and flexibility in the face of ongoing changes in requirements.

Data persistence is a CPU-intensive operation, so both data access and storage will involve a large number of asynchronous operations. Therefore, it is crucial to use asynchronous waits or register then callbacks to correctly manage the sequential relationship of read and write operations.

I have packaged the knowledge points covered in today’s sharing on GitHub, you can download it, run it multiple times, and deepen your understanding and memory.

Discussion Questions #

Finally, I’ll leave you with two discussion questions.

  1. Please introduce the applicable scenarios for each of the three data persistence methods: files, SharedPreferences, and databases.
  2. Our application has gone through three versions: 1.0, 1.1, and 1.2. In version 1.0, a database was created with a table named “Student”. In version 1.1, the “Student” table was modified to add a field called “age” (ALTER TABLE students ADD age INTEGER). Please write the database upgrade code for versions 1.1 and 1.2.
// Database creation code for version 1.0
final Future<Database> database = openDatabase(
  join(await getDatabasesPath(), 'students_database.db'),
  onCreate: (db, version) => db.execute("CREATE TABLE students(id TEXT PRIMARY KEY, name TEXT, score INTEGER)"),
  onUpgrade: (db, oldVersion, newVersion) {
    // do something for migration
  },
  version: 1,
);

Feel free to leave your thoughts in the comments section. I’ll be waiting for you in the next article! Thank you for listening and feel free to share this article with more friends to read together.