24 HTTP Networking and JSON Parsing #
Hello, I am Chen Hang.
In the previous article, I took you through the mechanism and implementation principles of asynchronous and concurrent programming in Dart. Similar to other languages, Dart achieves asynchronous programming through event loops and queues, and we can use Futures to encapsulate asynchronous tasks. On the other hand, although Dart is based on a single-threaded model, it also provides “multi-threading” capabilities such as Isolates, which allow us to fully utilize system resources and handle CPU-intensive tasks in concurrent Isolates, notifying the main Isolate of the execution result through messaging.
One typical application scenario for asynchronous and concurrent programming is network programming. A good mobile application not only needs a good interface and user-friendly interactive experience, but also requires the ability to interact with the outside world. Through the network, a bidirectional communication channel can be established between the isolated client and server, enabling operations such as resource access, interface data requests and submissions, and file uploading and downloading.
To facilitate the rapid implementation of information exchange based on network channels and real-time update of app data, Flutter also provides a series of network programming libraries and tools. Therefore, in today’s sharing session, I will use some small examples to explain how to implement data exchange with the server in a Flutter application and how to format the response data of the interaction.
HTTP Network Programming #
When we interact with server data over the network, we inevitably need to use three concepts: addressing, transmission, and application.
Among them, addressing defines how to accurately find one or more hosts on the network (i.e. IP address); transmission is mainly responsible for efficient and reliable data communication after finding the host (i.e. TCP, UDP protocols); and application is responsible for identifying the content of the communication between the two parties (i.e. HTTP protocol).
When we communicate data, we can choose to use only the transport layer protocol. However, the data transmitted by the transport layer is in binary format, and without the application layer, we cannot identify the content of the data. If we want the transmitted data to be meaningful, we must use an application layer protocol. Mobile applications typically use the HTTP protocol as the application layer protocol to encapsulate HTTP information.
In programming frameworks, an HTTP network call is usually divided into the following steps:
- Create a network call instance
clientand set general request behaviors (such as timeout). - Construct the URI, set request headers and body.
- Make the request and wait for the response.
- Decode the response content.
Of course, Flutter is no exception. In Flutter, there are three main ways to implement HTTP network programming: using HttpClient from dart:io, using the native Dart http library, and using the third-party library dio. Next, I will explain these three methods one by one.
HttpClient #
HttpClient is a network request class provided in the dart:io library, which implements basic network programming functionalities.
Next, I will share an example with you to demonstrate how to use HttpClient according to the steps mentioned above.
In the code below, we create an instance of HttpClient for network call and set its timeout to 5 seconds. Then we construct the URI of the Flutter official website and set the user-agent header to “Custom-UA”. After that, we make the request and wait for the response from the Flutter official website. Finally, after receiving the response, we print the result:
get() async {
// Create a network call instance and set general request behavior (timeout)
var httpClient = HttpClient();
httpClient.idleTimeout = Duration(seconds: 5);
// Construct the URI and set user-agent to "Custom-UA"
var uri = Uri.parse("https://flutter.dev");
var request = await httpClient.getUrl(uri);
request.headers.add("user-agent", "Custom-UA");
// Make the request and wait for the response
var response = await request.close();
// Received response, print the result
if (response.statusCode == HttpStatus.ok) {
print(await response.transform(utf8.decoder).join());
} else {
print('Error: \nHttp status ${response.statusCode}');
}
}
As you can see, it is relatively simple to make network calls using HttpClient.
One thing to note here is that since network requests are asynchronous operations, in Flutter, all network programming frameworks use Future as the wrapper for asynchronous requests. Therefore, we need to use await and async to perform non-blocking waiting. Of course, you can also register then to handle the corresponding events in a callback manner.
http #
Although the usage of HttpClient is simple, its interface exposes some implementation details. For example, the asynchronous call is too detailed, the client needs to actively close the connection, the response is in string format but needs to be manually decoded, etc.
http is another network request class provided by Dart official, which is much easier to use compared to HttpClient. Similarly, let me introduce the usage of http with an example.
First, we need to add http to the dependencies in pubspec.yaml:
dependencies:
http: '>=0.11.3+12'
In the code below, similar to the HttpClient example, we also construct an http network call instance and the URI of the Flutter official website. After setting the user-agent to “Custom-UA”, we make the request and finally print the request result:
httpGet() async {
// Create a network call instance
var client = http.Client();
// Construct the URI
var uri = Uri.parse("https://flutter.dev");
// Set user-agent to "Custom-UA" and immediately make the request
http.Response response = await client.get(uri, headers : {"user-agent" : "Custom-UA"});
// Print the request result
if(response.statusCode == HttpStatus.ok) {
print(response.body);
} else {
print("Error: ${response.statusCode}");
}
}
As you can see, compared to HttpClient, the usage of http is simpler, requiring only one asynchronous call to achieve basic network communication.
dio #
Although the usage of HttpClient and http is simple, they both have relatively limited customizable capabilities, and many common functions are not supported (or implemented in a cumbersome way), such as canceling requests, custom interceptors, cookie management, etc. Therefore, for complex network request behaviors, I recommend using dio, a popular third-party library in the Dart community.
Next, I will introduce the usage of dio through several examples. Similar to http, we need to add dio to the dependencies in pubspec.yaml.
dependencies:
dio: '>2.1.3'
In the following code, similar to the previous examples of HttpClient and http, we first create an instance of dio for network calls, then create a URI, set Header, make the request, and finally wait for the request result:
void getRequest() async {
// Create an instance for network calls
Dio dio = new Dio();
// Set URI, set user-agent header, and make the request
var response = await dio.get("https://flutter.dev", options: Options(headers: {"user-agent" : "Custom-UA"}));
// Print the request result
if(response.statusCode == HttpStatus.ok) {
print(response.data.toString());
} else {
print("Error: ${response.statusCode}");
}
}
It is worth noting that the actions of creating a
URI, settingHeader, and making the request are all performed using thedio.getmethod. Theoptionsparameter of this method provides the ability to finely control network requests, including settingHeader, timeout,Cookie, and request method. This part is not the focus of today’s presentation. If you want to understand it in depth, you can visit its API documentation to learn more about the specific usage.
For common file upload and download requirements, dio also provides good support. File upload can be achieved by building a FormData form, while file download can be done using the download method.
In the following code, we use FormData to create two files to be uploaded, and send them to the server using the post method. The usage of download is easier. We directly provide the URL of the file to be downloaded and the local file name to dio. If we need to monitor the download progress, we can add the onReceiveProgress callback function:
//Build a FormData form for file upload
FormData formData = FormData.from({
"file1": UploadFileInfo(File("./file1.txt"), "file1.txt"),
"file2": UploadFileInfo(File("./file2.txt"), "file2.txt"),
});
//Send to the server using post method
var responseY = await dio.post("https://xxx.com/upload", data: formData);
print(responseY.toString());
//Download files using the download method
dio.download("https://xxx.com/file1", "xx1.zip");
//Add the download progress callback function
dio.download("https://xxx.com/file1", "xx2.zip", onReceiveProgress: (count, total) {
//do something
});
Sometimes, our page consists of multiple parallel requests, and we need to wait for all these requests to return before refreshing the page. In dio, we can easily achieve this by combining it with the Future.wait method:
//Send two parallel requests simultaneously
List<Response> responseX = await Future.wait([dio.get("https://flutter.dev"), dio.get("https://pub.dev/packages/dio")]);
//Print the response result of request 1
print("Response1: ${responseX[0].toString()}");
//Print the response result of request 2
print("Response2: ${responseX[1].toString()}");
In addition, similar to Android’s okHttp, dio also provides request interceptors. Through interceptors, we can perform some special operations before the request is sent or after the response is received. For example, we can add a custom user-agent header for each request or return cached data for the requested URI that has been locally cached, thus avoiding the need for redundant downloads:
//Add an interceptor
dio.interceptors.add(InterceptorsWrapper(
onRequest: (RequestOptions options){
// Add the user-agent header for each request
options.headers["user-agent"] = "Custom-UA";
// Check if there is a token, and if not, return an error
if(options.headers['token'] == null) {
return dio.reject("Error: Please log in first");
}
// Check if there is cached data
if(options.uri == Uri.parse('http://xxx.com/file1')) {
return dio.resolve("Return cached data");
}
// Allow the request to proceed
return options;
}
));
//Add try-catch to prevent request errors
try {
var response = await dio.get("https://xxx.com/xxx.zip");
print(response.data.toString());
} catch(e) {
print(e);
}
It should be noted that since there may be exceptions during network communication (such as domain name resolution failure, timeouts, etc.), we need to use try-catch to catch these unknown errors and prevent the program from crashing.
In addition to these basic usage examples, dio also supports advanced features such as request cancellation, proxy configuration, and certificate verification. However, these advanced features are not the focus of this presentation, so I will not go into further detail here. For more information, you can refer to the GitHub page of dio to learn about specific usage.
JSON Parsing #
After establishing a connection between a mobile application and a web server, there are two important tasks: how the server describes the returned communication information in a structured way, and how the mobile application parses this formatted information.
How to describe the returned communication information in a structured way? #
To express information in a structured way, we need to use JSON. JSON is a lightweight data exchange language used to express objects composed of attribute values and literals.
Here is a simple JSON structure representing a student’s score:
String jsonString = '''
{
"id": "123",
"name": "张三",
"score": 95
}
''';
Please note that Flutter does not support runtime reflection, so it does not provide libraries like Gson or Mantle that automatically parse JSON to reduce parsing costs. In Flutter, JSON parsing is completely manual, so developers have to do more work, but it is relatively flexible to use.
Next, let’s see how a Flutter application parses this formatted information.
How to parse formatted information? #
Manual parsing refers to the process of using the built-in JSON decoder in the dart:convert library to parse a JSON string into a custom object. Using this approach, we need to first pass the JSON string to the JSON.decode method to parse it into a Map, and then pass this Map to the custom class to assign the relevant properties.
Taking the JSON structure representing a student’s score as an example, let me demonstrate how to use manual parsing.
First, we define a Student class based on the JSON structure and create a factory class to handle the mapping between the Student class’s properties and the values of the JSON dictionary object:
class Student {
// properties: id, name, and score
String id;
String name;
int score;
// constructor
Student({this.id, this.name, this.score});
// JSON parsing factory, initializes object with dictionary data
factory Student.fromJson(Map<String, dynamic> parsedJson) {
return Student(
id: parsedJson['id'],
name: parsedJson['name'],
score: parsedJson['score'],
);
}
}
Once the data parsing class is created, the remaining tasks are relatively simple. We just need to convert the JSON text to a Map using the JSON.decode method, and then pass it to the Student’s factory class to complete the parsing of the Student object.
loadStudent() {
// jsonString is the JSON text
final jsonResponse = json.decode(jsonString);
Student student = Student.fromJson(jsonResponse);
print(student.name);
}
In the above example, all the properties in the JSON text are primitive types, so we can directly take the corresponding elements from the JSON dictionary and assign them to the object. But what if the JSON has nested object properties, such as in the example below, where Student has a teacher property?
String jsonString = '''
{
"id": "123",
"name": "张三",
"score": 95,
"teacher": {
"name": "李四",
"age": 40
}
}
''';
Here, teacher is no longer a primitive type, but an object. In this case, we need to create a parsing class for each non-primitive type property. Similar to the Student class, we also need to create a Teacher parsing class for its property teacher:
class Teacher {
// properties: name and age
String name;
int age;
// constructor
Teacher({this.name, this.age});
// JSON parsing factory, initializes object with dictionary data
factory Teacher.fromJson(Map<String, dynamic> parsedJson) {
return Teacher(
name: parsedJson['name'],
age: parsedJson['age'],
);
}
}
Then, we just need to add the teacher property and its corresponding JSON mapping rule to the Student class:
class Student {
// ...
// add teacher property
Teacher teacher;
// constructor
Student({/* ... */, this.teacher});
factory Student.fromJson(Map<String, dynamic> parsedJson) {
return Student(
// ...
// add mapping rule
teacher: Teacher.fromJson(parsedJson['teacher']),
);
}
}
After adding the mapping rule for the teacher property, we can continue to use the Student class to parse the above JSON text:
final jsonResponse = json.decode(jsonString); // decode the string into a Map object
Student student = Student.fromJson(jsonResponse); // manual parsing
print(student.teacher.name);
As you can see, with this method, we can handle non-primitive type properties of any complexity by creating corresponding parsing classes.
However, so far, our JSON data parsing is still done in the main isolate. If the JSON data format is complex and the data size is large, this parsing method may cause short-term unresponsiveness in the UI. For such CPU-intensive operations, we can use the compute function mentioned in the previous article to perform the parsing work in a new isolate:
static Student parseStudent(String content) {
final jsonResponse = json.decode(content);
Student student = Student.fromJson(jsonResponse);
return student;
}
doSth() {
// ...
// use the compute function to parse JSON in a new isolate
compute(parseStudent, jsonString).then(
(student) => print(student.teacher.name)
);
}
By transforming with compute, we no longer need to worry about the UI being unresponsive due to long JSON parsing times.
Summary #
Alright, that’s all for today’s sharing. Let’s briefly review the main points.
Firstly, I have taken you through three ways to achieve communication between Flutter applications and the server: HttpClient, http, and dio. Among them, dio provides more powerful features, such as request interception, file upload/download, and request merging. Therefore, I recommend using dio in actual projects.
Next, I shared related content about JSON parsing. JSON parsing is relatively simple in Flutter, but because it does not support reflection, we can only parse it manually. That is, first convert the JSON string to a Map, and then assign the properties to a custom class.
If you have experience in native Android or iOS development, you may find that the JSON parsing solution provided by Flutter is not as convenient. In Flutter, there is no library like Gson or Mantle provided in native development to directly convert JSON strings into corresponding entity classes. And all these capabilities invariably require runtime reflection, which Flutter does not support from its design. The reasons are as follows:
- Runtime reflection breaks class encapsulation and security, which brings security risks. Just recently, the Fastjson framework had a major security vulnerability. This vulnerability allowed carefully crafted string texts to execute arbitrary code when deserialized, directly leading to remote control of business machines, intrusions into intranets, and theft of sensitive information.
- Runtime reflection increases the size of binary files. Because it is not clear which code may be used at runtime, after using reflection, all the code will be built into the application by default. This prevents the compiler from optimizing unused code during compilation, making it impossible to further compress the size of the application installation package, which is unacceptable for Flutter applications with the built-in Dart virtual machine.
Reflection brings convenience to developers, but it also brings many difficult-to-solve new problems. Therefore, Flutter does not support reflection. What we need to do is to manually parse JSON diligently.
I have packaged the knowledge points involved in today’s sharing on GitHub, which you can download and run repeatedly to deepen your understanding and memory.
Thought Question #
Finally, let me leave you two thought-provoking questions.
- Please use
dioto implement a custom interceptor that checks thetokenin the header: if there is no token, the request should be paused, and at the same time, visit “http://xxxx.com/token” to obtain a new token before continuing with the request. - Write the corresponding parsing class for the following Student JSON:
String jsonString = '''
{
"id":"123",
"name":"张三",
"score" : 95,
"teachers": [
{
"name": "李四",
"age" : 40
},
{
"name": "王五",
"age" : 45
}
]
}
''';
Feel free to leave a comment in the comments section to share your opinions. I’ll be waiting for you in the next article! Thank you for listening, and feel free to share this article with more friends to read together.