42 How to Construct an Efficient Flutter App Packaging and Distribution Environment #

Hello, I am Chen Hang. Today, let’s talk about the topic of delivering Flutter applications.

The delivery of a software project is a complex process, and any reason can lead to a failed delivery. One common phenomenon that small and medium-sized development teams often encounter is that the app works fine during development and testing but encounters frequent issues during the final packaging and delivery. Therefore, every time a new version is released, we not only have to wait for the packaging result, but also often need to work overtime to fix temporary issues. Even if the packaging is successful, the delivery process is still very tense without a good emergency strategy for live systems.

As you can see, product delivery is not only a headache for engineers, but also a high-risk action. In fact, failure is not scary, what is scary is that the reasons for failure are different every time. Therefore, in order to ensure reliable delivery, we need to pay attention to the entire process from source code to release, provide reliable deployment support, and ensure that the app is built in a stable and repeatable manner. At the same time, we should advance the packaging process and increase the frequency of builds, because this can not only detect problems earlier, but also reduce the cost of fixing them, and better ensure that code changes can be successfully released.

In fact, this is the idea of continuous delivery.

Continuous delivery refers to the establishment of a set of processes and mechanisms that automatically monitor source code changes, and automatically perform builds, tests, packaging, and related operations to ensure that the software can be maintained in a deployable state at any time. Continuous delivery enables faster and more frequent building, testing, and releasing of software, exposing problems and risks earlier, and reducing the cost of software development.

You may think that continuous delivery is only used in large software projects. Actually, that’s not the case. By using some free tools and platforms, small and medium-sized projects can also enjoy the convenience of automated development tasks. Among these tools, Travis CI is the largest in terms of market share. Therefore, I will use Travis CI as an example to share with you how to introduce the ability of continuous delivery for Flutter projects.

Travis CI #

Travis CI is an online hosted continuous delivery service. With Travis, you can perform continuous delivery without the need to set up your own server. It’s very convenient, just a few clicks on the webpage.

Travis and GitHub are a perfect combination. Once you have tied your GitHub project to Travis, any code changes will be automatically fetched by Travis. Then, Travis provides a running environment to execute the predefined tests and build steps defined in the configuration file. Finally, the build artifacts generated from these changes are archived on GitHub Release, as shown below:

Figure 1: Travis CI continuous delivery process diagram

As you can see, with the continuous build and delivery capabilities provided by Travis, you can directly see the results of code updates without the need to accumulate them for packaging and building before release. This not only allows for early error detection but also makes problem localization easier.

To provide continuous delivery capabilities for your project, you first need to tie GitHub to Travis. Open the Travis website and log in using your GitHub account. After logging in, a “Activate” button will appear on the page, which will redirect you to GitHub to set the project access permissions. Keep the default settings and click “Approve&Install”.

Figure 2: Activate GitHub integration

Figure 3: Authorize Travis to read project change history

After completing the authorization, the page will redirect to Travis. The Travis homepage will list all your repositories on GitHub, as well as the organizations you belong to, as shown in the following figure:

Figure 4: Complete GitHub project binding

After completing the project binding, the next step is to add a Travis configuration file to the project. The configuration method is also very simple, just place a file named .travis.yml in the root directory of the project.

.travis.yml is the Travis configuration file, which specifies how Travis should handle code changes. Once Travis detects new changes after committing the code, it will look for this file, determine the execution steps based on the project type (language), and then execute the commands inside it in order, including dependency installation (install), build commands (script), and deployment (deploy). The workflow of a Travis build task is shown below:

Figure 5: Travis workflow

As you can see, in order to have finer control over the continuous build process, Travis also provides corresponding hooks (before_install, before_script, after_failure, after_success, before_deploy, after_deploy, after_script) for install, script, and deploy, allowing you to execute special operations before or after them.

If your project is simple, without any third-party dependencies, and you don’t need to publish to GitHub Release, but just want to see if the build fails, you can omit the install and deploy parts in the configuration file.

How to integrate Travis into your project? #

As you can see, the simplest configuration file requires only two fields: language and script, which allows Travis to automatically build your project. The following example demonstrates how to integrate Travis into a Dart command-line project. In the configuration file below, we set the language field to Dart and define dart_sample.dart as the program entry point in the script field:

#.travis.yml
language: dart
script:
  - dart dart_sample.dart

By submitting this file to your project, you have completed the Travis configuration.

Travis will automatically run the commands specified in the configuration file for each code submission. If all the commands return 0, it means the validation is successful and there are no issues. Your commit record will be marked with a green checkmark. Otherwise, if an exception occurs during the command execution, it means the validation has failed and your commit record will be marked with a red cross. At this point, you need to click on the red checkmark to enter the Travis build details, investigate the failure cause, and fix the problem as soon as possible.

Figure 6. Code change validation

As you can see, to introduce the ability to automate tasks in a project, you only need to extract the commands required for the project to run automatically.

In the 38th article, I introduced the command to run automated test cases for a Flutter project, which is flutter test. Therefore, if we want to configure automated testing tasks for a Flutter project, we can simply include this command in the script field.

However, it is important to note that Travis does not have a built-in Flutter runtime environment. Therefore, we also need to install the Flutter SDK for the automated tasks in the install field. The following example demonstrates how to configure automated testing capability for a Flutter project. In the configuration file below, we set the os field to osx. In the install field, we clone the Flutter SDK and set the Flutter command as an environment variable. Finally, we add the flutter doctor and flutter test commands in the script field to complete the configuration:

os:
  - osx
install:
  - git clone https://github.com/flutter/flutter.git
  - export PATH="$PATH:`pwd`/flutter/bin"
script:
  - flutter doctor && flutter test

In fact, it is relatively easy to introduce automated testing capability for a Flutter project. However, considering Flutter’s cross-platform nature, how can we verify the automated building capability of a project on different platforms (i.e., building an ipa package on iOS and an apk package on Android)?

We all know that the command for building a Flutter project is flutter build. Similarly, we just need to include the commands for building iOS and Android in the script field. However, considering that these two build commands take relatively long execution time, we can use the concurrent task option matrix provided by Travis to split the iOS and Android builds and deploy them to separate machines for execution.

The following example demonstrates how to split build tasks using the matrix option. In the code below, we define two concurrent tasks: the Flutter build APK task, which runs on Linux using the command flutter build apk, and the Flutter build iOS task, which runs on OS X using the command flutter build ios.

Considering that different platforms require specific runtime environments, such as setting up JDK, installing Android SDK, and build tools for the Android build task, and setting Xcode version for the iOS build task, we provide corresponding configuration options for these two concurrent tasks.

Finally, it is worth mentioning that both tasks require the Flutter environment, so the install field does not need to be duplicated in each task:

matrix:
  include:
    # Declare the Android runtime environment
    - os: linux
      language: android
      dist: trusty
      licenses:
        - 'android-sdk-preview-license-.+'
        - 'android-sdk-license-.+'
        - 'google-gdk-license-.+'
      # Declare the Android components to install
      android:
        components:
          - tools
          - platform-tools
          - build-tools-28.0.3
          - android-28
          - sys-img-armeabi-v7a-google_apis-28
          - extra-android-m2repository
          - extra-google-m2repository
          - extra-google-android-support
      jdk: oraclejdk8
      sudo: false
      addons:
        apt:
          sources:
            - ubuntu-toolchain-r-test 
          packages:
            - libstdc++6
            - fonts-droid
      # Make sure sdkmanager is up-to-date
      before_script:
        - yes | sdkmanager --update
      script:
        - yes | flutter doctor --android-licenses
        - flutter doctor && flutter -v build apk

    # Declare the iOS runtime environment
    - os: osx
      language: objective-c
      osx_image: xcode10.2
      script:
        - flutter doctor && flutter -v build ios --no-codesign
install:
    - git clone https://github.com/flutter/flutter.git
    - export PATH="$PATH:`pwd`/flutter/bin"

How to automatically publish packaged binary files? #

In this case, our build command is to package the binary files. Can the packaged binary files be automatically published?

The answer is yes. We only need to add the deploy field to the two build tasks and set the skip_cleanup field to tell Travis not to clean up the built artifacts after the build is complete. Then, we can specify the files to be published through the file field, and finally upload them to the project homepage using the API token provided by GitHub.

The following example demonstrates how to use the deploy field. In the code below, we obtain the app-release.apk built by the script field, specify it as the file to be published using the file field. Considering that not every build requires automatic publishing, we added the on option in the configuration below to tell Travis to only automatically publish a release version when the corresponding code update is associated with a tag:

...
# Declare the commands to be executed for the build
script:
  - yes | flutter doctor --android-licenses
  - flutter doctor && flutter -v build apk
# Declare the deployment strategy, i.e., uploading apk to GitHub release
deploy:
  provider: releases
  api_key: xxxxx
  file:
    - build/app/outputs/apk/release/app-release.apk
  skip_cleanup: true
  on:
    tags: true
...

It should be noted that since our project is an open-source library, the API token for GitHub cannot be placed in the configuration file in plain text. Instead, you need to configure an API token environment variable on Travis and then set this environment variable in the configuration file.

First, open GitHub, click on the personal avatar in the upper right corner of the page to enter Settings, and then click on Developer Settings to enter the developer settings page.

Figure 7 Enter developer settings

On the developer settings page, click on the Personal access tokens option in the lower left corner to generate an access token. The token settings page provides rich access permission controls, such as repository restrictions, user restrictions, read and write restrictions, etc. Here, we choose to only access public repositories. Fill in the token name as cd_demo, click confirm, and GitHub will display the token content on the page.

Figure 8 Generate an access token

It should be noted that you will only see this token on GitHub once, and it will be gone once the page is closed, so copy the token before closing the page.

Figure 9 Access token interface

Next, open the Travis homepage, find the project you want to configure for automatic publishing, and click on More options in the upper right corner, then select Settings to open the project configuration page.

Figure 10 Open Travis project settings

In the Environment Variable section, add the token copied just now as an environment variable with the name GITHUB_TOKEN.

Figure 11 Add Travis environment variable

Finally, we just need to replace the api_key in the configuration file with ${GITHUB_TOKEN}.

...
deploy:
  api_key: ${GITHUB_TOKEN}
...

This case demonstrates the publishing of APK as the build artifact for Android. For iOS, we need to process the build artifact app slightly to make it into the more universal IPA format before publishing. This is where the before_deploy field of deploy comes in handy, allowing us to perform specific artifact processing before the official release.

The example below shows how to process the build artifact app through the before_deploy field. Since the IPa format is a wrapper on top of the app format, we first copy the app file to Payload and then compress it to complete the preparation work before publishing. Then we can specify the file to be published in the deploy stage and officially enter the release phase:

...
# Preprocess the build artifact before publishing by packaging it into IPA format
before_deploy:
  - mkdir app && mkdir app/Payload
  - cp -r build/ios/iphoneos/Runner.app app/Payload
  - pushd app && zip -r -m app.ipa Payload  && popd
# Upload the IPA to GitHub release
deploy:
  provider: releases
  api_key: ${GITHUB_TOKEN}
  file:
    - app/app.ipa
  skip_cleanup: true
  on:
    tags: true
...

After submitting the updated configuration file to GitHub, create a tag. After waiting for the Travis build to complete, our project will be able to automatically publish the build artifact.

Figure 12 Publishing build artifacts for Flutter App

How to add automatic release capabilities to a Flutter Module project? #

This example is about traditional Flutter App projects (i.e., pure Flutter projects). If we want to add automatic release capabilities to a Flutter Module project (i.e., a mixed development Flutter project), how should we set it up?

Actually, it’s not complicated. The Android build artifact of a module project is an AAR, and the iOS build artifact is a Framework. Automatic release of the Android artifact is relatively simple, as we can directly reuse the APK release by specifying the file as the AAR file. Automatic release of the iOS artifact is a bit more involved, as we need to process the Framework and convert them into the Pod format.

The following example demonstrates how to achieve automatic release of the iOS artifact of a Flutter Module. Since the Pod format itself is just a wrapper on top of the App.Framework and Flutter.Framework files, we only need to copy them to a unified directory called FlutterEngine and place the FlutterEngine.podspec file, which declares the component definitions, in the top-level directory. Finally, we compress everything into a zip file.

...
# Preprocess the build artifacts and compress them into a zip file
before_deploy:
  - mkdir .ios/Outputs && mkdir .ios/Outputs/FlutterEngine
  - cp FlutterEngine.podspec .ios/Outputs/
  - cp -r .ios/Flutter/App.framework/ .ios/Outputs/FlutterEngine/App.framework/
  - cp -r .ios/Flutter/engine/Flutter.framework/ .ios/Outputs/FlutterEngine/Flutter.framework/
  - pushd .ios/Outputs && zip -r FlutterEngine.zip  ./ && popd
deploy:
  provider: releases
  api_key: ${GITHUB_TOKEN}
  file:
    - .ios/Outputs/FlutterEngine.zip
  skip_cleanup: true
  on:
    tags: true
...

After submitting this code, you can see that the Flutter Module project can also automatically release the native components.

Figure 13. Automatic release of the build artifacts for a Flutter Module project.

Through these examples, we can see that the key to configuring tasks is to extract a collection of commands needed for project automation and determine their execution order. As long as we arrange these command collections in the install, script, and deploy stages, we can leave the rest to Travis to complete, and we can enjoy the convenience brought by continuous delivery.

Summary #

As the saying goes, “90% of failures are caused by changes,” which highlights the value of continuous delivery in ensuring stability during deployment. By establishing a continuous delivery pipeline, we can link code changes with automation, making testing and deployment faster and more frequent. This not only helps to identify risks early but also keeps software in a continuously stable and deployable state.

In today’s presentation, I introduced how to introduce continuous delivery capabilities to our projects using Travis CI. Travis’ automated workflow relies on the .travis.yml configuration file. After defining the commands needed for the build task, we can break down the execution process into three steps: install, script, and deploy. Once the project is configured, Travis will automatically execute the tasks whenever code changes are detected.

A simple and clear deployment process is essential for software reliability. If we simultaneously deploy 100 code changes, causing the performance of the app to decline, we may need to spend a lot of time and effort to determine which changes are responsible and how they affect the app. However, by deploying the app continuously, we can measure and understand the impact of code changes at a smaller granularity, whether they improve or degrade the performance. This allows us to identify problems earlier and have more confidence in faster releases.

It is important to note that in the example analysis today, we built an unsigned IPA file, which means we need to sign it before running it on a real iOS device or publishing it to the App Store.

iOS code signing involves private key and multiple certificate verification, as well as corresponding encryption and decryption steps, making it a relatively complex process. If we want to deploy automated signing operations on Travis, we need to export the distribution certificate, private key, and provisioning profile, and package them into a compressed file to be encrypted and uploaded to the repository in advance.

Then, in the before_install step, we need to decrypt the compressed file and import the certificate into the keychain of the Travis environment. This way, the build script can use the temporary keychain to sign the binary files. For the complete configuration, you can refer to the integration doc provided by the internal testing service provider, Pgyer, for further details.

If you prefer not to expose the distribution certificate and private key to Travis, you can download the unsigned IPA package, extract it, use the codesign command to re-sign the App.Framework, Flutter.Framework, and Runner individually, and then repackage them into an IPA file. This article provides detailed step-by-step instructions on how to do this.

I have uploaded the Travis configuration files mentioned in today’s presentation to GitHub. You can download these projects, Dart_Sample, Module_Page, Crashy_Demo, and examine their configuration files, as well as view the corresponding build processes on the Travis website to deepen your understanding and memory.

Thinking Question #

Finally, I’ll leave you with a thinking question.

In the Travis configuration file, how can you choose a specific Flutter SDK version (such as v1.5.4-hotfix.2)?

Feel free to leave a comment in the comment section sharing your thoughts. I’ll be waiting for you in the next article! Thank you for listening, and please feel free to share this article with more friends to read together.