Android Test Station – A Complete Guide on Running CTS

YouTube video

android test station

Are you looking to run Android Compatibility Test Suite (CTS) using Android Test Station (ATS)? In this article, we will provide detailed instructions on how to run CTS with Android Test Station and ensure a successful test run. Stay tuned to discover the step-by-step process to customize your test run parameters, choose device actions, and analyze test results.

Introduction to Android Test Station

Android Test Station is a powerful tool that allows you to run CTS with ease. All supported CTS configurations are bundled with the tool, ensuring a seamless testing experience. Before we dive into the details, make sure you have downloaded and installed Android Test Station on your machine.

Step 1: Selecting the CTS Version

To initiate a CTS test run, you need to select the CTS version you want to use. Within Android Test Station, navigate to the test suites page. From there, you can find all the bundled CTS configurations. Once you have chosen the desired version, click on the “Run Task” button.

Step 2: Customizing Test Run Parameters

Android Test Station provides a user-friendly step-by-step UI that allows you to customize your test run parameters. In the first step, you can check and modify the CTS command line. By default, Android Test Station runs a full plan, but you can customize it according to your requirements. This corresponds to the plan name and options used with the run command in the CTS console.

Let’s say you want to run a permission task case module. You can modify the CTS command line to reflect this change. Additionally, you can adjust the maximum retry number by expanding the advanced settings.

To make your test run easily identifiable, you can set labels for your run. This way, you can quickly find your test run later on. For example, if you are running a test on a Pixel 2 XL, you can label it with the model name.

Step 3: Selecting Devices for Test Suite

In this step, you can choose the device(s) on which you want to run the test suite. Android Test Station allows you to select one or more devices for sharding. You don’t need to worry about sharding options, as ATS automatically sets them for you.

Step 4: Selecting Device Actions

To ensure a smooth test run, you can select device actions to be executed before the first run and subsequent retries. For CTS, it is recommended to add reboot and CTS device setup actions. These actions help minimize interference between retries.

Step 5: Choosing CTS Package Location

In the last step, you can customize the location from which the CTS package should be downloaded. By default, Android Test Station is set to download the latest release version from Google’s server. However, you can override this and use a pre-downloaded CTS package on your machine.

Starting the Test Run

Once you have finalized all the parameters, click on the “Start Test Run” button to initiate the test run. Android Test Station will provide you with a test run detail page where you can monitor the status of your test run.

Monitoring Test Runs

While the test run is in progress, you can view live logs and leave the page if needed, as you can always come back later. Android Test Station also allows you to run multiple test runs in parallel and displays all the current and previous test runs on the test runs page.

Analyzing Test Results

Once the test run is completed, you can analyze the result summary from the test results tab. Android Test Station provides detailed information about which test cases have passed and which ones have failed. If needed, you can also browse test result files by clicking on the “View Output Files” button.

Rerunning Failed Test Cases

In some cases, you might encounter failed test cases even after retries. To ensure these are real failures and not false negatives, Android Test Station allows you to rerun the failed test cases. Simply click on the “Rerun” button, and you will be presented with the same step-by-step UI used for scheduling the first run.

Exporting Test Results

Once you have obtained the desired test results, you might want to export them for further analysis or to retry on a device connected to another workstation. Android Test Station allows you to export the test results by clicking on the “Export Result” button. This will download the results in a convenient format.

Rerunning Test Results from Remote Sources

If you have previously executed CTS manually or on another test station server, you can rerun the test results using Android Test Station. Select the “Remote” option when scheduling a new test run and upload the test results ZIP file. The rest of the configuration remains the same.

Conclusion

Android Test Station simplifies the process of running CTS and ensures a smooth testing experience. By following the step-by-step instructions provided in this article, you can confidently run CTS and analyze test results. Remember to customize your test run parameters, select suitable device actions, and explore the variety of options within Android Test Station. Happy testing!

Keywords: com.android.cts.ctsshim, Android Test Station, CTS, test run, test results, running CTS, test run parameters, device actions, analyzing test results