Creating a scheduled XCUITest(iOS) regression pipeline using GitLab

Taking full advantage of the DevOps Pipeline

Fastlane simulator’s screenshot

As your app grows and packs on new features, it becomes harder to test manually with new and existing features before every deployment. And, if you’re planning to deploy to prod every 2 or 3 weeks, it’s become very crucial to have a solid and robust UI regression pipeline in-place to prevent bug leakage.

In this article, I will go over the step-by-step process in setting up an XCUITest (iOS) pipeline that will run each of the following events:

1. CRON scheduled to run every evening, morning, or whenever you like.
2. Each time a new merge request is added (optional)

Now, we all know pre-prod environments can be somewhat unreliable due to scheduled/unknown outages & glitches. No matter which company you may work with (I’m sure your QA team will agree with it). In order to tackle this problem, we will use fastlane-plugin-test_center to re-run flaky tests up to 3 times before publishing results.

1. Install Fastlane using Bundler

• Install Bundler by running gem install bundler
• Create a ./Gemfile in the root directory of your project with the content

source "https://rubygems.org"
gem "fastlane"

• Run bundle update and add both the ./Gemfile and the ./Gemfile.lock to version control

Official Fastlane installation instructions: https://docs.fastlane.tools/getting-started/ios/setup/

2. Install Fastlane-Plugin-Test_Center

1. Create folder fastlane in the root directory.

2. Add fastlane-plugin-test_center to your fastlane by running following in your terminal

fastlane add_plugin test_center

You will notice Pluginfile will be auto-generated by fastlane, and will get added to ./fastlane directory. Ensure this file gets added to version control as well.

Official fastlane-plugin-test_centerinstallation instructions: https://github.com/lyndsey-ferguson/fastlane-plugin-test_center

Special Thanks Lyndsey Ferguson for developing this Plugin. Please consider sponsoring his project if you decide to use the plugin.

3. Setup FastFile

Add a new file called Fastfile in the ./fastlane folder created above, and add the following code to the file.

Test the setup by running fastlane tests in your terminal

https://github.com/lyndsey-ferguson/fastlane-plugin-test_center

At this point, if your setup is correct, Fastlane will pick tests specified in your scheme, and execute them.

4. Install Gitlab runner on MacOS

We need macOS to run XCUITest (iOS), so if you don’t have any available macOS as Gitlab runner, then you may need to build one.

Please follow official instructions on GitLab docs to setup your MacOS: https://docs.gitlab.com/runner/install/osx.html#manual-installation-official

Add tag xcuitest when registering the runner.

4. Setup GitLab YAML file

Add .gitlab-ci.yml file in your root directory of the project, and copy the following code to the file.

5. Add Scheduled Pipeline on Gitlab portal

1. Go to gitlab.com/yourproject
2. Go to CI/CD/Schedules
3. Click New Schedule
4. Select branch as master
5. Set your desired time, or custom CRON
6. Save, and that’s it

Now, your XCUITest (iOS) Pipelines will run every day at 10 PM (in my case), as well as anytime a developer added a new merge against.

Test reports can be found in Gitlab Artifacts and can be browsed directly from. the browser. Path: ./fastlane/output/YourProjectUITest/report.html

Refer ./fastlane/output/YourProjectUITest/report.xcresult for detailed report, along with screenshots of failed tests.

That’s all. Enjoy your next level testing!!

Next Level Swift

We are always looking for talented and passionate Swift developers! Feel free to check out our writer’s section and find out how you can share your knowledge with the Next Level Swift Community!