How-To

How to Run Mobile App Tests in CI/CD: 7 Steps for iOS and Android

Mobile CI needs a Mac for iOS, signing for device builds and an emulator that boots slowly. Seven steps, with working GitHub Actions YAML for both platforms.

Adithya Aggarwal

CTO & Co-founder at Pie

12 min read Updated Oct 30, 2026

Why Mobile CI Breaks Where Web CI Does Not

Mobile CI is harder than web CI because the app does not run where it was built. A web test job serves the build and drives it in a headless browser on the same Linux box. A mobile app compiles to a binary that has to be installed on an emulator, a simulator, or a physical device first, each with its own boot time, OS version, and hardware behavior.

Three costs fall out of that gap, and together they are why testing mobile apps is its own discipline rather than web testing on a smaller screen.

• Toolchain: Building for iOS requires Xcode, and Xcode runs only on macOS. The iOS job cannot share the cheap Linux runners carrying the rest of your pipeline. Apple’s Xcode Cloud does not change that. It moves the Mac somewhere you do not have to buy it.
• Signing: A build for a physical device will not install without a matching certificate and provisioning profile, and a fresh CI runner has neither. A simulator build skips the profile. Signing belongs in the job that feeds a device farm or TestFlight, and the per-commit job can leave it out.
• Device layer: It is slow, and it is not deterministic. Google’s emulator documentation says the emulator “might take a minute or so to launch for the first time, but subsequent launches use a snapshot and should launch faster”, and on a fresh CI runner every launch is the first one.

What You Need Before the First Run

Before you write a single pipeline step, four things have to be in place. If any of them is wrong, no amount of YAML will make the job stable.

• Runners for both platforms: GitHub Actions, CircleCI, and the mobile-first Bitrise all offer Linux and macOS runners. GitLab has hosted macOS runners too, but they are in beta and limited to Premium, Ultimate and open source accounts. Check your tier before you plan around them. Split the platforms into separate jobs so the cheap Android job is not queued behind the expensive Mac.
• A headless build command: ./gradlew assembleDebug assembleDebugAndroidTest for Android and an xcodebuild invocation for iOS. If it does not build from the command line with no Android Studio or Xcode window open, it will not build in CI.
• Managed signing: For device and TestFlight builds, store the iOS certificate and provisioning profile as encrypted secrets, or let Fastlane Match sync them from a private git repo, Google Cloud Storage, or S3. Never commit them.
• A test framework: Appium drives both platforms through the W3C WebDriver protocol, so one suite covers iOS and Android. Espresso and XCUITest run natively on their own platform and lock you to it. Pick before you build the pipeline, because the run command differs for each.

Seven Steps to Run Mobile Tests in CI/CD

Running mobile tests in CI/CD comes down to seven steps in a fixed order. Check out, cache, build, sign, boot the device, run, report. Each step assumes the one before it finished cleanly, and the per-commit simulator job skips step 4.

1. Trigger on pull requests: Run on every pull request, gated to paths that touch app or test code, so you are not booting an emulator to fix a README. Coding agents that open pull requests all day need a tighter gate, and testing AI-generated mobile apps shows how to build only the ones that change a screen.
2. Cache the toolchain: Cache Gradle, CocoaPods, and dependency directories keyed on your lockfile hash. A cold dependency download can be the largest single chunk of a mobile build.
3. Build both binaries: ./gradlew assembleDebug assembleDebugAndroidTest on Android, xcodebuild build-for-testing on iOS.
4. Sign hardware builds: Android debug builds are signed with the SDK’s debug certificate automatically, and the per-commit simulator job skips this step. A TestFlight job runs fastlane match appstore --readonly (adhoc or development for a device farm), or imports the certificate into a temporary keychain.
5. Boot the device layer: Start the emulator or simulator, or reserve a device on a farm. On a Linux runner, enable KVM first, and restore the emulator from a cached snapshot instead of cold-booting it.
6. Run the tests: Point Appium, Espresso, or XCUITest at the installed build. Disable animations in the test build so timing stays predictable.
7. Collect artifacts and report: Pull logs, screenshots, the Android XML results and the iOS result bundle back into the CI run. A failure should be debuggable without a re-run.

Here is the Android half as a working GitHub Actions job. It uses the reactivecircus emulator runner. One action boots the emulator and runs the suite, and steps 5 and 6 collapse into that block.

Community wrappers like it and custom scripts account for most of the 481 repositories in that study. On GitHub’s Linux runners, the action needs the KVM step that runs before it for hardware acceleration.

name: android-e2e
on:
  pull_request:
    paths: ['app/**', 'gradle/**']
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-java@v6
        with:
          distribution: temurin
          java-version: '17'
      # Step 2: cache Gradle so dependencies are not re-downloaded
      - uses: gradle/actions/setup-gradle@v6
      # Step 3: build app + test binary once
      - name: Build
        run: ./gradlew assembleDebug assembleDebugAndroidTest
      # Step 5: give the emulator hardware acceleration on the Linux runner
      - name: Enable KVM
        run: |
          echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
          sudo udevadm control --reload-rules
          sudo udevadm trigger --name-match=kvm
      # Steps 5 + 6: boot emulator and run in one action
      - name: Run instrumented tests
        uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 34
          arch: x86_64
          disable-animations: true
          script: ./gradlew connectedDebugAndroidTest
      # Step 7: keep the HTML report and the XML results even on failure
      - uses: actions/upload-artifact@v7
        if: always()
        with:
          name: test-results
          path: |
            app/build/reports/androidTests/
            app/build/outputs/androidTest-results/

On repeat runs, cache the AVD directory with actions/cache so the next boot loads a saved snapshot. The action’s README carries the pattern.

The iOS job splits compiling from running, so the simulator is not idling through the build. Pin Xcode to the version you release with. Since April 28, 2026, App Store Connect only accepts builds made with Xcode 26 or later. Testing on Xcode 16 now means testing a binary you cannot ship. Read the runner image manifest for what is installed before you hardcode a path.

name: ios-e2e
on:
  pull_request:
    paths: ['App/**', 'AppUITests/**']
jobs:
  test:
    runs-on: macos-26
    steps:
      - uses: actions/checkout@v7
      # Runner images ship several Xcode versions. Pin the one you release with.
      - name: Select Xcode
        run: sudo xcode-select -s /Applications/Xcode_26.6.app
      # Step 3: compile the app and the test bundle for the simulator, no device or profile needed
      - name: Build for testing
        run: |
          xcodebuild build-for-testing \
            -scheme App \
            -destination 'generic/platform=iOS Simulator' \
            -derivedDataPath build
      # Steps 5 + 6: the simulator boots here, with the binary already waiting
      - name: Run tests
        run: |
          xcodebuild test-without-building \
            -xctestrun build/Build/Products/*.xctestrun \
            -destination 'platform=iOS Simulator,name=iPhone 17,OS=26.5' \
            -resultBundlePath TestResults.xcresult
      # Step 7: the result bundle carries the logs, screenshots and test report
      - uses: actions/upload-artifact@v7
        if: always()
        with:
          name: ios-test-results
          path: TestResults.xcresult

Emulators on Every Push, Real Devices Before Release

Use emulators and simulators for speed on every commit, and real devices for confidence before release. That much is standard advice, and it holds. Where the standard advice goes stale is the reasoning, because it keeps citing gaps emulators closed years ago.

• Camera: The Android Emulator can use a webcam on the host as the device camera, and its extended controls load pictures into the scene the emulator camera shows.
• Network: The same controls simulate network types from GSM through 5G, with signal strength from none to great.
• Push: The iOS Simulator has accepted simulated pushes through xcrun simctl push since Xcode 11.4, and since Xcode 14 a simulator running iOS 16 or later on macOS 13 or later receives real remote notifications on Apple silicon or T2 Macs.
• Purchases: Apple’s StoreKit testing in Xcode runs against locally configured product data, and its test steps run the app “in the simulator or on a device.”

If your reason for renting hardware is push or purchases, check it against the simctl and APNs commands and Apple’s StoreKit docs before you sign the contract.

The real gaps are narrower and harder to work around. An emulator renders on the host’s GPU or in software, and anything that looks different on Adreno than on Mali will not surface. It runs Google’s system images instead of a manufacturer’s customized build, and it never throttles like a hot phone, drains a real battery, or hands off from Wi-Fi to cellular mid-test.

An emulator can also fail where a phone does not. A customer’s new Android build crashed often on our emulators while the same flow held up on a phone the team tested by hand. The emulator crashes then spiked the team’s crash dashboard. Rerun an emulator-only crash on hardware before anyone files it, and keep crashes from test runs out of the dashboard your team reads.

So the split has a rule. If a flow touches the GPU, the radio, the battery, or an OEM skin, it belongs on hardware. Everything else runs against an emulator on every push, and a broad hardware suite runs nightly through Google’s Developer Device Platform (the successor to Firebase Test Lab, in preview), AWS Device Farm, or BrowserStack App Automate.

Build that hardware matrix from your own analytics. The devices your users carry decide it, and the differences between the two platforms decide how much of it you need on each side.

How to Keep the Run Under Fifteen Minutes

Set a fifteen-minute budget for the per-commit suite. Bitrise’s mobile CI guide puts commit-level pipelines at roughly 10 to 15 minutes to keep the feedback loop fast.

Hitting it is about setup cost. The table splits the budget into four cost centers, each with the lever that moves it.

Sharding is the one that keeps paying. Ten minutes of tests split across five runners finishes in about two, plus each runner’s boot. AndroidJUnitRunner takes numShards and shardIndex arguments for exactly this split, and Google’s Developer Device Platform can shard a suite for you. Once caching has done what it can, sharding is the biggest lever left that does not cost coverage.

How to Stop Mobile CI Tests From Flaking

Before you fix a red mobile CI run, ask whether it is timing or the selector. The two need different fixes. Timing you can engineer away in an afternoon. A stale selector is a design problem that grows every sprint you leave it.

Timing: The Runner Is Not Your Laptop

A CI emulator boots cold and shares CPU with the build. On a runner without KVM it also gets no hardware acceleration, so animations and network calls finish on a schedule your laptop never showed you. Three changes fix it, and none of them is clever.

1. Explicit waits: Replace every fixed sleep with a wait on a condition.
2. No animations: Disable animations in the test build.
3. Pinned images: Pin the emulator image and the OS version so every run starts from an identical state.

Do those three before you add a single retry. Emulator drift, gesture timing and permission dialogs sit on this side of the split too, and why mobile end-to-end tests flake walks through each one.

Selectors: The Map That Goes Stale

The second cause has no equivalent fix. A test that locates a button by accessibility ID or XPath is a hardcoded map of the UI, and a redesign that moves or renames elements invalidates part of that map. The app moved, and the map did not move with it.

Mobile application test automation built on scripts takes this hit twice. The same product flow renders through a different element tree on each platform. One design change arrives as two separate repair jobs, both of them pure upkeep.

Removing the Selector Layer From Mobile CI

Scripted mobile CI stops scaling at the selector layer. Every speed lever above makes the run faster, and none of them stops a test from breaking because a button moved two rows down.

Pie drops that layer on mobile. It is an autonomous QA platform that drives your iOS and Android app almost like a new user would, reading the screen, deciding what to tap, and tapping a coordinate instead of resolving a selector. Runs happen on iOS simulators and Android emulators, the same environment your per-commit job targets. The hardware-only flows from the emulator section stay on the farm.

Wiring it into the pipeline you just built adds one upload step to the job you already have.

1. Build: The test job already produces a simulator .app and an .apk.
2. Upload: A GitHub Action uploads each build, zipped or not.
3. Run: With auto-run on build configured, your existing tests run against the new build.
4. Decide: Your team reads the result in Pie and makes the release call.

The upload step is a hand-off. A green upload job means the build arrived. The tests run after that, so your pipeline can be green before a single test has finished.

A Pie test holds no element ID or XPath for a relocated button to break. Re-run, read the result, and edit a test’s steps when the intended behavior changed.

Four Problems, Three of Them Solved

Mobile CI is four problems stacked into one job, and the seven steps above answer the first three. The fourth has no YAML fix. Pin every OS version and restore every snapshot, and a redesigned screen still takes a selector suite down on Monday morning.

Pie is our answer to the fourth. It works from what is on screen and picks up each build from one upload step in the job you already wrote, so a moved button can cost a re-run instead of a locator hunt. Build the seven steps this sprint, and hand Pie the part no YAML fixes.

Adithya Aggarwal

CTO & Co-founder at Pie

Eight years building search and delivery systems at Amazon. The kind of scale where flaky tests block billion-dollar releases. Now CTO at Pie, building AI agents that adapt when your UI changes. LinkedIn →