CI support in TestFly
In a continuous delivery pipeline, flaky tests and a falling pass rate should never go by unnoticed. TestFly has three components for CI:
-
CiEnvironmentDetectordetects GitHub Actions, Jenkins, GitLab CI, CircleCI, TeamCity and Bitbucket pipelines. -
BuildThresholdEnforcerchecks quality gates when the build finishes. -
JUnitXmlReporterwrites a machine-readableTEST-TestFly.xmlthat CI dashboards can pick up.
Quality gates in testfly.yml
execution:
mode: local
parallel: methods
threadCount: 4
maxActiveSessions: 4
ci:
failOnPassRateBelow: 95.0 # Fails the build if pass rate < 95%
maxFlakyTests: 2 # Fails if more than 2 tests are flaky
If a run breaks one of these thresholds, TestFly throws a BuildQualityGateException and exits with a non-zero code.
Multi-browser matrix in GitHub Actions
This workflow runs the suite headless on Chrome and Firefox and uploads the HTML report for each browser, even when tests fail.
name: TestFly E2E Regression
on:
push:
branches: [ master, main ]
pull_request:
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
browser: [ chrome, firefox ]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '17'
cache: 'maven'
- name: Run TestFly Tests
run: mvn clean test -Dbrowser.name=${{ matrix.browser }} -Dbrowser.headless=true
- name: Upload TestFly HTML Report
if: always()
uses: actions/upload-artifact@v4
with:
name: testfly-report-${{ matrix.browser }}
path: target/testfly-report.html
Top comments (0)