DEV Community

Cover image for TestFly Part 8: Flakiness quarantine engine and Java SPI extensibility
Hakan GÜL
Hakan GÜL

Posted on Edited on Originally published at hakangul.lovable.app

TestFly Part 8: Flakiness quarantine engine and Java SPI extensibility

#testfly #observability #devops #java

Quarantine engine and Java SPI extensibility

The last part of the series covers two features aimed at long-term maintenance: flakiness quarantine and Java SPI extension points.

Flakiness quarantine (testfly-quarantine.yml)

You can quarantine a flaky test instead of deleting it or commenting it out, and the CI pipeline keeps moving. List the test in testfly-quarantine.yml in the project root:

# testfly-quarantine.yml (Project Root)
quarantine:
  enabled: true
  tests:
    - className: io.testfly.examples.PaymentTest
      methodName: testStripeWebhook
      reason: "Waiting for third-party webhook fix (JIRA-4021)"
      owner: "@payment-team"
Enter fullscreen mode Exit fullscreen mode

CI still runs quarantined tests. When one fails, TestFly flags it as QUARANTINED and the build does not fail.

Java SPI extension points

You can extend TestFly without forking it or changing framework internals. Each extension point is registered through its registry class or through SPI:

Extension Point Interface Registration
Custom Driver Provider NamedDriverProvider DriverProviderRegistry.register() or META-INF/services
Report Adapter ReportAdapter ReportAdapterRegistry.register() or SPI
Lifecycle Hook ExecutionHook HookRegistry.register() or SPI
Full Plugin TestFlyPlugin PluginRegistry.register() or SPI

Example: a Slack report adapter 

package io.testfly.examples.plugins;

import io.testfly.reporting.ReportAdapter;
import io.testfly.metrics.ExecutionMetrics;

public class SlackReportAdapter implements ReportAdapter {

    @Override
    public String getName() {
        return "slack";
    }

    @Override
    public void onSuiteFinish(ExecutionMetrics metrics) {
        // Send Slack webhook with pass rate and failed test summaries
    }
}
Enter fullscreen mode Exit fullscreen mode

Series summary

The eight parts covered:

  1. Part 1: zero-boilerplate setup with BaseTest and testfly.yml
  2. Part 2: semantic locators (getByRole), BasePage and WaitEngine
  3. Part 3: CI/CD quality gates (BuildThresholdEnforcer) and parallel execution
  4. Part 4: AI test generation and self-healing with testfly-mcp
  5. Part 5: BDD with Cucumber 7, BaseCucumberSteps and @retryable
  6. Part 6: the built-in ApiClient, BaseApiTest and hybrid workflows
  7. Part 7: bundled axe-core accessibility checks (accessibility()) and VisualAssert
  8. Part 8: the flakiness quarantine engine and Java SPI extensibility

The source code is on GitHub.

Top comments (0)