Blog / Methods

How to check self-healing claims in AI testing tools

Published by AI QA Directory on . Built from the catalogue data as of that date; tool pages carry the current facts.

Self-healing is the ability of a test to repair a broken locator or step when the application changes. Many AI testing tools mention it, but not all of them document how it works. This guide shows how the catalogue separates a documented mechanism from a vendor-stated claim, and how to run the same check.

How the catalogue records self-healing

The self-healing fact has three values: documented, vendor-stated and none. Its evidence level follows the same rules as every other fact.

Counts as of 2026-09-24, across 330 tools:

How the catalogue records self-healing
Evidence levelTools
Documented93
Vendor-stated49
Not documented188

Of the 93 documented facts, 90 record a documented mechanism. The other 3 record docs that describe no self-healing.

What a documented mechanism looks like

A documented mechanism is a docs page, help article or release note that explains how healing works. A sentence on a homepage or feature page is a claim, not a mechanism. When we read a docs page, we look for answers to these questions:

  • What triggers healing: a failed locator, an exception or a changed element?
  • What does the tool try instead?
  • Is the repair saved, proposed for review or applied only to that run?
  • Is it on by default, and on which plans?

A page does not need to answer all four. It does need to describe healing on a documentation-class page, not only name it.

Documented examples

Sorted by name A-Z.

CloudQA. The CloudQA release notes record that auto-heal was enabled by default for all users in October 2025. Release notes count as documentation.

Cypress. The AI Test Generation guide documents that cy.prompt self-heals a step when the element it needs has changed.

Healenium. The How Healenium works page describes the trigger and the response. When a test fails with a NoSuchElement exception, Healenium runs its algorithm to generate a list of healed locators.

mabl. The mabl help centre has a dedicated article, How auto-heal works.

Documented as none

BugBug records self-healing as none, and the evidence level is documented. The How BugBug AI works page asks whether BugBug self-heals broken selectors at run time. The answer states that "you may still need to update the step manually".

This is the case where a "no" is documented. The docs say it, so the catalogue records it with the source.

Vendor-stated examples

testRigor records self-healing as vendor-stated. The description is on a marketing feature page, AI-Based Self-Healing for Test Automation, not in the testRigor docs.

ProdPerfect records self-healing as vendor-stated. The source is the ProdPerfect homepage, and the site links to no public documentation.

A vendor-stated value is not necessarily wrong. It means we found no documentation page that explains the mechanism.

Not documented examples

Checkly records self-healing as not documented. We found no self-healing feature in the Checkly docs.

TestRail records self-healing as not documented. We found no self-healing feature on the pages we opened.

Numbers stay out of the fact

Some vendors attach figures to self-healing, such as a share of maintenance time saved. Those figures never go into the self-healing fact. They appear only in the vendor claims block on the tool page, verbatim and linked to the page that states them.

A short checklist

  1. Search the docs, help centre and release notes for "self-healing", "auto-heal", "heal" and "locator".
  2. Open the page and look for a trigger, a repair action and what the user sees.
  3. If a documentation-class page describes it, record the URL and the date.
  4. If only a homepage or feature page mentions it, treat it as vendor-stated.
  5. If the docs say healing is not supported, record that as documented none.
  6. If no page mentions it, treat it as not documented.

The documented self-healing feature page lists the tools whose self-healing fact has the value documented. The rules behind every level are on the methodology page.

Sources

Spotted an outdated or wrong fact? Send the source and we apply it at the next monthly update.

Report a change