Methodology

Evidence levels

• Documented The fact is stated in public documentation: docs, help centre, changelog, pricing page, terms, or an official repository or registry. The source link goes to that page.
• Vendor-stated Only the vendor's marketing pages say it, and no documentation confirms it. The source link goes to the page where the vendor says it.
• Not documented We could not confirm the fact in documentation or on the vendor's pages. It has no source and is never filled with a guess.

Marketing pages, press releases about product features and third-party reviews do not count as documentation. A fact is never upgraded from vendor-stated to documented without a documentation source.

Vendor claims such as coverage percentages or time savings appear only in a bordered block labelled “Vendor-stated, verbatim” on the tool page, in the vendor's own words, with a link. They are never used as facts elsewhere on the site.

Verification cadence

The catalogue is rebuilt monthly. Each run re-checks pricing pages and company status for every tool and looks for new tools. Every fact shows the date it was verified.

A fact older than 90 days carries the flag “Verify before relying” until it is checked again. Pricing pages change often, so check the vendor's page before you buy.

Inclusion rules

A tool is listed when it markets AI-assisted software testing and has a public homepage. Facts come from documentation, pricing pages and public records; anything only the vendor says is labelled vendor-stated. Tools that shut down or are acquired stay listed with the date.

• The tool has a public homepage.
• It markets AI-assisted software testing, or exposes a testing capability to AI agents (for example an MCP server).
• A general AI assistant is included only when it ships a testing-specific product or plugin that is documented.
• Candidates come from public directories, category pages, open-source topics, registries and vendor lists. Rejected candidates are logged with a reason.

Each tool has one primary category, assigned from its documented scope. Some also list secondary categories. The categories and facet values are fixed in advance and listed on the categories page.

Status

The status page lists 67 dated events with sources. A tool is “Acquired” when another operating software company now owns and sells it. A tool owned by a private-equity or holding company stays “Active”, and the event is still recorded. “Renamed” applies when the product itself was renamed in the 24 months before verification. “Shut down” needs a source, such as a vendor notice or an archived official repository. A homepage that does not load is noted, but does not change the status by itself.

Comparisons and alternatives

The 120 comparisons are generated from the tool facts, not written by hand. Each cell carries its own evidence level, source and date, or reads “Not documented”. A difference is stated only when it is documented for at least one side and vendor-stated for neither. “When each fits” lists only each tool's documented scope. No comparison names a winner.

Pairs come from a fixed input list, from comparison searches, and from pairs among the five most searched tools in each category, capped at 120 comparisons in total. “Most searched” is measured by how many distinct public sites target the query, because we use no paid keyword tool.

Independence and affiliate policy

This is a personal project of the author, not a company. No affiliate links, no paid placement, no sponsored listings. Tools related to the author carry a disclosure line and follow the same fact standard.

Default sorts never favour a tool. A tool related to the author is sorted like every other tool. See disclosures.

How the catalogue is compiled

The catalogue is compiled with the help of Claude Code (Anthropic). AI agents open vendor pages, record each fact with its evidence level, source and date, and draft descriptions from documentation. Scripts check every fact for a level, a date and, where required, a source before the site is built. A fact that fails these checks stops the build.

Blog posts are built from the same data and dated. They state counts and documented facts only, and link their sources.

Questions