How we tested accuracy
Method, results and disputed findings · Last updated: 2 November 2026
Before selling anything, we ran the audit on 22 live websites and had every finding checked against the live site. This page explains exactly what was counted, shows real disputed findings and how they were resolved, and lists what is still open.
Who did the checking
The reviewers were separate AI agents, who hadn't built the tool, briefed to be sceptical. They were not people, and not an outside firm. Each one re-fetched every flagged page itself, with the same user agent the tool uses, read the HTML, and judged each finding.
What counted
We counted each distinct failure or warning per site (repeats of the same check on one site count once). Each was given one verdict:
- Correct: true, clear, and the right thing to do.
- Misleading: true, but confusing, overstated, or wrong for that kind of business.
- Wrong: not true when checked on the live site.
The reviewers also spot-checked passes that are easy to get wrong, looking for problems the tool missed.
Results (final round, 1 November 2026)
| Findings checked | Correct | Misleading | Wrong |
|---|---|---|---|
| 110 | 92 (84%) | 15 (14%) | 3 (3%) |
The reviewers also found 11 problems the tool missed, from spot-checks and from reading the pages. One site blocked the audit; the tool said so plainly and reported nothing about its pages, which the reviewers judged correct. One site was left out of the final count.
Earlier we described this as "97% accurate". That figure counted the misleading findings as accurate. It was true in the narrow sense that 97% of findings were factually true, but it was too generous, so we've replaced it with the breakdown above.
Disputed findings, and how they were resolved
Sites are described rather than named, because these were not our customers.
1. "No About or contact page" (wrong)
The tool said: an AI app builder's website had no About page and no contact route, a high-severity failure.
The reviewer found: the homepage linked to a page called "Info", titled "Company Info & Contact", naming the company and its office addresses.
Resolution: the tool only looked for pages called "about" or "contact". It now also recognises "info" and "company info" pages.
2. "Page has no main heading" (wrong)
The tool said: a page on a large payments company's site had no main heading (H1).
The reviewer found: the heading was there, visually hidden but readable by crawlers and screen readers.
Resolution: the tool had ignored hidden elements. It now counts headings that crawlers can read even when they aren't shown on screen.
3. "Article page without article markup" (wrong)
The tool said: a consultancy's articles page needed article markup.
The reviewer found: it was the list of articles, not an article. Adding article markup there would have been bad advice.
Resolution: listing pages are now excluded from that check.
4. "Two pages share a title" (misleading)
The tool said: two pages on a shop's site had the same title.
The reviewer found: true, but both were "Page Not Found" error pages listed in the sitemap. The real problem was broken pages, not titles.
Resolution: pages that are really error pages are no longer judged on their titles.
What is still open
- A page with lots of header and footer text but an empty main section can still pass the "content without JavaScript" check.
- A redirect that takes two steps to reach the main address (for example http://www → https://www → https://) is passed, when our own rule says it should be a warning. Our own site has this one.
- Only the first 50 pages in a sitemap are checked for pages that shouldn't be listed.
- Sites that change between visits (different content on different fetches) can give different results on a re-run.
We re-run this test set before every engine change and compare results, so a fix can't quietly break something else. The figures above are from the final round before launch; they haven't been re-measured since the fixes, so we don't claim a higher number.
Questions
If you think a finding in your own report is wrong, reply to your report email or write to hello@vendorlens.in. We'll check it by hand and tell you what we find.