What automated accessibility testing actually finds

Every scanner on the market runs on the same kind of engine we do, and every one of them has the same ceiling. Most vendors put that fact in the footer. We put it on a page with the studies attached, because a customer who understands the limit gets more out of the scan — and because the honest sentence "automation found 34 issues, a human audit will find more, and we do those too" is a better business than the dishonest one.

One page, no account, no card. Full-site scan (up to 150 pages): $49 — see pricing.

The numbers, and where they come from

Published measurements of automated coverage

Study	What it measured	Result
Deque coverage study	Roughly 300,000 issues across 13,000+ pages and 2,000+ audits, comparing what the axe engine caught against full audits.	Automation fully caught about 57% of issues by volume.
UK Government Digital Service tool test	One page seeded with 142 documented barriers, run through 13 automated tools.	The best tool found 40%; WAVE found 30%; axe found 29%.
Common industry framing	Share of WCAG success criteria that are machine-testable at all.	Roughly 30–40%.

Those numbers look contradictory and are not. They measure different things: share of issue volume (one missing-alt rule can fire five hundred times on a catalogue) versus share of barrier types or criteria. Volume flatters automation, because the errors machines find are exactly the ones that repeat. However you count it, the conclusion is the same: a scan is the start of an audit, not the audit.

What machines are genuinely good at

This is not a case against automated testing — we built a scanner. Machines are fast, cheap, repeatable and never bored, and they are reliable on anything with a measurable answer:

• Images with no text alternative at all.
• Colour contrast below the required ratio — computed, not eyeballed.
• Form fields with no associated label.
• ARIA misuse: invalid attributes, invalid values, roles that contradict the element.
• Structural faults: missing page language, list markup, frame titles, nested interactive controls.

On a real storefront that is usually dozens to hundreds of findings, concentrated in a handful of templates, and it is the cheapest accessibility work you will ever buy. It is simply not all of the work.

What no scanner can catch

The failures below are not exotic. They are ordinary things a human auditor writes up in the first ten minutes, and they are invisible to every automated engine — including ours.

Real WCAG failures that automated testing cannot detect

WCAG criterion	The failure	Why automation misses it
1.1.1 Non-text Content	The alt text is present but meaningless — a courier photograph described as "image123.jpg".	A rule can check that an alt attribute exists and is non-empty. Judging whether the words describe the picture requires seeing the picture.
2.4.4 Link Purpose	A link labelled "click here" with nothing to indicate where it goes.	The link-name rule passes: the name is present and unique. Whether it is informative is reading comprehension.
1.3.2 Meaningful Sequence	Two columns painted in the reverse of their source order by CSS, so the visual reading order and the keyboard/screen-reader order disagree.	The markup is valid and the CSS is valid. Nothing in the page declares which order the author intended.
2.4.7 Focus Visible	The focus outline is suppressed site-wide, so a keyboard user cannot see where they are.	Every control is still focusable and still named, so no rule fires.
1.3.1 Info and Relationships	Section headings built as styled <div>s — they never appear in a screen reader's list of headings, so the page cannot be skimmed.	A large bold div is legitimate markup. There is no signal that it was meant to be a heading.
1.4.1 Use of Colour	Stock status conveyed by red or green text alone, with a legend telling the reader to go by the colour.	Both colours pass the contrast check, so nothing fires. Noticing that colour is the only carrier of the meaning needs a person.
1.3.3 Sensory Characteristics	An instruction that identifies a control purely by position: "press the button on the right-hand side".	It is prose. Nothing in the DOM marks it as an instruction, let alone a positional one.
3.3.2 Labels or Instructions	A button labelled "OK" that books a courier and cancels the order — a destructive action behind a neutral label.	The button has an accessible name, so the rule passes. The mismatch is between the label and a real-world consequence that is not in the page.
3.1.2 Language of Parts	A French care instruction inside an English page, not marked as French, so it is read aloud with English pronunciation.	Language rules can only inspect language attributes that exist. A missing one means detecting the language of the prose.

We keep a page of these in our own test suite

Those nine failures are not hypothetical examples we wrote for this page. They are seeded into a fixture page that ships inside our repository, and our automated test suite asserts that a full scan of it reports zero violations, zero best-practice findings and zero needs-review findings — a page full of genuine, documented barriers that our scanner declares clean.

The fixture exists so nobody here can quietly drift into believing the marketing. If a future change to the scanner starts reporting findings on that page, we investigate the finding before celebrating it, and we never "fix" the page — repairing it would destroy the evidence behind everything on this page.

So what do you do about the other half?

1. Start with the scan, because it is free for one page and $49 for the site, and because the machine-findable issues are real barriers that repeat across every template you own.
2. Fix what it finds — that work is code work, and it is the bulk of the volume. Fixed-price bands.
3. Add a human pass where the risk sits: checkout, account creation, search. A person drives the site by keyboard and screen reader and writes up what no engine can see. We sell that as an add-on precisely because our own disclosure creates the need for it.

Anyone who tells you an automated result settles the question is either not reading their own tooling's documentation or is selling something. The US regulator has already required a vendor in this market to pay $1,000,000 over exactly that kind of claim — the overlay story is here.

That disclosure is on this page, in every report we send, and in every quote — see how we work.

Keep reading

• EAA compliance checker: what the law requires and who it covers.
• How BarriScan works: engine, evidence, fixes, verification.
• Pricing, including the human audit add-on.