Automated vs manual accessibility testing, criterion by criterion

Automated accessibility testing runs rules against your code and rendered pages. A rule can establish some failures outright, such as an image with no text alternative or text whose contrast is too low. Other problems a rule can only flag for a person to review, and for many criteria there is no automated rule at all: no tool can tell whether your reading order makes sense or your error messages help.

Manual testing, by someone who knows WCAG and uses assistive technology, covers every criterion but costs time. The table below shows, for each WCAG 2.2 Level A and AA criterion, which automated sources check it and what a person still has to check.

Our guide to the website accessibility audit shows where both kinds of testing sit in an audit.

Why we give no percentage

Many articles on this topic lead with a single figure for how much of WCAG automation covers. The published figures disagree widely, because they count different things: some count success criteria, and others, such as Deque’s 2021 study of more than 2,000 audits, count issues by volume across the pages audited. None of them tells you whether a given criterion is tested on your site.

Two more reasons make the figures misleading:

• A rule that exists is not a criterion covered. axe-core’s image-alt rule fails an image with no text alternative. It cannot tell whether “photo123.jpg” or “image” serves the image’s purpose, which is what 1.1.1 asks. The criterion has a rule, and still needs a person.
• A pass is not a conformance result. When no rule fails, that means no rule found a failure on the states the scan reached. It does not mean the page meets the criterion.

So we give no total. Read the row for the criterion you care about.

How to read the table

Each row is one WCAG 2.2 success criterion at Level A or AA. The columns:

• axe-core 4.14 rules: the rules in axe-core, an open-source engine many testing tools build on, tagged with that criterion at version 4.14.0. “Review only” marks a rule that can only flag something for a person, never report a failure. “Experimental” and “deprecated” mark rules axe-core does not run unless a tool turns them on, and target-size is off by default: a tool runs it by selecting axe-core’s WCAG 2.2 checks or turning it on.
• W3C ACT rules: whether the W3C’s Accessibility Conformance Testing rules include a test procedure for the criterion, and whether axe-core’s implementation is consistent with it, according to the W3C’s implementation report for axe-core. That report covers axe-core 4.14.0, the version these tables cite, runs it with its experimental rules on, and was last updated on 24 October 2026. ACT rules are informative and are not part of WCAG. “Proposed rules only” marks a criterion whose ACT rules are all still proposed, not yet approved by the W3C.
• Our rules: the rules of ours that our rule catalogue maps to the criterion. “Off by default” marks a rule you have to switch on in your audit settings. By default, every audit we run also uses axe-core’s WCAG A and AA checks alongside these, including target-size, and turns on the experimental css-orientation-lock.
• What a person still checks: the part of the criterion no rule decides, in short. Even a row full of rules has one.

Our WCAG 2.2 checklist has the code that passes and fails each criterion.

Every A and AA criterion

Perceivable

Operable

Understandable

Robust

Why tools disagree

Run two tools on the same page and they report different things. Three reasons account for most of it.

1. How sure a rule has to be. axe-core reports a failure only when its checks are conclusive, and reports “needs review” otherwise. A tool that reports uncertain results as errors shows more of them. Neither approach is wrong, but their totals are not comparable.
2. What the scan reaches. A rule tests the page in the state the scan sees: usually the first load, at one screen size, with nothing open. Menus, dialogs, error messages, hover and focus states, and the steps of a checkout need a scan that opens them, or a person.
3. What each rule tests. Two rules for the same criterion can test different things. That is why the W3C publishes ACT rules and reports whether each tool’s implementation is consistent with them.

Where AI-assisted testing fits

Language and vision models can help with the rows a rule cannot decide. A model can point out a text alternative that does not seem to match its image, a link whose text says nothing on its own, or a heading that does not describe its section. Each of those is a candidate for a person to look at, not a result.

A model’s output is judgment. It can be plausible and wrong, and it gives no evidence a rule could repeat. Keep it apart from measured failures: report it as an observation, give it no severity, and never count it as a pass or a fail.

How to combine them

1. Run the rows with failure rules on every release. Put axe-core or a tool built on it in your build or on a schedule. These checks are cheap to repeat and catch regressions in what a rule can decide. A default axe-core run skips target-size and the experimental and deprecated rules: select axe-core’s WCAG 2.2 checks to run target-size, and turn the others on, if you want their rows.
2. Have a person triage the review-only results. A “needs review” result is a list of places to look, not a finding.
3. Test the rest by hand, with assistive technology. Keyboard only, a screen reader with a browser, zoom and text spacing, and a pass through every process from start to finish. That is where much of an audit’s time goes, and fixing the automated failures first leaves more of it for this; see what drives the cost of an audit.
4. Write down what was not tested. A report that lists only failures leaves the rest unknown. We cover what goes in an audit report, including a result for every criterion.