Testing Tools

Overview

There are two approaches to accessibility testing: manual and automated.  Accessibility test tools make automated testing possible and manual testing much easier.  Most tools are free, but some, especially the fully automated test tools, have a cost.  There is no one test tool that can check everything so you will want to build a suite of tools to use when testing the accessibility of your website.

  • Automated test tools cover about 30% of the required accessibility checks, but they can catch up to half of the errors when they include customized tests. Fully automated test tools such as the enterprise-level tools listed below will automatically test, monitor, and report on the accessibility of an entire site or even multiple sites, making it easy to test a website and track changes over time.
  • Manual testing allows for a much better understanding of how accessible a site really is, but the testing takes more time and more knowledge about accessibility is generally required.  Many of these tools perform the same exact checks as the automated test tools, but they must be started manually each time, only test one page at a time and will not provide comprehensive reports that track accessibility for your site.
  • Simulators and Assistive Technology such as screen readers are also used in manual testing and can provide you with valuable insights to the true usability of your web pages for people with disabilities.

WebAIM provides a good breakdown of considerations when choosing between Accessibility Evaluation Tools.

Novice-friendly Test Tools

These novice-friendly manual test tools will take you a long way in accessibility testing with only a small amount of knowledge or training needed.

  • Your Keyboard (Free) – Your keyboard provides one of the best manual tests there is for checking accessibility as many people with disabilities rely on keyboard navigation. Simply hide your mouse and see if you can get to every link, form field and button using the Tab key. You should be able to complete forms using the Enter key, space bar and arrow keys. See WebAIM’s write-up on Keyboard Testing for more information. 
  • ANDI (Most Browsers, Free) – This tool works from the bookmarks bar in the browser. It displays the accessible name of elements on the page so novices don’t have to guess. It also lists links, provides a grayscale view, and marks the reading and tab order.
  • Tota11y (Most Browsers, Free) – This tool works from the bookmarks bar from the browser. It has limited test capabilities but is novice friendly and provides a quick look at a web page’s headings, links, labels, landmarks, images, and color contrast. It’s a great tool to get started with but supplement it with other tools.
  • The Web Developer Extension (Chrome, Free) – This tool focuses on testing specific items and bridges the novice and developer expertise. It allows you to highlight or turn off content within the page such as page styles or image alt text without getting into the code, which can be useful to developers as well as novices.
  • WAVE Extension (Chrome / FireFox, Free) – This robust tool from WebAIM has the same testing capabilities as the expert-level tools, but with a novice-friendly interface.  It displays errors directly on the web page with a high-level summery and detailed description available. It also displays a web page’s heading structure or color contrast errors. A great tool to get started if you have no development experience, it is also used by expert testers and, like the fully automated tools, tests around 30% of the required criteria for accessibility.
  • Colour Contrast Analyser (Windows / MacOS, Free) – The Colour Contrast Analyser (CCA) helps you determine the legibility of text and the contrast of visual elements, such as graphical controls and visual indicators. Simply select any two colors with the eyedropper and pass/fail results are provided.  It also helps when designing color schemes.  Because it is browser independent, this tool from The Paciello Group (TPG) can be used evaluate color contrast in websites, Word documents, PowerPoint presentations, etc.
  • Color Oracle (Windows / MacOS / Linux, Free) – Color Oracle is a color blindness simulator that shows you in real time what people with common color vision impairments will see.  Because it is browser independent, this tool can be used evaluate color blindness in websites, Word documents, PowerPoint presentations, etc.

Expert-level Test Tools

These manual test tools are intended for use by developers or expert accessibility testers. They all provide the testers with the ability to run tests against a single web page and then drill down into the code to determine what is causing the error and correct it.  Like the fully automated test tools, these tools test around 30% of the required criteria for accessibility; the difference is that tests must be run manually one page at a time.  Each of these free manual tools can be used in conjunction with a fully automated tool from the same company.

  • Accessibility Resource Center (ARC) Toolkit (Chrome, Free) – This tool from TPG works alone or with the fully automated ARC Accessibility Management Platform (described later on this page). It requires a bit of browser know-how to use as it is embedded in the developer tools, but it is one of my favorite tools. It includes tests, such as testing reflow and text spacing, that are not included in the other tools listed here at the time of this evaluation.  ARC test results are divided into errors and warnings, where most of the warnings require manual verification. This tool uses TPG’s ARC accessibility ruleset and other rulesets such as the aXe accessibility ruleset below.
  • aXe Extension (Chrome, Free) – This tool from Deque works alone or in conjunction with the automated aXe tools from Deque (described later on this page).  It requires a bit of browser know-how to use as it is embedded in the developer tools.  This tool uses only the aXe accessibility ruleset which is designed to avoid false positives in the results, resulting in fewer reported errors than the ARC Toolkit but also resulting in little need for manual verification.
  • Siteimprove Extension (Chrome, Free) – This tool from Siteimprove works alone or in conjunction with the automated tools from Siteimprove (described later on this page).  Like the two other expert-level tools, it requires a bit of browser know-how to use as it is embedded in the developer tools.  Siteimprove evaluates a web page for accessibility issues and displays visual feedback about your content by highlighting detected issues on the page.  It provides an clear overview of your page’s accessibility issues, explanations of how they affect your users, and recommendations on how to fix them – and lets you drill down from the overview to the details and then to the code. Siteimprove uses a broad ruleset and tends to find more errors than ARC or aXe, but it also requires more manual verification than either of the other two tools.

Enterprise-level Test Tools

Once configured, these tools will automatically test, monitor, and report on the accessibility of an entire site or sites at scheduled intervals.  The in-depth reports generated by these tools provide novice testers and accessibility managers with wide-ranging insights for managing accessibility errors over time, while still allowing expert testers access to information needed to start troubleshooting errors. This expanded functionality has a cost, but the ease of use offered by this solution may well be worth the price.  Contact the vendor for the pricing for all these tools.

  • ARC Platform (Cost) – This tool is TPG’s accessibility management platform which includes accessibility analytics, defect reports, eLearning modules, an extensive knowledge base of developer techniques, and support.  These services are modular, and you can choose those that best serve your priorities. ARC allows for long term automated monitoring and tracking. It pulls together results from the same rulesets used by the ARC Toolkit, with dashboards that enable you to easily prioritize error remediation and uncover trends.  It tests all digital assets, highlighting violations and key insights about the underlying causes of accessibility defects.  Additionally, TPG offers a free report which allows you to try the tool and get a snapshot of the current accessibility of your site.
  • aXe Accessibility Tools from Deque all work in conjunction with the aXe Extension and use the same aXe ruleset that is designed to avoid false positives in the results.
    • aXe Monitor (Cost) – This tool performs automated enterprise-level accessibility audits to meet your compliance needs with advanced reporting, monitoring, and management of accessibility issues. It dynamically scans, monitors, and reports on the accessibility status of your sites across business and development teams.
    • aXe Auditor (Cost) – This tool provides guided step-by-step manual accessibility testing and reporting tool that enables functional testers with minimal accessibility knowledge to create comprehensive and consistent accessibility issue reports for developers.
    • aXe DevTools (Cost) – This tool finds and fixes accessibility defects while you code. It integrates into an existing development framework and can catch up to 70% of accessibility issues while coding.
  • Siteimprove Accessibility (Cost) – This test tool uses the same rulesets as the Siteimprove Extension and lets you see all your web assets, including PDF files, so you can fix accessibility issues.  You can prioritize issues based on responsibility, conformance level, and severity.  It pinpoints issues via on-page and in-code highlights so you can jump right into your code and fix the issue, with guidance and recommendations.  Issues are broken down into manageable tasks that can be filtered by role or responsibilities for quicker delegation and clear ownership.  Customizable dashboards and automated reports give you an easy way to measure progress.

Simulators and Assistive Technology for Testing

Simulators and assistive technology such as screenereaders let you experience firsthand what people with disabilities experience when they visit your website.  They are an invaluable part of your accessibility test suite.  WebAIM provides basic tutorials to get you started with screenreader testing for JAWS, NVDA, and VoiceOver.

  • JAWS – (Windows, Cost) Using a screenreader to test requires some training. This screenreader allows you to interact with your webpage in the same way as a blind user.  You can download a demo version of JAWS and test for 40 minutes at a time but need to buy a license to use it without rebooting regularly.
  • JAWS Inspect – (Windows, Cost) JAWS Inspect helps sighted testers evaluate sites for screenreader compatibility.  It is not a screenreader, but “reads” a web page in much the same way that a screenreader would and then generates a written report of what the JAWS screenreader would have said.  For testers who are not used to listening to screenreaders this tool can be extremely helpful.
  • NVDA (Windows, Free) – Using a screenreader to test requires some training. This free screenreader allows you to interact with your webpage in the same way as a blind user.
  • Talkback (Android, Free) – This screenreader comes with the Android operating system and can be turned on using settings.
  • VoiceOver (iOS / MacOS, Free with Operating System) – This screenreader comes with the iOS and MacOS operating systems and can be turned on using settings.