Evaluation Tools

This page highlights the most popular accessibility evaluation tools and explains their basic strengths and weaknesses. Rather than using just one tool, you will want to use a suite of tools to test your pages. Not all tools are specialized software. The humble keyboard is a powerful accessibility testing tool that will flag many accessibility shortcomings. You can purchase expensive and feature-rich programs such as Freedom Scientific's JAWS and NetCentric's CommonLook - but we will not emphasize these programs, because for most testing scenarios there are free tools that are capable of doing the job.

The Keyboard

One of the most useful accessibility checking tools is the keyboard. Keyboard accessibility is one of the cornerstones of web accessibility. Screen reader users and those with motor impairments cannot use a mouse. Many people prefer using the keyboard to navigate web pages; for instance, the popular web browser Opera has excellent set of keyboard shortcuts.

Before using any other accessibility evaluation tool, first make sure that you can easily navigate to any part of your web pages or website and perform all functionality using just the keyboard. Unplug your mouse so you won't inadvertently cheat. You will find that this practice demonstrates clearly the difference between technical (yes, it can be done) and functional (but it might be a real pain) accessibility.

Note: If you are using a Mac, you will want to look at these tips for enabling keyboard functionality in the common Mac browsers.

View more information on testing for keyboard accessibility.

Web-Based Evaluators

W3C Validator

Validation is the first step in evaluating web accessibility. If your website doesn't validate to W3C standards, you may preventing assistive technology users from accessing your web pages. Syntax errors that do not affect the visual presentation of your page can hobble screen readers and other assistive technology. The particular W3C standard is not critical. You can validate your site to a variety of standards using the W3C Validator, which will give you a listing of errors and remediations.

AChecker online accessibility evaluator

One of the best one-click accessibility checkers is AChecker. It is extremely efficient because on a single page listing, it cites the line number of the accessibility violation, shows the errant code, gives the appropriate remediation, and links to a resource page specific to the problem. You can set the type and level of conformance you would like to achieve. It is very accurate as well. One limitation of this tool is that it is stymied by private content. However, this limitation can be overcome by copying the "view source" listing from the authenticated page, pasting it into a local HTML file, and uploading it to the checker. Note that the AChecker output is very comprehensive. Do not be distressed by the "Likely" and "Potential" error listings. Concentrate on the "Known Problem" listing, which will be the default listing on the results page.

WAVE Evaluator

The WAVE Web Accessibility Tool is produced by WebAIM, an accessibility industry leader. WAVE uses a distinctive icon approach in displaying accessibility information. It can display 80 possible icons to show missing alternative text for images, missing form labels, table structure, script elements and event handlers, document structure and reading order, and much more. 80 icons are too much to remember, but fortunately, Tooltips are used to explain the icon meanings, and the toolbar has an option to view an icon key.

Besides an icon display mode, the tool offers views of the structure and tab order of the page, an outline view of the headers, and a "disable styles" view.

Browser Extensions

WAVE Accessibility Toolbar

The WAVE Toolbar for Chrome is produced by WebAIM, an accessibility industry leader. It is essentially a toolbar version of the WAVE Evaluator tool.

Besides an icon display mode, the tool offers views of the structure and tab order of the page, an outline view of the headers, and a "disable styles" view.

The currently rendered document is evaluated directly within Chrome, allowing accessibility evaluation of CSS, scripted, AJAX, DHTML, dynamically generated, and secure web content.

One weakness of this tool is that the page rendering with the icons displayed can obscure certain page features and sometimes icons don't appear on the page. The tool does not have an option for listing all errors. Also, WAVE does not reference related accessibility documentation, which is ironic given that the WebAIM website is rich in content. However the tools tab on the menu bar does contain a link to the WebAIM home page, which has a very good search utility.

Firefox Accessibility Extension

This excellent accessibility add-on for Firefox covers most aspects of accessibility, and therefore requires some time to become acquainted with its tool set. The add-on is based on the Illinois Center for Information Technology and Web Accessibility (iCITA) best practices and comes with support documentation. One advantage that this extension has over AChecker is that it allows you to check CSS, scripted, AJAX, DHTML, dynamically generated, and secure web content.

Gez Lemon color contrast analyzer add-on

The Color Contrast add-on for Firefox is a very convenient way of testing the color contrast of your web pages. The tool is accessed conveniently through a toolbar or a Tools menu pick in Firefox and can be used in conjunction with other Firefox accessibility and web development tools. It generates an easy-to-read HTML report in seconds that can be saved and forwarded to web developers or webmasters. It is based on the WCAG 2.0 Level Double-A standard.

This tool cannot be used to evaluate graphic or Flash content, and it cannot evaluate text over a background image.

Screen Readers

Most web developers think of screen readers as being expensive and difficult to use and therefore not practical for web production work. This is no longer true. There are several free options (discussed below) that take a minimal amount of time to learn, and this is time well spent. There is no substitute for doing hands-on testing with a screen reader to evaluate the accessibility of a web page. Validating your pages is necessary—but not sufficient. A page might check out with a validator and be theoretically accessible, but it may well not be functionally accessible (i.e., it's a pain and time sink to find stuff, navigate, or operate the controls).

When using one of the screen reading options below, it is important to unplug the mouse and turn off your computer monitor. It is very tempting to "cheat" because, as you will quickly discover, people using assistive technology have to work much harder than able-bodied folks to navigate the web. Every moment of frustration you feel when using a screen reader is like reading an accessibility book. In, fact you may very well awaken a sense of moral outrage inside yourself if you use a screen reader often enough. It is truly remarkable the amount of nonsense on webpages that the disabled community are forced to contend with.

Web Anywhere

Web Anywhere is a web-based screen reader for the web. It requires no special software to be installed on the client machine and, therefore, is extremely convenient to use. Web Anywhere will run on any machine, even heavily locked-down public terminals, regardless of what operating system it is running and regardless of what browsers are installed. WebAnywhere has some limitations: namely, its inability to access sites with a frame-killer, a slow voice rate, and a limited command set (also an advantage programmers!). However, it is still very useful as a low-overhead screen reader. The home page contains good instructions and a listing of the command keys used.

NVDA (Windows-only)

NonVisual Desktop Access (NVDA) is a free and open source screen reader for the Microsoft Windows operating system. It also has the ability to run entirely from a USB drive with no installation. It has similar functionality to its much more expensive counterparts, but has a slimmer command set. It is gaining a lot of traction lately in the disability community because of the responsiveness of the programmers to end-user feedback and also their dedication to interoperability and to web standards—and the fact that it is free, of course. Free or not, it wouldn't be heavily used if it weren't a good program.

The Yahoo! Developer Network Blog contains an excellent article by Tom hughes-Crouchner, the Yahoo! Accessibility Project Manager, on testing web accessibility with NVDA, providing invaluable information. You can also visit the WebAIM NVDA page.

NVDA is such a good screen reading option that Mac users should seriously consider installing VMWare or Virtual Box to run it.

Mac VoiceOver

If you use a Mac computer, and you don't want to run a virtual Windows machine on your computer to run NVDA, you can use Apple's native screen reader, called VoiceOver, to test your site. It is easy to activate: click on the Universal Access icon in the System Preferences pane and select the On button for VoiceOver. Alternatively, you can enter Command + Fn + F5 from the keyboard. The first time you use VoiceOver, you will be offered the opportunity to take the online tutorial, which you will want to do. It's short and will have you off and running in no time. Voiceover takes a different approach than other screen readers, using menu-driven system instead of "single-key" navigation used by JAWS or NVDA, for example. This can be a more cumbersome method of getting around websites for disabled users, but it is perfectly adequate and easy to use for those doing testing.

JAWS

Last, but not least, JAWS is the most popular screen reader in the world today. It has a large feature set that mostly has been developed as a set of work-arounds to deal with sub-standard web pages. In terms of testing your web pages for accessibility, however, the expanded command set may actually be a liability. A page's accessibility shouldn't depend on arcane JAWS commands (nor the commands of any specific screen reader). There is also the issue that most web production folks don't have the time to learn JAWS adequately nor the budget to pay for it.

Lynx text-only browser

Lynx is a Unix-based, text-only web browser. (You can also install Lynx on your Mac or Windows machines.) Once popular with visually impaired users, it is still useful as an accessibility and usability tool. Lynx will not run javascript code. Tables are linearized (as they would be for a screen reader), and frames are identified by title name and are treated as separate pages. It is a very lightweight and convenient tool that has no learning curve.

PDF Accessibility Checker (PAC)

Short of testing a PDF document with a screen reader or opening the document in Adobe Acrobat Pro, it has been difficult to verify the accessibility of a PDF document. This situation has improved with the recent introduction the PDF Accessibility Checker. PAC conducts tests using 14 criteria:

  • Document is marked as tagged
  • Document Title available
  • Document Language defined
  • Accessible Security Settings
  • Tab follows Tag-Structure
  • Consistent Heading Structure
  • Bookmarks available
  • Accessible Font Encodings
  • Content completely tagged
  • Logical Reading Order
  • Alternative Text available
  • Correct Syntax of Tags / Rolls
  • Sufficient contrast for Text
  • Spaces existent

PAC has a webpage of instructions to assist you. One disadvantage to this package is that it is Windows-only. However, this is a huge improvement over doing a simplistic accessibility quick check in Adobe Reader to see if the document is tagged. In general, if you have PDF questions you should contact Maria Feeley at mfeeley@wlu.edu, the University's Chief Legal Officer and General Counsel.