Accessibility Testing: The Ins and Outs and How To Do It

In America, 61 million people currently live with disabilities; globally that number reaches over a billion, according to the World Health Organization (WHO) (WHO/Y. Shimizu, Disability and Health). Testing your software to ensure that every person, disability or not, can browse and engage with your website is known as web accessibility testing. These disabilities include vision, physical, cognitive, literacy, and auditory disabilities. Understanding the unique testing considerations for each can be overwhelming, however, that understanding is important when creating your accessibility testing plan. In this post I’ve outlined the importance and types of accessibility testing, specific things you can test for, and free tools that can help you execute your tests.

Why is Accessibility Testing Important?

The World Wide Web continues to grow and seed itself into all aspects of our lives. You can pay bills, schedule doctors’ appointments, find your soul mate, and shop for groceries all from the comfort of your desktop or phone. Imagine being one of the billion people in the world who live with a disability and not being able to use these resources effectively, or at all. In 2008, access to information and technology over the web was deemed a basic human right in the United Nations Convention on the Rights of Persons with Disabilities (UN CRPD) (Convention on the Rights of Persons with Disabilities (CRPD). Accessibility consideration while designing and implementing your website or application is the right thing to do.

If ensuring accessibility for all, because it’s a human right, isn’t enough, a strong business case for its importance can also be made. According to the American Institute for Research (AIR):

The total after-tax disposable income for working-age people with disabilities is about $490 billion, which is similar to that of other significant market segments, such as African Americans ($501 billion) and Hispanics ($582 billion)

https://www.air.org/system/files/downloads/report/Hidden-Market-Spending-Power-of-People-with-Disabilities-April-2018.pdf

If you’re not prioritizing accessibility when developing new products, you could be missing out on a large segment of income.

Another section of the population to consider are those with temporary disabilities or situational challenges. Maybe you’ve broken your arm, are in a meeting trying to check something on a smartwatch, trying to watch a video in class without turning the sound on, etc… All of these temporary disabilities or situational challenges happen to most of us at some point. To get more context, check out the video put out by the Web Accessibility Initiative talking specifically about situational or temporary disabilities: https://www.youtube.com/watch?v=3f31oufqFSM

The last reason, but certainly not the least, is the law. In some cases, adhering to the outlined standards isn’t a choice, it’s a requirement. Not adhering to it can bring on unnecessary lawsuits for your organization. In American, the following laws are in place:

• Americans with Disabilities Act (ADA): This law states that all the domains like public buildings, schools, and organizations should make technology accessible to everyone.

• Rehabilitation Act, section 504 and section 508: Section 504 accommodates all people with disabilities to access information in workplace, education, and other organizations and section 508 requires accommodations and access to technology.

What Disabilities Should I Consider When Testing?

• Vision: Color blindness, full blindness or partial blindness are disabilities to consider when testing. In addition, some visual effects such as strobe lighting or flashes on screen can trigger things such as seizures.
• Physical: Decreased motor skills can make tasks like using a mouse or keyboard difficult.
• Cognitive/Learning/Neurological: Learning disabilities may make it harder for some to engage with complex functionality such as captchas.
• Auditory: Auditory issues such as hearing impairments or deafness could make it difficult or impossible for someone to hear your content.
• Speech: Speech issues such as impediments, limited, or no vocalization might make it difficult or impossible for someone to interact with your application.

W3C/WAI – What Do These Mean?

As you begin to research accessibility testing you will quickly find yourself at the World Wide Web Consortium (W3C). As they state simply on their website, they “develop international standards for the Web: HTML, CSS, and many more” (w3c_wai, Making the Web Accessible).

The W3C put together the W3C Web Accessibility Initiative (WAI) to develops standards and support materials to help understand and implement accessibility considerations across the web. As you’re looking for ways to ensure accessibility standards are being met, it should be your second stop, after this blog post, of course!

Tools for Accessibility Testing

Web accessibility tools can help an organization quickly identify potential issues. While tools on the market will tout full automation of testing, the process still requires a manual review for potential risks. As with any automation, tools can produce false failures or bring back negative results that actually behave as the functionality was designed. There are many considerations and each project or organization’s needs may vary.

When selecting a tool keep in mind what features you will need to consider:

• Language and license types are two of the first decisions to be made. If you’re looking for a low-to no-cost tool, an open-source tool could be ideal but will limit your available choices. Also, selecting a tool that fits your programming language is important and will limit the available choices overall.
• Determining what type of tool would best fit into your testing processes is important. There are add-ons that you can simply plugin and start testing with. Additionally, there are tools that are useful for testing mobile vs. desktop platforms.
• Selecting a testing tool that analyzes based on the guidelines you’re following is key. For example, the W3C Web Content Accessibility Guidelines (WCAG) is recognized as the standard for web accessibility and supported by most tools.
• Decide what type of feedback you’re looking for. Some tools have great reporting outputs, making it easier to identify issues, begin analyzing, and making changes. Make certain the tool you choose has an easy way to get the information you need, in the format you need (CSV, PDF, etc…).
• Understand what degree of automation you’re looking for. Some tools can only test one page at a time, while others have the ability to test large groups of pages. Depending on the frequency of use and how large your site is, you’ll have to decide what is best.
• Authoring tool plugins makes it super simple for non-technical people to check results within the tools they are using to author content. Depending on who in your organization owns accessibility, this option might make a lot of sense.
• If what’s needed is to have someone outside of your organization own accessibility testing and they simply give you the results, you might choose to partner with a third party vendor.
• Lastly, if you’re looking for some simple tooling to allow you to manually check some easy things, review Common Things to Validate Before Singing Off on Accessibility Testing section. I’ve given some links to free tools where applicable.

W3C has a published list of accessibility testing tools found here.

Common Things to Validate Before Signing Off on Accessibility Testing

VISION CONSIDERATIONS

Depending on the severity of a vision disability, there are varying needs to consider. If a person is older, they may be able to see larger images, but not smaller fonts, while a person that is completely blind may rely on a screen reader assistant to vocalize on-screen text. In addition, color blind people might not have any problems seeing the words on the screen, but if the color of the font and the color of the background don’t provide enough contrast, they may not be able to see that there is text on-screen at all. Making sure you’re validating each of these scenarios is important in ensuring vision accessibility is covered:

• Screen Readers allow a person to hover over text or a link on a website and the reader will vocalize the words for the person. In addition, it will let the user know what type of element is on screen, like a button or link. There are many different tools to test this, but I’ve found Chrome has a nice plugin for this called ChromeVov. Validating your feature works well with a screen reader is a crucial step in your test plan. iOS devices also have accessibility assistance. A full listing of those, including their screen reader, can be found here. Likewise, Android has the same, and can be found here.
• Pop-ups can confuse anyone trying to navigate a web page. Ensure when pop-ups are shown to the user, the screen reader can account for it properly, and the pop-ups aren’t disruptive to the user experience.
• Unless you have a form of colorblindness it is hard to understand what that experience feels like. Using a website such as this will allow you to test your own website to validate the experience for a colorblind individual. Similar tools exist for validating mobile apps.
• Some people cannot read the text if there isn’t sufficient contrast between the text and background, therefore, validating there is sufficient color contrast on your website or app is important. According to the W3C, “foreground text needs to have sufficient contrast with background colors. This includes text on images, background gradients, buttons, and other elements. This does not apply for logos, or incidental text, such as text that happens to be in a photograph” (w3c_wai, Easy Checks – A First Review of Web Accessibility). A minimum contrast ratio of at least 4.5:1 meets acceptable standards. Chrome has an extension that will help you understand your ratios: WCAG Color contrast checker. For mobile devices, you can also use this site to input your colors and get the ratio.
• A common tool people with vision impairment use is a text enlarger. It lets you enlarge the text of your screen, on any device, so that you can more easily read the text. Most browsers and devices allow you to change the size of your text to allow you to test this user experience. As you’re testing this you’ll want to make sure you are looking for column or row overlaps, section overlaps, or text simply disappearing. In addition, make certain buttons and links are clickable and all forms and fields remain useable.
• Moving, flashing, or blinking content can cause a host of visual issues for some. These visuals include ads/videos, auto-updating information on the screen, flashing sale signs, etc… If a user cannot stop moving animated content it can become distracting or even dangerous. When testing, you should ensure there is no flashing or moving of information that lasts more than five seconds and cannot be stopped by the user. In addition, check to make sure no content is flashing more than 3 times per second, as this can cause seizures in some.

PHYSICAL CONSIDERATIONS

Physical disabilities refer to the partial or complete loss of movement. This can be a temporary loss such as a broken arm, or long term disabilities such as a loss of a limb or arthritis.

• Many people with physical limitations, whether temporary or permanent, cannot rely on a mouse to navigate a website. They may use a keyboard to interact instead. Ensuring that your site allows those people to use a keyboard to fully interact with your website is important. Testing this is simple; go to your website, click the address bar, and then put your mouse to the side. Using only your keyboard navigate the pages and interact with each element. When testing, you will validate you have proper outlining of fields, focus logically shifts through your forms when tabbing, media controls function, drop-down lists allow navigation and selection, and calendars allow for manual input.
• Some people may need more time to complete an interactive task than others. This could include temporary limitations such as a carpal tunnel flare-up, or more serious long term limitations. In such instances, testing to make certain your site accommodates slower inputs is important. When testing make certain you aren’t timing out too soon (unless a specific security requirement requires this). Furthermore, ensure the user has the ability to extend the timeout if required. Also, testing to validate that interruptions are kept to the bare minimum is important to allow people the time to interact with your site, whether it’s filling out a form or reading your terms and conditions. W3C has a good link to timeout guidelines.
• For those who have trouble using both a keyboard and a mouse, they might choose to use a speech-to-text tool. This allows a person to use speech to navigate a website. To test compatibility with this software, you can enable Speech Recognition on your Windows machine. Speech Recognition for Windows. For Mac users, there is Enhanced Dictation. Enhanced Dictation for Safari.

COGNITIVE, LEARNING AND NEUROLOGICAL CONSIDERATIONS

This section represents a large range of disabilities some of which impact the mental or intellectual intelligence of a person. These disabilities can alter how people express or receive information and their ability to understand and consume such information.

• Your website should have a simple and consistent flow throughout every page. Labels, buttons, and other elements should be accurately named. This is important so individuals understand how to move around your site at all times and it limits confusion.
• Using words and phrases that are as simple as possible while still getting your point across is a good practice no matter what. If you’re writing to the general public and aren’t delivering highly specialized content, make certain your readers can understand the concepts or phrases you’re trying to convey. Several applications will help you understand your sentence complexity. I use an application called Grammarly for writing and testing grammar and complexity.
• Limit the number of distractions on screen. For someone with Attention Deficit Disorder, distractions can limit their ability to function within your site.
• Some neurological disorders such as Parkinson’s Disease can cause hand tremors, making it difficult for someone to use a mouse and point to small buttons or click links to interact. When testing for this, make sure you’re validating button/field/link sizes and that clicking anywhere within the element allows interaction.
• Moving, flashing, or blinking content can cause a host of visual issues for some. These visuals include ads/videos, auto-updating information on the screen, flashing sale signs, etc… If a user cannot stop moving or animated content it can become distracting or even dangerous. When testing, you should ensure there is no flashing or moving information that lasts more than five seconds and cannot be stopped by the user. In addition, check to make sure no content is flashing more than 3 times per second, as this can cause seizures in some.

AUDITORY CONSIDERATIONS

Auditory disabilities include partial or complete loss of hearing. In addition, some people may have issues hearing audio with background noise. When testing for auditory accessibility keep these things in mind.

• Your videos should have closed captions (CC). To test this, simply start your video and ensure captions clearly state what the video is saying. CC should account for vision disabilities as well. For example, the text for the closed captions should be large enough for everyone to read.
• Text transcripts should be easily accessible and available for all audio. The transcripts should match what is verbalized in your audio. Transcripts should allow for vision disabilities as well in that text is large enough for everyone to read.
• For those with partial hearing impairments, ensure there are audio controls for your video/audio. Allowing users to control the volume of your media is important. When testing, make sure you are testing compatibility between site audio controls and device audio controls.

SPEECH CONSIDERATIONS

Speech disabilities can impact a user’s ability to speak comfortably, audibly, or at all. In addition, there are instances where speaking is temporarily not an option. When testing for speech disabilities, make sure to keep these things in mind.

• Your chat platform should allow end-to-end flows that don’t require speaking. Make sure you are testing those thoroughly as chat is a very common tool for those with speech disabilities.
• Make certain your email functionality is user friendly and allows common email providers to integrate appropriately.

Common Myths on Accessibility Testing

Accessibility testing isn’t always a standard testing type in organizations. Some organizations require it based on their mandatory adherence to disability laws and standards. Other organizations perhaps have considered it but have shied away due to the initial investment of making your site accessible. Here are some common myths I’ve heard around accessibility testing:

• MYTH 1: Accessibility doesn’t allow your website to be fun and creative. Designing a website that is accessible for all doesn’t mean that you can’t have images or videos. Creating accessible content that will engage your customer base, while keeping your designs creative, is certainly doable.
• MYTH 2: We don’t have time to consider accessibility. Development teams are already crunched to get things out the door quickly and adding accessibility into the testing phase can slow that down even further, I get it. However, consider having your design team own accessibility up front and not just accounting for it during testing. If accounted for during design, testing should be straightforward and defect counts should be low.
• MYTH 3: Converting inaccessible web pages or features to be accessible doesn’t make business sense. I’ll remind you that disabled people make up as much of the disposable income market base as African Americans. $490 BILLION a year is nothing to overlook when running a successful company. And that doesn’t account for the temporarily disabled individuals.
• MYTH 4: Accessibility is the responsibility of the developer to consider and implement. Intentional web accessibility considerations start within the design phase. Having accessibility be standard and uniform across all development teams is important.
• MYTH 5: Automated evaluation tools are good enough for testing. Most automation tools will give you results but it’s up to a human to validate if those are actually changes your organization needs to implement. For example, an automation tool can call out if an image doesn’t have descriptive text but it can’t tell you if that text is informative to the user.

Conclusion

As more and more of our daily lives play out over the web, it becomes more important for those of use creating content to consider who will be using it, and being inclusive for all. As testers, considering accessibility in our test plans helps ensure we are validating all of the guidelines discussed above.

It’s important to start small. Pick some easy things that require little effort and focus on those during your testing. W3C has a great Easy Checks page that will help you, along with this blog, implement some simple testing.

Lastly, if your organization is ready to go all in and make accessibility a priority, ensure you’re selecting the right tool for the job. Keep in mind that validating accessibility across websites or applications should involve manual validations too. Don’t rely too heavily on automated tools because you may spend valuable time chasing the wrong things. After all, the goal of accessibility is to provide the best human experience for everyone, regardless of any limitations they may have. So make sure you’re using humans to validate that goal is being met.

Citations

• Convention on the Rights of Persons with Disabilities (CRPD) Enable. (n.d.). Retrieved June 2, 2020, from https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities.html
• WHO/Y. Shimizu. (n.d.). Disability and health. Retrieved June 3, 2020, from https://www.who.int/news-room/fact-sheets/detail/disability-and-health
• W3c_wai. (n.d.). Making the Web Accessible. Retrieved June 2, 2020, from https://www.w3.org/WAI/
• W3c_wai. (n.d.). Easy Checks – A First Review of Web Accessibility. Retrieved June 12, 2020, from https://www.w3.org/WAI/test-evaluate/preliminary/