This document describes general guidelines you should follow when testing pull requests in this repository. It is not exhaustive but should be the starting point that you adapt for each PR. You may also use your best judgement and skip things that are unrelated to a specific PR. However, please be careful when doing this as accessibility bugs especially are easy to slip through the cracks when we aren't doing our due diligence and testing changes thoroughly.
Please follow the instructions in the README for how to set up and run the application locally. If you want to just generally test the application in its current state (i.e., what exists on main as opposed to what has actually been deployed to production) you can find the staging deployment here:
https://search-staging.openverse.engineering
Staging is redeployed everytime we merge to the main branch, so if you're looking to test the latest version of the app without going through the local set up, that's the easiest way to go.
Once you have the application running, you can visit it in your browser at http://localhost:8443.
You can also access it from other devices in your same network (like a mobile phone) for additional testing. See the finding your local IP address section of the README for how to identify the local IP adress Nuxt is served on. Once you have identified your local IP address, you can access the website running on your computer by visiting https://<local IP>:8443 replacing <local IP> (including the brackets) with the value you found using the instructions above in your mobile devices browser.
Testing from multiple different devices as often as possible is a great way to contribute to Openverse's frontend development.
In addition to testing on multiple devices, we also strive to test on almost all widely used browsers excluding Internet Explorer. Please be ready to regularly test your work and the work of others in Firefox, Chrome, and Safari on a desktop computer. On mobile devices, Mobile Safari on iOS, Firefox for Android and Mobile Chrome on iOS are all important targets for testing as well. A significant amount of web traffic is mobile these days.
Please review the WordPress Accessibility Handbook.
The WAI-ARIA spec. This document describes, in detail, all of the documented types of interactions that happen on most websites and the accessibility properties of them. Many of them also include examples.
Gutenberg also has an excellent Accessibility Testing Guide with specific instructions for setting up screen readers for testing with.
Practice using keyboard navigation when you run the app locally generally. This will reveal to you some of the initial hurdles that the app currently presents to users who rely on assistive technology. Note that keyboard accessibility is part of the bare-minimum in accessibility for a website along with accessible color contrasts.
If you are a regular contributor, at least once a week, attempt to use the site using a screen reader like VoiceOver on macOS, NVDA on Windows, or Orca on Linux. If you do not regularly rely on a screen reader for navigating the web, it can also stretch your comfort level a lot by closing your eyes or turning off your monitor while navigating using the screen reader. Keep in mind that many people who rely on screen readers to navigate the web do not have any of the visual context that a sighted user is using to interpret a website. This especially applies to directionality and the broad context of a page. Screen readers can't "see" what's at the "end" of the page unless the user navigates all the way there. Sighted users have a huge privilege in being able to take in the broader context of a website almost immediately through visual information.
Buttons, form fields, and other interactive elements should all have visible and high contrast focus styles applied. Please note that hover styles are not the same as focus styles and are often distinct. Note also that hover styles and focus styles may not always be applied at the same time. It is best to test the following scenarios:
- Hover over the element, unfocused
- Focus the element using the keyboard, no mouse hover
- Focus the element using the keyboard and also hover over it with the mouse
That will exhaust 95% of the interactions that visible focus styles need to cover.
Buttons should be able to be activated using mouse click, Enter and Space keys. Links should be able to be activated using a mouse click and Enter but not Space.
Arrow keys are common methods for navigating distinct UI elements, especially composite groups like field sets, radio groups, menus, and other composite elements. Please test these interactions and compare them against the WAI-ARIA examples for the same UI components.
When testing a new piece of UI, please test it thoroughly with a screen reader paying close attention to what the screen reader is saying, in particular how it is describing parts of the page.
Buttons, for example, should have appropriate labels. If the visible text of the button relies on some wider visual context to be intelligible, ensure that it has an appropriate aria-label that a screen reader can use to give more information about the button.
The Openverse frontend is a Nuxt SSR application. This means the initial Vue page rendering when you make a request is processed by a server and then delivered to you to be "hydrated" with the current state of the page. The implication of this is that there are two ways for every single page to be rendered, and we should test with that in mind. Please make sure that you are testing client side navigation as well as SSR. To test SSR for a page, simply reload the page: it will be rendered in SSR and then delivered to your browser. To test client side rendering for a page, navigate to that page from another page without reloading the page in between. For example, to test the search route client side, you can execute a search from the homepage and it will redirect you client-side to the search page.
Please note that these are non-expert and non-exhaustive recommendations. Spend time reading the WAI-ARIA spec and other web accessibility materials. Even just knowing about specific roles and interactions that exist and are meant to be developed in consistent ways is a good first step to learning what to look out for when testing.
Openverse uses Vue Testing Library for unit testing with Jest, and Playwright for end-to-end and visual-regression testing.
There are also legacy unit tests written in Vue Test Utils but those are slated to be re-written using testing library.
Please see the Playwright test README.md for instructions on running and maintaining the Playwright test suite.