Accessibility test standardization, improvements and documentation

[rahimabdi]

Tracker:

• Share draft documentation with @boazsender @cookiecrook (done Apr 28 2024)
• Determine publication platform (likely our GitHub Wiki page)
• Determine/publish updates to related ARIA testing documentation
• Determine if we need to merge this existing ARIA WPT documentation into the new document: https://github.com/w3c/aria/blob/main/documentation/wpt.md

---

@cookiecrook Consolidating general thoughts here on improving the quality/consistency of our WPT accessibility test files and providing supporting documentation for test authors (also to make it less daunting for WPT contributors). All feedback welcome! Happy to take this forward after you and other folks have provided input.

I know that detailed guidance is already captured here and this may be moved to web-platform-tests/interop-accessibility: Running the WPT Tests for ARIA and Related Specs.

• Writing tests, guidance and implementation practices
  • Boilerplate comment header for all test files (e.g., "This file tests X...")
  • Providing documentation links to relevant parts of a spec being tested
  • Comment links to where relevant testing is located (and documentation scheme for referencing files in other locations)
  • Naming .html test files (usually after a spec's section)
  • Readability improvement guidance: e.g., alphabetization where possible, prescriptive order of element attributes (e.g., <div id="id" data-expectedtestname="someTest" data-expectedlabel="label">, spacing and indentation, etc.
  • Subtest names must be globally unique (the more descriptive the better)
  • id naming scheme for elements/containers (e.g., many elements with unique ids referenced via aria-labelledby)
  • Performing HTML validation
  • Using PR checks (primarily ensuring that lint is clean, removing whitespace)
  • PR commit message (description and co-author attribution)
• Supporting documentation for test authors/maintainers
  • Overview of the WPT project
  • Understanding/viewing CI results
  • More on testharness.js
  • Index page with all a11y related tests (e.g., accname, wai-aria, core-arm, html-aam, svg-aam, etc.)
  • WPT Debugging Q&A and common issues (e.g., troubleshooting bad data* attribute on test, non-unique test name, improper use of aria-utils utility functions)
  • Explainer for aria-utils.js and what it does

[cookiecrook]

Generally all sounds good. I think some of the non-accessibility-specific stuff you mentioned above may already be documented in the main WPT project... Any accessibility-specific docs could go here, along with pointers to the main WPT docs where relevant.

[boazsender]

@rahimabdi @lolaodelola and I met and discussed a few additional ideas:

• We talked about creating a guide for where to put tests
• Create an index page of all the tests using the tags
• Going back and tagging existing tests
• Linking to these docs from web-platform-tests.org

We also talked about our multiple audiences

• People talking about different accessibility tests (e.g. aria test writers)
• people in other groups who want to test accessibility

[cookiecrook]

Adding Agenda+ for docs discussion in the May meeting.

[rahimabdi]

@boazsender @cookiecrook I've emailed you (and Lola) a link to the draft documentation I'm proposing. Your thoughts and feedback are welcome!

I've also added a tracker to the issue description with outstanding tasks/questions.