a11y-tests failing

[gabalafou]

The accessibility tests are failing.

A conversation started on #2100. That conversation should continue here.

[gabalafou]

What we know so far:

• Tests pass locally for me (tox run -e a11y-tests, on Mac OS against the latest commit on main, b55ec41)
• The failure has to do with the PyData examples page
• Both the Plotly and Ipyleaflet examples are broken but the Ipyleaflet example seems to be broken only because the Plotly example is broken (because removing the Plotly example locally causes the Ipyleaflet example to work locally.)
• The last merge to main that passed was on 24 Jan.

[gabalafou]

The Plotly and Ipyleaflet examples are working at stable/ but broken on latest/:

• Broken: https://pydata-sphinx-theme.readthedocs.io/en/latest/examples/pydata.html
• Working: https://pydata-sphinx-theme.readthedocs.io/en/stable/examples/pydata.html

When I look at the commits between the v0.16.1 tag (currently stable) and HEAD, it's not obvious to me where the bug is. My guess is that it has something to do with the new Read the Docs configuration.

[gabalafou]

This is curious. The Plotly and Ipyleaflet examples fail locally for me with v0.16.0 and v0.16.1, for example:

git checkout v0.16.1
git clean -xfd
tox -e run docs-dev
python -m http.server -d docs/_build/html
open http://localhost:8000/examples/pydata.html

[gabalafou]

My best guess right now is that this has to with the fact that Plotly.js 3.0.0 was released a week ago (Jan 27). I'm guessing that at build time nbsphinx pulls in the latest version of plotly.js and inlines it. This means that builds before Jan 27 would be fine whereas builds after Jan 27 would fail, even if we made no changes to our codebase during that time.

Here's some HTML snippets that differ depending on whether you curl examples/pydata.html from latest or from stable:

Latest:

<div class="output_area rendered_html docutils container">
        <script type="text/javascript">
        window.PlotlyConfig = {MathJaxConfig: 'local'};
        if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: "STIX-Web"}});}
        </script>
        <script type="module">/**
* plotly.js v3.0.0
* Copyright 2012-2025, Plotly, Inc.
* All rights reserved.
* Licensed under the MIT license
*/

Stable:

<div class="output_area rendered_html docutils container">
        <script type="text/javascript">
        window.PlotlyConfig = {MathJaxConfig: 'local'};
        if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: "STIX-Web"}});}
        if (typeof require !== 'undefined') {
        require.undef("plotly");
        define('plotly', function(require, exports, module) {
            /**
* plotly.js v2.35.2
* Copyright 2012-2024, Plotly, Inc.
* All rights reserved.
* Licensed under the MIT license
*/