E2E tests flaky as of v0.23.0

[dlqqq]

Description

Check release is now failing -- this doesn't seem related, likely some upstream change that we need to pin in this step:

Ensuring mock GitHub
Collecting fastapi
  Using cached fastapi-0.139.0-py3-none-any.whl.metadata (26 kB)
Collecting uvicorn
  Using cached uvicorn-0.51.0-py3-none-any.whl.metadata (6.6 kB)
Collecting starlette>=0.46.0 (from fastapi)
  Using cached starlette-1.3.1-py3-none-any.whl.metadata (6.4 kB)
Collecting pydantic>=2.9.0 (from fastapi)
  Using cached pydantic-2.13.4-py3-none-any.whl.metadata (109 kB)
Requirement already satisfied: typing-extensions>=4.8.0 in /opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages (from fastapi) (4.16.0)
Collecting typing-inspection>=0.4.2 (from fastapi)
  Using cached typing_inspection-0.4.2-py3-none-any.whl.metadata (2.6 kB)
Collecting annotated-doc>=0.0.2 (from fastapi)
  Using cached annotated_doc-0.0.4-py3-none-any.whl.metadata (6.6 kB)
Requirement already satisfied: click>=7.0 in /opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages (from uvicorn) (8.1.8)
Collecting h11>=0.8 (from uvicorn)
  Using cached h11-0.16.0-py3-none-any.whl.metadata (8.3 kB)
Collecting annotated-types>=0.6.0 (from pydantic>=2.9.0->fastapi)
  Using cached annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)
Collecting pydantic-core==2.46.4 (from pydantic>=2.9.0->fastapi)
  Using cached pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting anyio<5,>=3.6.2 (from starlette>=0.46.0->fastapi)
  Using cached anyio-4.14.1-py3-none-any.whl.metadata (4.6 kB)
Requirement already satisfied: idna>=2.8 in /opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages (from anyio<5,>=3.6.2->starlette>=0.46.0->fastapi) (3.18)
Using cached fastapi-0.139.0-py3-none-any.whl (130 kB)
Using cached uvicorn-0.51.0-py3-none-any.whl (73 kB)
Using cached annotated_doc-0.0.4-py3-none-any.whl (5.3 kB)
Using cached h11-0.16.0-py3-none-any.whl (37 kB)
Using cached pydantic-2.13.4-py3-none-any.whl (472 kB)
Using cached pydantic_core-2.46.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB)
Using cached starlette-1.3.1-py3-none-any.whl (73 kB)
Using cached anyio-4.14.1-py3-none-any.whl (124 kB)
Using cached typing_inspection-0.4.2-py3-none-any.whl (14 kB)
Installing collected packages: typing-inspection, pydantic-core, h11, anyio, annotated-types, annotated-doc, uvicorn, starlette, pydantic, fastapi

Successfully installed annotated-doc-0.0.4 annotated-types-0.7.0 anyio-4.14.1 fastapi-0.139.0 h11-0.16.0 pydantic-2.13.4 pydantic-core-2.46.4 starlette-1.3.1 typing-inspection-0.4.2 uvicorn-0.51.0
INFO:     Started server process [2733]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000/ (Press CTRL+C to quit)
Mock GitHub started
INFO:     127.0.0.1:58420 - "GET / HTTP/1.1" 200 OK
Getting GitHub connection for jupyterlab/jupyter-chat
Ensuring mock GitHub
INFO:     127.0.0.1:58430 - "GET / HTTP/1.1" 200 OK
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/actions/prep_release.py", line 9, in <module>
    setup(False)
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/actions/common.py", line 19, in setup
    return prepare_environment(fetch_draft_release=fetch_draft_release)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/util.py", line 605, in prepare_environment
    gh = get_gh_object(dry_run=dry_run, owner=owner, repo=repo_name, token=auth)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/jupyter-server/jupyter_releaser/v2/jupyter_releaser/util.py", line 679, in get_gh_object
    return core.GhApi(**kwargs)
Prepare Environment
             ^^^^^^^^^^^^^^^^^^^^
    File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/ghapi/core.py", line 105, in __init__
      funcs_ = L(funcs).starmap(_GhVerb, client=self, kwargs=kwargs)
               ^^^^^^^^^^^^^^^^
    File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/fastcore/basics.py", line 578, in __getattr__
      if attr is not None: return getattr(attr,k)
                                  ^^^^^^^^^^^^^^^
  AttributeError: 'list' object has no attribute 'starmap'

The E2E tests are also failing after #455 was merged, which is strange:

  2 failed
    tests/side-panel.spec.ts:93:9 › #sidepanel › #openingClosing › should list existing chat ───────
    tests/side-panel.spec.ts:138:9 › #sidepanel › #openingClosing › should list existing chat in default directory 
  2 flaky
    tests/placeholder.spec.ts:59:9 › #placeholder › #chatList › should list multiple chat files sorted alphabetically 
    tests/ui-config.spec.ts:273:7 › #typingNotification › should display several typing users ──────
  1 skipped
  101 passed (5.9m)

Reproduce

See 19b69c2

Expected behavior

Checks pass.

[dlqqq]

Check release bug was fixed in Jupyter Releaser ; the current E2E test suite just seems to be a bit flaky, so retitling this issue.

[dlqqq]

I think #placeholder › #chatList › should list multiple chat files sorted alphabetically has been fixed by #473 , which now places certain test suites in unique directories so parallel runs don't affect the filebrowser state.

The #typingNotification test is definitely the most flaky. Also seeing issues with #drag-drop-attachments.

Here are the flaky tests on v0.24.0:

··············
Notice:   2 flaky
    tests/drag-drop-attachments.spec.ts:109:7 › #drag-drop-attachments › Should attach a file via drag & drop from tab-bar into chat input 
    tests/ui-config.spec.ts:232:7 › #typingNotification › should not display typing users if disabled 
  1 skipped
  105 passed (5.4m)

  1) tests/drag-drop-attachments.spec.ts:109:7 › #drag-drop-attachments › Should attach a file via drag & drop from tab-bar into chat input 

    Error: expect(locator).toHaveCount(expected) failed

    Locator:  locator('.jp-chat-sidepanel').locator('.jp-chat-input-container').locator('.jp-chat-attachment')
    Expected: 1
    Received: 0
    Timeout:  5000ms

    Call log:
      - Expect "toHaveCount" with timeout 5000ms
      - waiting for locator('.jp-chat-sidepanel').locator('.jp-chat-input-container').locator('.jp-chat-attachment')
        14 × locator resolved to 0 elements
           - unexpected value "0"


      141 |
      142 |     const attachments = input.locator('.jp-chat-attachment');
    > 143 |     await expect(attachments).toHaveCount(1);
          |                               ^
      144 |     await expect(attachments.first()).toContainText(FILE);
      145 |   });
      146 |
        at /home/runner/work/jupyter-chat/jupyter-chat/ui-tests/tests/drag-drop-attachments.spec.ts:143:31

    attachment #1: video (video/webm) ──────────────────────────────────────────────────────────────
    test-results/tests-drag-drop-attachment-af6a2-rom-tab-bar-into-chat-input/video.webm
    ────────────────────────────────────────────────────────────────────────────────────────────────

    Error Context: test-results/tests-drag-drop-attachment-af6a2-rom-tab-bar-into-chat-input/error-context.md

  2) tests/ui-config.spec.ts:232:7 › #typingNotification › should not display typing users if disabled 

    Error: locator.textContent: Test ended.
    Call log:
      - waiting for getByRole('main').getByRole('tabpanel', { name: 'ui-config.chat' }).locator('.jp-chat-writers')


      259 |     try {
      260 |       await page.waitForCondition(
    > 261 |         async () => !!(await writers.textContent())?.trim(),
          |                                      ^
      262 |         3000
      263 |       );
      264 |     } catch {
        at /home/runner/work/jupyter-chat/jupyter-chat/ui-tests/tests/ui-config.spec.ts:261:38
        at Timeout.check (/home/runner/work/jupyter-chat/jupyter-chat/ui-tests/node_modules/@jupyterlab/galata/src/utils.ts:183:33)

    Error Context: test-results/tests-ui-config--typingNot-4057d-ay-typing-users-if-disabled/error-context.md

    attachment #2: video (video/webm) ──────────────────────────────────────────────────────────────
    test-results/tests-ui-config--typingNot-4057d-ay-typing-users-if-disabled/video.webm
    ────────────────────────────────────────────────────────────────────────────────────────────────

  2 flaky
    tests/drag-drop-attachments.spec.ts:109:7 › #drag-drop-attachments › Should attach a file via drag & drop from tab-bar into chat input 
    tests/ui-config.spec.ts:232:7 › #typingNotification › should not display typing users if disabled 
  1 skipped
  105 passed (5.4m)