Skip to content

Conversation

@markbrockhoff
Copy link
Collaborator

@markbrockhoff markbrockhoff commented Dec 5, 2025

TaskWPB-22129 [Web] Investigate and improve login e2e flow (its currently flaky)

Pull Request

Summary

Waiting for the navigation to the app on login caused issues in cases where the history info was shown in between, causing the test to timeout without interaction. So the stand alone util will no longer wait for it, only the page plugin will ensure the login finished, optionally also confirming the history info.


Security Checklist (required)

  • External inputs are validated & sanitized on client and/or server where applicable.
  • API responses are validated; unexpected shapes are handled safely (fallbacks or errors).
  • No unsafe HTML is rendered; if unavoidable, sanitization is applied and documented where it happens.
  • Injection risks (XSS/SQL/command) are prevented via safe APIs and/or escaping.

Standards Acknowledgement (required)


Notes for reviewers

  • The authentication tests manually add the timeout extension since they can't use the page plugin as they're testing the login itself

@codecov
Copy link

codecov bot commented Dec 5, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.75%. Comparing base (b1fb668) to head (86ae5a3).
⚠️ Report is 1 commits behind head on dev.

Additional details and impacted files
@@           Coverage Diff           @@
##              dev   #19847   +/-   ##
=======================================
  Coverage   43.74%   43.75%           
=======================================
  Files        1300     1300           
  Lines       32764    32764           
  Branches     7272     7272           
=======================================
+ Hits        14334    14335    +1     
  Misses      16707    16707           
+ Partials     1723     1722    -1     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2025

🔗 Download Full Report Artifact

🧪 Playwright Test Summary

  • Passed: 27
  • Failed: 69
  • Skipped: 5
  • 🔁 Flaky: 8
  • 📊 Total: 109
  • Total Runtime: 679.1s (~ 11 min 19 sec)
specs/2FA-for-TeamsSpecs/2fa-for-teams.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ f2a for teams > I want to verify that verification code is not required after login if 2FA has been disabled (tags: TC-8749, regression)
specs/Accessibility/Accessibility.spec.ts (❌ 2 failed, ⚠️ 1 flaky)
  • ❌ Accessibility > I want to see collapsed view when app is narrow (tags: TC-48, regression)
  • ❌ Accessibility > I should not lose a drafted message when switching between conversations in collapsed view (tags: TC-51, regression)
  • ⚠️ Accessibility > I want to see typing indicator in group conversation (tags: TC-46, regression)
specs/AccountSettingsSpecs/accountSettings.spec.ts (❌ 2 failed, ⚠️ 1 flaky)
  • ❌ account settings > I should not be able to change email of user managed by SCIM (tags: TC-60, regression)
  • ❌ account settings > Verify links to manage and create teams are shown when logged in as team owner (tags: TC-1723, regression)
  • ⚠️ account settings > Verify I can retrieve calling logs (tags: TC-1725, regression)
specs/AppLock/AppLock.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ AppLock > Web: App should not lock if I switch back to webapp tab in time (during inactivity timeout) (tags: TC-2752, TC-2753, regression)
specs/ArchiveSpecs/archive.spec.ts (❌ 1 failed, ⚠️ 2 flaky)
  • ❌ Accessibility > I want to archive the 1on1 conversation from conversation details (tags: TC-105, regression)
  • ⚠️ Accessibility > Verify the conversation is not unarchived when there are new messages in this conversation (tags: TC-99, regression)
  • ⚠️ Accessibility > I want to archive the group conversation from conversation details (tags: TC-104, regression)
specs/Authentication/authentication.spec.ts (❌ 5 failed, ⚠️ 0 flaky)
  • ❌ Authentication > I want to be asked to share telemetry data when I log in (tags: TC-8780, regression)
  • ❌ Authentication > Verify current browser is set as temporary device (tags: TC-3460, regression)
  • ❌ Authentication > I want to keep my history after refreshing the page on permanent device (tags: TC-3472, regression)
  • ❌ Authentication > Make sure user does not see data of user of previous sessions on same browser (tags: TC-1311, regression)
  • ❌ Authentication > Verify session expired info is visible on login page (tags: TC-1311, regression)
specs/Block/block.spec.ts (❌ 7 failed, ⚠️ 0 flaky)
  • ❌ Block: User A and User B are NOT in the same team > I want to cancel blocking a 1:1 conversation from conversation list (tags: TC-137, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you can block a user who is not in your team (tags: TC-140, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you still receive messages from blocked person in a group chat (tags: TC-141, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you can unblock user from 1:1 conversation details (tags: TC-142, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you cannot add a person who blocked you to a group chat (tags: TC-143, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you can unblock someone from conversation list options (tags: TC-148, regression)
  • ❌ Block: User A and User B are in the same team > Verify you can not block a user from your team (tags: TC-8778, regression)
specs/Connections/Connections.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Connections > Verify 1on1 conversation is not created on the second end after you ignore connection request (tags: TC-365, TC-369, TC-370, TC-371, regression)
specs/CriticalFlow/accountManagement-TC-8639.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Account Management (tags: TC-8639, crit-flow-web)
specs/CriticalFlow/addMembersToChat-TC-8631.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Team owner adds whole team to an all team chat (tags: TC-8631, crit-flow-web)
specs/CriticalFlow/backupRestoration-TC-8634.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Setting up new device with a backup (tags: TC-8634, crit-flow-web)
specs/CriticalFlow/Cells/uploadingFileInGroupConversation.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Uploading an file in a group conversation (tags: crit-flow-cells, regression)
specs/CriticalFlow/channelsCall-TC-8755.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Calls in channels with device switch and screenshare (tags: TC-8754, crit-flow-web)
specs/CriticalFlow/channelsManagement-TC-8752.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Channels Management (tags: TC-8752, crit-flow-web)
specs/CriticalFlow/conversationManagement-TC-8636.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Conversation Management (tags: TC-8636, crit-flow-web)
specs/CriticalFlow/groupCalls-TC-8632.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Planning group call with sending various messages during call (tags: TC-8632, crit-flow-web)
specs/CriticalFlow/groupVideoCall-TC-8637.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Group Video call (tags: TC-8637, crit-flow-web)
specs/CriticalFlow/joinTeam-TC-8635.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ New person joins team and setups up device (tags: TC-8635, crit-flow-web)
specs/CriticalFlow/messagesIn1On1-TC-8750.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Messages in 1:1 (tags: TC-8750, crit-flow-web)
specs/CriticalFlow/messagesInChannels-TC-8753.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Messages in Channels (tags: TC-8753, crit-flow-web)
specs/CriticalFlow/messagesInGroups-TC-8751.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Messages in Groups (tags: TC-8751, crit-flow-web)
specs/CriticalFlow/oneOnOneCall-TC-8754.spec.ts (❌ 0 failed, ⚠️ 1 flaky)
  • ⚠️ 1:1 Video call with device switch and screenshare (tags: TC-8754, crit-flow-web)
specs/CriticalFlow/personalAccountLifecycle-TC-8638.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Personal Account Lifecycle (tags: TC-8638, crit-flow-web)
specs/Edit/edit.spec.ts (❌ 7 failed, ⚠️ 0 flaky)
  • ❌ Edit > I can edit my message in 1:1 (tags: TC-679, regression)
  • ❌ Edit > I see changed message if message was edited from another device (tags: TC-682, regression)
  • ❌ Edit > I cannot edit another users message (tags: TC-683, regression)
  • ❌ Edit > I can edit my last message by pressing the up arrow key (tags: TC-686, regression)
  • ❌ Edit > Editing a message does not create unread dot on receiver side (tags: TC-690, regression)
  • ❌ Edit > I can see the changed message was edited from another user (tags: TC-692, regression)
  • ❌ Edit > I want to see the last edited text including a timestamp in message detail view if the message has been edited (tags: TC-3563, regression)
specs/LoginSpecs/login.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Verify you can sign in by email (tags: TC-3461, regression)
specs/RegressionSpecs/archive.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Archive status of conversation should only change by user explicitly changing it (tags: TC-97, TC-99, TC-104, TC-105, regression)
specs/RegressionSpecs/block-messages.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Block specs (tags: TC-141, regression)
specs/RegressionSpecs/block-search.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Block from search specs (tags: TC-144, regression)
specs/Reply/reply.spec.ts (❌ 14 failed, ⚠️ 3 flaky)
  • ❌ Reply > I want to see a placeholder text as quote when original message is not available anymore (tags: TC-2994, regression)
  • ❌ Reply > I should not see the quoted message when searching for original message in collections (tags: TC-2996, regression)
  • ❌ Reply > I want to see truncated quote preview if quote is too long (tags: TC-2997, regression)
  • ❌ Reply > I want to reply to a picture (tags: TC-3002, regression)
  • ❌ Reply > I want to reply to an audio message (tags: TC-3003, regression)
  • ❌ Reply > I want to reply to a video message (tags: TC-3004, regression)
  • ❌ Reply > I want to reply to a link (tags: TC-3005, regression)
  • ❌ Reply > I want to reply to a file (tags: TC-3006, regression)
  • ❌ Reply > I want to reply to a reply (tags: TC-3007, regression)
  • ❌ Reply > I want to reply to a link mixed with text (tags: TC-3008, regression)
  • ❌ Reply > I want to reply to a location share (tags: TC-3009, regression)
  • ❌ Reply > I want to click the quoted message to jump to the original message (tags: TC-3013, regression)
  • ❌ Reply > I should not be able to send a reply after I got removed from the conversation (tags: TC-3014, regression)
  • ❌ Reply > I want to reply with mention and tap on the mention in the reply opens the user profile (tags: TC-3016, regression)
  • ⚠️ Reply > I should not be able to reply to a ping (tags: TC-8038, regression)
  • ⚠️ Reply > I should not be able to reply to timed messages (tags: TC-8039, regression)
  • ⚠️ Reply > I want to send a timed message as a reply to any type of a message (tags: TC-3011, regression)
specs/SelfDeletingMessages/selfDeletingMessages.spec.ts (❌ 10 failed, ⚠️ 0 flaky)
  • ❌ Self Deleting Messages > Verify sending ephemeral text message in group (tags: TC-658, regression)
  • ❌ Self Deleting Messages > Verify timer is applied to all messages until turning it off in 1:1 (tags: TC-662, regression)
  • ❌ Self Deleting Messages > Verify that message with previous timer are deleted on start-up when the timeout passed in 1:1 (tags: TC-664, regression)
  • ❌ Self Deleting Messages > Verify the message is not deleted for users that didn't read the message (tags: TC-675, regression)
  • ❌ set globally in group conversation > I want to set a global group conversation timer (tags: TC-3715, regression)
  • ❌ set globally in group conversation > I want to see the current timer in conversation details (tags: TC-3716, regression)
  • ❌ set globally in group conversation > I want to see timed message disable in an input bar when global settings conversation options are set (tags: TC-3718, regression)
  • ❌ set globally in group conversation > I want to see the ephemeral indicator is updated in the input field if someone sets a global timer in conversation options (tags: TC-3719, regression)
  • ❌ set globally in group conversation > I want to see a system message that a global timer was set or changed or removed in conversation options (tags: TC-3720, regression)
  • ❌ in search results > I want to see ephemeral messages in the search results (tags: TC-3717, regression)

@markbrockhoff markbrockhoff force-pushed the test/fix-login-util branch 3 times, most recently from a1ecdae to d0b3c80 Compare December 8, 2025 15:13
Waiting for the navigation to the app on login caused issues in cases where the history info was shown in between, causing the test to timeout without interaction.
@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 9, 2025

Copy link
Contributor

@zskhan zskhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@markbrockhoff markbrockhoff merged commit b4bd3e0 into dev Dec 9, 2025
23 of 51 checks passed
@markbrockhoff markbrockhoff deleted the test/fix-login-util branch December 9, 2025 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants