-
Notifications
You must be signed in to change notification settings - Fork 72
[E2E][QIT] Migrate to Test Packages infrastructure #11175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mgascam
wants to merge
37
commits into
develop
Choose a base branch
from
dev/qit-use-test-packages
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,706
−743
Open
Changes from 34 commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
5940c8a
[E2E][QIT] Migrate remaining shopper specs
mgascam 8832b55
Add changelog entry
mgascam 774085a
Fix Alipay and pay-for-order test failures
mgascam 7977468
Skip Alipay tests due to QIT environment limitation
mgascam 61d20ed
Skip cardTestingPreventionEnabled: true case in pay-for-order test
mgascam 1de9aa1
Merge branch 'develop' into dev/qit-e2e-shopper-remaining-specs
mgascam 698a093
Remove unused emptyCart import
mgascam 1b5c186
add test package
5a5874e
remove development files
c21f5c9
fix failing tests
33c7a61
chore: update woocommerce/qit-cli to dev-trunk and add .gitignore for…
mgascam f65a0fa
rename tests folder
mgascam ca9195a
Rename parent folder
mgascam 698ac2f
feat: add .nvmrc file and update setup script for WooPayments E2E tests
mgascam 4ee499e
feat: update test configuration and add shopper subpackage for WooPay…
mgascam 9626216
fix: update home page test to verify visibility of site title
mgascam de9cfaa
chore: remove outdated migration guide for QIT test packages
mgascam d147c6d
fix: update QIT test package path and clean up environment configuration
mgascam 3d95523
fix: update plugin entries in QIT test package configuration
mgascam 5696698
feat: add QIT test package configuration and E2E test scripts
mgascam 0c525a2
Move wc blocks tests
mgascam 4efa989
Remove QIT custom E2E configuration.
mgascam a0f8100
fix: improve reliability of save payment method interaction in WC Blo…
mgascam 92e4117
docs: update README for WooPayments QIT tests setup and execution ins…
mgascam a3f67df
remove comment
mgascam f3580e4
revert qit-cli to stable version for CI compatibility
mgascam 0831dbe
Remove comment
mgascam 5b19283
docs: add note about dev version of qit-cli requirement for E2E tests
mgascam 1d9a361
Fix internal dependencies comment
mgascam 0b6d0b8
Remove migration readme
mgascam c8ff3a6
Amend test instructions
mgascam beebe58
refactor: simplify E2E test command and update README instructions
mgascam 3597932
fix: update test directory path for shopper project in Playwright config
mgascam 7355c44
Migrate remaining shopper tests
mgascam dfccf99
Merge branch 'develop' into dev/qit-use-test-packages
mgascam c3d144c
update: change version of woocommerce/qit-cli to dev-trunk in compose…
mgascam 9db1b35
update: add package-lock.json for QIT dependency management
mgascam File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,3 +10,4 @@ vendor/* | |
| release/* | ||
| tests/e2e/docker* | ||
| tests/e2e/deps* | ||
| tests/qit/test-package/* | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| Significance: patch | ||
| Type: dev | ||
|
|
||
| Migrate remaining shopper E2E specs to QIT (my account, multicurrency, alternative payment methods) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,40 +1,61 @@ | ||
| ## WooCommerce Payments QIT tests | ||
| ## WooPayments QIT Tests | ||
|
|
||
| We use the [QIT toolkit](https://qit.woo.com/docs/) for automated testing including security, PHPStan, and E2E tests. | ||
|
|
||
| #### Setup | ||
| - Create `local.env` inside the `tests/qit/config/` directory by copying the variables from `default.env`. | ||
| - To get the actual values for local config, refer to this [secret store](https://mc.a8c.com/secret-store/?secret_id=11043) link. | ||
| - Use standard `KEY=VALUE` format (no `export` keyword needed). | ||
| - Once configured, the first time you run a test command, it will create a local auth file for subsequent runs. | ||
| ### Setup | ||
|
|
||
| #### Running Tests | ||
| 1. Create `local.env` inside the `tests/qit/config/` directory by copying the variables from `default.env`. | ||
| 2. Use standard `KEY=VALUE` format (no `export` keyword needed). | ||
| 3. Configure the required credentials: | ||
| - **QIT authentication**: Get credentials from the [secret store](https://mc.a8c.com/secret-store/?secret_id=11043). These authenticate you with the QIT service. | ||
| - **E2E Jetpack credentials** (`E2E_JP_SITE_ID`, `E2E_JP_BLOG_TOKEN`, `E2E_JP_USER_TOKEN`): Get these from a Jurassic Ninja site already onboarded in test mode. | ||
| 4. Once configured, the first time you run a test command, it will create a local auth file for subsequent runs. | ||
|
|
||
| **Note:** E2E tests require the dev version of `qit-cli` (test packages are not yet in stable releases). Run `composer require woocommerce/qit-cli:dev-trunk --dev --ignore-platform-reqs` to install it locally. | ||
|
|
||
| ### Running Tests | ||
|
|
||
| #### Security and PHPStan tests | ||
|
|
||
| **Security and PHPStan tests:** | ||
| ```bash | ||
| npm run test:qit-security | ||
| npm run test:qit-phpstan | ||
| npm run test:qit-phpstan-local # Against local development build | ||
| ``` | ||
|
|
||
| **E2E tests:** | ||
| #### E2E Tests | ||
|
|
||
| E2E tests use the [QIT Test Packages](https://qit.woo.com/docs/test-packages/) approach. Tests are located in `tests/qit/test-package/`. | ||
|
|
||
| Before running E2E tests, build the plugin package: | ||
|
|
||
| ```bash | ||
| # Run all E2E tests | ||
| npm run test:qit-e2e | ||
| npm run build:release | ||
| ``` | ||
|
|
||
| This creates `woocommerce-payments.zip` which is used by QIT. Then run the tests with the required environment variables: | ||
|
|
||
| ```bash | ||
| # Run all E2E tests (prepend with env vars from local.env) | ||
| E2E_JP_SITE_ID='<value>' E2E_JP_BLOG_TOKEN='<value>' E2E_JP_USER_TOKEN='<value>' npm run test:qit-e2e | ||
|
|
||
| # Run specific test file | ||
| npm run test:qit-e2e tests/qit/e2e/specs/woopayments/shopper/shopper-checkout-purchase.spec.ts | ||
| # Run specific test file (passthrough to Playwright) | ||
| E2E_JP_SITE_ID='<value>' E2E_JP_BLOG_TOKEN='<value>' E2E_JP_USER_TOKEN='<value>' npm run test:qit-e2e -- -- shopper-checkout-purchase.spec.ts | ||
|
|
||
| # Run tests with specific tag | ||
| npm run test:qit-e2e -- --tag=@basic | ||
| # Run tests filtered by tag (e.g., @blocks, @shopper) | ||
| E2E_JP_SITE_ID='<value>' E2E_JP_BLOG_TOKEN='<value>' E2E_JP_USER_TOKEN='<value>' npm run test:qit-e2e -- -- --grep "@blocks" | ||
| ``` | ||
|
|
||
| **Note:** E2E tests require valid Jetpack credentials in `local.env` (`E2E_JP_SITE_ID`, `E2E_JP_BLOG_TOKEN`, `E2E_JP_USER_TOKEN`). | ||
| **Tip:** You can export the variables once per shell session instead of prepending each command: | ||
|
|
||
| - The commands use `build:release` to create `woocommerce-payments.zip` at the root of the directory which is then uploaded and used for the QIT tests. | ||
| ```bash | ||
| set -a && source tests/qit/config/local.env && set +a | ||
| npm run test:qit-e2e | ||
| ``` | ||
|
|
||
| ### Analyzing Results | ||
|
|
||
| #### Analysing results | ||
| - Once the test run is done, you'll see a result URL along with the test summary. | ||
| - Look at any errors that might have been surfaced and associate with PRs that has introduced the same by using `git blame`. | ||
| - Once the test run completes, you'll see a result URL along with the test summary. | ||
| - Look at any errors that might have been surfaced and associate with PRs that introduced them using `git blame`. | ||
| - Ping the author for fixing the error, or fix it yourself if it is straightforward enough. | ||
| - For failed tests, check the artifacts directory for screenshots and error context. | ||
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] The command shown uses double dashes
-- --which may be confusing. The first--is npm's argument separator to pass arguments to the underlying script, and the second--is likely Playwright's separator. Consider adding a brief explanation or simplifying the example to make it clearer for users, such as: