Add ability to reuse workspace across parallel test processes to speed up e2e #4466

marrobi · 2025-04-03T14:28:25Z

Add ability to reuse workspace across parallel test processes to speed up e2e

Overview

This PR introduces the ability to reuse workspaces across parallel end-to-end (E2E) test processes, significantly improving the speed and efficiency of E2E test execution. The changes include updates to configuration, scripts, Makefile targets, and documentation to support parallelization and workspace reuse.

Key Changes

Parallel E2E Test Support:
- Added support for running E2E tests in parallel by reusing existing workspaces.
- Introduced the E2E_TESTS_NUMBER_PROCESSES variable to control the number of parallel pytest processes.
- Updated workflows and scripts to use this variable and enable parallel execution.
Configuration and Environment Updates:
- Added new configuration options in config.sample.yaml for E2E test parallelization and workspace reuse.
- Updated environment variable handling and consolidated environment scripts.
Makefile and Script Enhancements:
- Added new Makefile targets for preparing and running E2E tests with parallelization.
- Updated and refactored scripts to support the new parallel test execution model.
CI/CD Workflow Adjustments:
- Modified GitHub Actions workflows to use the new parallelization variables and logic.
- Improved job selectors and environment setup for E2E test jobs.
Documentation:
- Updated developer and admin documentation to reflect the new E2E test process and configuration options.
- Removed outdated references to the old E2E test user/app registration flow.
Cleanup and Refactoring:
- Removed deprecated E2E test user/app registration logic and related environment variables.
- Cleaned up and renamed scripts for clarity and maintainability.

github-actions · 2025-04-03T14:29:13Z

Unit Test Results

0 tests 0 ✅ 0s ⏱️
0 suites 0 💤
0 files 0 ❌

Results for commit 836071e.

♻️ This comment has been updated with latest results.

Copilot

Pull Request Overview

This PR introduces the ability to reuse a workspace across parallel test processes to speed up end-to-end tests by implementing a file locking mechanism and additional logic in the test setup/teardown routines. Key changes include:

Enhancements to e2e_tests/conftest.py to support parallel workspace setup and teardown with locking.
Documentation updates and GitHub workflow modifications to expose and use the configurable E2E_TESTS_NUMBER_PROCESSES variable.
Adjustments in the devcontainer action to include a fallback default for the case where the E2E test process count is not supplied.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
e2e_tests/conftest.py	Added functions for JSON file handling, workspace locking logic, and modifications to support parallel execution.
docs/tre-admins/setup-instructions/workflows.md	Documentation updated to include the new E2E_TESTS_NUMBER_PROCESSES environment variable.
.github/workflows/pr_comment_bot.yml	Workflow updated to parameterize the number of E2E test processes.
.github/workflows/deploy_tre.yml	Workflow updated to use the variable for E2E test processes.
.github/actions/devcontainer_run_command/action.yml	Action updated to provide a default value for E2E_TESTS_NUMBER_PROCESSES.

e2e_tests/conftest.py

Fixes microsoft#2868

…k manager assignemnt.

Copilot

Pull Request Overview

This PR enables parallel E2E test execution by reusing a single workspace across pytest worker processes, reducing setup time and improving test throughput.

Introduces a file‐lock based fixture to share and reuse the base workspace among workers
Adds E2E_TESTS_NUMBER_PROCESSES support, updates CI/workflows, Makefile, and docs for parallel runs
Adds assign_airlock_manager_role helper and refactors Airlock tests to use a shared base workspace

Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
e2e_tests/test_performance.py	Switched workspace auth to Automatic and updated service ID var
e2e_tests/test_airlock.py	Refactored Airlock tests to use new shared base workspace fixture
e2e_tests/resources/workspace.py	Added `assign_airlock_manager_role` for MS Graph role assignments
e2e_tests/resources/resource.py	Annotated `post_resource` return type
e2e_tests/pytest.ini	Updated pytest markers (removed `extended_aad`, added `manual`)
e2e_tests/helpers.py	Simplified token helpers and added `get_msgraph_token`
e2e_tests/conftest.py	New parallel‐workspace fixture with file locking logic
e2e_tests/config.py	Introduced MS Graph and Guacamole service config variables
e2e_tests/.env.sample	Sample env file cleared; needs restoration
docs/tre-developers/end-to-end-tests.md	Updated VS Code debug instructions for new .env and config paths
docs/tre-admins/setup-instructions/workflows.md	Documented `E2E_TESTS_NUMBER_PROCESSES` variable
devops/scripts/setup_local_debugging.sh	Conditional env cleanup for new variables
devops/scripts/get_access_token.sh	Removed password‐flow branch
config.sample.yaml	Added commented e2e_tests section
Makefile	Added airlock and workspace-services targets; load parallel .env
.github/workflows/pr_comment_bot.yml	Inject parallel process var from repository vars
.github/workflows/deploy_tre.yml	Updated e2e test selectors and parallel var usage
.github/actions/devcontainer_run_command/action.yml	Exposed `E2E_TESTS_NUMBER_PROCESSES` to devcontainer runs
.devcontainer/devcontainer.json	Cleaned up outdated E2E AAD debug configuration

Comments suppressed due to low confidence (2)

e2e_tests/pytest.ini:5

The 'airlock' pytest marker is missing in pytest.ini, so tests decorated with @pytest.mark.airlock will trigger unknown‐mark warnings. Please add 'airlock' to the markers list.

manual

e2e_tests/resources/workspace.py

e2e_tests/conftest.py

Copilot

Pull Request Overview

This PR enhances the E2E testing framework by enabling workspace reuse across parallel test processes, thereby accelerating end-to-end test execution. Key changes include updating authentication types in test resources, refactoring test workflows and fixtures (including file locking for shared workspaces), and adjusting related configuration, CI/CD, and documentation updates.

Reviewed Changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
e2e_tests/test_performance.py	Switches workspace auth type from "Manual" to "Automatic"
e2e_tests/test_airlock.py	Updates workspace path references and variable names for clarity
e2e_tests/resources/workspace.py	Introduces get_msgraph_token and role assignment via MS Graph API
e2e_tests/resources/resource.py	Adds return type annotations to post_resource for improved clarity
e2e_tests/pytest.ini & config files	Updates environment variable names to support parallel test execution
conftest.py	Refactors workspace setup with file locking and asynchronous cleanup
CI/CD & Makefile files	Adapts workflows and build targets to integrate parallel test parameters
Documentation files	Updates instructions and references for new E2E parallelization options

e2e_tests/test_performance.py

e2e_tests/test_airlock.py

e2e_tests/resources/resource.py

e2e_tests/conftest.py

marrobi · 2025-05-20T06:22:45Z

/test-extended

github-actions · 2025-05-20T06:23:04Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/15130362385 (with refid 0704a77e)

(in response to this comment from @marrobi)

marrobi · 2025-05-20T08:35:28Z

/test-extended

github-actions · 2025-05-20T08:35:45Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/15132819569 (with refid 0704a77e)

(in response to this comment from @marrobi)

marrobi · 2025-05-20T11:23:51Z

/test-extended

github-actions · 2025-05-20T11:24:08Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/15136238334 (with refid 0704a77e)

(in response to this comment from @marrobi)

marrobi · 2025-05-20T14:17:07Z

/test-extended

github-actions · 2025-05-20T14:17:27Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/15139994904 (with refid 0704a77e)

(in response to this comment from @marrobi)

… into marrobi/issue2868

marrobi · 2025-05-20T18:27:14Z

/test-extended

github-actions · 2025-05-20T18:27:29Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/15145080127 (with refid 0704a77e)

(in response to this comment from @marrobi)

marrobi · 2025-05-21T07:27:39Z

/test-extended

github-actions · 2025-05-21T07:27:56Z

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/15156157858 (with refid 0704a77e)

(in response to this comment from @marrobi)

Add ability to reuse workspace across parallel test proceses

dda6ab6

marrobi requested review from Copilot and tamirkamara April 3, 2025 14:28

Copilot AI reviewed Apr 3, 2025

View reviewed changes

e2e_tests/conftest.py Show resolved Hide resolved

e2e_tests/conftest.py Outdated Show resolved Hide resolved

marrobi added 6 commits April 29, 2025 20:49

Merge branch 'main' into marrobi/issue2868

ba1930b

WIP on E2E

9dbf25d

Session fixtures don't have effect with pytest-xdist

b10ee4c

Fixes microsoft#2868

Fix linting.

88990cf

Threads for workspace creation, remove AAD specific tests, add airlco…

e4fdf2c

…k manager assignemnt.

Get token after role assignment.

5ff91a8

marrobi requested a review from Copilot May 19, 2025 15:16

marrobi added 2 commits May 19, 2025 16:16

Merge branch 'main' into marrobi/issue2868

42bad27

Revert workspace change.

98714a4

marrobi marked this pull request as ready for review May 19, 2025 15:19

Copilot AI reviewed May 19, 2025

View reviewed changes

e2e_tests/resources/workspace.py Show resolved Hide resolved

e2e_tests/conftest.py Show resolved Hide resolved

Fix writing workspace ID

84f8dfd

marrobi requested a review from Copilot May 19, 2025 15:34

Copilot AI reviewed May 19, 2025

View reviewed changes

e2e_tests/test_performance.py Show resolved Hide resolved

e2e_tests/test_airlock.py Show resolved Hide resolved

e2e_tests/resources/resource.py Show resolved Hide resolved

e2e_tests/conftest.py Show resolved Hide resolved

Fix az login

51d8c5d

Skip writing to file if set to use env vars

6b728b6

marrobi added 3 commits May 20, 2025 11:54

Remove verify.

ab28fbb

fix linting

263d08d

Add APPLICATION_ADMIN_CLIENT_ID and secret to e2e tests workflow.

d957753

marrobi added 6 commits May 20, 2025 14:46

make verify keyword argument.

fbdca20

Merge branch 'main' into marrobi/issue2868

a9596a0

Fix linting

ba500a7

Merge branch 'marrobi/issue2868' of https://github.com/marrobi/AzureTRE…

c23446f

… into marrobi/issue2868

fix linting

1dcea7b

Prevent deadlock.

8d3993e

add verify keyword

ed9782e

marrobi added 4 commits June 6, 2025 09:42

Merge branch 'main' into marrobi/issue2868

c46d186

Merge branch 'main' into marrobi/issue2868

7e2d2c1

Merge branch 'main' into marrobi/issue2868

02f2596

Merge branch 'main' into marrobi/issue2868

836071e

marrobi requested a review from jonnyry June 26, 2025 12:09

Add ability to reuse workspace across parallel test processes to speed up e2e #4466

Are you sure you want to change the base?

Add ability to reuse workspace across parallel test processes to speed up e2e #4466

Uh oh!

Conversation

marrobi commented Apr 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!