-
Notifications
You must be signed in to change notification settings - Fork 1
@W-19444022@ E2E tests base level #121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 17 commits
d85a16c
0faf642
fcf0e4d
ac2118a
c6fe687
83a7f67
fbc8c2b
5ca3dee
d7c3fd0
e4861f1
27273df
b16269d
4ccdcb4
c10fe78
c5dfdf9
29a6b11
5b2e9af
2717088
e3512e6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| name: E2E Tests | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main, kyledev/e2eTests] | ||
| pull_request: | ||
| branches: [main] | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| e2e-tests: | ||
| name: End-to-End Tests | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '20' | ||
| cache: 'npm' | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci | ||
|
|
||
| - name: Build all packages and extension | ||
| run: | | ||
| echo "🔧 Building all packages..." | ||
| npm run compile | ||
| npm run bundle | ||
| echo "🔧 Building VS Code extension..." | ||
| cd packages/apex-lsp-vscode-extension | ||
| npm run build | ||
| echo "✅ Build completed" | ||
| ls -la dist/ | ||
|
|
||
| - name: Verify extension build | ||
| run: | | ||
| echo "🔍 Verifying extension build artifacts..." | ||
| cd packages/apex-lsp-vscode-extension | ||
| if [ ! -f "dist/package.json" ]; then | ||
| echo "❌ package.json not found in dist" | ||
| exit 1 | ||
| fi | ||
| if [ ! -f "dist/extension.js" ]; then | ||
| echo "❌ extension.js not found in dist" | ||
| exit 1 | ||
| fi | ||
| if [ ! -f "dist/extension.web.js" ]; then | ||
| echo "❌ extension.web.js not found in dist" | ||
| exit 1 | ||
| fi | ||
| echo "✅ All required extension files present" | ||
|
|
||
| - name: Install Playwright browsers | ||
| run: npx playwright install --with-deps chromium | ||
|
|
||
| - name: Run E2E tests | ||
| run: | | ||
| echo "🧪 Starting e2e tests..." | ||
| echo "Environment variables:" | ||
| echo " CI: $CI" | ||
| echo " RUNNER_TEMP: $RUNNER_TEMP" | ||
| echo " TMPDIR: $TMPDIR" | ||
| echo " PWD: $PWD" | ||
| echo "Test workspace path: $RUNNER_TEMP/apex-e2e-workspace" | ||
| echo "Running Playwright tests with proper CI configuration..." | ||
| cd e2e-tests && npx playwright test | ||
| env: | ||
| CI: true | ||
| DEBUG: pw:webserver | ||
|
|
||
| - name: Upload test results and artifacts | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: playwright-results-${{ github.run_number }} | ||
| path: | | ||
| e2e-tests/playwright-report/ | ||
| e2e-tests/test-results/ | ||
| retention-days: 30 | ||
| if-no-files-found: warn | ||
|
|
||
| - name: Debug artifact directories | ||
|
||
| if: always() | ||
| run: | | ||
| echo "🔍 Debugging artifact directories..." | ||
| echo "Current working directory: $(pwd)" | ||
| echo "e2e-tests directory contents:" | ||
| ls -la e2e-tests/ || echo "e2e-tests directory not found" | ||
| echo "playwright-report directory:" | ||
| ls -la e2e-tests/playwright-report/ || echo "playwright-report directory not found" | ||
| echo "test-results directory:" | ||
| ls -la e2e-tests/test-results/ || echo "test-results directory not found" | ||
| echo "Searching for any Playwright artifacts..." | ||
| find . -name "*.png" -o -name "*.webm" -o -name "*.html" | head -20 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| # E2E Tests for Apex Language Server Extension | ||
|
|
||
| This package provides comprehensive end-to-end testing for the Apex Language Server Extension in VS Code Web environments. The test suite validates that the extension correctly integrates with VS Code's language server protocol and provides essential Apex language features. | ||
|
|
||
| ## Purpose | ||
|
|
||
| The e2e test suite ensures the Apex Language Server Extension works correctly in real-world browser environments by testing: | ||
|
|
||
| - **Extension Activation**: Verifies the extension properly activates when Apex files are opened | ||
| - **Language Server Integration**: Confirms the LSP worker starts and initializes without errors | ||
| - **Symbol Parsing**: Validates that Apex code is correctly parsed and symbols are identified | ||
| - **Outline View**: Tests that the VS Code outline view displays Apex class structure | ||
| - **Workspace Integration**: Ensures Apex files are recognized and handled in the workspace | ||
| - **Stability**: Confirms the extension doesn't cause VS Code crashes or performance issues | ||
|
|
||
| ## Test Philosophy | ||
|
|
||
| These tests focus on critical user-facing functionality rather than internal implementation details. They simulate real user interactions with the extension in a browser environment, providing confidence that the extension will work correctly when published. | ||
|
|
||
| The test suite prioritizes: | ||
|
|
||
| - **Reliability**: Tests are designed to be stable across different environments | ||
| - **Performance**: Fast execution with parallel test runs where possible | ||
| - **Maintainability**: Clean abstractions and reusable utilities | ||
| - **Comprehensive Coverage**: Core functionality is thoroughly validated | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Node.js >= 20.0.0 | ||
| - Extension must be built before running tests | ||
| - VS Code Web test server capability | ||
|
|
||
| ## Running Tests | ||
|
|
||
| ```bash | ||
| # Run all tests (recommended) | ||
| npm run test:e2e | ||
|
|
||
| # Debug mode with browser UI | ||
| npm run test:e2e:debug | ||
|
|
||
| # Visual mode for test development | ||
| npm run test:e2e:visual | ||
| ``` | ||
|
|
||
| ## Test Environment | ||
|
|
||
| The tests run against a real VS Code Web instance with the extension pre-loaded. This provides high confidence that the extension will work correctly in production browser environments. | ||
|
|
||
| **Supported Browsers**: Chromium (primary testing target) | ||
|
|
||
| **Environment Support**: | ||
|
|
||
| - Local development with detailed debugging | ||
| - CI/CD with stability optimizations | ||
| - Debug modes for test development | ||
|
|
||
| ## Architecture | ||
|
|
||
| The test suite uses Playwright for browser automation and is structured with: | ||
|
|
||
| - **Utilities**: Reusable functions for common test operations | ||
| - **Test Helpers**: Specialized functions for extension-specific testing | ||
| - **Configuration**: Centralized settings and selectors | ||
| - **Type Safety**: Full TypeScript support throughout | ||
|
|
||
| ## Debugging and Development | ||
|
|
||
| The test suite includes comprehensive debugging capabilities: | ||
|
|
||
| - Console error monitoring with intelligent filtering | ||
| - Network failure tracking | ||
| - Screenshot and video capture on failures | ||
| - Detailed logging for test analysis | ||
|
|
||
| For manual debugging, tests can be run against a standalone VS Code Web server with full developer tools access. | ||
|
|
||
| ## CI/CD Integration | ||
|
|
||
| Tests are configured for continuous integration with: | ||
|
|
||
| - Retry logic for flaky test handling | ||
| - Environment-specific timeouts and worker configuration | ||
| - Comprehensive reporting and artifact collection | ||
| - Headless execution with debugging artifact generation | ||
|
|
||
| ## Contributing | ||
|
|
||
| When adding new tests: | ||
|
|
||
| 1. Use existing test utilities and patterns | ||
| 2. Focus on user-facing functionality | ||
| 3. Ensure tests are reliable across environments | ||
| 4. Include proper error handling and logging | ||
| 5. Follow TypeScript best practices | ||
|
|
||
| The test suite is designed to grow with the extension while maintaining reliability and performance. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| /* | ||
| * Copyright (c) 2025, salesforce.com, inc. | ||
| * All rights reserved. | ||
| * Licensed under the BSD 3-Clause license. | ||
| * For full license text, see LICENSE.txt file in the | ||
| * repo root or https://opensource.org/licenses/BSD-3-Clause | ||
| */ | ||
| import { defineConfig, devices } from '@playwright/test'; | ||
|
|
||
| /** | ||
| * Playwright configuration for Apex Language Server Extension e2e tests. | ||
| * | ||
| * Configures test execution for VS Code Web environment with proper | ||
| * browser settings, timeouts, and CI/CD integration. | ||
| */ | ||
| export default defineConfig({ | ||
| testDir: './tests', | ||
|
|
||
| fullyParallel: !process.env.DEBUG_MODE, | ||
| forbidOnly: !!process.env.CI, | ||
| retries: process.env.CI ? 2 : 0, | ||
| workers: process.env.CI || process.env.DEBUG_MODE ? 1 : undefined, | ||
| reporter: process.env.CI | ||
| ? [['html'], ['line'], ['junit', { outputFile: 'test-results/junit.xml' }]] | ||
| : 'html', | ||
|
|
||
| use: { | ||
| baseURL: 'http://localhost:3000', | ||
| trace: process.env.CI ? 'retain-on-failure' : 'on-first-retry', | ||
| screenshot: process.env.CI ? 'on' : 'only-on-failure', | ||
| video: process.env.CI ? 'on' : 'retain-on-failure', | ||
| actionTimeout: 15000, | ||
| }, | ||
|
|
||
| projects: [ | ||
| { | ||
| name: 'chromium', | ||
| use: { | ||
| ...devices['Desktop Chrome'], | ||
| launchOptions: { | ||
| args: [ | ||
| '--disable-web-security', | ||
| '--disable-features=VizDisplayCompositor', | ||
| '--enable-logging=stderr', | ||
| '--log-level=0', | ||
| '--v=1', | ||
| ...(process.env.CI || process.env.DEBUG_MODE | ||
| ? [ | ||
| '--no-sandbox', | ||
| '--disable-dev-shm-usage', | ||
| '--disable-background-timer-throttling', | ||
| '--disable-backgrounding-occluded-windows', | ||
| '--disable-renderer-backgrounding', | ||
| ] | ||
| : []), | ||
| ], | ||
| headless: process.env.CI || !process.env.DEBUG_MODE ? true : false, | ||
| slowMo: process.env.DEBUG_MODE ? 300 : 0, | ||
| }, | ||
| }, | ||
| }, | ||
| ], | ||
|
|
||
| webServer: { | ||
| command: 'npm run test:e2e:server', | ||
| port: 3000, | ||
| reuseExistingServer: !process.env.CI, | ||
| timeout: 120_000, | ||
| cwd: process.cwd().endsWith('e2e-tests') ? '..' : '.', | ||
| }, | ||
|
|
||
| timeout: process.env.CI ? 120_000 : 60_000, | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not a
cdfan because it becomes state you hae to keep track of.-won npm commands to run certain commands in a certain workspace.If you really need the ls output (?) then you can also give it a directory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've not noticed/used that before so I can definitely add it to the commands.