|
1 | 1 | # E2E Tests for Apex Language Server Extension |
2 | 2 |
|
3 | | -This directory contains end-to-end tests for the Apex Language Server VSCode extension running in a web environment. The tests use Playwright to automate browser interactions with VSCode Web and verify that the extension works correctly. |
| 3 | +This package provides comprehensive end-to-end testing for the Apex Language Server Extension in VS Code Web environments. The test suite validates that the extension correctly integrates with VS Code's language server protocol and provides essential Apex language features. |
4 | 4 |
|
5 | | -## 📁 Project Structure |
| 5 | +## Purpose |
6 | 6 |
|
7 | | -``` |
8 | | -e2e-tests/ |
9 | | -├── fixtures/ # Test data and sample files |
10 | | -│ └── apex-samples.ts # Sample Apex files for testing |
11 | | -├── tests/ # Test files |
12 | | -│ └── apex-extension-core.spec.ts # Core functionality tests |
13 | | -├── types/ # TypeScript type definitions |
14 | | -│ └── test.types.ts # Test-related interfaces |
15 | | -├── utils/ # Utility functions and helpers |
16 | | -│ ├── constants.ts # Test constants and selectors |
17 | | -│ ├── test-helpers.ts # Core test helper functions |
18 | | -│ ├── outline-helpers.ts # Outline view specific helpers |
19 | | -│ └── global.ts # Global setup and teardown functions |
20 | | -├── test-server.js # VS Code Web test server |
21 | | -├── playwright.config.ts # Playwright configuration |
22 | | -├── tsconfig.json # TypeScript configuration |
23 | | -└── README.md # This file |
24 | | -``` |
| 7 | +The e2e test suite ensures the Apex Language Server Extension works correctly in real-world browser environments by testing: |
| 8 | + |
| 9 | +- **Extension Activation**: Verifies the extension properly activates when Apex files are opened |
| 10 | +- **Language Server Integration**: Confirms the LSP worker starts and initializes without errors |
| 11 | +- **Symbol Parsing**: Validates that Apex code is correctly parsed and symbols are identified |
| 12 | +- **Outline View**: Tests that the VS Code outline view displays Apex class structure |
| 13 | +- **Workspace Integration**: Ensures Apex files are recognized and handled in the workspace |
| 14 | +- **Stability**: Confirms the extension doesn't cause VS Code crashes or performance issues |
| 15 | + |
| 16 | +## Test Philosophy |
25 | 17 |
|
26 | | -## Overview |
| 18 | +These tests focus on critical user-facing functionality rather than internal implementation details. They simulate real user interactions with the extension in a browser environment, providing confidence that the extension will work correctly when published. |
27 | 19 |
|
28 | | -The e2e test suite verifies core extension functionality in VS Code Web: |
| 20 | +The test suite prioritizes: |
29 | 21 |
|
30 | | -- **VS Code Web startup** - Verifies the web environment loads correctly |
31 | | -- **Extension activation** - Confirms the extension activates when opening Apex files |
32 | | -- **LSP worker loading** - Ensures the language server starts without critical errors |
33 | | -- **File recognition** - Validates Apex files are detected in the workspace |
34 | | -- **Outline view** - Tests symbol parsing and outline generation |
35 | | -- **Stability** - Checks that VS Code remains responsive after extension activation |
| 22 | +- **Reliability**: Tests are designed to be stable across different environments |
| 23 | +- **Performance**: Fast execution with parallel test runs where possible |
| 24 | +- **Maintainability**: Clean abstractions and reusable utilities |
| 25 | +- **Comprehensive Coverage**: Core functionality is thoroughly validated |
36 | 26 |
|
37 | 27 | ## Prerequisites |
38 | 28 |
|
39 | | -1. Node.js >= 20.0.0 |
40 | | -2. npm packages installed (`npm install` from root) |
41 | | -3. Extension built (`npm run compile && npm run bundle` in `packages/apex-lsp-vscode-extension`) |
| 29 | +- Node.js >= 20.0.0 |
| 30 | +- Extension must be built before running tests |
| 31 | +- VS Code Web test server capability |
42 | 32 |
|
43 | 33 | ## Running Tests |
44 | 34 |
|
45 | | -### Quick Start |
46 | | - |
47 | 35 | ```bash |
48 | | -# Run all e2e tests (recommended - headless, parallel, fast) |
| 36 | +# Run all tests (recommended) |
49 | 37 | npm run test:e2e |
50 | 38 |
|
51 | | -# Run tests in debug mode with Playwright inspector and headed browser |
| 39 | +# Debug mode with browser UI |
52 | 40 | npm run test:e2e:debug |
53 | 41 |
|
54 | | -# Run tests visually (headed browser, slower execution for watching) |
| 42 | +# Visual mode for test development |
55 | 43 | npm run test:e2e:visual |
56 | 44 | ``` |
57 | 45 |
|
58 | | -### Current Test Status |
59 | | - |
60 | | -✅ **Core Tests (`apex-extension-core.spec.ts`):** |
61 | | - |
62 | | -**Test 1: Core Extension Functionality** |
63 | | - |
64 | | -- VS Code Web startup and loading |
65 | | -- Apex file recognition in workspace (2+ files) |
66 | | -- Extension activation when opening .cls files |
67 | | -- Monaco editor integration |
68 | | -- Language server worker initialization |
69 | | -- Critical error monitoring |
70 | | -- Extension stability verification |
71 | | - |
72 | | -**Test 2: Outline View Integration** |
73 | | - |
74 | | -- Opens Apex (.cls) file in editor |
75 | | -- Verifies outline view loads and is accessible |
76 | | -- Confirms LSP parses file and generates outline structure |
77 | | -- Validates specific Apex symbols (HelloWorld class, sayHello/add methods) appear |
78 | | -- Ensures outline view functionality works correctly |
79 | | - |
80 | | -**Test 3: Complex Symbol Hierarchy** |
81 | | - |
82 | | -- Opens ComplexExample.cls with advanced structure |
83 | | -- Tests parsing of static fields, instance fields, methods, and inner classes |
84 | | -- Validates proper symbol nesting and hierarchy display |
85 | | -- Comprehensive LSP symbol recognition testing |
86 | | - |
87 | | -**Browser Support:** Chromium (primary) |
88 | | - |
89 | | -### Manual Testing |
90 | | - |
91 | | -```bash |
92 | | -# Start the test server manually (for development) |
93 | | -npm run test:web:server |
94 | | - |
95 | | -# In another terminal, run specific tests |
96 | | -cd e2e-tests |
97 | | -npx playwright test apex-extension-core.spec.ts |
98 | | -``` |
99 | | - |
100 | | -## Configuration |
101 | | - |
102 | | -### Environment Configuration |
103 | | - |
104 | | -- **Development**: Fast retries, parallel execution |
105 | | -- **CI/CD**: Conservative settings, sequential execution |
106 | | -- **Browser**: Chromium with debugging features enabled |
107 | | -- **Timeouts**: Environment-specific values |
108 | | - |
109 | | -### Test Server (`test-server.js`) |
110 | | - |
111 | | -Starts a VS Code Web instance with: |
112 | | - |
113 | | -- Extension loaded from `../packages/apex-lsp-vscode-extension` |
114 | | -- Test workspace with sample Apex files |
115 | | -- Debug options enabled |
116 | | -- Fixed port (3000) for Playwright |
117 | | - |
118 | | -## Test Architecture |
| 46 | +## Test Environment |
119 | 47 |
|
120 | | -### Core Components |
| 48 | +The tests run against a real VS Code Web instance with the extension pre-loaded. This provides high confidence that the extension will work correctly in production browser environments. |
121 | 49 |
|
122 | | -#### **Utilities (`utils/`)** |
| 50 | +**Supported Browsers**: Chromium (primary testing target) |
123 | 51 |
|
124 | | -- `test-helpers.ts` - Core test functions (startup, activation, monitoring) |
125 | | -- `outline-helpers.ts` - Outline view specific functionality |
126 | | -- `constants.ts` - Centralized configuration and selectors |
127 | | -- `global.ts` - Combined setup/teardown logic (extension building, workspace creation) |
| 52 | +**Environment Support**: |
128 | 53 |
|
129 | | -#### **Types (`types/`)** |
| 54 | +- Local development with detailed debugging |
| 55 | +- CI/CD with stability optimizations |
| 56 | +- Debug modes for test development |
130 | 57 |
|
131 | | -- Strong TypeScript typing for all test interfaces |
132 | | -- Console error tracking types |
133 | | -- Test metrics and environment configurations |
134 | | -- Sample file definitions |
| 58 | +## Architecture |
135 | 59 |
|
136 | | -#### **Fixtures (`fixtures/`)** |
| 60 | +The test suite uses Playwright for browser automation and is structured with: |
137 | 61 |
|
138 | | -- Sample Apex classes, triggers, and SOQL queries |
139 | | -- Follows Apex language rules (no imports, namespace resolution) |
140 | | -- Comprehensive examples for testing parsing and outline generation |
| 62 | +- **Utilities**: Reusable functions for common test operations |
| 63 | +- **Test Helpers**: Specialized functions for extension-specific testing |
| 64 | +- **Configuration**: Centralized settings and selectors |
| 65 | +- **Type Safety**: Full TypeScript support throughout |
141 | 66 |
|
142 | | -#### **Configuration** |
| 67 | +## Debugging and Development |
143 | 68 |
|
144 | | -- Global setup/teardown combined in `utils/global.ts` - builds extension and creates test workspace |
145 | | -- Main Playwright configuration in `playwright.config.ts` with environment detection |
146 | | -- Test server (`test-server.js`) - VS Code Web instance with pre-loaded extension |
| 69 | +The test suite includes comprehensive debugging capabilities: |
147 | 70 |
|
148 | | -### Design Principles |
| 71 | +- Console error monitoring with intelligent filtering |
| 72 | +- Network failure tracking |
| 73 | +- Screenshot and video capture on failures |
| 74 | +- Detailed logging for test analysis |
149 | 75 |
|
150 | | -Following `.cursor` TypeScript guidelines: |
151 | | - |
152 | | -- ✅ Strong typing with `readonly` properties |
153 | | -- ✅ Arrow functions for consistency |
154 | | -- ✅ Descriptive naming conventions (camelCase, kebab-case) |
155 | | -- ✅ No enums (using string unions) |
156 | | -- ✅ Import type for type-only imports |
157 | | -- ✅ JSDoc documentation following Google Style Guide |
158 | | -- ✅ Error handling with proper filtering |
159 | | -- ✅ Constants for magic numbers |
160 | | -- ✅ Modular, maintainable code structure |
161 | | - |
162 | | -## Test Data |
163 | | - |
164 | | -The global setup creates a test workspace with sample files from `fixtures/apex-samples.ts`: |
165 | | - |
166 | | -- **`HelloWorld.cls`**: Basic Apex class with static methods (sayHello, add) |
167 | | -- **`ComplexExample.cls`**: Advanced class with fields, methods, and inner Configuration class |
168 | | -- **`AccountTrigger.trigger`**: Sample trigger with validation logic |
169 | | - |
170 | | -## Debugging |
171 | | - |
172 | | -### Console Errors |
173 | | - |
174 | | -Tests monitor browser console for errors. Non-critical errors (favicon, sourcemaps) are filtered out using centralized patterns. |
175 | | - |
176 | | -### Network Issues |
177 | | - |
178 | | -Tests check for worker file loading failures and report network issues with detailed logging. |
179 | | - |
180 | | -### Screenshots and Videos |
181 | | - |
182 | | -- Screenshots taken on test failures |
183 | | -- Videos recorded on retry |
184 | | -- Traces captured for failed tests |
185 | | -- Debug screenshots in `test-results/` directory |
186 | | - |
187 | | -### Manual Debugging |
188 | | - |
189 | | -1. Start server: `npm run test:web:server` |
190 | | -2. Open browser to `http://localhost:3000` |
191 | | -3. Open Developer Tools |
192 | | -4. Monitor console and network tabs |
193 | | -5. Interact with the extension manually |
| 76 | +For manual debugging, tests can be run against a standalone VS Code Web server with full developer tools access. |
194 | 77 |
|
195 | 78 | ## CI/CD Integration |
196 | 79 |
|
197 | | -The tests are configured for CI environments: |
| 80 | +Tests are configured for continuous integration with: |
198 | 81 |
|
199 | | -- **Retries**: 2 attempts on CI |
200 | | -- **Workers**: 1 (sequential execution on CI) |
201 | | -- **Reporting**: HTML report generated |
202 | | -- **Headless**: Default on CI |
203 | | -- **Timeout**: Extended for CI stability |
204 | | - |
205 | | -## Troubleshooting |
206 | | - |
207 | | -### Extension Won't Activate |
208 | | - |
209 | | -1. Verify extension is built: `npm run bundle` in extension directory |
210 | | -2. Check `dist/` directory exists with bundled files |
211 | | -3. Look for console errors in browser DevTools |
212 | | - |
213 | | -### Tests Timeout |
214 | | - |
215 | | -1. Check timeout configuration in `playwright.config.ts` |
216 | | -2. Verify VS Code Web server is responding |
217 | | -3. Ensure network connectivity |
218 | | - |
219 | | -### Worker Loading Errors |
220 | | - |
221 | | -1. Check worker files exist in `dist/` directory |
222 | | -2. Verify file URLs are accessible |
223 | | -3. Look for CORS or security policy issues |
224 | | - |
225 | | -### Port Conflicts |
226 | | - |
227 | | -- Change port in `playwright.config.ts` |
228 | | -- Ensure port is not in use by other services |
| 82 | +- Retry logic for flaky test handling |
| 83 | +- Environment-specific timeouts and worker configuration |
| 84 | +- Comprehensive reporting and artifact collection |
| 85 | +- Headless execution with debugging artifact generation |
229 | 86 |
|
230 | 87 | ## Contributing |
231 | 88 |
|
232 | 89 | When adding new tests: |
233 | 90 |
|
234 | | -1. Follow existing patterns using utilities from `utils/` |
235 | | -2. Add proper TypeScript types |
236 | | -3. Use centralized constants and selectors |
237 | | -4. Add JSDoc documentation |
238 | | -5. Update this README if needed |
239 | | -6. Follow `.cursor` TypeScript guidelines |
240 | | - |
241 | | -## Scripts Summary |
242 | | - |
243 | | -- **`test:e2e`**: Main test runner (headless, parallel) |
244 | | -- **`test:e2e:debug`**: Interactive debugging with Playwright inspector |
245 | | -- **`test:e2e:visual`**: Headed browser with slower execution for watching tests |
246 | | -- **`test:web:server`**: Start VS Code Web server manually for debugging |
247 | | - |
248 | | -## Known Limitations |
| 91 | +1. Use existing test utilities and patterns |
| 92 | +2. Focus on user-facing functionality |
| 93 | +3. Ensure tests are reliable across environments |
| 94 | +4. Include proper error handling and logging |
| 95 | +5. Follow TypeScript best practices |
249 | 96 |
|
250 | | -- VS Code Web has some differences from desktop VS Code |
251 | | -- Extension debugging capabilities are limited in web context |
252 | | -- Network-dependent features may be unreliable in test environments |
| 97 | +The test suite is designed to grow with the extension while maintaining reliability and performance. |
0 commit comments