|
| 1 | +# Testing Guidelines |
| 2 | + |
| 3 | +## Purpose |
| 4 | + |
| 5 | +Tests verify code changes work correctly and prevent regressions. Each test must: |
| 6 | + |
| 7 | +1. Validate production behavior |
| 8 | +2. Detect bugs before release |
| 9 | +3. Document feature intent |
| 10 | + |
| 11 | +Tests that fail these criteria likely provide low value. |
| 12 | + |
| 13 | +## Never Make Network Calls in Tests |
| 14 | + |
| 15 | +Tests must never make real network calls. Network calls cause test flakiness, slow execution, and external dependencies. |
| 16 | + |
| 17 | +## Mocking Strategy |
| 18 | + |
| 19 | +### Mock Only Inputs |
| 20 | + |
| 21 | +Mock constructor parameters, function arguments, and injected dependencies. Never mock internal logic, language-level |
| 22 | +APIs, or external libraries (except timing/dates and network operations). |
| 23 | + |
| 24 | +Mocking internal logic prevents validating production behavior. When you mock `Array.prototype.map` or internal library |
| 25 | +calls, tests validate fake implementations instead of actual code. If source code breaks, tests pass because they check |
| 26 | +mocks, not real behavior. |
| 27 | + |
| 28 | +**Mock in unit tests:** |
| 29 | + |
| 30 | +- Constructor parameters |
| 31 | +- Function arguments |
| 32 | +- Injected dependencies |
| 33 | + |
| 34 | +**Mock at all test levels:** |
| 35 | + |
| 36 | +- Network operations (AWS SDK, HTTP requests, WebSocket) |
| 37 | +- External processes (cfn-lint, cfn-guard) |
| 38 | + |
| 39 | +### When You Need to Mock Internal APIs |
| 40 | + |
| 41 | +If you need to mock language-level APIs or external libraries, the problem is likely either: |
| 42 | + |
| 43 | +1. Wrong test level (unit vs integration vs E2E) |
| 44 | +2. Poor function signature (Dependency injection) - refactor to accept the dependency as input |
| 45 | + |
| 46 | +Refactor class constructors or function signatures to accept dependencies as parameters. This isolates the unit under |
| 47 | +test and eliminates internal mocking. |
| 48 | + |
| 49 | +## Test Levels |
| 50 | + |
| 51 | +### Unit Tests (`tst/unit/`) |
| 52 | + |
| 53 | +**Objective:** Verify a single function or class in isolation. |
| 54 | + |
| 55 | +**Scope:** Mock constructor parameters, function arguments, and injected dependencies. Never mock internal runtime APIs. |
| 56 | +Tests validate algorithms, not the JavaScript runtime. |
| 57 | + |
| 58 | +**Key indicator:** Tests instantiate a single class or call a single function. |
| 59 | + |
| 60 | +### Integration Tests (`tst/integration/`) |
| 61 | + |
| 62 | +**Objective:** Verify collaboration between two or more internal modules. |
| 63 | + |
| 64 | +**Scope:** All internal modules interact normally. Mock external dependencies (cfn-lint, cfn-guard, AWS SDK, network |
| 65 | +calls). |
| 66 | + |
| 67 | +**Key indicator:** Tests instantiate and call multiple internal classes/functions. Does not use `MockServerComponents`. |
| 68 | + |
| 69 | +### End-to-End Tests (`tst/e2e/`) |
| 70 | + |
| 71 | +**Objective:** Validate the entire application as a black box, simulating a real LSP client. |
| 72 | + |
| 73 | +**Scope:** No internal modules are mocked. Tests send real LSP messages (`textDocument/didOpen`, `textDocument/hover`) |
| 74 | +and assert real LSP responses. Mock external dependencies (cfn-lint, cfn-guard, AWS SDK, network calls). Tests verify |
| 75 | +protocol compliance and artifact generation. |
| 76 | + |
| 77 | +**Key indicator:** Tests use `TestExtension` to spawn a real LSP server process and communicate via LSP protocol |
| 78 | +messages. |
| 79 | + |
| 80 | +## What to Test |
| 81 | + |
| 82 | +### Test Production Behavior |
| 83 | + |
| 84 | +Tests verify inputs and outputs. Unless testing integration or E2E scenarios, avoid testing side effects. |
| 85 | + |
| 86 | +**Test:** |
| 87 | + |
| 88 | +- Business logic correctness |
| 89 | +- Functional requirements |
| 90 | +- Edge cases and boundary conditions |
| 91 | +- Error handling |
| 92 | +- User workflows |
| 93 | + |
| 94 | +**Do not test:** |
| 95 | + |
| 96 | +- Object instantiation (`new MyClass()`) |
| 97 | +- Basic getters/setters without logic |
| 98 | + |
| 99 | +### Meaningful Assertions |
| 100 | + |
| 101 | +Tests must assert specific outcomes related to business logic. Tests covering many lines with weak assertions provide no |
| 102 | +value. |
| 103 | + |
| 104 | +## Tooling |
| 105 | + |
| 106 | +### Vitest |
| 107 | + |
| 108 | +Vitest is the test framework: |
| 109 | + |
| 110 | +- **Test Runner** - Executes test suites (`vitest`) |
| 111 | +- **Test Structure** - Provides BDD syntax (`describe`, `it`, `beforeEach`, `afterEach`, `beforeAll`, `afterAll`) |
| 112 | +- **Assertion Library** - Provides assertion functions (`expect`) |
| 113 | + |
| 114 | +### Sinon |
| 115 | + |
| 116 | +Sinon handles test doubles and fakes: |
| 117 | + |
| 118 | +- **Test Doubles** - Creates spies (`sinon.spy`), stubs (`sinon.stub`), and mocks |
| 119 | +- **Assertions** - Provides assertion functions for test doubles (`sinon.assert`) |
| 120 | +- **Time Control** - Manages system time via `sinon.useFakeTimers()` for debouncing and time-sensitive logic |
| 121 | + |
| 122 | +**Note:** Never use `vi.mock` for module mocking. Use only as a last resort. |
0 commit comments