OOM issues 100% of the time on spec files with many tests - Cypress 15.4.0+

[wmvsilva]

Current behavior

Hi Cypress team.

My team is a long-time Cypress user. After upgrading from Cypress 14.5.4 to Cypress 15.9.0, we noticed that spec files with a significant number of tests started failing 100% of the time. We determined that versions 15.3.0 and below do not have this issue. The issue starts in 15.4.0 and is still present in the latest Cypress 15.9.0 version.

To validate this issue, I made a basic example with 64 identical tests in a single spec file testing against our application. Each test just visits a "Dashboard" page for an application before passing.

The below examples are from a Apple M4 Pro Macbook with 24GB RAM, node 20.19.4, electron browser. The same issue was seen on a Linux server.

With this setup, Cypress 15.3.0 passes 100% of the time.

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        15.3.0                                                                         │
  │ Browser:        Electron 136 (headless)                                                        │
  │ Node Version:   v20.19.4 (/Users/[MY_USER]/.asdf/installs/nodejs/20.19.4/bin/node)              │
  │ Specs:          1 found (dashboard.spec.ts)                                                    │
  │ Searched:       ./src/integration/**/*.{js,jsx,ts,tsx}                                         │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  dashboard.spec.ts                                                               (1 of 1)

  #Dashboard
    ✓ #should render the dashboard link for all users (2650ms)
[...logs shortened for readability...]
    ✓ #should render the dashboard link for all users (2628ms)


  64 passing (2m)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        64                                                                               │
  │ Passing:      64                                                                               │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     2 minutes, 25 seconds                                                            │
  │ Spec Ran:     dashboard.spec.ts                                                                │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  dashboard.spec.ts                        02:25       64       64        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        02:25       64       64        -        -        -  

Meanwhile, Cypress 15.4.0 fails 100% of the time with this setup due to OOM and Electron Renderer process crashing:

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        15.4.0                                                                         │
  │ Browser:        Electron 138 (headless)                                                        │
  │ Node Version:   v20.19.4 (/Users/[MY_USER]/.asdf/installs/nodejs/20.19.4/bin/node)              │
  │ Specs:          1 found (dashboard.spec.ts)                                                    │
  │ Searched:       ./src/integration/**/*.{js,jsx,ts,tsx}                                         │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  dashboard.spec.ts                                                               (1 of 1)
[15859:0127/161200.071332:ERROR:electron/shell/common/node_bindings.cc:427] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.

DevTools listening on ws://127.0.0.1:58371/devtools/browser/55d7e7b9-5c56-43d5-841c-2d111d143585
<<<CYPRESS.STDERR.START>>>[baseline-browser-mapping] The data in this module is over two months old.  To ensure accurate Baseline data, please update: `npm i baseline-browser-mapping@latest -D`
<<<CYPRESS.STDERR.END>>>

  #Dashboard
    ✓ #should render the dashboard link for all users (2790ms)
[...logs shortened for readability...]
    ✓ #should render the dashboard link for all users (2043ms)

<--- Last few GCs --->

[18114:0x13c00700000]    87999 ms: Mark-Compact (reduce) 2106.7 (2545.1) -> 1995.9 (2419.1) MB, pooled: 0.0 MB, 510.33 / 0.00 ms (average mu = 0.713, current mu = 0.000) last resort; GC in old space requested
[18114:0x13c00700000]    88508 ms: Mark-Compact (reduce) 1995.9 (2419.1) -> 1985.4 (2363.6) MB, pooled: 0.0 MB, 508.67 / 0.00 ms (average mu = 0.559, current mu = 0.000) last resort; GC in old space requested

[18114:0127/161330.324652:ERROR:third_party/blink/renderer/bindings/core/v8/v8_initializer.cc:840] V8 javascript OOM (CALL_AND_RETRY_LAST).

We detected that the Electron Renderer process just crashed.

We have failed the current spec but will continue running the next spec.

This can happen for a number of different reasons.

If you're running lots of tests on a memory intense application.
  - Try increasing the CPU/memory on the machine you're running on.
  - Try enabling experimentalMemoryManagement in your config file.
  - Try lowering numTestsKeptInMemory in your config file during 'cypress open'.

You can learn more here:

https://on.cypress.io/renderer-process-crashed

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        64                                                                               │
  │ Passing:      36                                                                               │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      27                                                                               │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     1 minute, 21 seconds                                                             │
  │ Spec Ran:     dashboard.spec.ts                                                                │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✖  dashboard.spec.ts                        01:21       64       36        1        -       27 │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  1 of 1 failed (100%)                     01:21       64       36        1        -       27  

Below is the Cypress config for this test setup.

const { defineConfig } = require('cypress');

module.exports = defineConfig({
  fileServerFolder: '.',
  fixturesFolder: './src/fixtures',
  modifyObstructiveCode: false,
  trashAssetsBeforeRuns: true,
  video: false,
  videosFolder: '../../dist/cypress/apps/[APP-NAME]/videos',
  videoCompression: false,
  screenshotsFolder: '../../dist/cypress/apps/[APP-NAME]/screenshots',
  chromeWebSecurity: false,
  env: {
    isIntegrationTest: true,
  },
  retries: 2,
  viewportWidth: 1920,
  viewportHeight: 1080,
  e2e: {
    setupNodeEvents(on, config) {
      return require('./src/plugins/index.js')(on, config);
    },
    specPattern: './src/integration/**/*.{js,jsx,ts,tsx}',
    supportFile: './src/support/index.ts',
  },
});

Desired behavior

Cypress 15.4.0 should not fail with an OOM error.

Test code to reproduce

https://github.com/wmvsilva/cypress-test-tiny-oom

This repo follows a similar approach to the testing strategy I mentioned above where a URL is visited repeatedly in different it tests in a single spec file. Unlike my "Dashboard" example above, this forked repo just visits youtube. This issue appears to be more likely to happen quickly if you visit a resource-heavy URL repeatedly.

Cypress 5.3.0

  90 passing (2m)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        90                                                                               │
  │ Passing:      90                                                                               │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     1 minute, 53 seconds                                                             │
  │ Spec Ran:     spec.cy.js                                                                       │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  spec.cy.js                               01:53       90       90        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        01:53       90       90        -        -        -  

Cypress 5.9.0

[35510:0127/164719.019627:ERROR:third_party/blink/renderer/bindings/core/v8/v8_initializer.cc:840] V8 process OOM (ExternalEntityTable::AllocateEntry).

We detected that the Electron Renderer process just crashed.

We have failed the current spec but will continue running the next spec.

This can happen for a number of different reasons.

If you're running lots of tests on a memory intense application.
  - Try increasing the CPU/memory on the machine you're running on.
  - Try enabling experimentalMemoryManagement in your config file.
  - Try lowering numTestsKeptInMemory in your config file during 'cypress open'.

You can learn more here:

https://on.cypress.io/renderer-process-crashed

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        90                                                                               │
  │ Passing:      25                                                                               │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      64                                                                               │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     29 seconds                                                                       │
  │ Spec Ran:     spec.cy.js                                                                       │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

Cypress Version

15.9.0

Debug Logs

Other

No response

[eswanny22]

Can you try setting experimentalMemoryManagement: true in your Cypress config to see if that helps? That would be the first step before looking at other options.

[wmvsilva]

Hi @eswanny22 .

I tried setting experimentalMemoryManagement: true in the Cypress config for the test repo I provided - https://github.com/wmvsilva/cypress-test-tiny-oom.

After testing with experimentalMemoryManagement: true with different Cypress versions, I got the following results.

cypress-test-tiny-oom example (visiting youtube.com), running full test suite 3 separate times for each scenario

Cypress Version	experimentalMemoryManagement=false	experimentalMemoryManagement=true
15.3.0	3 full suite passes ✅	3 full suite passes ✅
15.4.0	3 full suite failures ❌	3 full suite passes ✅
15.9.0	3 full suite failures ❌	3 full suite passes ✅

---

This was very promising. Unfortunately, the experimentalMemoryManagement: true approach doesn't work for the Cypress tests for our application. These tests are similar to those in the cypress-test-tiny-oom except instead of visiting youtube.com, they visit our angular application and wait for it to load. Our Angular app is very resource-heavy.

Visiting our Angular app's dashboard, running full test suite 3 separate times for each scenario

Cypress Version	experimentalMemoryManagement=false	experimentalMemoryManagement=true
13.17.0	3 full suite passes ✅	3 full suite failures ❌
14.5.4	3 full suite passes ✅	3 full suite failures ❌
15.3.0	3 full suite passes ✅	3 full suite failures ❌
15.4.0	3 full suite failures ❌	3 full suite failures ❌
15.9.0	3 full suite failures ❌	3 full suite failures ❌

Observations about the failures:

• If I update the test so that the application URL is visited without giving it time to fully load before moving to the next it(...) test, there are no OOMs. Because of this, I suspect that when the application fully loads, some state may NOT be getting reset between Cypress it(...) tests.
• Other settings like CYPRESS_NO_COMMAND_LOG=1, numTestsInMemory: 0 do not fix the OOM issue.
• I wasn't able to replicate this OOM behavior with experimentalMemoryManagement=true with any public URLs. At this time, I have no reproducible example that I can share.

Note: All failures mentioned above are OOM errors similar to the below:

<--- Last few GCs --->

[8090:0x11400700000]    81005 ms: Scavenge 2186.1 (2243.6) -> 2178.4 (2259.4) MB, pooled: 0.0 MB, 3.42 / 0.00 ms (average mu = 0.551, current mu = 0.000) allocation failure; 
[8090:0x11400700000]    81160 ms: Scavenge 2206.6 (2264.1) -> 2196.1 (2273.9) MB, pooled: 0.0 MB, 3.29 / 0.04 ms (average mu = 0.551, current mu = 0.000) allocation failure; 

[8090:0129/225005.023108:ERROR:third_party/blink/renderer/bindings/core/v8/v8_initializer.cc:840] V8 process OOM (ExternalEntityTable::AllocateEntry).

We detected that the Electron Renderer process just crashed.

We have failed the current spec but will continue running the next spec.

This can happen for a number of different reasons.

If you're running lots of tests on a memory intense application.
  - Try increasing the CPU/memory on the machine you're running on.
  - Try enabling experimentalMemoryManagement in your config file.
  - Try lowering numTestsKeptInMemory in your config file during 'cypress open'.

You can learn more here:

https://on.cypress.io/renderer-process-crashed

<--- Last few GCs --->

[73097:0x10800ca0000]   212448 ms: Mark-Compact 3789.5 (3896.6) -> 3777.8 (3880.1) MB, pooled: 34 MB, 1124.25 / 0.00 ms  (average mu = 0.096, current mu = 0.013) background allocation failure; GC in old space requested
[73097:0x10800ca0000]   213586 ms: Mark-Compact 3779.3 (3880.1) -> 3765.9 (3870.3) MB, pooled: 39 MB, 1122.50 / 0.04 ms  (average mu = 0.057, current mu = 0.014) background allocation failure; GC in old space requested

[wmvsilva]

After switching the browser to Chrome, I find that our Angular app test suite (visiting our angular app over and over) is working on chrome on Cypress 15.9.0 without any issues with experimentalMemoryManagement=false. This same suite was unable to pass at all when using Electron.

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        15.9.0                                                                         │
  │ Browser:        Chrome 144 (headless)                                                          │
  │ Node Version:   v20.19.4 (/Users/[MY_USER]/.asdf/installs/nodejs/20.19.4/bin/node)              │
  │ Specs:          1 found (dashboard.spec.ts)                                                    │
  │ Searched:       ./src/integration/**/*.{js,jsx,ts,tsx}                                         │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
...
  (Results)

  ┌─────────────────────────────────────────────────────────────────────────────���──────────────────┐
  │ Tests:        58                                                                               │
  │ Passing:      58                                                                               │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     1 minute, 33 seconds                                                             │
  │ Spec Ran:     dashboard.spec.ts                                                                │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘