How to solve the Memory Leak problem on Node.js while running Jest

A step-by-step guide to solving the JavaScript heap out-of-memory error while running test cases on Node JS

A step-by-step guide to solving the JavaScript heap out-of-memory error while running Jest.

It looked like another typical day, with different deployments. However, we were surprised by a CircleCI error while running some of our E2E test cases. After re-running the deployment process multiple times, we realized it began to fail randomly without any explanation. The test cases worked perfectly in our local environments, but this is what appeared on CircleCI:

CircleCI is failing while running the test cases.

After looking at the logs, we found a JavaScript heap out-of-memory error. Reached heap limit allocation failed? Why is this problem happening in CircleCI, and it didn’t happen in our local environments? Let’s try to understand and optimize this process!

CircleCI Error: JavaScript heap out of memory.

This tutorial is going to be divided into the following points:

1. Problem explanation.
2. Solution 1: Change the NodeJS version.
3. Solution 2: Using the Garbage Collector.
4. CircleCI integration.

Problem explanation.

The heap size indicates the amount of memory allocated by the Node.js process during the test. We weren’t concerned about this problem until CircleCI complained about it, so we started doing some tests. Before looking at the problem, this is the configuration we had:

• NodeJS (v16.17.1)
• Typescript (ts-node v10.9.1)
• Jest (ts-jest v27.1.5)
• CircleCI

This is the command we used to run all the test cases of our Node application:

jest __tests__/e2e/ --logHeapUsage --runInBand --detectOpenHandles --forceExit

Just a few comments on this command:

• The --logHeapUsage logs the heap usage after every test. It’s crucial to see the memory allocated after every test. This is what allowed us to realize we had a real problem.
• The --runInBand flag runs all tests serially in the current process rather than creating a worker pool of child processes that run tests.

Looking at the logs, this was the result (please take a look at the heap size after every test case):

PASS __tests__/e2e/cases/user/user.get.test.js (1136 MB heap size)
PASS __tests__/e2e/cases/movie/movie.get.test.js (1334 MB heap size)
PASS __tests__/e2e/cases/author/author.get.test.js (1480 MB heap size)
...
PASS __tests__/e2e/cases/merhc/merch.get.test.js (1625 MB heap size)
PASS __tests__/e2e/cases/drinks/drinks.get.test.js (1806 MB heap size)
PASS __tests__/e2e/cases/author/author.get.test.js (2117 MB heap size)

More than 2GB of heap size! How is it possible, and why is it incrementally increasing? Some common reasons are:

• Memory leaks occur when objects or resources are allocated in memory but not properly released, leading to a gradual increase in memory usage.
• Each test case may generate or load data that adds up in memory over time.

Now that we know there is a problem, let’s work on solving it.

Solution 1: Change the NodeJS version.

It’s difficult to point out the exact error; Node might have introduced some unintended performance issues in this release, or maybe the version of this dependency didn’t correctly work with the Node version. Also, reading online, it seems like the “ts-jest” dependency tests get over 1GB of memory.

Just by changing the Node version, this is the improvement we got.

PASS __tests__/e2e/cases/user/user.get.test.js (1147 MB heap size)
PASS __tests__/e2e/cases/movie/movie.get.test.js (1259 MB heap size)
PASS __tests__/e2e/cases/author/author.get.test.js (1322 MB heap size)
...
PASS __tests__/e2e/cases/merch/merch.get.test.js (1439 MB heap size)
PASS __tests__/e2e/cases/drinks/drinks.get.test.js (1486 MB heap size)
PASS __tests__/e2e/cases/author/author.get.test.js (1560 MB heap size)

As you can see, there is still an incremental increase in the memory, but at least it’s smaller than in the previous case. So, for now, we’ll switch to the Node v16.10.0 version.

Solution 2: Using the Garbage Collector.

For us, this was the key point. We are going to modify the command that we use to run the test case to the following one:

node --expose-gc ./node_modules/.bin/jest __tests__/e2e/ --logHeapUsage --runInBand --detectOpenHandles --forceExit

The most important comment about this command is that we now directly run the test cases using “node” and expose the garbage collector using “--expose-gc”. You need to run the test cases using “node”; if you use “jest”, the “— expose-gc” command will not be recognized.

This means that when executing the Jest tests cases, the garbage collector can be explicitly invoked using the gc() function.

So this is what we are going to do! In each test file, after finishing the test cases, we call it. By manually triggering the garbage collection, you can free up memory and optimize memory usage.

afterAll(async () => {
    /* Other operations like closing or cleaning the DB
       ...       
    */
    if (global.gc) global.gc();
  });

After running all the test cases, this is the result:

PASS __tests__/e2e/cases/user/user.get.test.js (189 MB heap size)
PASS __tests__/e2e/cases/movie/movie.get.test.js (233 MB heap size)
PASS __tests__/e2e/cases/author/author.get.test.js (292 MB heap size)
...
PASS __tests__/e2e/cases/merch/merch.get.test.js (449 MB heap size)
PASS __tests__/e2e/cases/drinks/drinks.get.test.js (468 MB heap size)
PASS __tests__/e2e/cases/author/author.get.test.js (484 MB heap size)

As you can see, combining the previous step of changing the version and the garbage collector makes the results look way better!

In the end, we’ve decreased the memory heap size by around 75%.

CircleCI integration.

The last step is to run it on CircleCI. You might have other CD/CI tools, but it will be the same in the end. As we can see, everything works correctly, and our application can finally be deployed!

After fixing these memory heap size problems in Node JS, CircleCI is working again!

I hope this tutorial is useful for you. If this solution helps you, or you find other solutions, feel free to write them in the comments!

Thanks for Reading!

If you like my work and want to support me…

1. The BEST way is to follow me on Medium here.
2. Feel free to clap if this post is helpful for you! :)