Conquering Jest Performance 🏰
I recently took upon the task to improve our CI/CD speeds for our largest backend component in our SaaS CARO.
The biggest offender are our unit and e2e test jobs which almost take 20 minutes to complete. Is a quick CI/CD pipeline too much to ask for? This simple question led me down a dark path of understanding and finally conquering Jest performance.
Before I go further, I have to admit that we love writing tests for CARO. At least the core idea of it, is something we hold dear in our development lifecycle. To no surprise, this led to almost 750 unit and 100 e2e tests in one of our biggest backend components.
Anyone troubleshooting Jest performance will stumble across the same set of topics. Any Jest GitHub issue you cross paths with is applying yet another bandaid quick-fix without really tackling the core problem.
A good example of this would be the --runInBand flag.
If you are not running your tests in a very CPU and memory-constrained environment, this option should not be your solution. It will not only substantially slow down your test suite (it's now all running in a single thread without any additional workers) but further hide your real problem, which you encounter:
The memory consumption endboss 💻🔥
Investigating our sluggish tests, I found that our unit tests really, really really like gobbling up memory. Every unit test consistently adding about 50MB, with the node GC slowly losing control of the situation. If you want to learn on how to debug and analyze what exactly is consuming so much memory in between your spec files, I highly recommend Your Jest Tests are Leaking Memory.
Using:
node --expose-gc ./node_modules/.bin/jest --runInBand --logHeapUsage
You will be able to get a pretty good picture of your memory usage, and leveraging the chrome dev tools, you can even create some heap dumps to investigate and pinpoint which dependency, or your code is leaking memory.
So with the perfect tools at hand, I was able to fix the memory consumption of our tests, right? Nope. Even after rigorous testing, removing dependencies, and even running very simple test suites, I was unable to pinpoint the problem. It seems the general architecture and inner workings of Jest facilitate and encourage memory leaks to happen.
To understand why that happens, you need to understand how Jest handles running tests. When Jest runs a test file, it first sets up a clean environment by resetting the state of the module cache and clearing any mocked functions or objects. It then loads the test file and executes its code in a Node process, with access to the test file's dependencies.
Jest provides a test runner that is responsible for executing the test functions within the test file. You can configure your own custom runner via the runner: field in your jest.config.js as documented here.
Reading the docs, I wondered if maybe there are other Jest test runners on GitHub that focus on performance and memory consumption. Then I found something which really made my week.
Fastest Jest Runner 🏆
Fastest-Jest-Runner is a custom jest runner that aims to fix the core memory consumption problems of Jest's architecture. Instead of loading all dependencies of each test file again and again, this custom runner forks the Jest process for each spec file. This pretty much solves all memory management issues. Using pstree we can even see the forks while running our tests.
❯ pstree | grep jest
| | \--- 55921 lleifermann grep jest
| \-+- 55429 lleifermann node /node_modules/.bin/jest
| \-+- 55431 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
| \-+- 55432 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
| |--- 55907 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
| |--- 55908 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
| |--- 55909 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
| |--- 55910 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
| |--- 55912 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
| |--- 55913 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
| \--- 55914 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
\--- 52842 lleifermann node --max-old-space-size=4096 --expose-gc --v8-pool-size=0 --single-threaded /node_modules/fastest-jest-runner/dist/worker.js
Configuring my jest.config.js to use Fastest-Jest-Runner was pretty easy and can be done like this:
module.exports = {
...
runner: "fastest-jest-runner",
globals: {
"fastest-jest-runner": {
"snapshotBuilderPath": "<rootDir>/../jest-snapshot-builder.ts",
"maxOldSpace": 4096
}
}
};
Exchanging the default runner with this custom runner effectively runs all of our tests consistently with 40MB heap usage instead of 2000MB at peak 🤯
Typescript victory lap 🏎️
With memory now tamed and our tests faster than ever we still might encounter some sluggishness caused by CPU bound tasks. If you are testing a Typescript codebase with Jest you need to transpile your code before piping it through jest. This can be achieved 'on-demand' per spec file via a Jest transformer and looks like this in your Jest config:
"transform": {
"\\.[jt]sx?$": "babel-jest"
}
babel-jest and other popular alternatives like ts-jest seem particularly slow in doing the transpilation. Swapping this one out for @swc/jest increased my performance even further.
Using @swc/jest also further seems to reduce memory usage. My final jest.config.js now looks like this:
module.exports = {
...
transform: {
'^.+\\.ts$': ['@swc/jest'],
},
...
runner: "fastest-jest-runner",
globals: {
"fastest-jest-runner": {
"snapshotBuilderPath": "<rootDir>/../jest-snapshot-builder.ts",
"maxOldSpace": 4096
}
}
};
Conclusion 📖
With all of the combined changes i mentioned above, our unit and e2e tests now consume significantly less memory (allowing us to parallelise in the future) and run almost 300% faster. If you are also struggling with Jest performance issues i highly recommend you to check out Fastest-Jest-Runner and @swc/jest.