If you're like me, every once in a while your CI process crosses a magic threshold and you decide you have to carve it back down to size. I ran just over 1000 TeamCity CI builds on a less-than-beefy VM with a basic React/TypeScript/ASP.Net Core project, trying different combinations of npm calls, flags, and alternatives.

Here's the options that performed the best.

Suggested Options

If you do not clean the workspace on every build:

• Best: Use yarn install or pnpm install - 88% + 80% faster than npm install
• 2nd Best: Use npm install --prefer-offline --no-audit - 15% faster than npm install
• Do not: Do not use npm ci, see note below

npm install at 20 seconds, vs yarn and npm at under 4 seconds

If you clean the workspace on every build (or use a build service that doesn't cache environments):

• Best: Use yarn install or pnpm install - 77% + 63% faster than npm install
• 2nd Best: Use npm ci --prefer-offline --no-audit - 53% faster than npm install

vanilla npm install at 270 seconds vs vs yarn and npm under 1/2 of that

The biggest gain on the npm calls is due to --prefer-offline, which tells npm to use locally cached packages when available, only calling the registry if it isn't already available.

What if I use multiple agents? Update package versions very frequently?

You will get poorer performance from the machine cache, try using a private registry or caching proxy. This way, once a developer or alternate build agent has downloaded the package once, it will be available closer to your other systems.

Example: Verdaccio (Note: I haven't personally tested this system yet)

Can I get the faster build times if I have to clean my build every time?

Use yarn or pnpm (kidding, not kidding)

Another option I've seen has been to archive node_modules to a common directory at the end of your CI build, then add a first step to restore it if the archive exists. This moves you from the "clean" timings to the "dirty" timings, which is a 10x increase (then switch to yarn for another 8x).

Environment + Versions

The test data was produced using the following versions:

• node.js: 10.15.1
• npm: 6.4.1
• yarn: 1.13.0
• pnpm: 2.25.6
• TeamCity: 2018.2.2 (build 61245)

And the following project:

• github/BlogExample.Web/ClientApp: React 16.2 with TypeScript 3.3.3, Redux, Thunk, etc

Warning: npm ci performance

One result that really stood out during my tests was how poorly npm ci performed on non-clean builds. TeamCity can nuke and recreate the workspace in seconds, while running npm ci on a dirty folder was adding 50% (1 minute on this system) over running it on an empty folder (clean build).

Clean `npm ci` at 128 seconds, dirty `npm install` at 185 seconds?

One other interesting item was that the release notes for npm ci came with some incredible performance results over yarn and pnpm. I don't know whether those results were a fluke, ci has gotten slower, everything else has gotten faster, or what. npm ci was visibly slower than both of those tools under the same circumstances and latest versions.

Tested Options

Here are the details on the options I used:

npm --prefer-offline option

By default, verifies packages against the registry after a minimum cache time to see if the package is still good. The release notes say this is just a 304 check, but this still includes the network latency and lookup time in the registry. That minimum cache time is 10 seconds. 🙄

prefer-offline tells npm to ignore the cache minimum time and just go ahead and use the locally cached package if it's already been downloaded, without verifying it against the registry.

The official docs are light on this command now, but some details available here as well as in the release notes here

npm --no-audit option

The new feature to check packages for known vulnerabilities is great! However, the CI run may not be the best place to run that (is anyone looking at those results?). Skipping the audit step doesn't cut a lot off, but it does help.

I legitimately like the audit feature and think it should be checked regularly, more coming in a future post

npm --progress=false option

Older versions of npm produced a progress bar as it installed packages, slowing down some systems (Windows) with the console interaction.

I did not see a statistically significant impact from this flag in up to date npm and assume I have one of the environment variables set on the build server that turns this off OR that the performance has become less of a blocker.

npm install

Installs all packages specified in the package-lock.json file (or if that does not exist, package.json). By default it does not clean out existing packages in node_modules and performs a number of operations sequentially (like building the dependency tree of your millions of packages).

npm ci

"Continuous Integration" or "Clean Install", we may never know (both are used in the docs)

npm ci performs a clean install from your package-lock.json, with the goal of sort-of reproducing a deterministic result. It will install exactly the versions you specify in your package-lock, but the dependencies of those packages may be updated.

yarn

Yarn is an alternative to npm. It takes better advantage of caching, parallelizes operations (npm does a number of things sequentially), and produces more repeatable, deterministic results than npm install.

pnpm

pnpm aims to be a faster, more efficient package management client. The main advantages are that it links files from it's cache rather than copying them and does not use the flat structure that npm has moved to in recent years.

pnpm supports the same commands as npm. Operations not directly implemented by pnpm are passed through to npm to execute.

Details Test Results

From those options, I ran a minimum of 20 builds for each test case on "clean" and "dirty" working folders. "Clean" folders were cleared by TeamCity before the build started, "Dirty" folders came after at least one prior untracked run with the same package command.

24 Test Cases of npm install, npm ci, yarn, and pnpm

Test Case Labels:

• po: --prefer-offline
• na: --no-audit
• pf: --progress=false

Tests were run over the course of a week and a half. The build server was stable and running on a VM on a dedicated server, with no other active VMs or workload. Network performance was consistent over this time and some cases (especially that long ci) were run over more than one interval or for much longer sampling times. Time was measured both for the total run time as well as specifically just the npm step. In total, 1120 TeamCity builds were run.

Keep in mind, these won't be exactly the same on your environment, but they should be directionally correct enough to give you a solid start on your own environment.