If you're like me, every once in a while your CI process crosses a magic threshold and you decide you have to carve it back down to size. I ran just over 1000 TeamCity CI builds on a less-than-beefy VM with a basic React/TypeScript/ASP.Net Core project, trying different combinations of npm calls, flags, and alternatives.
Here's the options that performed the best.
If you do not clean the workspace on every build:
- Best: Use
pnpm install- 88% + 80% faster than
- 2nd Best: Use
npm install --prefer-offline --no-audit- 15% faster than
- Do not: Do not use
npm ci, see note below
If you clean the workspace on every build (or use a build service that doesn't cache environments):
- Best: Use
pnpm install- 77% + 63% faster than
- 2nd Best: Use
npm ci --prefer-offline --no-audit- 53% faster than
The biggest gain on the
npm calls is due to
--prefer-offline, which tells
npm to use
locally cached packages when available, only calling the registry if it isn't already available.
What if I use multiple agents? Update package versions very frequently?
You will get poorer performance from the machine cache, try using a private registry or caching proxy. This way, once a developer or alternate build agent has downloaded the package once, it will be available closer to your other systems.
Example: Verdaccio (Note: I haven't personally tested this system yet)
Can I get the faster build times if I have to clean my build every time?
pnpm (kidding, not kidding)
Another option I've seen has been to archive node_modules to a common directory at the end of your CI build, then add a first step to restore it if the archive exists. This moves you from the "clean" timings to the "dirty" timings, which is a 10x increase (then switch to yarn for another 8x).
Environment + Versions
The test data was produced using the following versions:
- node.js: 10.15.1
- npm: 6.4.1
- yarn: 1.13.0
- pnpm: 2.25.6
- TeamCity: 2018.2.2 (build 61245)
And the following project:
- github/BlogExample.Web/ClientApp: React 16.2 with TypeScript 3.3.3, Redux, Thunk, etc
Warning: npm ci performance
One result that really stood out during my tests was how poorly
npm ci performed
on non-clean builds. TeamCity can nuke and recreate the workspace in seconds, while
npm ci on a dirty folder was adding 50% (1 minute on this system) over
running it on an empty folder (clean build).
One other interesting item was that the release notes for
npm ci came with some
incredible performance results over yarn and pnpm. I don't know whether those
results were a fluke,
ci has gotten slower, everything else has gotten faster, or what.
npm ci was visibly slower than both of those tools under the same circumstances and
Here are the details on the options I used:
npm --prefer-offline option
By default, verifies packages against the registry after a minimum cache time to see if the package is still good. The release notes say this is just a 304 check, but this still includes the network latency and lookup time in the registry. That minimum cache time is 10 seconds. 🙄
prefer-offline tells npm to ignore the cache minimum time and just go ahead and use
the locally cached package if it's already been downloaded, without verifying it
against the registry.
npm --no-audit option
The new feature to check packages for known vulnerabilities is great! However, the CI run may not be the best place to run that (is anyone looking at those results?). Skipping the audit step doesn't cut a lot off, but it does help.
I legitimately like the audit feature and think it should be checked regularly, more coming in a future post
npm --progress=false option
Older versions of npm produced a progress bar as it installed packages, slowing down some systems (Windows) with the console interaction.
I did not see a statistically significant impact from this flag in up to date npm and assume I have one of the environment variables set on the build server that turns this off OR that the performance has become less of a blocker.
Installs all packages specified in the
package-lock.json file (or if that does not exist,
package.json). By default it does not clean out existing packages in
performs a number of operations sequentially (like building the dependency tree of your
millions of packages).
"Continuous Integration" or "Clean Install", we may never know (both are used in the docs)
npm ci performs a clean install from your
package-lock.json, with the goal of sort-of
reproducing a deterministic result. It will install exactly the versions you specify in
package-lock, but the dependencies of those packages may be updated.
Yarn is an alternative to npm. It takes better advantage of
caching, parallelizes operations (npm does a number of things sequentially), and produces
more repeatable, deterministic results than
pnpm aims to be a faster, more efficient package management client. The main advantages are that it links files from it's cache rather than copying them and does not use the flat structure that npm has moved to in recent years.
pnpm supports the same commands as npm. Operations not directly implemented by pnpm are passed through to npm to execute.
Details Test Results
From those options, I ran a minimum of 20 builds for each test case on "clean" and "dirty" working folders. "Clean" folders were cleared by TeamCity before the build started, "Dirty" folders came after at least one prior untracked run with the same package command.
Test Case Labels:
- po: --prefer-offline
- na: --no-audit
- pf: --progress=false
Tests were run over the course of a week and a half. The build server was stable and running on a VM on a dedicated server, with no other active VMs or workload. Network performance was consistent over this time and some cases (especially that long ci) were run over more than one interval or for much longer sampling times. Time was measured both for the total run time as well as specifically just the npm step. In total, 1120 TeamCity builds were run.
Keep in mind, these won't be exactly the same on your environment, but they should be directionally correct enough to give you a solid start on your own environment.