Bundling with the RequireJS Optimizer

When we build sites using an AMD library like RequireJS, we will have a long list of files that need to be downloaded when someone uses the site. More files means more trips to the server and more download time. Minifying files and using gzip can speed up the download times, but neither affects the Round Trip Time (RTT) that even a cache validation incurs, clogging up a request pipeline just to ask the server if an ETag or last modified date is still valid.

So let’s see one way we can improve things, with small (14 requests) and larger (194 requests) page loads as an example.

RequireJS Bundling

There are several different ways you can bundle scripts with RequireJS, but I wanted to start out exploring their optimizer. The advantage of using the optimizer is that it can intelligently trace the module dependencies and include them for me rather than requiring me to figure out all the dependencies and ensure I bundle them up in the right grouping and order.

I started with a very small application just to play around and see what the impact would be. It consists of an HTML page, a couple 3rd party libraries, and 10 JS files I wrote. On loading the page, it uses a require statement to load two of the files (and thus all of their dependencies). My plan is to build a replacement set so I can load a single file and have it also load in all of the dependencies, in the right order to prevent independent network requests for other dependencies.

Sample Site Structure:

town/
   index.html
   js/
      lib/
         jquery.js
         knockout.js
         require.js
         Squire.js
         jasmine/
            ... jasmine files ...
      src/
         app.js
         ... 9 more hand-written files ...
      test/
         ... several JS spec files ...
   styles/
         ... css file ...
   images/
         ... image files ...
tools/
   r.js
   ... my config will go here ...
js-built/ 
   ... bundle + minified files will be created here ...

This project also has some test files mixed in both the lib folder and a parallel test folder, which we want to exclude from processing at all (on a larger project this would be going through a build process, no point eating up CPU time minifying files that will never go to production).

You can feed the optimizer either command-line options or an options file, I suggest putting everything in a configuration file for repeatability (and readability).

{
    appDir: '../town/js',
    baseUrl: 'src',
    paths: {
        knockout: '../lib/knockout-3.0.0'
    },
    dir: '../js-built',
    fileExclusionRegExp: /(^test|Squire|jasmine|require)/,
    modules: [
        {
            name: 'app',
            include: ['app', 'townViewModel']
        }
    ]
}

The optimizer produces a new “app.js” file for me in the js-built folder and I can copy that over my existing source file. Notice how I did not have to define every single file, the optimizer will take the two modules in the “include” and trace all of their dependencies for me. There is also an option to exclude individual files or other defined bundles.

Config Translation:

• appDir: The path to the js file, relative to the tools directory where r.js lives (not relative to where we execute node from)
• baseUrl: Base URL used for RequireJs modules, relative to that appdir (Further note below)
• paths: RequireJS paths (had I been using a config file w/ RequireJS, I could have supplied that instead of redefining paths here)
• dir: Output directory (also the working directory for the optimizer), relative to the r.js file again
• fileExclusionRegExp: the optimizer ignores any file or directory that matches this regular expression (Further note below)
• modules: an array of modules to build, which can depend on earlier modules (this is a small app so I put everything in a single module)

As I worked with this smaller example and a much larger one, here are some issues I ran into:

• appDir: I ran into problems defining appDir too deeply and had to define it at the shared higher level (but only on the larger project, so this may be a side effect of the next item)
• paths: On the larger project I had a number of paths defined with a starting slash, which works fine for a site but the optimizer translates as “look on the root of the drive”, not seeing any reason for those to be root paths, I fixed them in my main RequireJS config to be relative
• fileExclusionRegExp: I attempted to invert this into an opt-in list using negative lookaheads, but was unable to get it to match more than one value for lookaheads, despite testing the expression elsewhere
• optimize: can be used to turn off minification, which was necessary before I figured out how to filter out some 3rd party files that the optimizer would exit with an error over

I did run into some other issues, here and there, but unfortunately was not keeping track of them at the time.

Results

To work around the “localhost is crazy fast” issue, we can use Chrome to load sites with throttled connections (Dev Tools, Toggle Device mode with the phone icon, change the Network dropdown). For these results I used the 3G option (100 RTT), which is only about 10% slower than the ping from my house to my personal website and at 750kbps, matches the type of shared bandwidth people might see if their company is keeping costs low and over-utilizing a cheaper internet connection. Improvements we make for our slower visitors only makes the experience that much better for our faster ones.

Sample Site Results

I ran the site with and without caching enabled, refreshing and capturing only the best possible result I saw. Here’s what I saw for the Vanilla site, with and without caching:

After using the optimizer (bundled and minified), the best results I received were:

The best vanilla load was 14 requests at 1.88 seconds, with a best cache time of 456ms. The optimized version reduced this to 4 requests at 1.44 seconds, with a best cache of 311ms.

Larger Site Results

While there was a visible difference in the small site, I also wanted to see what would happen in a larger example. The larger site has almost 200 requests, including AJAX calls to an external API and numerous image and CSS resources that have not been optimized yet. Like the small example above, we are not using gzip in this example. Using the same 100RTT setting in chrome (which also impacts us more in this case, due to the 750kbps speed), here are before and after timings:

After using the optimizer to create 2 minified bundles:

The best vanilla load is 194 requests at 16.18s, which drops to 2.91 seconds with cache. With bundling and minification, that drops to 31 requests at 10.3 seconds, which drops to 27 requests and 2.89 seconds with cache.

The configuration for this site continued to be almost as light-weight as the small sample site above, so despite the number of files increasing by greater than an order of magnitude, the ability for the optimizer to trace those dependencies for me meant that I was able to bundle all of these files with a configuration that was only about twice as long as the small sample site above.