Docker layer caching for Node: copy package.json first

In late 2016 Freebird's web app and API were a Node service shipped as a Docker container, deployed with Convox on AWS and built on CircleCI. Deploys were slow. Most of that time was npm install, running from scratch on every single build, because our Node Dockerfile defeated Docker layer caching.

You can see our frustration in the commit history for that Dockerfile. The messages from October 25th, 2016 read, in order: "speeding up deployments," "fixing morgan," "debugging global npm cache," "modding npm," "trying ot fix container," "fixing cig," and then three commits titled just "t." The day before that is a commit called "NPM INSTALLLL!!!!" with about sixty exclamation points.

The fix is Docker layer caching

Docker caches each instruction as a layer, and a layer is reused only if nothing it depends on has changed. If you copy your whole app into the image and then run npm install, any change to any source file invalidates the copy layer and every layer after it. Every deploy reinstalls every dependency.

The fix is to copy the one file that determines your dependencies first, install, and only then copy the rest:

Dockerfile

FROM node:6.9.1-slim
 
WORKDIR /app
 
COPY ./approot/package.json /app/package.json # [!code highlight]
RUN npm install --depth 0 --silent # [!code highlight]
 
COPY ./approot /app
RUN bower install --allow-root
 
EXPOSE 3000
CMD [ "node", "app.js" ]

Now a change to application code only invalidates the final COPY and what follows. The expensive npm install layer is reused until package.json itself changes, which is rare.

What we tried first

Before landing on that, the Dockerfile accumulated a small graveyard of commented-out attempts: tarring node_modules into a backup directory and restoring it on the next build, conditionally extracting the tarball if it existed, pruning and reinstalling. All of them were trying to hand-build a cache that Docker already provides if you order the instructions correctly.

We also moved the front-end asset build out of the image and into CircleCI, which built and uploaded assets once instead of inside every container build.

This was a team effort. Another engineer on the team had spent the weeks before moving global tool installs between the Dockerfile and CircleCI and bumping the Node version; my part was the reordering and the caching commit that finally stuck.

Still true

Every modern Node Dockerfile guide now opens with this pattern, usually with the lockfile copied alongside package.json and npm ci instead of npm install. It's worth understanding why it works, because the same principle applies everywhere you have layered caches: CI caches keyed on lockfile hashes, build systems keyed on inputs, even skipping native mobile builds when only JavaScript changed.