If you’ve been watching my
@scottnonnenberg/notate repo on Github, you might have noticed quite a bit of churn related to setting up CircleCI. I learned quite a lot, and I’m passing all of that on to you. Let’s talk about testing software built with Node.js on CircleCI.
Before we dig in, let’s answer that first very important question. Why use CircleCI? There are a large number of options in the continuous integration space!
First, let’s talk about the widely-used and totally free option, Jenkins. If you want full customizability and zero cost, this option is for you. But it takes time to set up and maintain! Most people won’t want to put that effort into it.
What’s next on the list? In the world of open source, TravisCI and Github play together very well. Many, many node modules use it. Those little badges frequently link to TravisCI build results. If you play in that world, it’s comfortable, perhaps even the default option. I used it for my thehelp libraries.
So, what advantages does CircleCI have over that that default option, TravisCI?
Put it all together and you have a quality tool!
And it’s free for open source too!
Like TravisCI, getting started is as easy as connecting to your GitHub account. Choose one of your organizations (you are considered an organization along with ‘real’ organizations), then click the Build project button and you’re off and running!
There are two options you’ll likely want to change. Select Project Settings in the top right:
Now we’re ready to go!
CircleCI supports Node.js out of the box, but it’s not quite what you expect. If you jump in and start running commands, these are the default versions:
node: "0.10.33" npm: "2.13.5"
That is quite old! v0.10.33 was released in October 2014!
The recommended way to access the version you want (and the only way support multiple versions in your build) is via
nvm. You’re already using a local node version manager, right?
nvm, or perhaps
CircleCI’s containers do come with Node.js 4.x installed, but it’s not the default. You’ll need to explicitly request it. If you want something newer, say for example, the now-necessary npm v3, you’ll need to install it yourself. In your
dependencies: override: - nvm install 6 && npm install
This is an ‘override’ because the default is a raw
npm install call.
test: override: - nvm use 4 && npm run test-server-all - nvm use 6 && npm run ci
The ‘test’ section is similar. The default is a raw
npm test call.
Because each statement underneath the ‘override’ key will be run with their own environment variables, they’ll use the default (very old) version of Node.js. To fix that you’ll need to use the
nvm use 6 && syntax for every command or set the default with
nvm alias default 6.
It’s also worth noting that CircleCI will auto-detect your project type, so you don’t even need a
circle.yml. But you probably want to at least choose your Node.js version. To update the default
npm available on the machine, you can set the default node version in your
circle.yml like this:
machine: node: version: 6.3.0
Out of the box, CircleCI is especially fast, because it caches your project’s
node_modules directory after doing that initial
npm install. You can manually request a cache-free build, but by default every build after the first for a given branch uses a cache provided by previous builds.
But this isn’t a very good idea. Good tests should match the real user experience as closely as possible. Let’s consider some scenarios:
npm installwill install it.
npm prunewould eliminate no-longer needed dependencies.
^3.2.0) you’ll need an
npm updateto get the version you expect.
npm installwon’t do anything. But
package.json, ranges inside your dependencies’
package.jsonfiles mean that you need an
npm updatefor this scenario. Basically you always need to be calling
Whew! All that to get the right versions of your dependencies!
We’re trying to match the user experience, right? Well, users are installing from nothing all the time! Here are my changes to remove all of this complexity and get back to a basic, from-scratch
dependencies: pre: - rm -rf ./node_modules cache_directories: - ~/.npm
This will remove the cache-provided
node_modules directory when starting up, necessary because we can’t stop CircleCI from caching that directory. What we can do is add directories to the cache. So we save the user-level
npm cache to make installs a bit faster.
Personally, I think this should be the default.
I had been successfully running the
@scottnonnenberg/notate project on my MacBook Pro for quite some time, so I was surprised when some core infrastructure didn’t work during my CircleCI builds.
python -m SimpleHTTPServer 8001
It’s not hard to remember, and available on any machine that has Python. It’s there on any OSX machine with no install required, and available on Windows with a quick install. I have run many successful
mocha-phantomjs runs on my machine with it. Even
broken-link-checker runs making many requests very quickly.
SimpleHTTPServer would hang my CircleCI build completely. No timeout from PhantomJS, no warning at all. Just the end of build output, then the build would be cancelled after 10 minutes of no activity. It wasn’t immediately obvious what the problem was, but I did find others talking about hangs.
And so, it was time to do what I should have done in the first place. Instead of using something based on Python in my Node.js project, I used something based on Node.js: the
http-server node module. It was a small change to my
npm serve script:
http-server -p 8001 -a localhost
Voila! No more hangs during my
I had been using a simple custom script to start my web server, then invoke
mocha-phantomjs to test against it. It had originally been a fun little bit of coding.
But it wasn’t fun anymore when my builds started to hang because of it. Coming back later, I now know that some of the hangs were due to
SimpleHTTPServer. But that wasn’t the only source of hangs. My script was attempting to start two different
npm scripts, then kill them gracefully when complete.
But the killing wasn’t going gracefully.
I tried a few changes, SSHed into the container to mess around, and did quite a bit of research into how
npm manages its
npm run child processes on Linux vs. other platforms. There were no clear answers here, and I didn’t want to spend any more time on it. It was time to move to a tried-and-true solution:
I had seen this library used in other open-source projects in the past couple months, and it came up as I was researching
npm’s behavior with child processes. My custom client test script became very simple:
npm-run-all --parallel --race serve test-client-all
It first runs
npm serve to run the server, then keeps that running while it runs
npm run test-client-all for the tests. The key is the
--race command, which tells
npm-run-all to kill all processes when the first one exits. It has been working smoothly thus far!
Having used the
time command on Ubuntu VPS machines and OSX as a simple way to get performance stats, I was surprised to find it causing errors when used in
npm scripts on Linux. Something about the way
npm calls commands on Linux prevents you from calling
> @email@example.com mocha /home/ubuntu/eslint-compare-config > NODE_ENV=test time mocha --recursive --require test/setup.js "-s" "15" "test/unit" "test/integration" sh: 1: time: not found npm ERR! Linux 3.13.0-91-generic npm ERR! argv "/home/ubuntu/nvm/versions/node/v4.2.2/bin/node" "/home/ubuntu/nvm/versions/node/v4.2.2/bin/npm" "run" "mocha" "--" "-s" "15" "test/unit" "test/integration" npm ERR! node v4.2.2 npm ERR! npm v2.13.5 npm ERR! file sh npm ERR! code ELIFECYCLE npm ERR! errno ENOENT npm ERR! syscall spawn npm ERR! @firstname.lastname@example.org mocha: `NODE_ENV=test time mocha --recursive --require test/setup.js "-s" "15" "test/unit" "test/integration"` npm ERR! spawn ENOENT
I can do it when I SSH into CircleCI machines, but I can’t do it from
npm. Could be
CircleCI keeps good statistics about the length of builds, so it’s not a big deal. It just prevents me from seeing that information during local runs. Disappointing.
The good news is that all three of these changes will make my projects more likely to run on windows.
It’s the modern era, and people want their systems to talk to each other. And as one of the leading players in the CI space, CircleCI talks:
There’s a whole lot to tweak, and nothing’s stopping you from adding a new development dependency and doing whatever you need!
CircleCI is a great option for open source, private repositories on GitHub, and on-premise with GitHub and CircleCI Enterprise.
Get those builds running on every pull request and commit, track results and performance over time with ‘build insights’, improve build performance with parallelization, then start deploying to staging and production!
It all adds up to easy continuous integration and deployment. Jump in!