npm Blog (Archive)

The npm blog has been discontinued.

Updates from the npm team are now published on the GitHub Blog and the GitHub Changelog.

npm@2.0.0

Last week, I released npm@2.0.0. If you’ve been using npm@1.4, it’s a substantial update, but that’s not why it’s 2.0.0. npm@1.0.1 was released on April 30th, 2011 – three and a half years ago. 1 That’s basically the entire lifetime of Node as a viable platform. Why bump the major version now? The answer is pretty boring:

why 2.0.0?

In npm@2.0.0, Ben changed npm run-script to allow you to pass arguments into scripts. That’s a breaking change. It’s as simple as that. Think of npm 2 as a step on the road towards getting npm right with semver. (There will be more. npm 3 will be out before the end of the year.)

But we did slip some other breaking and significant changes into npm 2, so please read on.

a note on npm’s release process

Those of you who follow me on twitter may have noticed that I’m doing something quirky with npm releases. This is because the npm CLI now has a release process based on a powerful, underused feature of npm: dist-tags. Every week, we publish the new version to the next dist-tag. After smoke-testing the release for a week, during the next week’s release, the release manager (me) promotes that version to the latest tag, which is what gets installed when you run npm install -g. 2

This means that every week, there are at least two versions of npm published – npm@latest, for everybody, and npm@next, for those who enjoy playing around with new things, or who want to help us test npm. (I would love it if more of you were running npm@next and giving me feedback on the next release. Just saying.)

revising node-semver

In npm 2, semver, the package that npm uses to deal with versioning, jumps straight from version 2 to version 4. Here’s why:

A while ago, we noticed that semver was at odds with the semver standard over how 0.x.y versions are interpreted. Isaac was fine with making node-semver follow the standard more closely, even though this was a breaking change and therefore meant that node-semver and the semver standard would have different major versions (this was for real his biggest objection to the change).

Isaac landed this change in npm@2.0.0-alpha.6. At first, it seemed like this wasn’t going to be a very painful change (especially because Isaac also made 1.0.0 the default version for npm init). It only affected the behavior of version ranges using ^, and only for pre-1.0.0 versions. Unfortunately, we’d overlooked our old frenemy peerDependencies. 3

Consider grunt: pretty much the entire grunt ecosystem uses versions around 0.4.x. All it takes is a few plugins peer-depending on "grunt": "^0.4.2" while others are peer-depending on "grunt": "^0.4.5" to shove ordinary developers into dependency hell. Toolchains that were working fine for months suddenly couldn’t be upgraded, or even reinstalled, and npm started complaining constantly.

That wasn’t going to work. Isaac tried to reconcile the standard’s semantics for 0.x.y versions with common practice in the npm community. While the discussion was thoughtful, it was also slow. To get npm@2.0.0 out the door, we came up with our own semver that matches our understanding of how you, our users, think about semver (which, as Isaac says, was the intention of the standard’s authors all along). semver@4 is easy to describe:

  1. Revert ^ behavior to semver@2’s.
  2. Exclude prerelease versions from matching ranges. You must pin against prerelease versions explicitly (=1.2.0-alpha.5) to include them in package.json.
  3. Make the documentation easier to follow.

This is much better, except few package developers understand how prereleases work in semver. The thing about semver is that it’s semantic – almost all of the parts of a npm package version mean something. When you consistently use - to join the upstream version to the npm version of phantomjs, you’re (probably unintentionally) marking those versions as prereleases, and therefore not installable for people using ^ and ~ ranges. Getting maintainers to stop doing this is a much smaller problem than boiling the Grunt ocean, so we’re handling this by contacting the maintainers of affected packages directly.

We went through a lot of work to end up not that far from where we started. But the result is that we have a much better understanding of the implications of semver, a more consistent semver, and much more accessible documentation. 4 Not too bad.

scoped packages

The most prominent feature driving the release of npm 2 didn’t actually need to be in a new major version at all: scoped packages. npm Enterprise is built around them, and they’ll also play a major role when private modules come to the public npm registry.

Scopes themselves are simple, and the best way to learn about them is in the fine new documentation on scopes, npm install, and npm adduser that Laurie put together. One thing that might not be obvious from the docs is that because of the need to tie scopes to registries, you can now be logged in to multiple registries simultaneously. Most people don’t need this, but it’s handy for those who do.

improved reliability

Because npm is a node application from the ground up (or is it?), everything it does is a giant wad of highly-concurrent, continuation-based silly string. This makes npm really good at soaking up your machine’s available disk and network bandwidth, and given how much npm is doing, it’s surprisingly problem-free.

One fun thing about concurrency is that the more you increase a concurrent process’s efficiency, the more likely you are to discover its buried race conditions and deadlocks. 5 Another fun thing is that eliminating one race condition will make smaller, more obscure race conditions more likely to bite you. And so it’s been for us over the past few months:

I am a pessimist, and also lack the hubris necessary to claim we’ve fixed everything. In fact, I know we have at least one more race condition lurking in there somewhere. I’m still very proud of the work the entire npm community has done in finding, analyzing, and fixing these bugs. If you’re using grunt, gulp, broccoli, Phonegap, Cordova, ember-cli, or Yeoman, you are now much less likely to run into baffling failures when using them. If you have complicated builds involving optionalDependencies or lifecycle scripts, they are much more likely to work right, every time. This is reason enough for me to celebrate npm 2’s release.

the full changelog

There are, of course, many other features (local dependencies! bearer-token auth!) and bug fixes (file permissions in the cache! allow % in passwords!). Here’s the semi-organized, complete changelog for npm 2.

BREAKING CHANGES

Semver:

Arguments passed to npm run-script <script>:

npm requires node >= 0.8:

large features

Scoped packages:

Fixes for race conditions:

Local dependencies:

smaller features

bug fixes

documentation fixes

dependency upgrades

  1. It’s pretty typical of npm’s history that it went right from 1.0.0rc9 to 1.0.1rc0. ↩︎

  2. If you saw this tweet, you may know that I have feelings about the current state of npm update -g. You should check in on the discussion I started to see what I intend to do about it. ↩︎

  3. It’s not like there are that many packages on the registry with versions under 1.0.0, are there? If this thought occurred to you, you probably see what’s coming next. ↩︎

  4. For instance, the CLI team (Isaac, Rebecca, and I) have decided that prerelease versions of npm are more trouble than they’re worth, and will be sticking to x.y.z versions in the future. ↩︎

  5. Why yes, I have done a lot of Java development. Why do you ask? ↩︎