My First npm Publish

My first npm publish was unusual. npm didn’t exist at the time, so that presented a bit of a challenge.

This is the story of helping to inventing a universe so that I could make an apple pie from scratch.

SSJS

Back in 2009, I was working at Yahoo! as a Front-End Engineer. That meant that I wrote a lot of PHP and JavaScript. I had just finished a project where we had front-end components generated on the back-end and shipped to the client based on some data being parsed into a template, and then later on, on the front-end, do the same work in JavaScript with the same templates and data services.

These days, that’d be called “fast boot” or “isometric templates” or something clever, but back in those dark days, it required tediously maintaining two implementations of a view layer, one in PHP and the other in JavaScript. Maintaining the same thing in two languages was downright awful.

“Well”, I figured, “JavaScript is a language, and we can control what’s on the server, why not just run JavaScript on the server?”

The state of the art in server-side JavaScript (SSJS) was Rhino on the JVM. The problem was, unless you compiled your JavaScript into JVM bytecode using arcane special magicks, it was godawful slow. I started messing around with V8 and SpiderMonkey, thinking “I want something like PHP, but JavaScript”.

The SSJS community at that time was a very different place than the Node.js of today. There were dozens of projects, any one of which could’ve seemed like it would be the breakout hit. SpiderApe and v8-juice were trying to make it easier to embed spidermonkey and v8, and add a standard library to each. v8cgi (renamed to TeaJS) provided a CGI binding to use v8 in Apache2. I started messing around with K7, which provided a bunch of macros for using V8 in various contexts, and Narwhal, which was the only one of these that seemed to be delivering a fully thought-out platform for making programs. There was also Helma and RingoJS, and probably a bunch of others I’m forgetting.

A few years ago, we used to joke that every Node.js dev had their own test framework and argument parser. Well, in 2009, every server-side JavaScript developer had their own SSJS platform.

The contributors to all of these platforms got together in a mailing list and tried to form some kind of standard for server-side JavaScript programming. Front-end JavaScript has the DOM, so we thought, and right now, writing server-side JavaScript suffers from a dearth of portability. What we need is a standards body, clearly! This was initially called “ServerJS”, but then expanded its scope to CommonJS.

The first proper “module” I wrote in JavaScript was a port of a url parser I wrote for YUI. I landed it in Narwhal. There was no userland, really. Just lots of little cores.

Some time later, in August of 2009, I gave a tech talk about SSJS and demonstrated using Narwhal and Jack, a Rack-like thing built on top of Narwhal, using the JSGI protocol.

After the talk, one of the people in the audience asked if I’d ever tried out Node.js. As it turned out, I had, but like so many SSJS platforms:

It had a single developer working on it, and no other contributors or community.
The documentation was extremely sparse
It failed to build on my mac laptop.

Ergo: Not a thing.

“I dunno,” he said. “Maybe try it again. It’s pretty nifty.”

He insisted that it was fast, and I was like, “Meh. JVM is fine.”

Node.js

I checked the website again, and they’d added a “Community” section. Also, the docs still sucked, but it was version 0.0.6 now, which was like, 4 more than it was the first time I’d checked, so whoever this Ryan guy was, he was at least working hard on the thing.

It compiled successfully, and I was hooked! It started up so fast compared to Rhino! And it had tests that ran when I did make test, and they passed!

3 important lessons for OSS success:

Docs and tests matter.
At least have a link to a mailing list or something. (Remember: this was before GitHub connected us all with Issues.)
It has to build and be fast.

I gradually stopped paying much attention to CommonJS, and instead just threw my efforts at Node. I hung out on the mailing list and in IRC during all my free time.

The problem with Node back then was that even though a growing number of people were all writing really interesting programs, it was hard to share them. So, I wrote this thing, which was a port of a bash script I was using to play with people’s code.

The Registry

Technically that wasn’t “publishing” though. In order to actually publish to npm there had to be an npm registry. Today, that registry is a webservice at https://registry.npmjs.org/, run by npm, Inc.. The first registry was a git repo called “npm-data”. I collected up the handful of modules that’d been shared from on the mailing list and in the Node.js wiki page, and made a JSON file with links to them.

One principle of package management that I felt was really important was that no one person should be the bottleneck in community growth. Especially if that person is me. Because I really hate that crap.

I don’t mind working really hard on lots of challenging stuff, but if I have to do some simple task over and over again, especially if other people are depending on me to do it, it’s like torture to me. The prospect of being in someone’s critical path for deploying their module was just… ugh. Gross.

I needed a web service type thing that would let people publish packages and then could download those packages and install them.

I got to talking to Mikeal Rogers, who worked at Couch.IO. He built the first npm registry CouchApp, and got it functional.

Fun fact! For a little while, anyone could publish any package, and we relied on the honor system to keep anyone from clobbering anyone else’s name. It was an ok system for a short while, since there were only about 4 or 5 people in the world who knew this thing existed, but we got an authentication and authorization system set up before anyone could take advantage of it.

By that time, I’d quit my job at Yahoo! and was taking a sabatical. If you can afford it, I highly recommend saving up a little nest egg and taking a few months off to see what comes out of you. Muses can be fickle, and tend to call when least expected.

I know what you’re thinking…

You’re thinking that the culmination of this story is that I published npm to npm and that was my first npm publish, and it’ll be super meta and awesome like that. It’d be a beautiful punchline.

Real life is sloppy sometimes.

I knew that I wanted npm to be able to accept abbreviated versions of commands, so that npm inst would do the same thing as npm install. (To this day, the friendly CLI shorthands are some of npm’s most beloved features.)

The first thing I published to npm was abbrev. I’d written it already, mostly as a sort of coding crossword puzzle some… Saturday? Wednesday? All the days were pretty identical during those two lazy/exhausting months of funemployment.

Since abbrev was only one module, no build command, it was really easy to publish and install repeatedly. Ever since then, it’s always been one of my go-to testing modules to make sure things are working properly. Not only was it my first npm publish, it was the first npm publish, and it was published probably dozens or hundreds of times to http://localhost:5984/ while I was working on npm. So, of course, when I had a registry running on my little DreamHost instance, abbrev was the first thing I published to it.

The really wacky part: despite it being the first thing I’d published with npm, I didn’t actually use abbrev in npm until 5 months later. That whole time I kept trying to figure out how to have proper dependencies in the thing that installed dependencies. Eventually, I gave up and threw it in a utils folder.

Looking back over abbrev now, it’s amazing to me how little it’s changed. Most of the code is still that initial implementation from May 2010.

The moral of the story is that you don’t know how it’s going to end.