The Future of the npm Website: Let's Map this Road!

After announcing the complete rewrite of npm-www, many of you have asked what else was next for your favorite website.

First, some data: Since January of this year, our web traffic has increased 47%. Approximately 35% of our visitors are coming to npmjs.org for the first time. On a given weekday, we’re approaching 40,000 daily active users (up from roughly 25,000 in January). That’s a lot of people, and those numbers are only going to increase over time.

We see these numbers, and we ask: what are our users doing? And how can we help them do it better?

Most of our users go to npmjs.org to find the packages they need for their projects. But are they finding those packages as well as they could?

When it comes to the npm website, our primary goal is discoverability and everything that it entails: education, presentation, exposure, curation, documentation, and more. At npm, Inc., we want to give you the tools you need and access to the community you love to be a successful Node user.

Enough dilly-dallying, here’s what we’re up to:

Total Redesign, From the Ground Up

New Architecture

The original npm-www was a great first-pass at the web front for npm. But over time, it sacrificed modularity, testability, and simplicity in the name of get-that-next-feature-out-the-door-asap. We want the npm website to be an example of one of The Best Ways To Build A Website. For that to happen, we need modular, testable, easy-to-add-to code, so we decided to start over.

We chose to use Hapi as our framework - the Hapijs team has thought really hard about the elements of a strong web application, and has an incredible number of modules to prove it.

The newww npm-www takes advantage of the Hapi framework to create a more reliable, stable, scalable, and secure web app. Plus, by making newww incredibly modular, we can test and easily maintain every component of our system with confidence.

Want to learn more? You can read all about the modularization of newww in the repo README.

Want to see it in action? Starting today, you can see what we’re up to at https://preview.npmjs.com! (Go ahead, you know you want to click around. And then log bugs as issues. Doooo iiiiiit. ;-))

New Look

In addition to the total rewrite of www’s back end, we’ve teamed up with the amazing people at Bold to give the front end a total makeover. In addition to more colors and a cleaner layout, the new design aims to make it easier for new community members, experts, and everyone in between to use our site.

We’re still in the process of implementing our new designs, but we’ll let you know when they’re ready on our preview site! In the meantime, here’s what to look forward to:

The biggest visual changes to the site were predominantly driven by our focus to improve the information architecture and user experience. Users shouldn’t have to guess how to use our site, or feel lost and confused when they’re visiting for the first time. We want your visit to the npm website to be awesome, the first time and every time.

With that in mind, we’ve improved package presentation, given documentation a major role, added context clues so you know what you’re looking at and how to find what you need, and oh so much more.

Some examples:

Package Management

Package authors are going to get what they’ve been asking for: online editing for basic package information! Clean up your READMEs and update package keywords, description, and URLs straight from your package page. No more version bumping and publishing because of typos.

In addition, the README, often the only documentation a package has, will be front and center on package pages. Smart parsing logic of READMEs will encourage cleaner documentation and more code examples as well.

Finally, we’ll be adding better statistics to help package seekers gauge a package’s overall “health.” We’ll use data like GitHub stars and issues, age, completeness of the README, existence of tests, and more to come up with a magical formula. (Yes, of course we’ll tell you what the magical formula is! We just have to come up with it first ;-))

Curation

Finding the right package for the job is hard. One way to make it easier is to rely on what other people are using.

The first thing we’re going to do is add public keyword tagging: in addition to the keywords package owners provide, members of the community at large can suggest their own keywords with customized tags.

The second thing we’re going to do is give you the ability to make your own collections:

Collections

Packages are awesome on their own, but their real magic comes when used in conjunction with other packages. Lots of folks write incredible blog posts about all the packages they use and how they’re used together - but like finding the right package in the first place, finding those blog posts can be difficult.

This brings us to npm collections: user-written guides about how to use multiple packages together in one beautiful layout. Think of a collection as an extensive README, a souped-up blog post, or a peek into the soul of an application - we’ll leave that as an exercise to the reader.

But more importantly, collections will provide another level of meta-data that will make finding the right packages easier. Collections and packages will go hand in hand, both linking from a collection to the packages mentioned in it but also linking from a package page to all the collections in which it is mentioned.

Need an example? The dat project has their Modules We Use page, which would make a great collection. Alternatively, a collection could look something like this tutorial on how to create a Node web app with Hapi and Twilio Integration. Honestly, we leave it up to you, the collection writer. :-)

We’ll be featuring the best collections on the homepage, which will not only give you credit for putting together awesome writeups, but will also help beginners learn how to get up and running!

We’re so pumped for collections, we can’t even. Though that does bring us to the third thing we’re doing for curation:

Ecosystems

One of the most common questions we receive is, “How do I know if a given module will work with X?” where X is one of [‘Browserify’, ‘Windows’, ‘Grunt’, ‘Gulp’, ‘LineRate’, ‘Tessel’, ‘one of the many other ecosystems that already exist’]. Keywords aren’t enough, especially if you have to depend on a package author to remember to update their package’s keywords.

Introducing ecosystems: subsets of npm packages that all have a common foundation. We expect ecosystems to be rather large and constantly changing; as such they will be maintained programmatically via an API, instead of being hand-curated like collections.

Eventually, ecosystems will also have their own dedicated search and customized weighting to make finding an ecosystem-relevant package even easier.

We’ll be looking for partners to help manage ecosystems, so if you currently run a third-party site that curates packages for a given ecosystem and want to increase your own discoverability while we take care of the hosting bill, please get in touch!

Last But Not Least: Better Search

Let’s be real: search is hard. But we can make it easier by adding filtering, better metrics, improved ranking algorithms, and more. And we will!

Our awesome Faiq the Intern has spent his summer cleaning up our search code, adding tests, and leaving search much better than he found it. We’ll continue building on his work to give you more control over your search results along with enough information so you can get what you need. And we will be working with the CLI team to bring the powerful new search to the command line as well.

Timeline

If the above is any indication, we’ve got a lot of work to do. While I won’t set any hard deadlines, I can say that a good chunk of it is already done, and will continue to be done in stages:

Feature parity of newww with npm-www (current site)
Implementation of redesign
Collections & Ecosystems
Search

We know change can be tough, so we want to give you some time to adjust. As we get close to finishing a stage, we’ll share it with you, our users. At that point, you can start playing around and letting us know when something doesn’t quite work right. When we think it’s ready to go live, we’ll push it to production! And, as always, you can see everything we’re doing and help contribute at the newww GitHub repo.

Have questions? We’ve got answers! Reach out via email, Twitter, or IRC (#npm on Freenode). We look forward to hearing from you :-)