Publishing what you mean to publish

Editor’s note: This is a guest post from Adam Baldwin of ^Lift Security and the Node Security Platform. As we discussed in earlier posts, Adam conducts constant security reviews of the Registry and its contents and keeps us appraised of anything that might compromise our security.

Over the years I’ve spent a lot of time digging through the half million public packages on npm. There are millions of public tarballs to go with those public packages, which means there’s a lot of code to look through and experiment with. Buried in that code is a surprising amount of sensitive information: authentication tokens, passwords, and production test data including credit card numbers.

You, as a developer publishing to npm, want to avoid leaking your data like this. I’ll share some tips for how to control what you’re publishing and keep your secrets out of the public registry.

npm publish

Let’s explore the behavior of npm publish, because understanding how it chooses which files to include is critical to controlling what gets published. If you want to dive in deeper, the npm documentation goes into more detail, but I’ll cover the important points here.

When you run npm publish, npm bundles up all the files in the current directory. It makes a few decisions for you about what to include and what to ignore. To make these decisions, it uses the contents of several files in your project directory. These files include .gitignore, .npmignore, and the files array in the package.json. It also always includes certain files and ignores others.

npm will always include these files in a package:

package.json
README and its variants like README.md
CHANGELOG and its variants
LICENSE and the alternative spelling LICENCE

npm will always ignore these file name patterns:

.*.swp
._*
.DS_Store
.git, hg , .svn, CVS version control directories
.npmrc
.lock-wscript
.wafpickle-*
config.gypi
npm-debug.log

.gitignore vs .npmignore

One of the most common ways to exclude files and folders is to specify them in a .gitignore file. This is because files you do not want to commit to your repository are also typically files you do not want to be published.

npm also honors a file called .npmignore, which behaves exactly the same as .gitignore. These files are not cumulative. Adding an .npmignore file to your project replaces .gitignore entirely. If you try to use both, you will inadvertently publish a file you thought you had excluded.

This is how that might happen:

You add your production.json configuration file to your .gitignore file because it contains sensitive information.
Then you add .npmignore but forget to add any files to it.
You run npm publish.
Because .npmignore exists, it is consulted instead of .gitignore, but it includes no files to ignore!
Your production.json file is therefore published, and your sensitive information is leaked.

Stick to using .gitignore if you can! If you are using a different version control system, use .npmignore. If you are using git and have an ignore file but wish to publish some of the files you’re not committing— perhaps the result of build steps— start by copying your .gitignore file to .npmignore. Then edit to remove the files you don’t want in git but do want in your package.

Whitelisting files with the files array

There’s an even better way of controlling exactly which files are published with your package: whitelisting with the files array. Only 57,000 packages use this method of controlling what goes into them, probably because it requires you to take inventory of your package. It’s by far the safest way to do it, though.

The files array specifies each file or directory to include in your publish. Only those files are included, plus the ones npm always includes no matter what (such as package.json), minus the ones denied by another rule.

Here’s a package.json file with a files array:

{
  "name": "@adam_baldwin/wombats",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "files": [
   "index.js"
  ],
  "keywords": [],
  "author": "Adam Baldwin <baldwin@andyet.net> (https://liftsecurity.io)",
  "license": "ISC"
}

No matter what other files exist in this project directory during npm publish, only the index.js file will be packed up into the tarball (plus the readme and package.json, of course!)

Double-check your work

You can use the npm-packlist module to programmatically get a list of the files npm would include for a specific directory. You can also run npm itself to find what it would include. This command lists the files that would get packed up:

tar tvf $(npm pack)

Using read-only tokens

If you are using private modules in a continuous integration (CI) system you will need to provide that service with an authentication token. In the past you had to provide a regular authentication token out of your .npmrc, which gives your CI system the ability to do everything you can do with your npm account. Now it’s possible to generate a read-only token that can limit the damage if a token is leaked via CI.

This feature isn’t yet supported by the npm CLI, but you can use the public registry API to generate a token by hand. Here’s an example with curl:

curl -u [USERNAME]:[PASSWORD] https://registry.npmjs.org/-/npm/v1/tokens \
-X POST -H 'content-type: application/json' \
-d '{"password":"[USERNAME]", "readonly": "true"}'

You can review and delete the tokens you have created via the npm website. under Your profile > Tokens.

What to do if you’ve published sensitive data

If you accidentally published a module containing sensitive information, you should consider that data compromised. It would be nice if you could just unpublish the module and hope that nobody saw the mistake, but the reality is as soon as you publish a module it’s replicated to all of the registry mirrors and and other third parties, like ^Lift. The only way to ensure that your services and data aren’t compromised is to invalidate and change any API keys or passwords that were published.

If a secret you can’t change has been leaked, your first step should be to unpublish the package to limit the damage, then take any other actions that are appropriate for the kind of data that leaked. The npm support team will help you unpublish any packages that are outside the 24 hour window for unpublication.

Try whitelisting!

I hope this has been a good refresher on how to help protect what you publish in the registry and ensure that you don’t accidentally leak sensitive data. I encourage you to try using whitelists to control which files go into your next package.

Preventing the mistaken distribution of sensitive information via the npm Registry or any other format requires you and your team to continually educate yourselves on best practices. We will continue to discuss common security issues or mistakes on the npm blog, so be sure to follow along and stay informed on the best ways to secure your code.