some notes on RFCs

For about a year now, npm’s engineering teams have been writing RFCs as a routine part of our design process. I’m going to talk a little bit about why we do this, what we get from it, and what it can’t do for us.

I’ve been doing something like this for a while on my own, as part of my own design process for coding projects. I often practice readme-first development: I write the README for a project, with usage examples and a sketch of the API, before I write any code. This helps me think through how I want my users to interact with my library. This works for small projects, or projects where the discovery process is limited to how people will want to use it.

For larger projects, or ones where I’m not sure in advance what the solution should look like, I use a different process. Here, I write things I call one-pagers, named for about how long they should be. These start with a problem statement, continue with an exploration of useful facts about the problem, then move on to a proposal for a specific solution. These are useful to me even if nobody else ever reads them, because the act of writing them makes me slow down and think about what I’m proposing to do.

npm’s RFC process is an elaboration of the one-pager process, but renamed to reflect that seeking out feedback on the document is just as important as writing it. It’s a request for comment!

Let’s get some comments!

The structure of an RFC

npm’s more formalized RFC process adds a bit more to the document than I used to, because it is intended for a wider audience. Our RFCs have four sections:

• problem statement
• background
• possible solutions
• specific proposal

I’ll go into detail about what each of these are and why they’re helpful for us.

the problem statement

Often, your hardest task is identifying the problem clearly and specifically.

Sometimes the “problem” arises from non-engineering concerns: “we need to implement this new feature” is a good trigger for an engineering design proposal. Sometimes you’re fixing a bug. Other times, it’s more vague to start with: “this system needs to scale” or “this user action is too slow.”

The goal of writing the problem statement is to focus on this problem instead of others that might be just to the side. It also pushes you to consider the properties of a good solution, which you must understand when you’re deciding what action to take.

background/root causes

I think of this as the research section: How did we get here? Why do we need to act on this now? What’s contributing to the situation? Writing this section pushes you to discover constraints on the solution or requirements that might be lurking. Knowing why a current system behaves the way it does is handy when you’re evaluating a replacement for it.

Writing this section is meant to help you understand the problem fully and describe it clearly enough that your colleagues can understand it too.

possible solutions

Is there more than one way to solve this problem? Probably! If you haven’t come up with more than one solution yet, try to do so now. Ponder the possibilities and pull apart their tradeoffs. The research work you did writing the background section sets you up to do this.

The goal of writing this section is to shift you into thinking about what you’re going to do next.

the proposed solution

Okay: now you drive a stake into the ground and have an opinion. What’s your proposal for solving the problem?

Answer these questions if you can: * Why does this proposal meet the constraints better than other solutions? * What are your solution’s requirements? These might range from the technical (“this will be a breaking change, so we’ll need to release a major version number bump”) to the human (“the support team will need a new tool to view this data”). * What are the consequences of your choice? * Which systems will need to change? * Which systems will be replaced?

Go into as much detail as you can, given that you haven’t started implementation yet. By trying to answer these questions, you may conclude that you don’t know enough yet, and you need to do more research — and proposing to do more research is a perfectly reasonable proposal.

Be prepared: this section will be the focus of your colleague’s commentary when they read it. Also, don’t be afraid to make a strawman proposal that you know will be replaced. Reacting to a strawman might be just what your team needs to inspire some new ideas.

Making the request

A request for comment is not complete until you’ve requested the comments!

Share your RFC with your colleagues and solicit their feedback. At npm, we do this by posting the RFC to a specific channel in our company Slack. We tag in people whom we especially want feedback from, but the entire company is welcome to read and comment on RFCs.

Your colleagues will have insights you don’t, on both the problem and your proposed solution. Use this feedback to revise your RFC as necessary and tune up your proposal. You might need to revise significantly and get another round of feedback, or you might be ready to start implementation work.

What it’s not

The goal of writing an RFC is not to produce a document that gets followed as if it were a spec. The goal also is not to be correct and marvelously brilliant in your first proposal. The journey is the point of writing an RFC.

Writing an RFC pushes you to think clearly about the problem you want to solve and to do research into it. By discussing it with your colleagues, you improve both your understanding of the problem and your understanding of the tradeoffs required by possible solutions.

I guarantee that your team is collectively smarter than any one person in it, even you. Together, you can solve harder problems than any of you could alone, so use this collective brain! It might take practice — it’s not always fun to have your ideas challenged — but once you can receive, consider, and incorporate others’ feedback, you will be smarter for it, and your eventual solution will be better.

Note the corollary: when you comment on an RFC, your job is to be constructive and collaborative.

Another interesting corollary: you can tune the RFC focus to suit the needs of your team. Do you, as a team, need to spend more time thinking through the operational requirements of your work? Make that a required section of the RFC. Do you always forget about support tools? Your RFC template can remind you. Encourage your team to think about the parts of the process it needs to think about more.

Don’t add too much overhead to this document, though. Don’t let this process consume too much time or be an excuse for procrastinating or over-architecting.

This is the beginning

A map is not the territory it represents, but, if correct, it has a similar structure to the territory, which accounts for its usefulness. —Alfred Korzybski, Science and Sanity (1933)

You should emerge from writing an RFC with a better map of the path ahead of you, but you still need to put on your boots and hike. Happy trails.