Creating processes for API clarity and consistency

A clear, consistent API makes a huge difference for developers. As Plaid has grown, we've added more products and endpoints, and now have many engineers over more than a dozen teams that work on developer-facing surfaces. With this scaling, we needed a way to ensure all Plaid API endpoints use the same principles and interfaces. This post will discuss some of the procedures we've set up, in the hopes that other engineering organizations with this problem will find it useful.

The API Review Council

The primary way we enforce API consistency is through the API Review Council (APIRC). The council contains a rotating team of about a dozen members from teams across Plaid. Teams that tend to make a lot of API changes or own unusual surface areas have a permanent seat on APIRC, with the specific engineer representing the team rotating every year or so, unless they choose to stay on. The council is made up of team members across Engineering, Customer Engineering and Developer Support. While earlier iterations of the council were more cross-functional, we found having a more focused council helped us streamline the process and the output. Membership requires a minimum level of experience (E4 – the level right after a new college grad) and we monitor membership to ensure a good balance of seniority, aiming to have at least a third of the council be senior engineers. (This can be tough, as those are the engineers who tend to have many competing demands for their time!)

Rotating members through the council and representing a diverse array of teams has a number of benefits. Most obviously, we get input from people with a wide variety of perspectives and experience and who are familiar with different parts of the API and developer use cases. More subtly, every current or former council member serves as a representative and expert for API best practices. The more engineers at your company who have served on the API review council, the more advocates you will have for good API design.

API Style Guide

One of the first tasks of the council was to create an API Style Guide, which codifies the style decisions currently used by the API, as well as our recommendations in cases where the API didn't already set a clear precedent. As we make new decisions, each one is added to the Style Guide. The Style Guide covers topics such as:

• Representation of commonly used concepts, like names, dates, addresses, or money, with special attention to internationalization concerns

• Conventions around naming, spelling, capitalization, and abbreviations

• Conventions around structuring requests and responses, such as when to use nesting vs. flat objects and how to paginate response data

• What parts of the API are considered safe (backwards-compatible) to change and what parts are not

• Existing error classifications and what they should be used for

• How empty or missing return values should be represented 

Unsurprisingly, when creating the guide, we found a number of cases where the API was not consistent! We recommend you create an API Style Guide as soon as possible to track your decisions and make sure new APIs you create are consistent with older ones.

We also have a checklist for reviewers and submitters to help them ensure that API changes are consistent with the style guide. While we don't yet have automated tooling, such as Spectral linting rules, to enforce API style, we plan to add this soon. 

Blocking approval

Every change to the Plaid API must be approved by the APIRC. In our case, we use Jira – teams file new tickets to request a review. Changes can range in scope from adding a new error code or enum value, to entire new features or workflows that require multiple endpoints. Simple changes are typically approved promptly (within 2 business days) after brief discussion in the comments of the Jira ticket, often with the filer adopting small suggestions from the reviewer. Our internal documentation on APIRC sets out clear guidelines for how long submitters should expect to wait, and they are well within their rights to send us a friendly ping on Slack if they have waited past the initial 2 business day window for a response.

While many people are aware of the ticket-filing process, having an OpenAPI specification is also very helpful, since it provides a single place where we could set up blocking reviewership in GitHub. Teams need to update the OpenAPI file to have their endpoint added to client libraries or documentation, making it a bottleneck where review can be enforced. An on-call rotation, maintained via PagerDuty, lets members know when it's their turn to review changes. A secondary on-call is also in place so that the primary can get a second opinion or hand off reviews when they're swamped. 

Secondary review

For tickets that require more than a brief review, we have several options:

• For larger changes, the reviewer can bring in one or more additional reviewers. These reviewers are often the secondary on-call, a member of the APIRC with domain expertise in the surface area being reviewed, or just a volunteer on the APIRC Slack channel who responded to a request for help. Some changes (such as adding new endpoints) require at least two reviewers for approval.

• If the change requires discussion beyond back and forth over Jira or comments in a spec, the author can set up a meeting with the APIRC reviewer (our weekly office hours is time set aside for just this) or discuss their change at the weekly APIRC council meeting.

Newsletter

If a council exists but doesn't have a newsletter, did it even happen? The API Review Council sends out a newsletter on a monthly basis. Topics include fun API facts (fun API fact: Did you know that in addition to the usual RESTful vocabulary, HTTP also technically supports the obscure verbs LINK and UNLINK, which are used to manage the metadata of an object?), updates to the style guide, and praise for someone who contributed to API quality. Of course, the main purpose of the newsletter is to increase awareness of resources for developers changing the API – particularly the council review process and the style guide.

Recommendations

If you are working on a company that, like Plaid, has a lot of APIs that you'd like to keep consistent, an API review process is essential. Here's a recap of our tips:

• Create an API Style Guide early, before your API naturally evolves inconsistent style. Update it over time.

• Provide checklists to help reviewers and submitters ensure that their changes have been reviewed and are consistent with API style. Enforce these checklists through automated tooling as much as possible.

• Ensure your API Review organization has representation from many different teams and a mix of senior and junior (but not too junior) members.

• Rotate members through your API Review organization to instill API best practice expertise throughout engineering.

• Make sure the process for requesting API review is simple, and set up an on-call rotation with some redundancy in staffing to make sure small changes can be approved promptly. Save heavier-weight review processes for bigger or more contentious changes.

• Use newsletters or other forms of outreach to remind engineers about the API standards organization and resources – keep these messages lightweight, to-the-point, and fun!

Want to join our API Review Council?