An introduction to spec-driven API development

Historically, API design has been mainly implementation-driven (also called code-first). Nothing faster than quickly prototyping a few endpoints together before even thinking about putting together an API specification. Mo specs mo problems. Or is it?

You might recognise the side-effects of a typical product "drift". For example; You want to develop a new REST API for a service or a product, so you start with the beginning (designing the REST API) or with the end (development of the REST API). Along the way, ideas change or have to be adapted because of the iterative nature that is linked to development, but now your specification or documentation is outdated, and you have to rework your tests to keep up with the changes continuously.

Maintaining your specification, code, tests, and documentation as separate parts means that every change must stay in sync in all these layers of your product. This causes double or even triple the work effort and starts to feel like a waste of time. In addition, there is the risk that parts are missed entirely - causing frustration with anyone trying to use your newly developed API.

With spec-driven development you turn the implementation-first model around, by putting your API specification at the center.

The Stages of Spec-driven Development

Spec-Driven Development is where your application is built in 3 distinct phases:

1. Design: The creation of a specification
2. Build: Development of code & validation (testing) to match the Spec. Refine the specification to match the final result.
3. Management: The stage where you deploy & monitor the API, based on the specification

Instead of all layers (spec, code, tests, docs) evolving separately, the beauty of this approach is having a single source of truth to act on.

Spec-driven cycles (credits to Mulesoft for the diagram)

The Benefits of Spec-driven Development

Spec-driven development is a great strategy to keep design, development, documentation and testing, all in sync. Resulting in the following benefits:

• Better documentation
• Clearer requirements
• Shorter feedback loops
• Out of the box mocking
• Consistent API design
• Team alignment

Single Source of Truth

The API spec that forms the source for our cycles, can be any format (OpenAPI, RAML, API Blueprint, WADL, ...). The OpenAPI Specification (formerly called Swagger) has become the generally accepted format for formerly REST APIs.

At its core, the OpenAPI Specification is a JSON or YAML file that describes a REST API , which supports:

• Defining the API's endpoints and their available operations (GET, POST, PUT, PATCH, DELETE)
• Outlining the parameters necessary for the input and output of each operation
• Describing authentication methods
• Communicating other information associated with the API itself (contact, licensing, terms of use, etc.)

By using the "OpenApi " specification format, you use a worldwide standard. This affords a number of advantages. Tooling, code generation, documentation, validation, ... all out-of-the-box, and all driven by very active communities - both open-source and enterprise.

Designing & defining your REST API using OpenAPI, you benefit from:

• A clearly written overview of all requirements, which can be used for development
• A lead document for QA and tests to validate against
• Up-to-date and useable content for customer-facing documentation

Workflow Artifacts

On top of that, the use OpenAPI enables some automated workflows.

Automation workflows that can generate a number of "outputs":

• API reference documentation
• Postman Collections
• Swagger documentation
• Online playground
• Test suites
• Code samples
• Client SDK's

Credits to Joyce Lin

Let's use a concrete, recognisable example of a workflow: "Development of new API"

• A product manager writes the requirements
• A back-end engineer writes the code that powers the new API
• A QA engineer writes tests to verify the API
• A front-end engineer builds a nicely looking web application on top of the API
• A partner integrates by using your API and creates a client for it
• The marketing team creates the how-to's that describe the use-cases of your new API
• ...

Each step in this workflow requires a clear, documented and detailed description of the "contracts" (input/output aka request/response) that your API exposes. Describing the API contract at creation, is already challenging, but keeping track of the changes and communicating them can be time consuming. You want to make this a collaborative effort and as painless as possible. The OpenAPI specification format solves this quite well.

But with changes during development or after launch, it means that everybody involved needs to update their part:

• A QA engineer needs to rewrite his tests to verify the API
• The front-end engineer needs to modify his implementation
• The partner needs to rewrite the code of his client implementation
• All public documentation needs to be reviewed and corrected

And probably not once but multiple times, during development - so rinse & repeat.

Workflow automation

By leveraging the power of OpenApi workflows & automation, you can put all the focus of the iteration workload on the API specification.

Let's take the example of creating a Postman collection, which is commonly used by engineers, QA's, PM's to interact with API's.

The flow below shows the workflow to create and maintain the Postman requests & write some tests for it, which is mostly manual work. You can import the latest OpenAPI specification directly in Postman but it will still require some fine-tuning of the request parameters and writing the tests. This process then needs to be repeated after each iteration of your API.

Leveraging automation, you can have a CI/CD pipeline trigger a Postman generation with every GIT commit of an OpenAPI change.

The creation and maintenance, even the test writing, workload for the Postman collection, can be replaced by a CI/CD automation, which frees up time for the team members for improving the OpenAPI document.

OpenAPI powered automation

Instead of maintaining the Postman collection, it is more efficient to collaborate on the OpenAPI spec and add real examples that will actually work for your build API.

Because of the "spec driven development" cycle, everybody involved is benefiting from the collaboration:

• The QA engineer can immediately start with manually testing and create new test scripts, instead of figuring out how to create a request.
• The front-end engineer has a working example in Postman and can leverage it as documentation and playground
• ...

This is just a simple realistic example where the spec driven development benefits of time saving, always up-to-date information, ready to use execution and the skipping of copy/paste maintenance work; are showcased.

Extended workflow automation

But there is more, much more you can do with regards to workflow automation. Based on our OpenAPI specification, on which everybody contributes, we have defined a number workflow stages:

Workflow automation stages

1. Linting & validation: which makes sure the OpenAPI document is well formatted and the examples used are valid with the defined request/response models.

2. OpenAPI publication: we transform the OpenAPI document in YAML & JSON format so that they can used as the source for internal & external tools.

3. Publish a Postman collection: we generate a Postman collection that is ready to be used to execute API requests.

4. Generate online documentation: this is the UI of your API, by exposing all the details & properties the response & requests. It is a crucial aspect of the DX that comes with your API. Use it to generate an online Swagger reference, an API reference by Redocly and an interactive online playground by RapiDoc.

5. Testing: since all ins/out are described with real examples, we can leverage all this information to generate tests. The tests are automatically injected in Postman based on the example values in the OpenAPI document. This enables "Contract tests" through Newman, "Soak" and "Smoke" performance tests via K6.

All these stages are based on just one OpenAPI document, our single source of truth, packed with relevant details and examples. It builds upon a rich set of open-source tools around the OpenAPI format.

There is so much more to be said about this promising approach to api development, so stay tuned for a follow-up post in the upcoming weeks.

Did we convince you to try out spec-driven development? A few weeks ago, we released Portman to help you increase the ROI of using API specs. Make sure to give it a try!

We're looking forward to hearing your feedback. 🙏🏻