How to Build Better Software with Spec-Driven Development

This newsletter is sponsored by .

Your engineers weren’t hired to parse PDFs

Every few quarters someone proposes building document parsing in-house. It scopes like a sprint. Five years on and it’s roughly $1.6M more than buying, and an engineer still owns maintaining the pipeline.

KPMG, Sandisk, and Lovable run their documents through by LlamaIndex instead, handling messy tables, charts, scans and handwriting, on one API.

Use code SUMMERGIFT26 for $250 in credits on signup. You can try it yourself and point it at the documents that are currently costing you engineering hours. Closes October 30.

Thanks to LlamaIndex for sponsoring this newsletter. Let’s get back to today’s thought!

Intro

With AI-assisted engineering becoming the standard way of building software, the speed of writing (generating) code has significantly increased.

But faster code generation doesn’t necessarily mean better software.

What I like to say: You can generate thousands of lines of code in a matter of minutes, but if the requirements are unclear and the architecture is bad, you’ll spend more time fixing and rewriting code than you saved generating.

That’s why spec-driven development (SDD) can help a lot with this. You spend more time thinking about what needs to be built and how it should work, and almost no time fixing issues.

In this article, I’ll share 2 practical approaches to SDD: a simple workflow used by Michael Bolin, a tech lead on OpenAI’s Codex repository, and a more structured 7-step process used by the engineering team at Larridin.

You’ll also learn when SDD makes sense and when it’s better to keep things simple.

This is an article for paid subscribers, and here is the full index:

- Most of the code these days is AI-generated
- Option 1: Use a simple workflow
- Option 2: Use a detailed process
- Step 1: Start with a spike
- Step 2: Promote the spike to a reference implementation
- Step 3: Write the spec
🔒 Step 4: Write and refine the test plan before the implementation plan
🔒 Step 5: Create the implementation plan
🔒 Step 6: Implement
🔒 Step 7: Verify the implementation
🔒 When to Use SDD (and When Not To)
🔒 Last words

Let’s start!

E-book and pre-order of “The Multiplier Mindset” published by O’Reilly now available!

This article is an adapted version of one of the chapters of my book called “The Multiplier Mindset”. E-book or pre-order now available. Paperback version coming out in 3 weeks!

Most of the code these days is AI-generated

Based on my conversation with Katelyn Lesse, Head of Platform Engineering at Anthropic, engineers at Anthropic have begun using LLMs to generate 100% of their code.

Instead of adjusting the generated code themselves, they provide feedback and guide the LLM to the correct solution. That is considered an effective use of AI for creating code.

And that’s also a reality (or close to) in many other companies. Additionally, many of the engineers spend a lot more time creating the spec, instead of writing or generating code just based on pure prompts.

That’s what we call: Spec-driven development. Which is a way of building software where you focus on building a good specification first, and then use AI to implement the project.

Now, let me share 2 options for doing spec-driven development, a simple one and a more structured process, next.

Option 1: Use a simple workflow

Michael Bolin, a tech lead for OpenAI’s Codex open source repo, has a very simple workflow:

1. Write an initial tech spec in Notion, describing the requirements and functionalities that the code must support.
2. Ask colleagues to review and comment on the draft.
3. Use a simple prompt to tell the LLM tool to build code from the spec.
4. Review the code and make appropriate adjustments.

Michael’s workflow is a great example of a simplistic and effective use of spec-driven development. This works particularly well for smaller, less complex projects where the risks are relatively low.

It’s also important to mention that as AI coding tools use more and more tokens, you need to be mindful of the cost, as most of us don’t work with unlimited tokens.

With a great spec, you can usually get good output even if you use a lower-tier model, because a great spec is one that provides very clear instructions. Many companies use a higher-tier LLM to write the spec and a lower-tier LLM to build it.

Option 2: Use a detailed process

Earlier this year, I visited several companies in San Francisco, including Larridin, which is building an enterprise intelligence platform. The engineering team and CTO Ameya Kanitkar shared a 7-step process that they use when practicing SDD, and I’ll summarize it in this section.

Step 1: Start with a spike

It’s impossible to write a complete spec when dealing with unknowns. So the team creates what it calls a spike task, a throwaway prototype, to resolve unknowns before starting a new project.

The goal is not to build a full minimum viable product (MVP) that it can extend into a real product; a small, ugly prototype will do.

All it has to do is answer one question: “Does this idea actually work?” This helps to remove any technical uncertainty.

Building a spike task is a much better option than contaminating the spec with various options the team might not use.

There are many unknowns in a complex project, and a throwaway prototype can show you whether specific parts of projects are actually feasible within the current architecture and how they mesh with other constraints.

You can’t write a full spec with too many unknowns. For example, if the unknowns you’re dealing with are edge cases for the API, you can spike those edge cases and see if you can get an answer.

If your whole system depends on how fast your API is, spike the hot path and measure it. If you’re building a new view that has a lot of records in it, spike it to prove that the API can return the results fast enough.

The goal is to stop when you have an answer, not when the code is pretty.

If you’re worried about naming variables, functions, and file structure, stop! You’re putting too much thought into the implementation details.

Step 2: Promote the spike to a reference implementation

Once the spike works, don’t throw it away. It’s now your reference implementation. Save it in the repo in a known location, something like spec/spikes/reference/APIStressTesting.

You can point the LLM to it and say, “I know this pattern works, because here it’s working!”

The Larridin team called this one of the most underrated aspects of SDD: a working example is worth a lot more than a full page in the spec, because it helps the model stop guessing (and thus hallucinating).

When you provide the reference implementation to the model, comment on the specific parts that prove the approach, as well as any deficiencies, like parts where things are hardcoded or the error handling is missing.

The more info you give the model, the more accurate its overall implementation is going to be. Providing a spike without comments can be as dangerous as not providing one at all, because the model will just reproduce the wrong parts alongside the right ones.

Step 3: Write the spec

Now it’s time to write the spec, which you can and should do with AI’s help. It should be detailed enough to hand to a junior engineer or a small model.

Kanitkar suggests forcing the model to restate what needs to be done, list any assumptions, and flag any ambiguities. If it finds any remaining unhandled assumptions, you can create a new spike or update the existing one.

Kanitkar recommends that you “write the smallest spec that unambiguously specifies the system. If a section feels like fluff, cut it. If a decision feels obvious, write it down anyway. Obvious to you is not the same as obvious to the model.”

As an example, he shared a full spec for the Zero-Dependency Brainstorm Server.

Step 4: Write and refine the test plan before the implementation plan