What we optimise for when we reach for an LLM

Disclaimer: I wrote this blog post entirely by hand, but used Claude to review and provide feedback on it before seeking human review.

Most of the blog posts I read about LLMs are either gushing about a new era of infinite productivity, or bemoaning the enshittification of the entire software industry. I don’t often come across posts that take the even-handed view of evaluating these tools like any other tool, so I thought it was time to make a contribution to the discussion.

If I discovered one of our engineers repeatedly typing out boilerplate for JSON decoders, I’d be disappointed. If I discovered them spending an afternoon hand-crafting types that encode our business invariants into our systems and then letting LLMs draft the feature based on those types, I’d be delighted. The aim of this post is to communicate why I’d react differently in each circumstance.

Our context (and why yours may differ)

The lens I’m bringing to this has been shaped by the kind of company Bellroy is and the kind of CTO I strive to be. Bellroy is not a software company; it’s a physical-product company that happens to have technological proficiency as one of its core competencies. Since the very early days, Bellroy has employed an internal engineering team - initially to focus on automating the order fulfilment pipeline, but today to build and maintain an entire bespoke e-commerce platform. Bellroy has been continuously profitable since early in its history and isn’t sprinting towards the end of a funding runway. This substantially influences the way decisions are made day-to-day. One of Bellroy’s goals is sustainable, ethical profitability. To meet that goal - among other things - we must be evaluating choices based on long-term Return On Investment (ROI). That evaluation does a lot of heavy lifting: it means that the benefit of shipping more code more quickly must be offset against the cost of maintenance. This philosophy neatly fits my approach to running the Technology team at Bellroy: we must operate as a profit centre, not a cost centre.

Engineering as a profit centre

Some concrete examples of “profit centre” thinking in action:

• choosing statically typed functional programming languages to eliminate entire classes of production issues
• prioritising regular dependency updates of every single code repository we manage to avoid “drop everything” zero-day vulnerability scenarios and dependency rot
• employing a “fixed time, flexible scope” project methodology to eliminate project “overhang” while still delivering tangible, measurable outcomes
• using internally-built probabilistic Net Present Value (NPV) calculator tools that use Monte Carlo simulations to evaluate project benefits (another blog post, perhaps), and using those valuations to drive prioritisation decisions and push back on scope

This last point is important - we rely on our engineers not only to ship features and maintain our codebases, but also to help project stakeholders to identify what they actually need as opposed to what they asked for. Technology does not operate as a silo in Bellroy; we collaborate with other functional teams to deeply understand their processes - and how changes to their processes impact the rest of the business - before we write a single line of code.

This is where we find the current generation of LLMs still wanting. The “everyone’s a builder now” mentality sounds great if you believe that generating code is all benefit and no cost, or if it lets someone adept at business analysis - who has already validated the ROI of a particular change - bypass the engineering department and ship the thing themselves. That perspective ignores the opportunity cost of the next-best-thing that person could have been doing with their time, and the maintenance cost of the thing that the Technology team now maintains but had no part in designing.

“But Mike”, I hear you say, “surely the LLMs can do the maintenance and bug fixes for all those vibe-coded features as well?” Well, yes, you’re (partially) right. We do use LLMs for maintenance and bug fixes. But what we don’t do is rely on LLMs to resolve those rare-but-critical production issues at 3am on a Sunday morning. With so many horror stories floating around the internet of what unmonitored LLM tools can do with access to production infrastructure, we are unwilling to give them (write) access to ours. With this responsibility still in the hands of the humans, the humans need to have a good mental map of our systems architecture and our infrastructure. We need to be tracking how big the on-ramp is for new staff. And that’s exactly what we’d give up by ceding all engineering activities to the LLMs.

What maintainability actually means to us (and Claude!)

We use Claude at Bellroy, across the entire company. And it’s great for so many things.

One thing we’ve noticed about Claude-generated code is that Claude tends to want to provide verbose comments on most changes it makes. Despite explicit instructions not to do so, it uses these comments to document what the code does, or adds context from the prompts, or refers to previous states of the code that become irrelevant the second a change ships.

For us, this is bad practice. Code comments should only provide context that is current and that is not derivable from just reading the code - adding a comment that just describes what the code does increases the risk of the documentation diverging from the implementation, which just adds potential confusion for no real benefit. Our function, type and variable names should do enough of the work to allow an engineer to get oriented.

We use comments only to provide the “why” of a particular implementation choice so the intent - the “should” of the change - is preserved, because it is that information that is hardest to reverse-engineer from the code one year from now. We take a similar approach to pull request descriptions, using a template that asks for a problem statement (what problem exists prior to this change), a solution description (how this particular change solves the problem) and an account of what the engineer has done to verify that the problem is actually resolved. Without engineers having taken part in the scoping process and understanding the size and shape of the business problem, they can’t provide that context.

Another deliberate decision we’ve made to assist with maintainability is to work in a monorepo. This means that in our commit history, related changes are often shipped together. Coupled with the practices described above, this - at least in theory - should mean that an engineer working in a particular area should be able to answer 3 questions:

1. What does this particular area of the code do?
2. Why - from a business perspective - does it do it? And why in this particular way?
3. What else had to change to make this solution workable, and

Where LLMs earn their keep

Ironically by having our humans care about and document this stuff we can provide better context to LLMs. This makes them more efficient at feature building, bug fixing and triage.

LLMs are great, and you should use them! But - in my opinion - they are best employed producing what can be machine-verified. There are still many things better left to human judgement.

An example: writing a mechanical JSON encoder and decoder to match a wire format, repeatedly and by hand, is not something any human engineer should be doing in 2026. Declining to automate that is paying a tax for no good reason. In Haskell, we have a standard practice for testing such things - a golden test with real examples to anchor you to reality, and roundtrip property tests using hundreds of randomly generated values each time to catch edge cases. Provide the LLM with the real examples and let it do the work of writing the encoders, decoders and tests (and for the record, there are better tools than LLMs for deriving correct marshalling and unmarshalling strategies for JSON wire formats from OpenAPI specifications - I merely use this as an concrete example of boilerplate code).

A rule like “all currency amounts on a sales order are in the same currency, across its lines and associated payments” is something you can - and we do - encode in our type system. This means a program that mixes currencies in one order is unrepresentable in valid code; it simply won’t compile.

(An aside for the Haskellers - here’s a sample of Haskell code showing how that’s achieved using a phantom type index on our internally-defined Discrete type, a singleton type to reflect it at runtime and an existential unwrapped at serialisation boundaries - a simplified version of a mechanism from safe-money and probably worthy of its own blog post):

newtype Discrete (currency :: Currency) = Discrete Integer

data SomeDiscrete where
  SomeDiscrete :: forall currency. Sing currency -> Discrete currency -> SomeDiscrete

-- the `currency` type argument enforces a common currency across attributes
data OrderItem (currency :: Currency) = OrderItem
  { code :: AccountingSystemIdentifier,
    amount :: Discrete currency,
    description :: NonEmptyText,
    quantity :: Natural1,
    taxAmount :: Maybe (Discrete currency),
    taxCode :: TaxCode
  }

fromBaseUnit :: forall (currency :: Currency). Integer -> Discrete currency
fromBaseUnit = Discrete

instance FromJSON (OrderItem currency) where
  parseJSON = Aeson.withObject "OrderItem" $ \obj -> do
    -- ...
    amount <- fromBaseUnit <
gt; obj .: "amount"
    -- ...

But it’s not obvious to a machine - and frankly most humans, unless they’ve done the research - why you’d want to do such a thing. For some businesses, like ours, it’s a fundamental truth because of the way we’ve set up our platform. For others, they may be far more flexible about how they take payments. And it may change in the future, which should influence exactly how you implement it. And all of those factors I just described are not the sort of thing an LLM will intuit by default (yet), unless the person operating the LLM goes to great lengths to include that in the context. By having that rule encoded - with care and deliberate design - in the type system, you’re creating an environment where you can delegate smaller coding tasks to the LLMs with confidence. Where the fundamentals of the business are unbreakable contracts the LLM has to respect.

This combination of determinism and non-determinism is where you can have your cake and eat it too. Consider the problem of wanting to have an LLM triage an email inbox and auto-respond. Emails can come from anywhere and go to anywhere, and I would lose sleep at night just plugging Claude into Gmail with a set of instructions and saying, “have at it”. What I’m much more comfortable with is having a workflow automation tool like n8n pull each email requiring triage using the Gmail API, feed the email headers and body into an LLM whose only task is to choose the action to take from a finite list, and then having n8n execute that action via the Gmail API. The humans retain complete control over the input and possible outcomes, while the LLM is doing the previously-impossible task of dealing with every permutation of email content you can throw at it and turning that into something sensible.

A tool, not a strategy

Claude has been adopted across Bellroy and most Bellrovians use it every working day. They’re using Chat, Cowork and Code, or plugging in other tools into Claude’s API, such as n8n. I spend a good chunk of my working hours each week trying to work out ways we can more effectively leverage this technology right across the company - not just in the Technology team - and at the time of writing we’re actively hiring an AI Enablement Lead (among other roles - we’re growing!). These tools are game-changing. I intend for Bellroy to remain an AI-forward organisation and we currently have more great ideas than we have people to execute on them.

What these tools have done is change the distribution of costs, and that’s going to mean different things to different companies depending on their strategy. The cost of building things - ideas, processes, software tools - has clearly gone down, and to some extent the cost of maintenance has as well. But I would argue the cost of maintenance has not reduced to the same extent that the cost of building has, and in order to effectively manage that maintenance burden you need humans to understand the why and how of the things we build. For the time being - to avoid accumulating cognitive debt - Bellroy’s human engineers will still be writing code by hand… some of the time.