Every task my fleet completes now carries its cost

In late July I want to experiment with some open models on a spare Mac Studio in my house (a symptom of the times, I know), to see what they could achieve alongside my foundation model usage.

By the end, I knew what the experiment had burned in tokens. After all, every model call from every agent I run crosses a single agentgateway process on its way to the API, so the token count was never in doubt. But what the count couldn't tell me was what the tokens had accomplished.

• Which task was the expensive one?
• Did that bug cost more than the feature that shipped it?

Tokens are a fact about my usage; what they bought is a fact about my work, and the two live in different systems on my machines. So I set out to join them in one view.

The token count answers a question I stopped needing to ask

The cost conversation still runs on volume. The vernacular has split into tokenmaxxers, and even tokenminmaxers now, arguing about which direction the token spend should go: up or down.

Luis Morales, Head of AI at Super.com, gave me a shape for why that stalls. In "We Stopped Counting Tokens" he lays measurement out as a ladder, utilization then output then outcome, and puts an expiry date on each rung:

Utilization metrics are the first rung of the ladder. You need them early. Then you need to outgrow them fast.

His test for an expired metric landed hard here: "A question that always returns the same answer has retired itself." Am I using tokens? Yes. Tomorrow: yes. I kept asking it because it was the question my tooling could answer.

The spenders and the work already had names

I borrowed both halves from Steve Yegge, who has been writing about what it is like to run a crew of agents day to day. In "Model Welfare for Agentic Engineers" he portrays what I do, as well:

Sessions are days, and seats are people.

A seat, on my machines, is a session I keep handing the same name and the same memory file, pointed at the same kind of work.

The work has names too. Every unit of it is a bead, a small linkable record in an agent-native issue tracker. The concept is Yegge's (Beads); the engine I run is beads_rust, a fast Rust port from Jeffrey Emmanuel. A seat claims a bead, works it, closes it with a reason, and the beads link into a graph that remembers what the fleet did.

So I already had two graphs, one of identity and one of work, with the gateway's receipts sitting between them.

Cost breakdown for an actual epic in beads

Here's that actual experiment, broken down by bead and cost. The goal was to find out whether a local model can hold a real coding session, write the rules for what degrades and what stops, wire the second backend, and learn what a failover does to the bill:

Each node is one task with its own record:

• who claimed it
• why it closed
• what it depended on

The epic is the parent; research, specs, decisions, features, bugs, and reviews hang under it, each carrying its own work and its own review, the way any epic you have ever scoped does.

The gateway stamps every request with a seat and a task

Each seat's launcher stamps two identity headers on its traffic, and the gateway lifts them into columns on its request log with one line of CEL apiece:

standardAttributes:
  user:  'default(request.headers["x-session-identity"], "")'
  group: 'default(request.headers["x-billing-origin"], "")'

CEL is Common Expression Language, a small rule language proxies use for jobs exactly this size; the line reads as take this header, or fall back to an empty string. Anything that cannot identify itself lands in one bucket of unknown spenders. When I started, that bucket collected 1000+ requests a week. Last week it collected only 15 unidentified sessions. Effectively all the inference is identified and labeled now, letting me do some pretty interesting observations on what the seats accomplish day to day.

The second header names the task, which is a node in a graph of tasks. The value of pivoting on this graph cannot be understated: the entire graph of planned and finished work can be pinned to each inference call, at whatever stage is the focus of the task. This allows me to not only understand what tasks cost, but also which specific parts of the task were expensive. Then you start to learn things, like failure modes you'd otherwise miss, or the model doing something that should be hardened in a script.

For each subagent, the Claude process has the ID of the bead it is working in its exports, so the header rides every call that session makes, and the gateway writes it onto the row next to the dollars it derives from token usage. On a subscription those dollars are list-rate equivalents, which is great, it shows how much value I get from my flat subscription each week (I wouldn't be able to afford this on API costs, I'll just say that).

Each bead comes back wearing its cost

The same graph, with the dollars written on. Reading it takes no spreadsheet:

• Each task's number is its own spend, the calls made under that task and nothing else.
• The epic's figure is the plain total of the tasks underneath it.
• Coordination, the sessions that dispatch and merge everything else, carries its own line instead of being smeared across the work it touched.

The most expensive task in the experiment wrote no code at all: the research that benchmarked local models against real bugs from my own backlog, at $61.80, and it decided what everything else was built on.

Seat totals show where the three weeks went

Zoom out and the ranking is predictable, at least for an AI explorer who is constantly refining his own harness. The seat that tends the fleet's tooling leads, infrastructure follows, and the whole fallback epic is a thin sliver inside each box. My invoice already knew this much. It is where I used to stop, because it was as far as my numbers went.

Filter to one epic and the leaderboard flips

Filter the same seats down to the fallback work alone and the ranking inverts:

• The infrastructure seat jumps to 1st, which is what owning the gateway route costs.
• The fleet's biggest spender drops to 2nd, and more than a third of what it spent here was coordination rather than code.

My front desk turns out to be the epic's cheapest line: two decisions and a runbook on the process. The read-only critic bought the merge gate.

This is still an early experiment, and I only run one machine on this practice. But it opens an opportunity for you, as the harness operator, to ask higher-order questions about the efficiency of the system, because the entire thing is a feedback loop. Any session can connect work to tasks to inference to the seat responsible for owning it, and reason and act on it. Some of these questions for me have been:

• Is my research desk cheap because it is efficient, or because I under-use it?
• Why is the seat that makes my decisions the cheapest line on the graph?
• What opportunity am I missing?
• What is taking up most of my time and tokens?
• Of what I completed, what had the biggest impact?

I can picture these numbers across a whole engineering org, but I haven't gotten there yet.

The taps are the layer I actually pay

Steve Yegge named this layer too. His fleet drinks from what he calls a token tap, a pool of flat-rate subscription accounts his agents rotate through instead of metered API billing, and he runs a dozen of them.

I only have 3. (😇)

Since all of the inference is labelled even by tap in the gateway, I can segment any given week to find how much of each tap was consumed by which seat:

I can split that same tap across beads as well to find which bead epics took up my time and precious tokens that week:

Put a cost on a unit of your own work

If you're a software leader right now trying to map inference to outcomes, you really need a context layer to capture this same level of insight but across all of your engineers.

What surprised me is how loose the unit of work can be when you attribute cost to a bead. Our industry tends to ask for cost per PR, but my agents spend most of their time doing work that doesn't end up as a PR. That didn't stop me from trying to valuate it, however.

I found that attributing the cost to a bead, a unit of work that can be defined as truly anything, is the right tempo of scale to match the inference pattern used by models.

And even better: a PR can neatly contain a record of all the beads that produced it, thus earning the cost per PR by adding the cumulative work inside it. Because you're tracking at the inference level, you're getting a much more accurate sense of cost anyways, compared to the PR which is only going to capture costs associated with tasks around code gen.

I had my infra agent cut up some examples for you to study below, if you want to try running this experiment yourself. Be sure to try out agentgateway and beads_rust regardless. I also wrote a tutorial on how to stand a gateway like this up that's a good place to start after learning the tools.

gl;hf! 😎

Appendix: the pieces worth copying

1: lift two headers into columns. standardAttributes is read at startup only, so this is a restart, not a hot reload. Validate first, and watch the validator reject a deliberately broken control before you trust it.

config:
  database:
    url: "sqlite:///var/lib/agentgateway/requests.db?mode=rwc"
  standardAttributes:
    user:  'default(request.headers["x-session-identity"], "")'
    group: 'default(request.headers["x-billing-origin"], "")'

agentgateway --validate-only -f config.yaml   # non-zero exit names the bad field

2: the sanitizer that makes the unknown-spender bucket structural. The filter cannot emit ?, which frees ? to mean "nothing was derived here" and nothing else.

sanitize() { printf '%s' "${1:-}" | tr -s '[:space:]' '-' | tr -cd '[:alnum:]._-'; }

seat=$(sanitize "${MY_SEAT_NAME:-}")
export SEAT_ADDRESS="${seat:-?}"      # a real name can never contain '?'

3: group by seat, last 7 days. Read a snapshot, never the file the gateway is writing.

sqlite3 requests.db "VACUUM INTO '/tmp/snap.db'"

-- datetime() is load-bearing: started_at is ISO8601 with a 'T', and a bare
-- string compare matches every row with today's date without erroring.
SELECT agentgateway_user               AS seat,
       count(*)                        AS requests,
       sum(coalesce(total_tokens, 0))  AS tokens
FROM request_logs
WHERE datetime(started_at) >= datetime('now', '-7 days')
GROUP BY seat
ORDER BY tokens DESC;

4: the bead join. The identity schema is closed at two columns, so the per-task header rides the access log's attribute JSON, which is the path that hot-reloads. The sentinel lives in the config, not the client: a launch that never heard of beads still lands as none, honestly.

frontendPolicies:
  accessLog:
    database:
      add:
        bead_id: 'default(request.headers["x-bead"], "none")'

-- per-task spend, sentinel excluded; keep the extraction parenthesized
SELECT (attributes_json ->> '$.bead_id') AS bead,
       count(*)                          AS requests,
       round(sum(cost), 2)               AS dollars
FROM request_logs
WHERE (attributes_json ->> '$.bead_id') IS NOT NULL
  AND (attributes_json ->> '$.bead_id') != 'none'
GROUP BY bead
ORDER BY dollars DESC;

Andrew Zigler is a 2026 AAIF Ambassador. Find the cohort at aaif.io/ambassadors. I wrote up how to stand a gateway like this up in July; this piece is what its receipts taught me to ask for.