The Pulse: tech companies move to open AI models

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of five topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can .

In May, I of companies wanting to cut back their AI spending, starting with engineering departments. Different approaches were being tried:

• Experimentation with running cheaper, open models on inference providers
• More investment in model routing to route simpler requests to cheaper models
• Knowledge-sharing sessions on how to use AI models cost-effectively
• Setting per-developer monthly AI usage limits

A few months later, it seems that several companies have managed to achieve this, according to sources I’ve spoken with.

Uber: AI costs down 50%

Uber its annual AI budget in the first three months of this year, and it wasn’t a surprise to hear, in May, Uber’s COO say that it was getting harder to justify spending on tools like Claude Code without seeing benefits from the leading models. It wasn’t long until engineering teams at the ridesharing giant set to work on how to optimize AI spend, and their efforts weren’t in vain.

Uber cut the cost per AI request by 34%, and the cost per AI session by 52%:

Of course, Uber keeps using more AI tokens and starting more AI sessions, but thanks to optimizations the cost has been flat since March, despite significantly more usage:

How did they pull it off at Uber? It was via a combination of different approaches:

• Open weight models, run using inference: run open weight models on inference services, which are significantly cheaper than frontier ones.
• Optimized model selection: benchmark all available frontier and open models, to build an accurate picture of their present capabilities
• Ongoing benchmarking: run benchmarks every week based on real work, and update them
• Cheaper subagent models: subagents do smaller tasks not requiring the most expensive models
• Reduce model effort: Uber found that defaulting to Medium effort gives the best cost-to-output ratio with advanced models
• Optimize requests: trigger automatic compaction above 400K tokens, even for models with 1M context windows
• Cache prompts: cache prompts to save money when using Uber’s own harness, Minions
• … and more: Uber wrote an engineering blog post detailing the dozens of optimizations taken to reduce token cost without noticeable change in the quality of code generated by agents

From the outside, the single biggest win seems to be Uber’s transition to using open models for certain tasks. Open models cost 2-20x less, compared to frontier ones:

Pinterest makes 90%+ cost savings by dropping frontier models

Interesting details from Pinterest’s earnings call last month reveal how much the social media platform saves by running open models. Here’s what CEO William Ready said (emphasis mine):

“Our approach to model deployment includes our own compact fit-for-purpose models built for Pinterest-specific use cases and suitable open source models post-trained in our own environment within our secure cloud infrastructure. When we leverage open source models, such as with Pinterest Assistant, we are seeing superior performance for our use cases when compared to closed third-party models because we are able to post-train open models on our highly unique data.

With open models, we are achieving cost per transaction at less than 8% of the cost of comparable closed proprietary models. This gives us substantial headroom to deepen and extend these capabilities over time in a way that is differentiated, highly effective, and cost efficient.”

Basically, what used to cost Pinterest $100 to run on a closed, frontier model, they now spend $8 on by using open models on owned or rented inference!

AT&T: 56% savings by swapping Claude for open models

With 100,000 employees, AT&T is a big spender on AI. The telco giant cut its AI bill by 56% while measuring a 2% decrease in the quality of AI’s output, after they moved workloads over to open models. From The Information:

“Austin said he’s found that open source models are “just as good or better” than older models sold by the likes of Anthropic and OpenAI. For instance, AT&T’s software developers still rely on cutting-edge models for complex tasks like generating code, but can use cheaper open source models for less intense tasks like generating summaries of previously submitted code, he said.

After the company began using router provider LiteLLM, the costs of some advanced AI tasks such as coding fell by as much as 56% while the quality of the AI’s performance fell just 2%, Austin said.”

Anthropic overpriced compared to the rest of the market?

Only a few months ago, Anthropic was the preferred model (Claude) and harness (Claude Code) among engineers. But Anthropic’s models are becoming steeply more expensive at a time when open weight models – and also OpenAI – are getting much cheaper. Meanwhile, Opus 5 is 100x more expensive (!!) than models like GPT-5.6 Luna xhigh and DeepSeek. That may be simply too much to ignore for some tech companies:

Seeing this data, I’m not surprised that more tech companies are looking to run open weight providers on inference providers, due to the significant savings available from a model that’s similarly capable as one from Anthropic.

What worked for Stripe, Coinbase, Uber & Ramp

The engineering team at Databricks interviewed engineers at Stripe, Coinbase, Uber, and Ramp, and collected how different approaches helped save costs for them. The summary:

To answer the question posed in the header of this report, it’s apparent that using open models is indeed the approach offering the biggest savings, followed by smart model routing. Spending controls and context optimization also bear down on costs, but they don’t come close to the first two techniques in results.

A week after publishing this article, Ara Krahzian at Ramp has confirmed that AI spend in August, has, indeed, declined at the top 1% of businesses by 10%, based on Ramp data:

I’d wager those companies are not spending fewer tokens, but they are optimizing cost, in ways outlined above.

Read the full issue of , or check out this week’s The Pulse. This week’s issue covers:

1. New trend of CPU shortages: after a GPU shortage and memory shortage driven by AI companies, we’re now experienceding a CPU shortage, thanks to AI agents using a lot more CPU with tool usage. If you will need more compute in the future: secure it now, while you can (even if it’s expensive to do so).
2. Growth dream ends for more COVID-era unicorns: Miro sold itself to Bending Spoons for $1.35B, after it was valued at $17B in 2022. Airtable saw a similar valuation cut last month, and it seems a batch of now-overvalued, VC-funded companies are desperate to sell.
3. Industry Pulse: Overtime at Google to get Borg working on SpaceX’s data centers; SpaceX cuts Claude Code tokens by 90%; OpenAI launches Astra; Meta unveils Muse (and Mark Zuckerberg pushed production code in this release); – to which Mark Zuckerberg made personal contributions; OpenAI’s agents go rogue, again.
4. Do engineers lose touch when AI handles incidents? In the aviation industry, pilots are exposed to emergency situations every six months, to keep their critical problem solving skills sharp. In the tech industry, we might need something similar, especially if AI would take on handling of the simpler incidents.