LinkedIn Swapped Out OpenAI Models to Keep Its Server Costs Flat

The more people use AI features in apps, the more the companies selling those apps have to spend on servers to run them, potentially squeezing their profit margins.

But some firms have found ways to keep costs in check. For instance, Microsoft-owned LinkedIn says it has flattened the cost of running its AI-powered Hiring Assistant, which uses large language models to automate candidate search and outreach for recruiters, even as usage of the tool has steadily increased in the past year.

More than 25,000 businesses pay for Hiring Assistant, LinkedIn said late last month, up from 20,000 it disclosed in September. While the number of customers has grown steadily over the past year—each paying several hundred dollars to over $1,000 per user per month—the cost of serving the product has remained roughly flat, LinkedIn executives said in interviews.

And LinkedIn doesn’t plan to increase spending on graphics processing units in the fiscal year that ends June 2027, even as it expects an increase in the number of customers using its AI features, according to Chief Technology Officer Erran Berger.

“We can build these ever more powerful capable tools while keeping costs manageable,” he said.

That kind of cost control is critical to Microsoft, which like other large cloud providers is under pressure to cut expenses as it expands capital expenditures for data centers, denting its free cash flows.

To pull off the feat, LinkedIn swapped out general-purpose LLMs from OpenAI for smaller AI models it developed in-house through a process known as distillation, or customizing a small model to mimic the quality of a larger model within a more limited set of tasks. Specifically, LinkedIn used OpenAI’s models to “teach” smaller open source models how to automate such tasks, according to vice president of engineering Prashanthi Padmanabhan. (LinkedIn declined to say which open source models it used.)

When LinkedIn launched Hiring Assistant a year ago, it relied primarily on OpenAI models for most of its features. Now the smaller models automate features such as searching for job candidates, ranking them based on recruiters’ criteria, and drafting outreach messages to candidates, Padmanabhan said. The smaller models get similar results to what OpenAI models produce for a saving of up to 67% on what LinkedIn was paying to run the OpenAI models a year ago, she said. (Since then, OpenAI has also dropped prices for most of its models and released new models it says are more efficient; LinkedIn declined to specify whether its own models are more cost-effective than OpenAI’s current offerings.)

However, LinkedIn typically uses OpenAI models to power its newest AI features, until it can develop customized models to swap in, she said. That explains why the overall computing cost has remained steady rather than fallen.

“We still use frontier model for many of our use cases, but over time we‘ve figured out it’s not the best way to scale,” Padmanabhan said. “We want to make sure we're bringing innovation and engineering efficiency so the end to end cost stays flat even as the product grows.”

Other companies may be able to save even more money if they follow LinkedIn’s example. LinkedIn and the rest of Microsoft can use OpenAI models for free, and must only pay the computing cost of running them, thanks to Microsoft’s long-term business arrangement with OpenAI. In other words, LinkedIn’s savings came purely from running smaller, more efficient models. Companies that pay full freight to use OpenAI or Anthropic’s models stand to benefit even more by running smaller models from elsewhere.

To be sure, LinkedIn has such an advantage over other AI app developers because its agreement with OpenAI also allows it to distill that company’s models, whereas OpenAI bars other companies from distilling its models to develop competing ones.

Still, companies including AT&T, Cursor and others can customize open source models effectively without relying on distillation, we reported previously.

Thank you for reading the Applied AI newsletter! I’d love your feedback, ideas and tips: [email protected].

If you think someone else might enjoy this newsletter, please pass it forward or they can sign up here.