The Pulse: We need to talk about migrations with AI

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can .

OpenAI put an impressive-sounding case study about how they helped Asana save $5.9M with a single migration. From OpenAI (emphasis mine.)

“Asana cleared 5 years of engineering work in 2 weeks with Codex. Using OpenAI Codex, Asana replaced an outdated testing system in two weeks for about $12K.

For this project, Codex helped Asana’s engineers remove Enzyme, an outdated testing system that had made the company’s code harder to upgrade. Model and infrastructure costs came to about $12,000, compared with Asana’s roughly $6 million estimate for the previous staffing plan.

After 1.5 weeks of engineering effort spread across two calendar weeks, Enzyme was fully removed. Model and infrastructure costs totaled about $12K. For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M. The experience changed which long-running software projects the company believes are practical to take on.”

For context, Asana migrated from Enzyme to React Testing Library, which indeed would not be a simple migration. However, their estimate of four engineers (each on circa $300K/year, according to OpenAI’s arithmetic), spending five years on the project, had me like:

But thinking about this for longer raised the question: what does a migration from Enzyme to React Testing Library even look like?

Enzyme to React Testing Library migration

Let’s take a simple test and see how it looks in various testing libraries.

For our test, we want to verify that a button increments a counter. Here’s our button in React:

Now, the test to verify this, in Enzyme:

And let’s rewrite this test in React Testing Library:

The two tests do the same, but they have completely different syntax! Let’s see just how different they are, with a side-by-side comparison:

The main reason for this difference is that the two frameworks use a fundamentally different approach to testing. Whereas Enzyme is oriented towards component testing (notice how the test operates on a component instance), the React Testing Library operates on the rendered Document Object Model (a data structure representing the HTML shipped to the client) so the test sees the whole rendered page, not just the component its written for. That’s why the testing approach will differ radically between the two frameworks, especially when testing complex user journeys. You can learn more here on the tradeoffs between the two approaches from the React Testing Library author.

It took Airbnb 6 weeks to migrate 3,500 tests with AI

Last year, Airbnb revealed how they migrated their Enzyme test suite of 3,500 component test files within six weeks with LLMs. The estimate of doing this by hand was 1.5 engineering years. Airbnb did the LLM-aided migration in a multi-phase process:

Airbnb’s team had to build loops to keep retrying migrations; once they did, 75% of files were migrated in just four hours, and the migrations were straightforward. They then built a more sophisticated refactor pipeline for the remaining 25% of tests; after building the pipeline, the new loop migrated most of the remaining tests (97%) in total, after running over 4 days. The remaining 3% was done with LLM input, with engineers finishing it in a week.

This was in March 2025, when the frontier coding model was Claude 3.7 Sonnet. Today, models are a lot more capable, such as the likes of GPT-5.6 Sol and Claude Fable 5.

AI makes impractical migrations doable

On the basis that it took Airbnb six weeks, I find it credible that it took Asana two weeks to migrate what is probably a similarly complex test infrastructure from Enzyme to RTL, a year later.

The time and $6M cost as quoted by OpenAI feels inflated. I assume the numbers were based on an estimate that a fulltime engineer could do a maximum of X tests migrated per day, where X was between 5 and 10. Then, calculate the number of engineering years this takes (perhaps 20 engineering years), and multiply by the cost of an engineer. You estimate a project like this when it’s an undesirable project you really don’t want to do as an engineer!

So, looking at it from this point of view: does it matter if the estimate was 1.5 years (Airbnb) or half a decade (Asana, hypothetically)? Or if the estimated cost was $1M or $5M? It would still be an impractical migration and a foolish endeavor: too long and distracting, at least, in the “traditional” way!

Pre-AI, years-long migrations were rare. In 2021, Sentry took 1.5 years to convert their frontend codebase from JavaScript to TypeScript. That was a migration of 1,100 files and 95,000 lines of code! Around 10 engineers worked on the migration, so if we assume a $300K-per-engineer cost, that’s a $2–4M cost for circa 95,000 lines of code.

Indeed, Asana’s reported $12,000 migration cost could actually cost even less if there was ruthless focus on cost optimization. What about using a model that’s 10x cheaper than OpenAI, like an open model running on inference providers? After all, why use the most expensive model, especially if a company already owns GPUs, running models, making inference practically free (except for power costs, that is.) For a first run, $12,000 would be affordable for a company paying $300K for engineers. But for subsequent runs, I’d wager it’s worth spending time optimizing the cost, and saving $10K per migration (or more!)

A few more details from inside Asana

I managed to catch up with Dan Ubilla at Asana, who leads the Developer Productivity group at the company. He helpfully clarified a few things about the Enzyme migration post on the Asana site:

• Asana started the Enzyme migration in 2024. The team had 4,000+ Enzyme files to migrate, and they took a first stab at the migration with LLMs, at the time. Beyond migrating existing tests, much of this phase focused on getting the codebase into a state where future tests could be written more idiomatically. This included improving mocking, data set up, and coverage instrumentation. They finished migrating about 25% of the files to RTL, choosing the tests that had the highest return on investment: meaning easiest to migrate, or most frequently updated ones.
• The remainder of the migration became low priority. Asana prioritized migrations as critical (“must do soon”), important (“important to do soon”), and opportunistic (“nice to have”). The company has a good number of critical and important migrations, and the remainder of the Enzyme migration sat as one of the many opportunistic.
• Five years was about when the migration was projected to finish. The effort was not about “a team working for five years, nonstop on the project” – not at all! It was about the timeframe that the opportunistic Enzyme migration would have been completed, given its priority, and done, realistically. Given this migration was going on for two years, the five-year estimate was a rational one.
• The team wanted to prove that LLMs drastically speed up migrations, and demonstrated it with an opportunistic , low-priority one. The Enzyme migration was not all that important to do quickly, and would have sat around for years. So it was the perfect candidate to prove that with LLMs, long migrations can be sped up!

And finally, one addition from my end:

The $6M cost was a back-of-the envelope estimation. I asked Dan how this estimate of the migration costing $6M came together. Dan confirmed that the estimation was done the same way as most of us do these estimations:

• Estimate how long it takes an engineer to manually migrate a single Enzyme file (including rewrite + validation). Be generous with this estimation
• Multiply it by the outstanding files to migrate (the 3,000+ ones)
• Add in the time to remove any traces of the framework, monitoring, static analysis tools
• Multiply by the hourly rate of a typical engineer
• … and the number came out to $6M

This estimation does not take into account that migrating even ten files by hand takes less than ten times more than it takes to migrate one file (you become more efficient), nor does it assume LLM usage. It’s a baseline to get a sense of how the work would have been done, pre-AI, if distributed across all Asan engineers to pick up, and migrate few files, whenever they have time. My take is that this number is probably an overestimation, but the point is less about the number, and more to convey that this is a lot of work!

Internally, Asana’s team and leadership are now convinced about the usefulness of LLMs for migrations, and they’ll be using them for other, long-running, otherwise soul-sucking migrations.

Expect long-avoided migrations to finally happen

A few months ago, Uber shared that they executed a massive JUnit migration in four months with two engineers and AI: moving 600,000 unit tests spanning 15 million lines of code (!!) by moving from the unsupported JUnit 4 to JUnit 5. During the migration, 1.25M lines of code were modified. This type of migration used to be impractical; with AI tools, it took eight engineering months of effort, plus AI costs.

And there’s the (530,000 lines of code from Zig to Rust in two weeks for a $165K API cost) as another example of drastically faster migrations with AI.

The best part about rapid, AI-assisted migrations is not needing to worry about supporting “old” libraries. One of the main reasons library migrations have been such a pain was the need to keep supporting the old library or technology during the migration. Shortening this window could well be worth the additional cost, at least until we figure out how to drastically reduce the time and cost of these migrations, as well.

The shared characteristic of all of the above migrations is that engineers needed to plan for it, design verification loops, and be involved throughout. I always dreaded migrations, so I see it as good news that we have a new tool to use for “grunt work” like this.

Read the full issue of last week’s The Pulse, or check out . This week’s issue covers:

1. Did Meta really decide to reduce team sizes by 60% because of AI? An in-depth report by Reuters details how Meta’s leadership decided to slash team sizes by 60%, hatching plans in January to execute the social media giant’s largest-ever layoffs. But Mark Zuckerberg changed his mind at the last second, and now the company is stuck with all-time low morale, a wave of resignations, and its prized culture turning mercenary. Analysis.
2. More thoughts on Ramp’s in-house AI infra. Building AI tools in-house is akin to an “internal bootcamp” for AI engineering, which probably justifies the exercise at most tech companies.
3. Industry Pulse. GitHub’s load increase is speeding up, Stripe acquires OpenRouter, Ramp launches Ramp Router in public, a career “double boomerang”, App Store revenue drops for the first time ever at Apple – and what is a bug?