A proposal: how to hire junior engineers in the age of AI

About six weeks ago I wrote We are all Product Engineers now, an exploration of what LLM-written code is doing to the job of software development. A few weeks after that I turned it into an AI Engineer New York talk, which at a quarter-million views is now by far the most-watched talk I’ve ever given.

The talk and the post both end with an exhortation: we have to figure out how to hire new engineers to do the vastly-changed job of software development now that “programming” isn’t part of that job description. This is a monumental shift and there is an entire cohort of people who got into college three and four years ago who are emerging into a job market that has absolutely no use for the skills they learned.

Since then I’ve been thinking about how, exactly, one could go about hiring junior engineers who would be good at the new way of doing things, specifically about how to interview them. This post is about my best idea so far.

Junior hiring has collapsed, and old tech interviews went with it

Back in July I noted that hiring of junior engineers has collapsed starting in roughly 2023, probably not coincidentally when AI accelerated coding took off. Things continue to get worse for them. Stanford's "Canaries in the Coal Mine" research found employment of software developers aged 22 to 25 is down nearly 20% from its late 2022 peak, while hiring of senior engineers has continued to grow. Anthropic, surely at the forefront of AI-driven software development, has said that the share of engineers it hires who are junior has fallen from 27% to 9% since 2025.

A big driver of the collapse, I believe, is that there’s no way to know if a junior person is going to be any good at being what I termed a product engineer, especially if even the limited experience they gained at school was about old-fashioned programming, algorithms and data structures and other things the agents take care of now. Hiring a junior is always a risk, but the risk is greatly magnified when we only just defined the job and haven’t figured out how to interview for it.

Our previous tool for selecting promising juniors was the coding interview. I have said for more than a decade that coding interviews are stupid but the final nail is definitely in that coffin now. People are certainly trying to find replacements for it. In a much-noted blog post, an engineering manager at Anthropic said they have stopped giving coding interviews on their team (frequently mis-reported as being a whole-company policy). Canva replaced its computer science fundamentals round with a mandatory AI-assisted one in 2025, and Meta swapped one of its two onsite coding rounds for an AI-assisted version. Google is piloting code comprehension and AI-assisted rounds.

But nobody is trumpeting a solution yet, so it’s fair to say we’re all still figuring this thing out, and the observed reality is that a lot of companies are just opting out entirely and only hiring senior developers, a totally unsustainable approach.

Hiring seniors is easier, but there are only so many of them

The argument I made in the talk is that we have to hire junior engineers or the industry will shrink and die without them, and that’s true, but that’s an altruistic, big-picture argument that doesn’t work when each individual hiring manager is trying to work under capitalism, a point others have already made. Everyone benefits from a supply of trained seniors, but everyone would prefer that some other company paid to train them, and in the current environment, figure out how to train them at all.

Luckily, a selfish, capitalism-friendly argument can be made for hiring junior product engineers, which is that they are a lot cheaper than senior engineers, who are growing ever-more expensive as 71% of the growth in software job postings between May 2025 and May 2026 called for them. They are also simply available, while senior engineers are all extremely employed already.

So if you can find good product engineers, you can get them cheaply and in quantity, and that’s good because we’re about to need enormous numbers of them. Product engineering is not rocket science. We didn’t stop making people who can do this job, we just haven’t figured out how to identify them, because the job changed so much so quickly.

Internships are not a solution to the hiring problem

One solution a couple of companies have hit on to try and deal with being unable to use coding interviews to figure out who’s going to be good at this new job is paid internships: simply admit the interview doesn’t tell you much, hire a ton of interns, and see who actually performs on the job. Shopify grew its internship program from about 100 a year to over 1,000, and Cloudflare set a goal of 1,111 interns in 2026.

That’s certainly a solution, but it is both slow and expensive and totally unavailable to startups, who lack all of the time and the money and the level of adult supervision required to hire junior developers and see if they sink or swim in six months. Most companies can only afford a few hours of interviewing per candidate. Even if you’re hiring interns, the program is going to go better if you have an interview process that selects better candidates and doesn’t just randomly pile juniors into the room.

You have to interview for something, so what?

The usual shortcuts for picking juniors are just bias in a hat and glasses

Another way is to hope that somebody else has already done it for you, by hiring from a well-known elite school. This too is already happening: the Pragmatic Engineer found that . Hiring for a fancy school is also something I have railed against in the past: credentialism is a filter that simply rewards being born with rich parents and does not predict that they will be good at any job, including that of a product engineer.

Other recently touted approaches include hiring for things like “hunger” and “taste”. I don’t know what those things mean, and neither do the people hiring for them. “Hunger” sounds like a code-word for “youth” which is a) an illegal thing to hire for and b) not helpful when all your junior candidates are roughly the same age anyway.

(Did you know IQ tests are effectively illegal for hiring in the US because they disproportionately screen out minorities? I learn things when I research these posts.)

So a lot of people are falling back on “just have a free-flowing conversation, I’ll know it when I see it” which has been scientifically measured as one of the weakest possible approaches: Sackett and colleagues in 2022 put the predictive validity of unstructured interviews at .19, compared to .42 for structured ones. Google learned this the hard way with its famous brainteasers, which its head of people operations eventually admitted predicted nothing at all.

So the interview has to be structured: everybody gets the same interview. What are we even interviewing for?

We have to interview for the actual things that make you good at the job

To try and figure out what we can actually interview for in a couple of hours, I tried to more concretely break down what goes into product engineering. Here’s what I came up with:

Product sense: given an amorphous mass of unstructured information, can you figure out what the actual problem is that needs to be solved? This involves pushing back on what the user is explicitly asking for, which is often the wrong thing, and finding hidden assumptions and constraints that they don’t mention at all.

Intelligence: you do in fact need to be a particular kind of smart to be good at building systems, so go looking for that kind of smart rather than an IQ test. Can somebody hold a whole system in their head, spot problems, and figure out what needs to change when constraints shift?

The ability to explain yourself in writing: this is always something I’ve prized in hires, but it has suddenly become most of the job. If you can’t explain yourself in natural language, you cannot prompt an agent effectively. I expect this will garner some pushback, since there are a lot of experienced programmers who are notably bad at explaining their work, but the job has changed.

The best evidence of all of these, of course, is a portfolio of existing work where a candidate has done all of these things, and then getting them to explain it to you in writing. Every time you build a system badly you figure out how to build your next system better, so the more you’ve built, the better you’ll be. But most juniors don’t have a portfolio, and selecting for the ones that do is just our old friend Rich Parents again, selecting for kids who had a lot of free time and resources.

So now we’re homing in. It has to be the same fair, job-related task given to everyone who applies. What does that look like? Here’s what I’ve come up with.

Spend an hour figuring out what to build and then build it

The old way would have been to give a developer some requirements and get them to write the software. But with product engineering, figuring out what the requirements are is most of the job.

Luckily, the same thing that wiped out the coding interview can come to our rescue: we can give a big pile of requirements, constraints and exhaustive detail to an LLM, then get the LLM to role-play as the customer.

The candidate gets an hour (or two?) with a model that has everything they need to know to build the system, and a second model (it needs to be the same model for everyone) that will write whatever software they tell it to build. Their job is to get the information out of the customer and into working software, through the interface of natural language.

In the old way, the software built would have to be something very simple to fit into the time constraints of an interview. With LLM-driven development, we can demand much more: real, working software, possibly even deployed to a supplied target platform.

There are of course some rules.

• You can keep asking the customer questions as you go. Real life works that way. You can even show them a half-finished version and see if they like it.
• You can throw away your whole approach and start over, several times if you want to. The worst product people I’ve seen pick an initial approach and then stick to tweaking it when starting over would be the right call.
• You can’t jailbreak the customer. Kudos for thinking of that approach, very Kobayashi Maru of you, but real customers cannot be tricked into dumping a requirements document to the console.

The good thing about this approach is that it can be extremely domain-specific. It can be based on the real working system, or you can just make up a brand new problem. It can be arbitrarily detailed and complex, it can have edge cases galore, it can be like the real world as much as possible, and it will reward candidates who think of edge cases, approach things sideways, and think about whole systems at once. In short, the people who are good at product engineering.

It’s also relatively cheap, because you can build another agent to score the output. You give your judge agent the full requirements and the software, and get it to build and run as many tests as it can think of and report back to an actual human for final scoring. (Is this an eval, being recommended by a guy who currently works at an eval company? You bet your ass it is. Evals are super useful, which is why I work there.)

For bonus efficacy, maybe tack on another hour where the candidate and a human interviewer discuss the resulting software: what works, what doesn’t and why.

This isn’t actually a new approach

While thinking through this approach I discovered that it isn’t actually a novel idea in the least. In the world of medicine, this has been standard practice since the 1960s. In the OSCE, medical students take a history from a standardized patient: a trained actor portraying a specific complaint, who answers consistently from student to student so the assessment is fair. Students are scored on a checklist of the questions they should have asked, which isn’t very far away from being judged on how many requirements they found in the table.

Is anybody already trying this in tech? Not that I can find. Palantir's decomposition interview has long asked candidates to break a vague real-world problem into parts with little or no code, but it doesn’t have the building part. There’s also a startup called Promptster, which gives candidates an incomplete ticket and AI teammates who hold the missing context, then shows reviewers how many hidden requirements each candidate recovered, which isn’t the same but isn’t a million miles away either.

Don’t try to automate the whole thing

You’ll note that I gave an agent the task of writing tests but that the final judgement is given to a human. That’s very much deliberate. A strong model judge agrees with humans about as often as humans agree with each other but a 2026 evaluation of 21 judges found some that gave the same answer every time while still being badly biased. Using fully automated scoring would also be against the law in some places: New York City requires annual bias audits of automated hiring tools, and you don’t want that heat.

If there is one thing I have learned working in the world of evals, it’s that your judges have to be calibrated by human judgement.

Will this be good enough?

This entire approach is, right now, theoretical. I’m not hiring for software developers right now and I haven’t yet tried to persuade anybody else to give this a trial. So I don’t know if it will work, but I’m optimistic that it’s a step in the right direction.

The company that figures this out will have a huge advantage: a ready supply of affordable, capable software developers who can just figure out what to build for themselves, just when demand for that kind of developer is exploding. If you do give it a try, I would love to hear how it goes.