I audited 123 AI-generated designs. 74% failed real users

I audited 123 AI-generated real product designs. 74% failed real users.

I hunted 165 github repos and audited 123 of them to figure out what you need to know as a design engineer

AI coding tools have quietly become a default way screens get built. You describe what you want, the tool hands you a working interface in seconds. It looks clean. It runs. Someone ships it.

Not “does it look good in the preview,” but “would this survive a screen reader, a keyboard, or someone who’s turned animation off because it makes them sick.”

So I measured it. I pulled 165 real, publicly shared projects built with AI tools like v0, Lovable, and bolt, and ran the same automated check on every one of them: no committee, no human reviewer, just a deterministic tool applying the same rules every time. A project counts as having failed real users when the check flags at least one problem serious enough to genuinely stop someone: a screen reader user missing a label, a keyboard user who can’t reach a button, someone with motion sensitivity getting animation they explicitly turned off. Not a style opinion. An actual barrier. 123 of the 165 had enough of an actual interface to run the check against. That 123 is the denominator behind every number in this piece.

If you’re picturing “AI makes ugly interfaces,” that’s not the finding. It makes visually competent ones. What it forgets is the part you can’t see in a screenshot.

What I actually checked

I wasn’t grading taste. The check looked at eight things every interface has whether anyone designed for them or not: color contrast, screen-reader accessibility, typography, spacing, form design, navigation structure, motion behavior, and microcopy. Same rules, every project, no human judgment call in the loop. That’s what makes the numbers checkable run it again on the same 123 projects and you get the same answer, every time.

74% of the projects had at least one problem serious enough to genuinely fail a real user. Not a nitpick an actual barrier for someone trying to use it.

But don’t read that as “AI-generated interfaces are disasters.” Look at the full shape of the data, not just the headline number. Just over a quarter of projects, 26%, turned out completely clean. Another 37% had only one or two blocking issues. That’s 63% of the sample sitting at clean or nearly clean. The problems compound from there: 15% had three to five issues, and the remaining 22% had six or more, with a long tail running up to a single project carrying 194.

The typical (median) project had exactly one blocking issue and eight findings total once you count the smaller stuff. It’s that long tail, not the typical project, that drags the average up to 6.4 which is exactly why the median, not the average, is the number to trust for what a typical AI-generated project actually looks like.

The defect worth watching

Here’s the part that should reframe how you review AI-generated work. Of every issue the check found, the single largest category wasn’t visual. It was accessibility 48% of every issue found, more than every other category combined.

52% of projects had accessibility problems serious enough to genuinely fail a real user. Count the smaller warnings too, and 98% had at least one accessibility issue of some kind.

To be precise about what “accessibility” means here, because vague accessibility claims are exactly the kind of thing everyone’s learned to distrust: these are structural problems, the invisible skeleton a screen reader depends on. Here’s exactly what broke, ranked by how often it showed up across the 123 projects, each with the plain-language fix:

These are exactly the failures that look perfect on screen and are broken underneath. You would never catch them by looking.

The motion nobody asked for

The most surprising finding wasn’t accessibility, it’s well documented industry-wide. It was this: 35%, roughly one in three, animated without checking whether the person viewing the page had asked for less motion.

Most operating systems let people turn on a “reduce motion” setting, for migraines, vestibular disorders, or plain preference. A third of these interfaces animate regardless, as if that setting doesn’t exist. It’s one of the cheapest fixes in this entire report a single, small check an engineer can add in minutes and one of the least discussed. Worth putting on your review checklist specifically because it’s so easy to forget and so easy to fix.

Accessibility, motion, and contrast are the three categories serious enough on their own to genuinely fail someone using the interface. Four more categories don’t reach that bar, but they showed up constantly as warnings and one of them is worth calling out by itself: 95% of projects had weak or generic copy, things like “click here” calls to action, unexplained jargon, placeholder-sounding labels. Navigation issues turned up in 46% of projects, form issues in 45%, and typography inconsistencies in 31%. None of these genuinely fail someone by themselves. All of them are the kind of thing a person notices immediately and a generated interface doesn’t.

What also held up

Color contrast was mostly fine only 12% of projects had contrast problems, likely because most of these tools default to design systems like Tailwind, which come with accessible contrast built in. Spacing was clean across the board too.

So the story is AI generates visually competent ones. not credible ones.

What’s missing is the layer you can’t screenshot.

The 74% figure holds as the sample grows, and holds after removing the worst-performing projects.

I found out Im not the only one…

Weeks before this piece, Smashing Magazine argued that accessibility needs to become an operational capability rather than a one-time audit, precisely because AI-assisted development keeps generating new surface area to break faster than periodic reviews can catch it. The example they opened with could have come straight out of this dataset: a checkout button labeled “Pay Now” that was actually a

with a click handler bolted on, invisible to a screen reader, caught only after a blind customer tried to complete a purchase and couldn't. They cite Y Combinator's finding that a quarter of its Winter 2025 startup batch had codebases that were 95% AI-generated, and the same WebAIM 2026 number used above.

Evil Martians reached a similar conclusion from a different angle. Their open-source Storybook Workbench renders every component state in a product and audits it for exactly the kind of debris this study found: dead code and accessibility bugs a normal build pipeline never surfaces. On one internal app, it caught 31 dead components and 6 accessibility bugs in a matter of hours. I flagged this first in my own AIUX Daily newsletter, alongside a separate piece on where Figma’s own canvas renderer had been quietly skipping the browser’s accessibility tree entirely.

A more controlled version of the same test showed up this year in a peer-reviewed benchmark in Universal Access in the Information Society: GPT-4o, Copilot Pro, Claude 3.7 Sonnet, and Grok 3 were each asked to generate HTML, then the output was checked by hand with a keyboard and a screen reader against an accessible baseline. The gap they found lines up with what shows up here.

A genuine counterpoint I could find, since the picture isn’t unanimous. A separate comparative study found some models, GPT-4o and Qwen2.5-Coder among them, actually produced more accessible markup than the human-written code they were benchmarked against, though prompting techniques like self-criticism introduced their own new failure modes around ARIA attribute uniqueness. Im not sure these days I dont believe studies that push some agenda, thats why I conducted my own :) The honest read: this is prompt- and model-dependent, not a fixed law of nature, which is exactly why a check like this one is worth running on your own stack rather than assuming the answer either way.

What this means for you as a designer

If you’re the one reviewing what an AI tool just generated, the way you review it has to change.

A year ago, “does this look right” covered most of the job. Glance at the layout, check the spacing, click through the flow, sign off. This data says that instinct now only covers the part these tools already get right on their own contrast and spacing barely ever break. Signing off on “looks right” tells you almost nothing about whether the screen actually works, because the four things that do break (a field with no label, a button that isn’t really a button, a page with no landmarks, headings that skip levels) are invisible in a screenshot and invisible in a click-through demo. You have to go looking for them specifically, every time, because the tool isn’t going to flag them for you.

That’s a real shift in what you’re responsible for catching. It used to be reasonable to assume a developer’s habits covered the accessibility basics without anyone having to ask. When an AI tool writes that structure instead, that habit doesn’t exist. Nobody is quietly doing the thing you used to be able to take for granted. If you don’t check it, it doesn’t get checked — and in this sample, 74% of the time, it wasn’t.

Here’s a pdf that you can also use in your own designs, prompt your ai design tool to follow them based on the severity so you know what to fix first before you hand it to the developer

Practically, this isn’t a new discipline to learn from scratch. It’s a specific, short list added to every design review, the seven items in the checklist below, ranked by how often they actually broke in this study, plus one line for motion. A few minutes of checking, once you know what to look for.

None of it requires you to write code. It requires knowing what to ask an engineer, or an AI tool’s next prompt, to fix. That’s a different skill than “produce a beautiful comp,” and it’s the one this data says is becoming the actual differentiator.

The genuinely good news buried in this data: the failures are systematic. The same things break, project after project, not randomly. Systematic problems can become a checklist, and a checklist is something you enforce every time instead of catching by intuition when you happen to notice.

How this was measured, briefly

165 public projects that identify as AI-generated (built with tools like v0, Lovable, and bolt, pulled from more than 150 different accounts so no single builder dominates the sample), of which 123 had enough of an interface to analyze that 123 is the denominator for every percentage above. The check itself is automated: no model in the loop, no subjective call, same input in, same result out, every time. That’s what makes it reproducible instead of one team’s opinion, and it’s why the stability and outlier checks above were worth running before publishing a single number. Every one of the 123 projects, and what it scored, is public: see the full list and check any of them yourself.

One honest limitation: this was a static, automated check. It can catch a missing label or a skipped heading. It can’t fully replace a person actually navigating an interface with a keyboard or a screen reader, the way a full manual review would.

Two more, from the same place as the rest of this piece. The sample is self-selected toward projects that publicly advertise being AI-generated, which skews toward landing pages, portfolios, and smaller apps rather than complex internal tools. And I didn’t compute a per-tool breakdown, so this piece makes no claim about whether v0, Lovable, or bolt individually perform better or worse than the others.

We spent a decade making interfaces beautiful. The next decade of AI-assisted work asks something else of us too: catching what these tools can’t see, because increasingly, that’s the part only a person will check.

If you want to run this same check on your own AI-generated screens, the tool is free and open. The full study, with all the charts on this page plus the complete methodology, lives at designwithclaude.com/design-research/ai-generated-frontends. And if you work alongside engineers using AI coding tools day to day, this is exactly the review dwc is built to enforce automatically, before anything ships.

I run aiuxdesign.guide, a free AI UX pattern library and audit tool, and track what shipped in AI product design every day at aiuxdesign.guide/news.

Sources & further reading

• The full study, with methodology at dwic — Im the one building it
• All 123 projects and their scores, so you can check any of them yourself
• The WebAIM Million 2026 by WebAIM
• The European Accessibility Act comes into effect by the European Commission’s AccessibleEU Centre
• Why Accessibility Is An Operational Capability, Not A Feature by Smashing Magazine
• Storybook Workbench: audit vibe-coded UIs and find hidden bugs in hours by Evil Martians
• My AIUX Daily issue covering it, alongside Figma’s own accessibility gap
• Can generative AI create accessible web code? A benchmark analysis in Universal Access in the Information Society
• Human or LLM? A Comparative Study on Accessible Code Generation Capability (the counterpoint)
• Is WCAG Compliance Enough? by Vispero

I audited 123 AI-generated designs. 74% failed real users was originally published in Bootcamp on Medium, where people are continuing the conversation by highlighting and responding to this story.