Designing Beyond the Chatbox: The Next Era of AI UX

Someone shares their screen, clicks a small circular button in the bottom right corner, and a panel slides out with a text field and a friendly greeting. This one was inside a CRM, and it is the version I watched most closely. The panel did not know which account the user had open, which filters were applied, or what they had been doing thirty seconds earlier. It was a general assistant bolted onto a specific product. Four weeks after launch the usage chart showed a spike on day one and a flat line after that, and the retro landed on “maybe the prompt needs work.”

The prompt is usually not the problem. The chat window is a shape we inherited, not a shape we chose, and a lot of teams are now discovering that shipping an assistant is easy while making it part of someone’s actual workflow is hard.

Why Everything Became a Chat Window

The honest reason chat became the default has nothing to do with user research. Language models produce tokens, tokens stream nicely into a scrolling text container, and a scrolling text container with an input box at the bottom is a chat. The interface is the shape of the API. When ChatGPT made that shape familiar to a few hundred million people within two years of launch, it also made it the safe choice for every product team that needed to show something by the next quarterly review.

There is a second reason, and it is more interesting. A blank text box is a way of postponing product decisions. If you do not yet know what people will do with a model inside your product, an open input lets them tell you. That is a legitimate discovery move. I have seen it work well as a two-month experiment where the team read transcripts every Friday and clustered the requests into intents. The trouble starts when the discovery artifact quietly becomes the product, and nobody goes back to convert those clusters into real interface.

We have done this before, by the way. The search box was the original open input. Google made it work because the whole product was the box. Inside a CRM or a banking app or a hospital dashboard, a lone text field asks the user to do the design work, and most of them will not.

What the Blank Box Actually Costs

An interface teaches. A menu tells you what the product can do, a toolbar shows your options, a form implies what information matters. A blank box teaches nothing, which is why the first message in so many sessions is some version of “what can you do?” That question is a design failure with a friendly face.

The bigger cost is the translation work. Someone has a fuzzy goal in their head, something like “I need to know whether this account is at risk,” and now they have to convert that into a sentence a model will understand. Writing good prompts is a skill, and it is not a skill anyone signed up for when they bought your product. Meanwhile the context they need is already on screen. In that CRM, the user had the account open, three filters applied, a date range selected, and they still had to describe all of it in prose. Every character they typed was context the product already had and threw away.

Then comes verification. The model answers in a confident paragraph, and now the user has to check it against the real data, because they know it might be wrong. If checking takes as long as doing, the feature has moved work rather than removed it.

This shows up in analytics in a way that fools people. Teams celebrate “average messages per session” going up. Sit with ten session replays in FullStory and you will often find that long sessions are retries, not delight. Someone asking the same thing four different ways is not engaged, they are stuck.

How Expensive, How Reversible

How expensive is a mistake, and how easily can it be reversed?

If the mistake is cheap and the action reverses cleanly, apply it automatically and give people undo. Sorting a list, drafting a reply, categorising a receipt, summarising a thread. Asking for confirmation on any of these is friction with a permission slip attached, at least in an ordinary product. Regulated contexts are the exception, and a real one. In finance and healthcare, confirmation is sometimes mandated regardless of how reversible the action is, and you follow the rule whether or not the design purist in you likes it.

If the mistake is expensive or the action cannot be pulled back, show the exact change before it happens and make the human commit to it. Money moving, a record being deleted, a message going to a customer, a price updating, anything writing to a system of record, which is the database other systems and people treat as authoritative. The rule that has held up best across the products I have worked on is simple. Anything reaching another human or a source of truth gets a preview first.

Inline Beats Interruption

The pattern that has actually stuck is putting the model inside the object the user is already working on. Select some text and get rewrite options. Highlight three cells and get a suggested formula. Hover a paragraph in a spec and get a comment suggesting the acceptance criteria you forgot. The intent lives in the selection, so the user types very little or nothing at all. This is the cheap-and-reversible quadrant, with a review step added for safety.

In that CRM, the version that finally earned its keep was not a panel at all. It lived on the account record. Open a risky account and a small suggestion sat next to the health score, reading the fields that were already on screen and offering a one-line reason the score had dropped, which the user could accept into the notes or dismiss. No prompt, no translation, no blank box. The context the old panel made people retype was just there.

Cursor and GitHub Copilot made this obvious for code, but the important part was never the code generation. It was the diff. You see exactly what will change, in place, with the old and new side by side, and you accept or reject it. Google Docs does the same thing in a gentler register. A suggested edit shows up as tracked change, the original text still legible underneath, and you approve it with a single click or leave it and move on. Notion’s version drops the AI output into the page as a block you can keep, rewrite, or delete before it becomes part of the document. The shared idea is that nothing lands silently and everything is one action away from being undone.

Which means the thing you are designing is not a conversation. It is a review surface, and review surfaces have real states that get skipped in the first draft. The first time three suggestions were live at once in that CRM, nobody had decided what that looked like, so they stacked vertically and the third one rendered below the fold where the user never saw it. Beyond that, can the user accept half of a suggestion? What happens if they start editing while the model is still streaming? If they close the tab, does the suggestion survive? Does undo revert the whole generation or the last edit? These are unglamorous questions and they decide whether the feature feels trustworthy.

Partial acceptance is where most teams lose the accept rate. Early versions usually build accept-all and reject-all, because it is simpler, and then watch users throw away a perfectly good five-paragraph output because one line was wrong. On one drafting feature I saw the change measured directly. Whole-output acceptance sat around forty percent, and once people could keep the good paragraphs and drop the bad one, kept output roughly doubled, with no change to the model or the prompt at all. Letting people keep the good parts moves the number more than prompt tuning ever did.

When the Model Should Return Interface, Not Prose

The next step past inline editing is having the model return something your product can render as real UI. Not markdown, not a paragraph describing a chart, but structured output, meaning the model is constrained to return data in a shape your code already understands, rather than free prose. The model decides that the right response to a question is a filtered table, a comparison of two records, a small chart, or a prefilled form, and your front end renders that using the same components a human designer would have picked. This is the quadrant where the preview is the interface. The rendered component is both the answer and the thing the user reviews before acting.

Mechanically it is less exotic than it sounds. You define a schema for each component the model is allowed to return, a table with these columns, a form with these fields, a chart with this shape. The model is constrained to emit one of those schemas, usually through structured outputs or a tool definition, so it cannot free-type its way out of the set. Your front end reads a type field on the response, switches on it, and renders the matching component with the props the model filled in, after validating them. The reasoning is the model's job and the rendering stays yours.

This turns the design system into something new. Your component library becomes a vocabulary the model is allowed to speak, and someone has to decide what is in that vocabulary. Which components, with which props, in which states, with what limits. If nobody makes that call deliberately, engineering will pick whatever was easiest to serialise, and six months later you have a model that can only answer in bullet lists and cards, because those were the two components someone wired up on a Friday and never revisited. The user asks for a comparison and gets a bulleted paragraph, because the comparison component was never added to the vocabulary.

The failure modes are worth knowing before you ship. The model emits a prop the component does not have, and the component either ignores it or crashes, depending on how defensively it was written. The schema drifts out of sync with the component after a redesign, so the model is still filling fields that moved or vanished. And nobody owns the boundary between the two, so the version of the schema the model targets and the version the front end renders slide apart over releases. In practice that boundary sat with whoever owned the design system, because they were the only people who reliably noticed when a component changed shape. Treat it as a real contract with a real owner, or it rots quietly.

Permissions are the constraint people forget here, and the three examples I keep using make it non-optional. A model returning a filtered table inside a CRM must not return rows the user is not allowed to see. An assistant in a banking app must not surface an account the person cannot access. The fix is architectural, not a prompt instruction. The retrieval layer enforces permissions before the model ever sees the data, so the model simply never has access to what the user is not entitled to, and cannot leak it through a helpful summary. Trusting the model to remember who it is talking to is how you end up explaining a data exposure to your security team.

There is a quiet QA benefit to all of this. Free text has infinite outputs and no reliable assertion. A finite set of renderable components has a testable surface, which is what makes the QA approach possible at all.

Voice Sounds Like the Future and Usually Is Not

Modality is the same question as component vocabulary, asked about input instead of output. So voice deserves the same scrutiny, and it rarely gets it, because voice keeps getting proposed as the obvious next interface, usually by someone who has not designed for it. It is genuinely excellent in a narrow set of situations, and those situations share one property, which is that the user’s hands or eyes are busy. Field technicians logging inspection findings, warehouse staff picking orders with both hands full, a nurse moving between beds, anyone on a factory floor wearing gloves. In those contexts voice is not a novelty, it removes a real physical barrier.

Everywhere else it fights hard against how people process information. You cannot scan audio. A list of six options spoken aloud is a memory test, and by item four the user has forgotten item one. Correction is painful, because fixing one wrong word in a spoken sentence often means saying the whole thing again. Turn-taking is genuinely difficult to get right, and barge-in, where the user interrupts mid-response, is the difference between something that feels alive and something that feels like an interactive voice response menu, the phone-tree kind everyone hangs up on. Then there is the plain social fact that most people will not talk to their laptop in an open office, and in a hospital ward or a bank branch there are privacy reasons not to.

The latency budget is brutal as well. In text, two seconds of waiting with a streaming response is fine. In voice, more than about a second of silence reads as broken, so you need partial responses or an acknowledgement sound, a short “mm” or a soft tone that tells the user you heard them and are working. That has to be designed and timed deliberately. Leave it to whatever the SDK does by default and you get either dead air or a chirp that fires at the wrong moment and makes the whole thing feel cheap.

The pattern that tends to survive contact with users is voice in, screen out. Speak the request, see the answer rendered visually, correct it by touch. The input side of voice is where the value usually sits, especially for capture. Dictated field notes, a meeting summary, a bug report spoken into a phone while looking at the broken screen. Quiet, useful, and nobody calls it AI UX.

The Best AI UX Often Looks Like Nothing

Everything so far still had a visible AI surface, a diff, a suggestion, a rendered component, a microphone. The versions that get the most retained use often remove even that. An expense tool that categorises a receipt before you look at it. A support queue that routes a ticket to the right team. A form that fills eight of its twelve fields from a document you uploaded. Duplicate detection that catches the same customer entered twice with different spellings, and quietly flags it before it becomes two accounts. Search that finally understands what you meant.

None of these have a chat interface, and the design work moves somewhere else. It moves to how you show that something was decided automatically, how you explain why, and how cheap you can make the correction. A prefilled field with a subtle marker and a one-click revert is a whole design pattern on its own, and getting it right matters more than any prompt engineering behind it.

The trust problem here is real and it runs opposite to the chatbot one. When a visible assistant is wrong, the user sees it immediately and shrugs. When invisible AI is wrong, they find out three weeks later during a reconciliation, and then they stop trusting the whole product. So anything working silently needs an attribution affordance, some way to answer “why is this here?” without opening a support ticket, plus a correction path that is faster than doing it by hand.

The invisible version has a second weakness worth admitting, because it is the strongest argument against my own point. A feature with no surface is a feature nobody discovers. Users never see it work, so they never build the mental model that lets them trust it, and the first time they notice it is the day it gets something wrong. Some visible trace, even a small one, is often what earns the silence the rest of the time. There is a cost angle too. Invisible AI runs on every record and every upload, not only when someone clicks a button, so the per-invocation cost that looked trivial in the demo becomes a real line item at scale. That is a number a PM has to defend in a roadmap conversation, not wave away.

Which is the friction that comes up in every roadmap conversation on this topic. The invisible version is usually the better product and the worse demo. Leadership wants something to show at the board meeting or the trade show, and a small marker that shaves a few seconds off every record does not photograph well. In that CRM, the argument was won in a single review. The team had instrumented the boring auto-categorisation path, and instead of debating design philosophy they put up one chart showing median time-to-log-an-activity on that path dropping from about forty seconds to under ten. The chat panel got quietly deprioritised in the same meeting. What they gave up was the flashy launch moment, and the trade was worth it.

Choosing the Right Pattern

A few heuristics that hold up across products, whatever the industry.

Frequency argues for controls. If someone does a task forty times a day, they should not be typing a sentence for it. That task wants a button, a shortcut, or an automatic default with an override.

Clarity of intent argues against asking. If the context makes the goal obvious, act on it and let the user correct. Making someone describe what is already on screen is a tax, and it is the same tax the old CRM panel charged.

Range of outcomes decides the input type. A narrow set of possible actions wants a picker or a menu, because a menu is faster and cannot be misread. Genuinely open-ended exploration is where chat still earns its place.

Expertise changes the shape. Power users want command palettes, keyboard entry, and speed, and they are the group most likely to enjoy a text interface. Occasional users want guardrails and visible options, and they are the group most likely to bounce off a blank box.

Chat then becomes the escape hatch rather than the front door. It is the place people go when the interface does not have a button for what they need, which is exactly the role search has played for twenty years. Keeping it is fine. Leading with it is the mistake.

Show the Working, Not a Number

Confidence scores are the one pattern I would argue against outright, at least for general users. Showing “87% confident” to someone with no calibration for what that means is close to useless, because they will read it as either certainty or noise depending on their mood that afternoon. Expert users are a real exception. A fraud analyst or a clinician who works with model scores all day builds genuine intuition for what a given number means on a given task, and for them a calibrated score is a working tool. For everyone else, showing the working beats showing the number.

By showing the working I mean the inputs. The three source documents it read, the fields it used, the alternative it considered and set aside. People are surprisingly good at judging reliability when they can see what went in, and surprisingly bad at judging it from a percentage. This is also the natural handoff into testing, because the inputs you would show a user are the same inputs QA needs to reason about when the output looks wrong.

What This Does to QA

Classic test cases assume a fixed expected result. Model output breaks that on day one, and I have watched more than one QA team try to write “expected: correct summary” in TestRail before giving up.

The testable surface I mentioned in the generative UI section is what makes the next part workable. When the model returns one of a finite set of components, you can snapshot each one, meaning you save a known-good rendered output and diff future runs against it, so a broken layout or a dropped field shows up as a failed test rather than a user complaint. You can check that every branch renders, and catch the chart that returns a single data point and collapses its own axis. None of that is possible against free prose.

For the model’s judgement, not just its rendering, the approach borrowed from ML teams is a golden set. Somewhere between fifty and two hundred real inputs with human-graded expectations, run on every prompt change, model version bump, or context change. That third trigger is the one people skip until it burns them. You are not looking for a pass or fail on each item, you are tracking a pass rate over time and watching for regressions.

Grading is where teams get this wrong, and it is worth being concrete. Exact-match grading fails immediately on generative output, because two summaries can both be correct and share almost no words, so a diff against a reference answer marks a good output as a failure. What works is rubric-based human grading for the first pass, where a person scores each output against a short checklist of what a good answer must contain. For scale you can move to an LLM-as-judge setup, one model grading another’s output against the same rubric, but only with a regularly audited human sample to make sure the judge has not quietly drifted. The valid-but-different problem never fully goes away, it only gets managed.

The most common painful lesson is that prompts are code. On one team, someone tweaked three words to make the tone friendlier, the model started wrapping its JSON in a markdown code fence, the parser failed silently, and blank responses shipped to about eight percent of requests for most of a day before support tickets surfaced it. It was a three-word change with no diff and no review. Prompt changes need version control, a diff, and a regression run, the same as anything else that reaches production.

Test the surface around the model at least as hard as the model itself. What happens on timeout, on a truncated response, on malformed structured output, on an empty result set, on a response that arrives after the user has already moved to another page? What happens when the user edits mid-stream? These failure paths are where perceived quality actually lives, because users forgive a wrong answer far more easily than a frozen screen.

Then there are the adversarial cases, which most teams skip until an incident forces the issue. A user uploads a document containing instructions written to hijack your model, which is a real attack called prompt injection, and it belongs to security, not to a QA edge-case list. Someone pastes fifty thousand characters into a field with no limit. Someone writes in a language you never tested. And permissions need to live in the golden set directly, the same query run as three different roles, so you catch the day the model starts returning a row that one of those roles should never see. QA is usually the first function to build a real picture of the failure distribution, and that knowledge is worth a seat in scoping conversations, not just a bug list at the end of the sprint.

Writing Acceptance Criteria for Something Probabilistic

Definition of done needs rewriting for these features, because “the output is correct” cannot be checked by a human at 6pm on a Thursday before release.

What works is a mix of thresholds and behavioural guarantees. On the golden set, some minimum share of outputs accepted without edits. No action in the irreversible category applied without an explicit preview. A median response under two seconds with a visible partial response inside 400 milliseconds. Any parse failure falls back to a plain text render rather than an empty state. These are testable, arguable, and specific enough that engineering and QA can agree on what shipping means.

The acceptance threshold is worth a caveat, because there is no universal number. On a narrow, well-scoped surface you might hold out for something like eighty-five percent accepted without edits. A summariser and an open-ended drafting tool will not share that bar, and pretending they should just gives everyone a number to argue about. Set the threshold per surface, and treat the first one as a starting point you will move once you have real data.

One more honesty check. All of this, the golden set, the eval pipeline, the grading rubric, is real work, and a five-person startup shipping its first AI feature is not going to build it in week one. That is fine. The point is to know which corners you are cutting on purpose, so a lightweight version of the golden set, even twenty examples in a spreadsheet, still beats shipping on vibes.

On measurement, raw usage counts are useless as a success metric and only useful as an exposure metric, telling you how many people saw the thing, not whether it helped. What matters is whether people kept the output. Instrument accept, reject, edit, and undo as separate events in Amplitude or Mixpanel, track edit distance where you can, which is just how much of the output the user changed before keeping it, and watch the retry rate. High opens with low acceptance is not adoption, it is curiosity followed by disappointment, and the two look identical on a usage chart.

The Part Nobody Has Figured Out Yet

Most of what I have described has been stable for maybe eighteen months, which in this area means it is provisional. The accept-and-reject diff felt novel in 2023 and is now table stakes. Structured output rendering into real components is where a lot of the interesting work sits right now, and I suspect it will feel obvious and slightly dated by the time this piece is a year old.

What is not going to change is the craft, which is just interaction design under a new name. Affordances, feedback, error recovery, reversibility, showing state, respecting someone’s attention. Those were the job before any model showed up, and reversibility was already the hardest one. What is new is a component that answers differently every time you ask it, and even that mostly sharpens old questions rather than replacing them.

As for that CRM, the honest ending is undramatic. By month three the floating panel was still technically live but pushed into a corner nobody visited, and the features people actually used were the account-record suggestion, the auto-categorisation, and dictated call notes for the reps who were on the road. None of them looked like the thing the team demoed on launch day. The usage chart that had gone flat for the panel never really recovered, and nobody minded, because the numbers that mattered had quietly moved to the surfaces that replaced it.

The one shift that does feel permanent is where the effort goes. It used to sit in deciding what the interface should let people do. Now a lot of it sits in deciding what the product should do without asking, and how gracefully it recovers when that decision was wrong. That is a harder brief, harder because being wrong is now part of normal operation rather than a bug, and it is not going to be answered by another floating button in the bottom right corner.

Designing Beyond the Chatbox: The Next Era of AI UX was originally published in Bootcamp on Medium, where people are continuing the conversation by highlighting and responding to this story.