Express course · No. 30
Most AI courses are about the engine. This one is about the experience built on top of it — and the hard fact at its center: the model is confident, fluent, and sometimes wrong. Good AI product design isn't about hiding that; it's about designing for it — setting honest expectations, showing the work, keeping the human in control, and failing gracefully. The design, not the model, is what earns trust.
Essence only · One picture per idea · Trust by design
The whole discipline starts from accepting what the model is: brilliant, helpful, and unreliable in a particular way. Design that pretends otherwise sets users up to be burned.
The model is confident even when it's wrong
A charismatic expert who gives every answer in the same self-assured tone — the right ones and the made-up ones sound exactly alike.
A language model produces fluent, confident output whether or not it's correct — its mistakes don't come with a worried tone or a flag. This is the defining design challenge of AI products: the interface can't rely on the model to signal its own uncertainty, because it sounds equally sure when right and when hallucinating. So the design has to carry what the model won't: cues about reliability, room to verify, ways to recover. You're wrapping a UX around a confident, fallible component.
Trust is built by the design, not the model
You trust a good bank not because the teller is infallible, but because of receipts, statements, and the ability to dispute a charge — the system earns trust the person alone couldn't.
Users won't trust an AI feature because it's powerful; they'll trust it because the experience around it makes them feel safe — they can check it, correct it, undo it, and know what to expect. The model supplies capability; the design supplies trust. A brilliant model in a careless interface feels untrustworthy; a modest model in a thoughtful one feels dependable. Trust is a property of the product you build, not the model you used.
Match the design to the stakes
You'll happily let a tool auto-correct a typo, but you want to read every word before it sends a legal letter — the same engine, very different guardrails.
How much freedom you give the AI should track how costly a mistake would be. Low-stakes, easily-reversible output (a suggested tag, a draft sentence) can be more automatic; high-stakes, irreversible action (sending money, deleting data, publishing) demands review and confirmation. The single most important design decision is calibrating autonomy to stakes — and getting this wrong, in either direction, is what makes an AI feature either reckless or uselessly timid.
The model is confident even when wrong, so the design must carry what it won't: reliability cues, room to verify, ways to recover. Trust is built by the product, matched to the stakes.
A huge share of AI disappointment is really mismatched expectations. Telling users honestly what they're dealing with — before they're burned — is the cheapest trust you can buy.
Tell users it's AI, and it can be wrong
A weather forecast says "70% chance of rain," not "it will rain" — naming the uncertainty up front is what makes people trust it even when it's off.
Be upfront that a feature is AI-powered and that its output can be imperfect. This isn't an apology or a weakness — it's calibration. A user who knows they're getting an AI suggestion reads it appropriately: useful, worth checking, not gospel. Hiding that the output is AI-generated, or implying it's authoritative, sets people up to over-trust it and then feel betrayed when it's wrong. Honest framing turns a wrong answer from a betrayal into an expected, manageable event.
Frame outputs as drafts and suggestions
A handwritten "draft" stamp on a document changes how you read it — you treat it as a starting point to improve, not a final word to obey.
The words and framing around an output shape how much users trust it. Presenting AI results as drafts, suggestions, or starting points — rather than finished, authoritative answers — invites the user to engage critically and edit, which is exactly the right posture toward a fallible tool. "Here's a draft you can edit" produces better outcomes and fewer disasters than "here's the answer." The framing does quiet, constant work to keep the user in the right relationship with the model.
Don't oversell what it can do
A salesperson who promises a gadget does everything sets you up to feel cheated; one who tells you exactly what it's good and bad at earns lasting trust.
It's tempting to market an AI feature as magic, but overselling guarantees disappointment — the gap between the promise and the reality becomes the user's experience. Set expectations slightly below what the model can do, not above, and let it pleasantly surprise. Being honest about its limits, even prominently, builds far more durable trust than hype that the first failure shatters. Under-promise and let the model over-deliver, not the reverse.
Most AI disappointment is mismatched expectations. Say it's AI and can be wrong, frame outputs as drafts not verdicts, and under-promise — honest framing is the cheapest trust there is.
An answer you can't check is an answer you can't trust. Letting users see where the output came from turns a black-box claim into something they can verify — and verifiability is the foundation of trust.
Cite the sources behind an answer
A research paper with footnotes lets any reader trace a claim back to its source — the citations are what make it credible rather than just confident.
When the model answers from retrieved information, show the sources — link the documents, quote the passage, point to where the claim came from. Citations let users verify an answer instead of taking it on faith, and they turn the model from an oracle you must believe into a research assistant you can check. This is the design payoff of grounding: it's not enough for the answer to be sourced internally, the user has to be able to see and follow the source.
Make the answer verifiable, not just plausible
A good accountant doesn't just hand you a number — they show the line items, so you can confirm it rather than trust a confident total.
Design so the user can check the output, not just receive it. Surface what the model based its answer on, highlight the specific data it used, make it easy to confirm against reality. A verifiable answer is trustworthy even when occasionally wrong, because the user can catch the wrong ones; an unverifiable answer is dangerous even when usually right, because there's no way to tell the good from the bad. Build for checking, and the inevitable mistakes become catchable instead of costly.
Show confidence and reasoning where it helps
A doctor who says "I'm fairly sure, but let's confirm with a test" guides you better than one who states every diagnosis with identical certainty.
Where you can, surface signals of how reliable an output is — a confidence level, the reasoning steps, a flag when the model is unsure or working from thin evidence. This helps users calibrate how much to lean on each answer, instead of treating them all as equally solid. It won't be perfect, but even a rough "this one is shaky" cue dramatically improves how safely people use the feature. Giving the user the reliability signal the model's tone hides is some of the highest-value design you can do.
An answer you can't check is one you can't trust. Cite sources, make outputs verifiable, and surface confidence — verifiability turns the inevitable wrong answers from hidden risks into catchable ones.
The safest place for a fallible model is in the passenger seat. Designing so the user stays in charge — approving, editing, undoing — is what makes powerful AI safe to ship.
Suggest and assist, don't act unasked
A good assistant drafts the email and leaves it for you to send — they don't hit send themselves and tell you afterward.
For anything consequential, design the AI to suggest rather than act: propose the edit, draft the message, recommend the choice — and let the human decide whether to accept it. This keeps the person in the loop on decisions that matter, so a model mistake becomes a rejected suggestion instead of an executed error. The more an action matters or can't be undone, the more firmly the model should propose and the human should dispose. Assistance the user commands is safer than autonomy the user merely supervises.
Let users edit, not just accept or reject
A good template lets you change the parts that don't fit, instead of forcing an all-or-nothing choice between perfect and useless.
The best AI interactions let the user shape the output, not just take it or leave it. Make AI results easy to edit, refine, and partially keep — because the model often gets you 80% there, and the value is in letting the human finish the last 20% rather than discarding the whole thing over one flaw. Treating output as editable clay rather than a finished verdict matches how the model actually performs: a strong draft that a human improves. Edit-ability turns "almost right" from a failure into a head start.
Confirm before anything irreversible
A bank requires a second confirmation for a large transfer — a deliberate pause before the thing you can't take back.
Put a clear confirmation step before any action the user can't undo — sending, paying, deleting, posting, anything irreversible. This is the same blast-radius thinking as security, expressed in the UX: a model error should at most produce a suggestion the user declines, never an irreversible action taken on its own. And pair it with undo wherever you can, so even accepted actions stay recoverable. The ability to step back is what lets users act on AI confidently instead of fearfully.
Keep the human in charge: the model suggests, the person decides; outputs are editable, not take-it-or-leave-it; and anything irreversible needs confirmation and, ideally, undo.
A fallible component will fail, so how it fails is a design decision, not an edge case. Graceful failure — admitting limits instead of fabricating — is what separates a trustworthy product from a dangerous one.
Let it say I don't know
The expert you trust most is the one who says "that's outside what I know" instead of confidently making something up to fill the silence.
Design the experience to allow — and prefer — an honest "I don't know" or "I'm not sure" over a fabricated answer. The most dangerous AI failure is confidently inventing a response when it has no basis, and a product that makes refusal a valid, graceful outcome is far safer than one that pressures the model to always produce something. Surfacing uncertainty instead of papering over it isn't a failure of the feature — it's the feature working honestly. A model that knows its limits, and a design that respects them, earns more trust than one that always answers.
Handle the low-confidence case differently
A spam filter that's sure routes the message automatically, but one that's unsure flags it for you to check — the uncertain case gets handled, not hidden.
Don't treat every output the same regardless of how shaky it is. When the model is uncertain or working from weak evidence, design a different path: flag it, ask for confirmation, route it to a human, or ask the user a clarifying question. The uncertain cases are exactly where blind automation hurts most, so they deserve their own graceful handling rather than being shipped with the same confidence as the solid ones. Designing for the shaky middle, not just the happy path, is what makes the feature robust.
Degrade, don't collapse
When the GPS loses signal it shows your last known position and says "searching," instead of going blank or sending you off a cliff — it fails softly.
When the AI can't perform — an error, an outage, an out-of-scope request — the experience should degrade gracefully rather than break or, worse, produce garbage. Fall back to a simpler option, a clear message, a manual path, a cached result. A feature that fails into a sensible, honest state keeps the user's trust; one that fails into a confident wrong answer or a broken screen destroys it. Plan the failure modes as deliberately as the success path, because with a fallible component, failure is part of normal operation.
Failure isn't an edge case for a fallible model — it's normal. Let it say "I don't know," handle low-confidence cases differently, and degrade into an honest fallback instead of confident garbage.
A good AI product doesn't just serve outputs — it learns from how users react to them. Capturing that signal turns every interaction into a chance to improve, and gives users a way to be heard.
Give users an easy way to react
A thumbs-up and thumbs-down next to each answer — a one-tap way for the user to tell you it landed or missed, costing them nothing.
Make it effortless for users to signal whether an output was good or bad — a thumbs up/down, a quick rating, a flag. This does two things at once: it gives the user a sense of agency and being heard, and it gives you a continuous stream of real signal about where the feature works and fails. A feature that can't be rated is a feature you're flying blind on; a simple reaction control is the cheapest, richest feedback you'll get. Capture the reaction users would give anyway.
Let corrections teach the system
When you fix the address your phone keyboard suggested, it learns — the correction isn't just for this time, it improves the next time.
The richest feedback is the user fixing the output. When someone edits an AI suggestion, that edit is a precise signal of what "right" looked like — capture it. Corrections and rejections are gold: they tell you exactly where the model fell short, on real cases. Feed them back into your evals (the evals course) so the feature measurably improves, and the product gets better along the contours of where real users actually struggle. The correction is both a fix for now and a lesson for later.
Production feedback is your best eval set
The best test questions come from the real exam students sat, not from what the teacher imagined — reality writes better tests than you can.
The reactions and corrections flowing in from real use are the most valuable source of eval cases you have — they're what your users actually do, including the failures you'd never have invented. Mining this feedback to find where the feature breaks, and folding those cases into your evaluation set, closes the loop: production teaches you what to fix, you fix it, and the next release is better. A product with this loop running improves continuously; one without it is guessing. The feedback loop is what makes an AI feature get better instead of just existing.
A good AI product learns from use: give users an easy way to react, capture their corrections as precise signal, and feed real production failures back into your evals to improve continuously.
AI product design comes down to one posture, applied throughout: treat the model as a powerful, fallible assistant, and build the experience that lets a human use it safely and confidently.
Calibrate autonomy to the stakes, everywhere
You let the assistant file the paperwork on its own, draft the important letter for your review, and never sign the contract without you — autonomy scaled to consequence.
The unifying decision, made again and again across a product, is how much to let the AI do on its own versus how much to route through the human — and the answer is always: scale it to the stakes. Low-risk, reversible things can be more automatic; high-risk, irreversible things get expectations, verification, control, and confirmation layered on. A well-designed AI product isn't uniformly autonomous or uniformly cautious; it's calibrated, piece by piece, to how much a mistake there would cost.
Design for the wrong answer, not just the right one
A good safety engineer designs the car assuming a crash will happen, not hoping it won't — the airbags are for the bad day, and they're the point.
The mark of mature AI product design is that it's built around the model being wrong, not just being right. The demo shows the right answer; the product survives the wrong one. So you design the verification, the control, the graceful failure, the feedback loop — the machinery that makes a confident mistake catchable and recoverable. Get the happy path working and you have a demo; design for the unhappy path and you have a product people can actually trust with real work.
- Is autonomy matched to the stakes — more automatic where it's reversible, more controlled where it isn't? - Are expectations honest — users told it's AI, outputs framed as drafts, not oversold? - Can the user verify — sources cited, answers checkable, confidence surfaced where it helps? - Is the human in control — suggest not act, editable output, confirmation before the irreversible, undo? - Does it fail gracefully — able to say "I don't know," low-confidence handled, degrades not collapses? - Is the feedback loop closed — easy to react, corrections captured, fed back into evals?
- fallible component — the model is confident even when wrong; the design must account for it.
- calibrating autonomy to stakes — more freedom where a mistake is cheap, more control where it's costly. - honest expectations / drafts not verdicts — framing that keeps users in the right relationship with the model. - verifiability / citations — letting users check an answer instead of trusting it blindly. - human-in-the-loop — suggest, edit, confirm, undo — the person stays in charge. - graceful failure — admitting "I don't know" and degrading instead of fabricating. - feedback loop — capturing reactions and corrections to improve the feature over time.
- The experience earns trust through verifiability and control, not just model power. - Users are told honestly what it is, and outputs are framed as drafts to check. - The human stays in control — editing, confirming, and able to undo. - It fails gracefully, admitting uncertainty instead of confidently fabricating. - Real feedback flows back into your evals, so the feature improves with use.
AI product design is building the experience that makes a powerful, fallible model safe to use: autonomy matched to stakes, honest expectations, verifiable answers, the human in control, graceful failure, and a feedback loop that learns.