Draft v0.1

Trust and Confidence

Trust and confidence describe the user’s willingness to understand, rely on, and act with AI appropriately within a given context. Trust is not blind belief in AI capability, and confidence is not unconditional certainty in AI output. In AI UX, the goal is to build appropriate trust and informed confidence—so users neither underuse valuable AI nor over-rely on uncertain outputs. 1 2

Trust and confidence are outcomes of good design, not labels a product can claim for itself. Users build trust when AI behavior is understandable, consistent, bounded, and aligned with their goals. Users build confidence when they can see what the AI is doing, verify what matters, guide or override behavior when needed, and recover safely if something goes wrong. 1 3 2

As AI systems take on more responsibility, trust and confidence become more important—and more fragile. Higher-capability systems can create greater value, but they also increase the need for transparency, control, human override, and clear communication of limits. Trust should be calibrated to the task, the level of autonomy, and the level of risk in the workflow. 3 1 2

Principles

1. Build appropriate trust, not maximum trust

The goal of AI UX is not to make users trust AI as much as possible. The goal is to help users develop an accurate mental model of what the system can do, where it may be uncertain, and when human judgment is still required. Appropriate trust prevents both underuse and overreliance. 1 2

2. Trust is earned through behavior, not messaging

Trust does not come from aspirational language, branding, or human-like presentation alone. It is earned when AI behaves predictably, communicates clearly, respects boundaries, and performs in ways that match its stated capabilities and limitations. 1 2

3. Confidence depends on visibility and verification

Users are more confident when they can understand what the AI is doing, inspect the basis of an answer or action, and verify important outputs before acting. Confidence should increase when evidence is available and decrease when the system signals uncertainty or limited scope. 1 3

4. Trust should scale with risk and autonomy

The more consequential or operational an AI system becomes, the more carefully trust must be calibrated. Informational systems may rely on source grounding and clarity, while agent-assisted and autonomous systems require stronger confidence-building mechanisms such as action previews, approval checkpoints, rollback, auditability, and policy visibility. 3 1 2

5. User agency is foundational to trust

People trust AI more appropriately when they retain meaningful influence over outcomes. The ability to guide, modify, reject, pause, override, or recover from AI behavior helps users feel confident engaging with more capable systems without giving up authority. 1 3

6. Trust should be maintained over time, not assumed at launch

Trust and confidence are dynamic. They are shaped by repeated use, outcome quality, system consistency, error handling, and the user’s ability to understand and recover from failures. AI experiences should support ongoing feedback, learning, and measurement rather than assuming trust is established once and for all. 1 3 2

Guidelines

Set accurate expectations early

Communicate what the AI is for, what it can and cannot do, when it is active, and what level of autonomy it has. Clear expectation-setting helps users form an accurate mental model before they rely on the system. 1

Make AI behavior visible and understandable

Users are more likely to trust AI appropriately when they can see what it is doing, what it plans to do next, and what information, policies, or constraints shaped the output. Visible AI behavior reduces ambiguity and helps users decide when to rely on the system. 1 3

Provide evidence and rationale when decisions matter

When users need to evaluate a recommendation, explanation, or action, provide source grounding, rationale, confidence cues, or policy references as appropriate. Confidence increases when important outputs can be inspected and verified. 1 3

Make user control and override easy to find

Trust increases when users know they can shape outcomes and intervene when necessary. Controls for approval, rejection, pause, override, and recovery should be available at the points where users are most likely to need them. 1 3

Communicate uncertainty and limits directly

Do not present AI output as fully authoritative when verification is still needed. Where results may be incomplete, probabilistic, or uncertain, communicate that clearly so users can apply the right level of scrutiny. 1 2

Avoid design choices that inflate trust

Do not use anthropomorphic language, vague confidence signals, or interface patterns that imply more understanding, certainty, or autonomy than the system actually has. Overstated capability may increase short-term acceptance but erodes trust over time. 1

Preserve clear recovery paths

Users are more confident adopting AI when they know mistakes are recoverable. Safe rollback, compensating actions, action history, and understandable failure states all help maintain trust when conditions change or outcomes are wrong. 3 1

Measure trust through behavior, not sentiment alone

Trust and confidence should be evaluated through signals such as acceptance, rejection, override, verification, rollback, and safe adoption patterns—not only through stated satisfaction. These behavioral signals better reflect whether trust is appropriately calibrated in real workflows. 3 1

Trust and Confidence across AI capability models

Informational AI

Trust is built through clarity, source grounding, and honest scope. Users should understand that the AI is informing, summarizing, or explaining—not acting on the system’s behalf. Confidence comes from being able to inspect sources, ask follow-up questions, and verify important information. 3 1

Examples 3 - Documentation chat that provides citations and makes it easy to verify source material. 3 1 - “Why is this happening?” explanations that clarify context without overstating certainty. 3 1

Assistive AI

Trust is built through helpfulness, visible rationale, and low-friction choice. Users should understand why the AI made a recommendation, what alternatives exist, and whether the output still requires review or editing. Confidence comes from being able to compare, refine, accept, or reject suggestions easily. 3 1

Examples 3 - Configuration recommendations with rationale and clear accept, edit, or dismiss options. 3 1 - AI-assisted report generation that shows draft status, input sources, and areas needing user review. 3 1 - Step-by-step setup guidance that explains tradeoffs and leaves the decision with the user. 3

Agent-Assisted AI

Trust is built through supervised execution, visible plans, approval checkpoints, and strong recovery options. Users should feel confident delegating work because they can see what the AI intends to do, approve key steps, pause execution, inspect results, and intervene when needed. 3 1

Examples 3 - Guided remediation workflows that preview actions before execution and support interruption. 3 - Multi-step configuration automation with checkpoints, step status, and rollback paths. 3 1 - Ops orchestration flows with visible action histories and clear manual takeover options. 3 1

Autonomous AI

Trust is built through governance, boundaries, policy visibility, monitoring, and reliable recovery. Users may not approve every action in real time, but they must still understand what the system is allowed to do, what triggered action, what outcomes occurred, and how to suspend or override automation when needed. Confidence comes from strong safeguards, consistent behavior, and clear auditability over time. 3 1 2

Examples 3 - Auto-scaling systems with visible thresholds, rules, and change histories. 3 - Self-healing systems that show detected issues, applied remediations, and resulting system state. 3 - Policy-driven traffic routing adjustments with clear governing rules, monitoring, and override paths. 3

What good trust and confidence look like

A well-designed AI experience helps users answer five questions with confidence:

What is this AI designed to do—and what is outside its scope? 1 2
Why should I rely on this output or action in this context? 1 3
What evidence, rationale, or history can I inspect if it matters? 1 3
How can I guide, verify, override, or recover if needed? 1 3
Does the system behave consistently enough over time to earn continued trust? 1 2

Summary

Trust and confidence are the outcomes of AI experiences that are transparent, bounded, controllable, and aligned to workflow risk. The goal is not maximum trust, but appropriate trust: enough confidence for users to benefit from AI where it adds value, and enough clarity and control to question, verify, or intervene when needed. As AI systems become more capable, designing for trust and confidence means helping users understand the system, act with informed judgment, and recover safely over time. 1 3 2