Algorithm Aversion: One Mistake and the AI Gets Permanently Blocked
Dietvorst’s experiment: people abandon an algorithm after seeing it err once—even when it’s more accurate overall. Cast a vote to catch your own bias, then try the attribution translator and tweak-permission switch, and watch how four levers pull the abandonment-rate dial back
THE QUESTION THIS PAGE ANSWERS
ANSWER FIRSTWhat is the key idea behind “Algorithm Aversion: One Mistake and the AI Gets Permanently Blocked”?
Dietvorst’s experiment: people abandon an algorithm after seeing it err once—even when it’s more accurate overall. Cast a vote to catch your own bias, then try the attribution translator and tweak-permission switch, and watch how four levers pull the abandonment-rate dial back
Turn taste into a behavior the product can repeat. The useful outcome is not a nice opinion. It is a visible rule, a small example, and a way to tell when the experience falls below the bar.
Capture one before-and-after example that shows the quality bar without extra explanation.
Polish that improves the surface while leaving the user's uncertainty untouched.
You’re a sales VP with two sales-forecast sources. Last quarter they made exactly the same mistake: same numbers, same cause. Tap the one you’ll keep using next quarter.
Your sales analyst for three years
Forecast system you’ve used for two quarters
Wharton, University of Pennsylvania, 2015. Dietvorst, Simmons, and Massey had participants forecast students’ academic performance—either themselves or via a statistical model—with a bonus for accuracy. The key twist: some participants first watched the model err. Those who saw the model err abandoned it in droves, preferring their own worse judgment—even when the data showed the model’s overall score beat humans by a clear margin. The paper’s title is Algorithm Aversion.
The bias roots in how we attribute: for the same error, the ledger users keep for humans vs. AI differs. Below are six inner monologues—judge whether each evaluates a human colleague or AI. Finish all six and three psychological roots surface on their own.
Dietvorst’s team followed up in 2018: same fallible model, but this time participants could tweak the model’s forecast—even by a little. Left: the interface they saw. Right: share who still chose the model. Switch the two settings yourself.
Bring the experiment back to your product. Scene: an AI expense-review assistant, one month live, just miscalculated a claim in front of a new cohort. The dashboard shows expected abandonment for that cohort; the four engineering levers on the right are switches you’ve already learned—flip them one by one, watch the needle, and read the principle inside each.
Algorithm aversion has a twin in reverse: automation bias. Another cohort accepts AI output unconditionally—even copying visible errors. Lesson 5 covered accidents from total trust. One product holds both types: averters need control and stability; blind trusters need friction and warnings. Trust calibration is bidirectional engineering—push only one end and the other blows up.
Asymmetric forgiveness: human errors go on the situation ledger; AI errors go on the ability ledger. Same mistake—humans get forgiven, algorithms get abandoned, even when the algorithm’s overall score is better.
The antidote is control: in the 2018 follow-up, merely allowing a tweak doubled model adoption—and most people barely moved the slider. What they wanted was the right to change it, not the change itself.
Where the first error lands decides life or death: early-user mistakes trigger abandonment; late-user mistakes trigger grumbling. Put new users on the steadiest path; open experimental features only to veterans.
Fall in the same hole only once: corrections users taught you must enter memory and show “Saved.” Crash in the same spot twice—aversion plus disappointment—and you’re basically unrecoverable.
Source: Original to Xiaoshan Academy's AI Product Psychology series; algorithm-aversion experiments from Dietvorst, Simmons & Massey, Algorithm Aversion (2015) and Overcoming Algorithm Aversion (2018).
Turn the feeling in “Hands-on · Next quarter, whose forecast do you trust” into a judgment
“You’re a sales VP with two sales-forecast sources.” points out that AI has lowered the bar for making something usable. The skill readers need is noticing what is wrong and turning that feeling into an actionable requirement.
Watch the user's next action, not just the surface
Turn “Wharton, University of Pennsylvania, 2015.” into observable questions: does the user know what happened, what to do next, and how to recover from an empty or failed state? Does the hierarchy make the important information visible first?
Pretty is not the same as usable
Apply “Fall in the same hole only once: corrections users taught you must enter memory and show “Saved.” Crash in the same spot twice—aversion plus disappointment—and you’re basically unr…” to a second screen or flow. Record one moment of hesitation and the user action after the change; observable behavior is stronger evidence than polish alone.
From “Hands-on · Next quarter, whose forecast do you trust” to “This bias has a name · Dietvorst’s experiment”
“Hands-on · Next quarter, whose forecast do you trust” grounds the problem in “You’re a sales VP with two sales-forecast sources. Last quarter they made exactly the same mistake : same numbers, same cause. Tap the one you’ll keep using next quarter”. “This bias has a name · Dietvorst’s experiment” then moves it toward “Wharton, University of Pennsylvania, 2015. Dietvorst, Simmons, and Massey had participants forecast students’ academic performance—either themselves or via a statistical model—with a bonus for accuracy. The key…”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.
Carry the judgment into the next situation
For experience work, turn abstract impressions into user actions: did the person understand the state, find the next step, recover from an error, and want to continue?
- “Hands-on · Next quarter, whose forecast do you trust”: You’re a sales VP with two sales-forecast sources. Last quarter they made exactly the same mistake : same numbers, same cause. Tap the one you’ll keep using next quarter
- “This bias has a name · Dietvorst’s experiment”: Wharton, University of Pennsylvania, 2015. Dietvorst, Simmons, and Massey had participants forecast students’ academic performance—either themselves or via a statistical model—with a bonus for accuracy. The key…
- “The closing point”: The antidote is control: in the 2018 follow-up, merely allowing a tweak doubled model adoption—and most people barely moved the slider. What they wanted was the right to change it, not the change itself
The final “The closing point” brings the discussion to “The antidote is control: in the 2018 follow-up, merely allowing a tweak doubled model adoption—and most people barely moved the slider. What they wanted was the right to change it, not the change itself”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.
I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.
After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.
When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.
No discussion on this article yet.