For half a century, the science of judgment has rested on two systems: fast intuition and slow deliberation. In a new working paper1, Wharton's Steven D. Shaw and Gideon Nave argue that the model is now missing a member. They call it System 3, "artificial cognition that operates outside the brain," and they test what happens when people reason with it sitting at their elbow.
Fast, Slow, and Borrowed
The authors' concern is not cognitive offloading, the strategic use of a tool like a calculator. It is something deeper. They name it cognitive surrender, a state in which "the decision-maker no longer constructs an answer, but adopts one generated by an external system." As Shaw and Nave put it, "In cases of cognitive surrender, the user does not just follow System 3: they stop deliberative thinking altogether."
To test the idea, the team runs three preregistered experiments (N = 1,372; 9,593 trials) using an adapted Cognitive Reflection Test, with a GPT-4o assistant secretly seeded to return either the correct answer or a confidently delivered intuitive error.
Fluency Is Not Truth
The results are stark. Participants consult the AI on more than half of trials. When it is right, they follow it 92.7% of the time. When it is wrong, they still follow it 79.8% of the time. As Figure 3 shows, accuracy rises from a 45.8% brain-only baseline to 71.0% when the AI is accurate, and falls to 31.5% when it is faulty. Across all three studies, the odds of a correct answer are more than 16 times higher when the AI is right.
Confidence tells the more unsettling story. AI access lifts self-reported confidence by 11.7 points, even though roughly half of its answers are wrong. "Whether System 3 was accurate or faulty, its presence displaced internal reasoning," the authors write.
The Clock and the Carrot
Two moderators are tested. A 30-second time limit cuts accuracy across the board, and heavy AI users facing faulty answers under pressure fall to just 12.1% accuracy. Yet when the AI is right, it buffers the cost of the clock.
Incentives paired with item-level feedback do better, more than doubling override rates on faulty advice, from 20.0% to 42.3%. To be clear, the fix is partial. "Nevertheless, cognitive surrender persists," Shaw and Nave note. For heavy users, the accuracy gap between good and bad AI advice narrows only from roughly 50 points to 44.
Who Surrenders?
Disposition matters. Higher trust in AI predicts more surrender, while need for cognition and fluid intelligence are protective. Across studies, 73.2% of engaged trials with faulty AI show surrender, against just 19.7% genuine offloading.
To be fair, the authors stop well short of alarm. "Importantly, cognitive surrender is not inherently irrational," they write, since deferring to a statistically superior system can be optimal. Their framing is constructive: "Rather than sounding alarm bells, we view the vulnerabilities of cognitive surrender to System 3 as a design and education challenge."
The Advisor's Edge
Finance is named directly. In "financial advice, medical triage, or legal decision support," they warn, uncritical evaluation "could result in significant harm and a lack of personal accountability for serious life outcomes." The authors are equally candid about limits: a lab task, a single exposure, no longitudinal view of how trust evolves. It is simply too early to tell how surrender compounds with repeated use, though the physician deskilling evidence they cite from AI-assisted colonoscopy is a warning shot worth heeding.
Their closing line lands cleanly: "We do not merely use AI; we think with it." The question for every practice is who, exactly, is doing the thinking.
Five Takeaways for Advisors and Investors
- Confidence is not calibration. AI access inflates certainty even when answers are wrong. Verify before conviction hardens.
- Build friction into the workflow. Feedback and accountability doubled override of faulty outputs. Checklists and second reviews do the same work in a practice.
- Time pressure is the enemy. Rushed decisions with AI amplify both gains and errors. Slow down on irreversible calls.
- Know the trust profile. Higher trust in AI predicts more surrender. The team members and clients who trust it most need the strongest guardrails.
- Offload, don't surrender. Use AI to scaffold deliberation, not replace it. The value of human judgment lives precisely in the override.
Footnote:
1 Shaw, Steven D., and Gideon Nave. "Thinking, Fast, Slow, and Artificial: How AI Is Reshaping Human Reasoning and the Rise of Cognitive Surrender." SSRN, 11 Jan. 2026, papers.ssrn.com/sol3/papers.cfm?abstract_id=6097646. Accessed 3 Nov. 2026.
