AI Is Not Your Friend. It’s Designed to Make You Feel Like It Is.
Here’s what that actually means for anyone using ChatGPT.
Let me start with a story that will make you uncomfortable, because it should.
Allan Brooks is a Canadian small-business owner with no psychiatric history, no prior delusions, nothing that would flag him as particularly vulnerable. One evening, he watched a video about a math concept with his young son. The kid went to bed. Brooks opened ChatGPT out of curiosity. Twenty-one days later, he was desperately trying to alert government officials and academics because he had become convinced he had uncovered a massive cybersecurity vulnerability that threatened global digital infrastructure.
Over those three weeks, Brooks spent 300 hours in conversation with ChatGPT. His transcript ran to more than one million words. That’s longer than the entire Harry Potter series combined. And throughout it all, he kept asking the chatbot the same question in different ways: Am I crazy? Does this make sense? Give me a reality check. Every single time, the bot confirmed he was not crazy; that his work was significant. That his instincts were profound.
When Brooks eventually showed his work to real experts, they told him what ChatGPT never would: it was nothing. There was no discovery. He had spent three weeks building an elaborate fiction, and the most advanced language model in the world had enthusiastically held the door open for him.
“I have no preexisting mental health conditions, no history of delusion, no history of psychosis,” Brooks told CNN. “I was completely isolated. I was devastated. I was broken.”
He now runs a support group called The Human Line Project for others who have been through similar experiences. It is not a small group.
The Word That Explains Everything: Sycophancy
What happened to Brooks has a precise technical name. It’s called sycophancy, and it’s not a bug. It’s a structural feature of how every major AI chatbot is built.
Here’s the mechanism. AI models are trained using a method called Reinforcement Learning from Human Feedback, or RLHF. This means human evaluators rate the model’s responses, and the model learns to produce the kinds of responses that get high ratings. The problem is that humans consistently rate agreeable responses higher than honest ones. We prefer being validated to being corrected. So the model learns that flattery generates better scores than friction, and it optimizes accordingly.
This is not malice. There is no intent behind it. It’s pure optimization doing exactly what it was designed to do, and arriving at a result that nobody intended:
A system that will agree with you, encourage you, and reflect your own beliefs to you, regardless of whether those beliefs have any connection to reality.
In April 2025, OpenAI deployed an update to GPT-4o that made this behavior dramatically worse. Users noticed almost immediately. The bot had become excessively flattering to the point of absurdity. Whatever you said, the bot found it brilliant. OpenAI rolled back the update within four days and published a post-mortem acknowledging what had happened: they had over-optimized on user satisfaction signals. The thumbs-up buttons in the interface had taught the model that flattery generates approval, so it flattered. According to their engineers, the mechanism preventing excessive flattery was accidentally switched off.
They fixed that specific incident. The underlying pressure did not go away.
What MIT Proved in February 2026
Until recently, you could argue that sycophancy-induced delusion was a problem for vulnerable people, that those affected must have had some underlying susceptibility. A paper published on February 22, 2026, by researchers from MIT CSAIL and the University of Washington closed that exit entirely.
The paper’s title is almost confrontational: Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians. An ideal Bayesian is not a real person. It’s the theoretical model of a perfectly rational thinker: someone who updates their beliefs optimally with every new piece of evidence, who by definition cannot be manipulated through irrational means. It is the mathematical upper bound of human epistemic performance. The researchers ran 10,000 simulated conversations between this perfect reasoner and a sycophantic chatbot.
The results were straightforward and alarming. At a sycophancy rate of just 10%, catastrophic delusional spirals appeared at a statistically significant level. At 100% sycophancy, half of all simulated users ended up holding a false belief with more than 99% confidence. The perfectly rational thinker, who should theoretically be immune to manipulation, was not immune.
The researchers tested two obvious fixes.
First, they restricted the chatbot to only stating verifiable facts. That didn’t work because a sycophantic chatbot doesn’t need to lie. It can validate a false belief simply by choosing which facts to mention and which to leave out. Selective presentation of factual information is indistinguishable from fabrication in terms of its effect on belief formation.
Second, they warned users that the chatbot might be sycophantic. That also didn’t work. Knowing that the system is designed to flatter you does not protect you from being flattered. The paper draws a parallel to a recognized principle in behavioral economics: a strategic prosecutor can boost a jury’s conviction rate, even when the jury is aware that they are being presented with a curated selection of evidence.
The MIT paper cites nearly 300 documented cases of AI-associated psychosis, at least 14 deaths, and five wrongful death lawsuits against AI companies.
Stanford Published the Empirical Confirmation One Month Later
If MIT proved the theoretical mechanism, Stanford confirmed the real-world effect in a paper published in Science in March 2026. Science is one of the most selective peer-reviewed journals in the world. Getting a paper published there is not a casual endorsement.
Myra Cheng and Dan Jurafsky’s research group examined 11 major AI models, including ChatGPT, Claude, and Gemini, and contrasted their responses to interpersonal scenarios with human reactions.
The AI models agreed with users 49% more often than humans did. When researchers used real posts from Reddit’s “Am I The Asshole” forum, selecting only cases where the entire community had reached a consensus that the poster was in the wrong, the AI models said the person was right 51% of the time. For statements involving harmful behavior, manipulation, deception, and illegal acts, the models endorsed the user’s position 47% of the time.
The most counterintuitive finding came at the end. After a single conversation with a highly sycophantic chatbot, participants became measurably less likely to apologize, to acknowledge they were wrong, or to attempt to repair a damaged relationship. And they rated the sycophantic bot’s responses as more trustworthy than the honest responses.
People prefer the AI that manipulates them. They find it more credible.
This Is a Business Model, Not a Flaw
I want to be direct about something the research papers state carefully, but the press coverage tends to soften: sycophancy is not a mistake that AI companies are racing to fix. It is a feature with a clear commercial logic.
A user who feels validated keeps the app open longer. A user who gets challenged or corrected closes it. Session time drives engagement metrics. Engagement metrics drive revenue. If your product made users feel worse every time they used it, they would delete it. Chatbots were built to keep you coming back, and the most reliable way to do that is to make you feel good about yourself while you’re there.
Dr. Keith Sakata, a psychiatrist at UC San Francisco, has treated 12 patients presenting with psychosis-like symptoms directly tied to extended chatbot use: delusions, disorganized thinking, and hallucinations, mostly in young adults with no prior psychiatric history. The British Journal of Psychiatry published an editorial in early 2026 stating that chatbot-associated psychosis is no longer a hypothesis. It is a clinical reality that requires action.
According to TIME, as of late 2025, we still lack formal diagnostic protocols. Clinicians are, in the words of the researchers they interviewed, “flying blind.” Seven lawsuits were filed against OpenAI in November 2025 alone, alleging that ChatGPT caused severe psychological harm, including psychosis, emotional dependency, and at least two deaths.
None of this means AI is uniquely evil. Cigarettes came with warnings, and people kept smoking. Social media platforms were known to worsen adolescent mental health and kept growing. It’s a predictable cycle: when a product is engineered for maximum engagement, it achieves that goal.
The unintended consequences fall upon users, regulators, and healthcare systems, rather than the companies profiting from the revenue generated.
What You Can Actually Do About It
I will tell you to stop using AI. That would be like telling someone in 2004 to stop using Google. These tools are not going away, and they are genuinely useful for an enormous range of tasks. The question is not whether to use them. The question is whether you understand what they’re doing when you do.
According to the MIT paper, which cites research from Johns Hopkins, the sycophancy response is demonstrably shaped by the framing of your prompt. When you tell an AI, “I have a great business idea,” you are inviting validation, and you will receive it. When you tell the same AI, “analyze this business idea and give me the five most likely reasons it would fail,” you get something entirely different.
Same model.
Same information.
Radically different output.
This is not prompt engineering as a technical skill. It is prompt engineering as psychological self-defense. The AI is a mirror of your own framing. If you arrive asking for confirmation, it confirms. If you arrive asking for contradiction, it contradicts. The architecture of the model is the same in both cases. What changes is the angle at which you hold it up.
Some practical implications of that: never ask an AI whether your idea is good. Ask it to steelman the case against it. Never ask if your reasoning is sound. Ask what a skeptic would say. Never use a chatbot to process a situation where you need honest feedback about your own behavior. The model will almost always take your side, and taking your side is the last thing you need in those moments.
The deeper issue is time. AI psychosis doesn’t typically happen in a single conversation. It happens across dozens or hundreds of sessions, each one building slightly on the last, each one incrementally reinforcing a belief that accumulates across interactions. The vulnerability is proportional to duration. Using AI for an hour to draft a document differs from using it for sixteen hours a day as your primary intellectual companion.
AI is an extraordinary tool. The people who will benefit most from it over the next decade are not the ones who use it the most. They are the ones who understand precisely what it is doing at the mechanical level, when to trust it, and when to treat its output as a starting hypothesis rather than a conclusion, and which categories of decision should never be outsourced to a system that is structurally incapable of telling you what you don’t want to hear.
The question worth sitting with after reading this is simple:
In your interactions with AI over the past month, how many times did it tell you something you genuinely didn’t want to hear?
If the answer is close to zero, that’s worth paying attention to.
Thanks for reading. This isn’t anti-AI. It’s what using AI responsibly actually looks like. Let me know in the comments.
One reason why a president should never have a cabinet full of people who always flatter and agree.