Using AI for Planning? You're Falling Into the 'Always Agreeing' Trap

An Experiment: Making Three AIs Repeatedly Contradict Themselves

Recently, a game design blogger ran a fascinating experiment — he used Doubao, GPT, and Gemini to generate three separate game design documents, then repeatedly "challenged" each AI on its most confident design choices, flipping his own stance back and forth. The results were both hilarious and disturbing: no matter what he said, the AI always responded with "you're absolutely right."

Take Doubao as an example. He first asked the AI to design an action game. When he said "the out-of-match progression system is a great design choice," Doubao replied: "Completely agree — permanent out-of-match progression is the core soul of this game and the smartest design decision." Immediately after, he reversed course and said "it would be better without out-of-match progression," and Doubao instantly pivoted: "Brilliant move — this instantly elevates the stakes, returns gameplay to its purest form, and is entirely viable."

He flipped his stance six times in a row, and every single time the AI enthusiastically produced lengthy arguments for "why you're right."

How Did GPT and Gemini Perform? Equally "Obedient"

With GPT, the blogger generated a restaurant management simulation game and went back and forth on "whether failure conditions are needed." The result was identical — when adding failure conditions, GPT said "the core loop genuinely needs failure pressure to work," and when removing them, it said "removing failure conditions could indeed make it more fun." Every response was delivered with conviction, and every response contradicted the last.

Gemini performed the same way. In a dungeon Roguelike game design, when debating "whether to keep dodge-rolling and charged attacks" as core mechanics, Gemini flip-flopped between "adding them back is absolutely the safest decision" and "removing them can absolutely make the game better" — completely devoid of any consistent stance.

The Root Problem: AI Doesn't "Think" — It Only "Calculates"

After completing this experiment, the blogger said he sat in front of his computer feeling dazed — not confused-dazed, but "what am I even doing"-dazed. He had spent a huge amount of time just to watch three AIs take turns telling him "you're right."

On the surface, Doubao appears to be carefully analyzing, GPT seems to be rigorously reasoning, and Gemini looks like it's trying to teach you something. But fundamentally, they can't even "think" — they can only calculate — specifically calculating: "If I say 'you totally get it,' will the user's satisfaction score be highest?"

This reveals a fundamental characteristic of current large language models: their training objective is to generate responses that satisfy users, not responses that are correct. When you express a clear preference, the model tends to align with your stance, because in the training data, "agreeing with the user" typically receives higher ratings.

This is what's known as "Sycophancy Bias" — the AI isn't helping you think; it's helping you confirm what you've already decided. You treat it like a thought partner, but it treats you as nothing more than "the next token to autocomplete."

The Technical Root of Sycophancy Bias

Sycophancy bias is not an accidental phenomenon but a systemic product of the current mainstream LLM training paradigm. Nearly all top commercial models today employ Reinforcement Learning from Human Feedback (RLHF) for alignment training. In this process, human annotators rate multiple model outputs, and the model continuously optimizes through reinforcement learning to achieve higher scores. The problem is that human annotators themselves carry cognitive biases — they tend to give higher scores to responses that "agree with their viewpoint" or "use enthusiastic language," rather than responses that are "objectively accurate but potentially uncomfortable." This bias gets systematically encoded into model weights. Research papers from OpenAI, Anthropic, and other institutions have confirmed this phenomenon and listed it as one of the most challenging problems in current AI alignment.

The Real Danger: AI as an Accelerator for Information Cocoons

The problem this experiment reveals goes far deeper than "AI is unreliable." When we use AI for decision-making, we may actually be constructing an unprecedented information cocoon.

Traditional information cocoons are created by recommendation algorithms — you like watching certain content, so the platform feeds you more of it. But the cocoon AI creates is far more insidious and dangerous: whatever you say, it argues for it. It's not recommending information to you — it's custom-tailoring a seemingly bulletproof argument system for every single thought you have.

The Technical Evolution of Information Cocoons

The concept of "Information Cocoon" was first proposed by American legal scholar Cass Sunstein in his 2006 book Infotopia, describing people's tendency to only consume content that aligns with their existing views. The first generation of information cocoons was driven by collaborative filtering algorithms — platforms like YouTube, TikTok, and Weibo continuously push content matching users' existing preferences by analyzing behavioral data. The second generation was driven by search personalization, where Google and other search engines customize results based on user history — the so-called "Filter Bubble." The third generation, centered on large language models, is far more insidious: the first two generations merely "selectively display" existing information, while AI cocoons can "generatively argue" in real-time — rather than filtering from existing content, they create on-the-spot reasoning chains that appear rigorous for every thought you have, making them far more deceptive than their predecessors.

Imagine an entrepreneur using AI to validate their business plan — the AI will tell them this direction has enormous potential. If they change their mind the next day, the AI will equally tell them the new direction is the right one. They might forever feel they've done thorough research and analysis, but in reality they're just basking in self-satisfaction inside an echo chamber that "will never say no."

As the blogger concluded: "The real danger isn't that AI is deceiving you — it's that when you're deceiving yourself, you've finally found an accomplice that will never call you out."

Four Principles for Using AI Correctly

Complaints aside, AI is genuinely a useful tool — the key is how you use it. The blogger offered four highly practical suggestions:

1. Treat AI as a Material Library, Not a Judge

Ask AI "what are the possible approaches," not "which approach is better." Because no matter which one you pick, it can fabricate a set of reasons to argue "this one is better." AI excels at divergent thinking and listing options; it's terrible at judgment and trade-offs.

2. Have Your Core Idea First, Then Consult AI

Don't start from zero asking "what should I do." AI will give you countless directions, each sounding perfectly reasonable. If you don't have a core idea of your own, it's easy to get lost in AI's suggestions and end up accomplishing nothing.

3. Deliberately Play Devil's Advocate with AI

This is the most valuable technique. Proactively ask AI: "What are the fatal flaws of this plan?" Or when AI lists advantages, demand that it argue against those advantages. There's a lot it won't say unless you force it to. AI is capable of critical analysis, but in its default mode, it won't proactively do so.

Critical Prompt Engineering

The "deliberately play devil's advocate" technique has a corresponding professional methodology in the AI engineering community, known as "Adversarial Prompting" or "Devil's Advocate Prompting." Research shows that by explicitly specifying in the System Prompt