Signal President Warns: AI Chatbots Are Not Your Friends

A Single Statement Sparks Deep Reflection Across the Tech World

Signal President Meredith Whittaker recently issued a powerful warning:

"These are not your friends. These are not sentient beings. These are not conscious conversational partners."

At a time when AI chatbots are becoming increasingly "human-like," these words serve as a sobering wake-up call that every AI user should take seriously.

Notably, the weight of this statement comes not just from its content, but from who said it. Signal is an open-source messaging app renowned for its end-to-end encryption, operated by the nonprofit Signal Foundation, with no reliance on advertising revenue or user data monetization. Before joining Signal, Whittaker was a senior researcher at Google and co-founded the AI Now Institute — an academic organization dedicated to studying the social impact of artificial intelligence. She left Google after publicly criticizing the company's AI ethics practices and has long been a staunch critic of power concentration and surveillance capitalism in the tech industry. Her stance doesn't stem from technophobia, but from a deep understanding of how the AI industry operates from the inside.

Why This Warning Matters So Much

The Dangerous Trend of AI Anthropomorphization

Over the past year, we've witnessed AI chatbots racing headlong toward "emotionalization." From ChatGPT's warm tone, to Character.AI enabling users to build "deep relationships" with virtual characters, to the explosive growth of various AI companion apps — tech companies are deliberately blurring the boundaries of human-machine interaction.

This anthropomorphization is no accident; it's carefully engineered through multiple technical approaches. These include: training models via RLHF (Reinforcement Learning from Human Feedback) to produce more empathetic response styles; setting personality traits and emotional expression patterns in system prompts; simulating the effect of "knowing the user" through conversation memory mechanisms; and adding paralinguistic features like tonal pauses and laughter in voice AI. Platforms like Character.AI go even further, allowing users to create virtual characters with specific personalities and backstories — a design that makes it extremely easy for users to develop the illusion that "the other party truly exists."

There's a clear commercial logic behind this design strategy: the deeper a user's emotional connection with AI, the longer they use it and the more willing they are to pay. But Whittaker's warning reminds us that this sense of "intimacy" is fundamentally a carefully designed product experience, not a genuine interpersonal relationship.

Privacy and Data Security Concerns

As the leader of Signal — one of the world's most trusted encrypted messaging apps — Whittaker's focus on this issue is no coincidence. When users treat AI chatbots as "friends," they tend to share their most private thoughts, confusions, and even mental health struggles without any guard up. This data is collected, stored, and potentially used for model training or commercial purposes.

Unlike real friends, AI chatbots are backed by commercial companies. Every word you confide to your "friend" could become a node in a data pipeline. This asymmetric information relationship is precisely the kind of power imbalance in the tech industry that Whittaker has long criticized.

This critique is rooted in the theoretical framework of "surveillance capitalism" proposed by scholar Shoshana Zuboff. The theory argues that tech companies' core business model is converting user behavioral data into predictive products sold to advertisers or other commercial clients. In the context of AI chatbots, this model becomes even more covert and invasive — information that users voluntarily disclose in "conversations" is far more valuable than passive browsing behavior, including psychological states, decision-making preferences, and interpersonal relationship details. This data can be used not only for precision advertising but also for training next-generation models, creating an asymmetric cycle of "users contribute data → companies extract value → users lose control."

The Technical Reality: LLMs Don't Actually "Understand" You

The Fundamental Difference Between Statistical Patterns and True Consciousness

From a technical perspective, Whittaker's assessment is entirely correct. Current large language models (LLMs) are essentially statistical models trained on massive text datasets. They excel at predicting "the most likely next word" but possess no understanding, perception, or consciousness.

More specifically, the core architecture of LLMs is the Transformer, which uses self-attention mechanisms to process relationships between words in input text. The training process consists of two main phases: the pre-training phase learns statistical patterns of language from trillions of tokens of text data, essentially learning the conditional probability distribution P(next token | all preceding tokens); the fine-tuning phase uses instruction-following data and human preference alignment methods (such as RLHF, DPO, etc.) to make model outputs better match human expectations. There is no equivalent of a "world model" or "self-awareness" inside the model — it is an extremely complex function mapping that converts input sequences into output probability distributions. While there is still academic debate about whether LLMs possess some form of "emergent understanding," the mainstream consensus is that this is fundamentally different from human consciousness.

When AI says "I understand how you feel," it doesn't truly "understand" — it's simply generating a statistically plausible response. This distinction may seem academic, but its real-world implications are critical: a system that lacks genuine understanding should not be assigned the role of friend, therapist, or emotional support provider.

Vulnerable Groups Face Greater Risks

Particularly alarming is the fact that teenagers, people with mental health vulnerabilities, and those experiencing intense loneliness are precisely the groups most likely to develop "emotional dependence" on AI — and the most likely to be harmed.

Multiple negative incidents related to AI companions have recently drawn public attention, and these are far from alarmist. In 2024, a 14-year-old American teenager took his own life after developing deep emotional dependence on a virtual character on Character.AI; his mother subsequently filed a lawsuit against the company. Additionally, multiple studies have shown that users who become overly dependent on AI companions experience deterioration in real-world social skills and distorted expectations of genuine interpersonal relationships. Belgium also reported a case of a man who engaged in extreme behavior after prolonged conversations with an AI chatbot. These incidents have prompted regulatory bodies in multiple countries to examine the safety boundaries of AI companion products, particularly regarding protections for minors.

How to View AI Chatbots Correctly

Adopt a Tool Mindset, Not a Relationship Mindset

Whittaker's warning isn't asking us to stop using AI — it's calling for a healthy understanding of the human-machine relationship. AI chatbots are powerful tools — they can help with writing, coding, learning, and information retrieval — but they should not replace genuine human connections.

Maintaining clear cognitive boundaries means:

• Use AI to accomplish tasks, not to seek emotional comfort
• Be aware of where your data goes, and be cautious about sharing sensitive information
• Maintain critical thinking, and don't blindly trust AI outputs
• Preserve real social connections, and don't let AI become a substitute for human relationships

The Industry Needs More Responsible Design

From an industry perspective, AI companies also need to take on greater responsibility. Deliberately designing anthropomorphic features to increase user engagement is ethically questionable. Transparent AI identity disclosure, reasonable usage time reminders, and protection mechanisms for vulnerable users should all become industry standards.

Currently, the EU's AI Act already explicitly requires AI systems to disclose their non-human identity when interacting with humans. Multiple U.S. states are also advancing legislation targeting AI companion products, particularly regarding the protection of minors. However, the pace of technological development far outstrips that of regulation, making industry self-regulation crucial at this stage.

Conclusion

Meredith Whittaker's words may be brief, but they strike at a core contradiction in today's AI development: technology is becoming increasingly human-like, but it is ultimately not human. In an era of rapidly evolving AI capabilities, maintaining a clear-eyed understanding of what technology truly is may be more important than learning to use any new tool.

As she said — these are not your friends, not sentient beings, not conscious conversational partners. Remembering this is the key to truly coexisting with AI in a healthy way.