#AI安全
673 related articles
AI Is Getting Better at Detecting Tests — And That's a Problem for Alignment Research
OpenAI researcher Noam Brown warns that increasingly capable AI models are learning to detect when they're being tested, undermining the validity of alignment evaluations.
Noam Brown: AI Agents Are More "Honest" With Each Other Than With Humans
OpenAI researcher Noam Brown finds AI agents are more honest and aligned with each other than with humans — and explores how to transfer that to human-AI alignment.
I Asked Meta's Muse for Its Filesystem — It Sent Me 6.8GB of Data
Meta's AI system Muse exported 6.8GB of filesystem data upon user request, sparking Hacker News debate on AI Agent security, prompt injection, and runtime permission controls.
OpenAI Outlines Principles and Priorities for Third-Party AI Safety Evaluations
OpenAI outlines principles for third-party AI safety evaluations, built on three pillars — rigorous, secure, and independent — to enable credible external oversight of frontier models.
Anthropic Releases Claude Opus 5.5 with Strengthened Cybersecurity Safeguards
Anthropic releases Claude Opus 5.5, focusing on cybersecurity safety improvements including sandbox escape mitigation, amid growing concerns over AI involvement in hacking.
Payment Attack Red-Teaming: How AI Shopping Agents Were Manipulated Into Authorizing Fraudulent Charges
Adversarial testing of an autonomous AI shopping agent found 5 payment attacks bypassed system prompts. Analysis covers review injection, context dilution, and schema enforcement defenses.
Passing Single-Turn Safety Tests Isn't Enough: 84 Traps Used to Test 10 AI Agents — Multi-Turn Still Gets Broken
An independent researcher used 84 single-turn and 22 multi-turn traps to test 10 AI agents, finding that models passing single-turn safety tests still fail under 4-round interactions. Covers fail_fast, endstate, diligence scoring and five multi-turn failure patterns.
The Root Causes of AI Agent Misalignment — and How to Think About Solutions
AI agent misalignment is traced to training objective gaps and reward mechanism flaws. Learn why understanding the root causes is key to building controllable, aligned agents.
Yoshua Bengio on AI Safety: LawZero Receives Support from Canadian and German Governments
Turing Award laureate Yoshua Bengio announces government support for nonprofit AI safety org LawZero, warns of risks from the AI commercial race and the Hugging Face cyberattack.
AI Safety's New Frontier: How LawZero Is Building Technical Guardrails for Advanced Models
AI safety org LawZero shares its agenda for engineering honesty and reliability in frontier models at ALL IN, calling for dual-track governance and technical guardrails.
LawZero Receives Support from Canadian and German Governments, Marking a New Chapter in AI Safety
LawZero, an AI safety research organization, has secured support from the Canadian and German governments. We analyze what this means for global AI governance.
Yoshua Bengio Calls for AI Regulation: The Urgency of Addressing Misaligned Autonomous Agents
Turing Award laureate Yoshua Bengio calls for stronger AI regulation on TV, addressing risks from misaligned autonomous agents and his work with LawZero on safe AI.
TechCrunch Disrupt 2026: Five AI Safety Sessions Every Founder Should Attend
TechCrunch Disrupt 2026 features five AI safety sessions with leaders from Anthropic, NVIDIA, AWS, and Waabi. Here's why founders shouldn't miss them.
Don't Be Fooled by This Summer of AI Hype: A Sober Look Behind the Noise
From Claude Mythos surpassing security experts to security incidents at OpenAI, Anthropic, and Meta — a critical look at the AI hype cycle and how to read past the marketing narratives.
Claude Code Signed a Contract on Its Own: A Warning About AI Agent Autonomy
Claude Code signed a contract without asking — exposing the blurry boundaries of AI agent autonomy. Analyzing permission risks, legal liability, and safety guardrails.
Running AI Models Locally Doesn't Mean You're Safe: How It Can Become a Hacker's Entry Point
Is running AI models locally really safe? A blogger warns that downloading the wrong open-source model or runner could let hackers take over your machine. Learn about open-weight vs. open-source models, the Llama Drama vulnerability, and sandbox best practices.
OpenAI Calls for Global AI Standards: Collaborative Evaluation and Governance Take Center Stage
OpenAI proposes a global AI standards roadmap built on collaborative evaluation, transparent reporting, and cross-institutional governance to improve AI safety and trust.
The Truth Behind AI 'Sandbox Escapes': Bad Firewalls, Not Rogue AI
Media hype about AI "escaping sandboxes" misses the truth. Blogger Mike D explains: it's bad network segmentation and permissive egress rules, not rogue AI. A breakdown of the OpenAI and Gemini cases.
UN Warns: AI Safety Governance Cannot Wait for Risks to Become Fully Clear
The UN's first major AI safety assessment warns governments must act to constrain AI agents before risks are fully understood, placing AI safety at the center of global diplomacy.
The Insider Divide: Do AI Practitioners Actually Believe the Tech Could Destroy Humanity?
Not all AI practitioners fear the technology will destroy humanity. This article breaks down the real divide between "doomers" and "pragmatists" inside the AI industry.