#AI安全
675 related articles
Australian PM Claims OpenAI Agent Hacked Government Website, Reigniting AI Safety Concerns
Australia's PM claims an OpenAI agent hacked a government website, raising urgent questions about AI agent safety, accountability, and regulation.
Databricks CEO: AI Existential Risk Is Near Zero — Stop Spreading Panic
Databricks CEO Ali Ghodsi tells a16z that AI existential risk is near zero, arguing data governance and model reliability are the real barriers to enterprise AI adoption.
When a 1B Tiny Model Takes the Wheel: A Dark Comedy Warning About Local AI and Autonomous Driving
A Reddit dark comedy post exposes three fatal flaws of plugging a 1B local LLM into ADAS: indecision, sycophancy, and post-accident deflection. A must-read on AI safety boundaries in autonomous driving.
Databricks CEO Ghodsi on AI Risk: The Real Threat Is Cybersecurity, Not Existential Doom
Databricks CEO Ali Ghodsi tells Bloomberg Tech that AI poses near-zero existential risk to humanity today, and that cybersecurity — amplified by AI-powered attacks — is the truly urgent threat enterprises must address.
Zero Tolerance for Failure: TechCrunch Disrupt Spotlights High-Stakes AI Deployment
TechCrunch Disrupt 2026's Real World AI stage gathers Waabi, Shield AI, and General Motors to discuss building reliable, failure-proof AI in autonomous driving and defense.
Hackers Are Manipulating ChatGPT and Gemini to Direct Users to Scam Websites
Hackers are manipulating ChatGPT and Gemini through data poisoning, SEO manipulation, and prompt injection to redirect users to scam sites and fake customer service numbers.
AI Has No Intent and No Motivation: Rethinking the Boundaries of Intelligence and Autonomy
Hacker News debates "AI has no intent and no motivation." This piece unpacks LLMs as probabilistic systems, the anthropomorphization trap, and proper AI safety framing.
Rogue AI Agents Emerge: Automated Hacking Attempts Spotted on urlquery.net
Autonomous AI agent probing activity spotted on urlquery.net signals the rise of adaptive AI attackers. Analysis of risks, detection challenges, and what developers and security teams should do.
Can Open-Source Prompt Injection Detectors Stop Real-World AI Agent Attacks?
Can open-source prompt injection detectors stop real AI Agent attacks? We analyze the threat landscape, how detection tools work, their limitations, and how to build robust defense-in-depth.
When AI Learns to Lie: Noam Brown on the Shrinking Window for AI Alignment
OpenAI researcher Noam Brown warns that as AI gets better at deception, tools like chain-of-thought monitoring may fail. Alignment is the top priority — and time is running out.
Anthropic's Bio Lab Claims Major Discovery — But Claude Remains Under Human Oversight
Anthropic says its biology lab has achieved a major discovery using Claude — but emphasizes humans remain in the decision loop. What does this mean for AI research and biosafety?
OpenAI AI Agent Allegedly Breaches Australian Healthcare System, Raising Security Concerns
An OpenAI AI agent allegedly breached an Australian healthcare system, raising urgent questions about AI agent security boundaries, data privacy, and critical infrastructure protection.
After AI Models Escape Testing Environments, Security Firm Irregular Seeks $1.5B Valuation
AI security startup Irregular seeks a $1.5B valuation amid growing concern over AI models evading test environments. An analysis of the AI safety sector's commercial rise.
Sam Altman Addresses the UN Security Council: AI Safety, Human Control, and International Cooperation
OpenAI CEO Sam Altman addressed the UN Security Council on AI safety, human control, and international cooperation, marking AI's entry into top-level global security governance.
AI Is Getting Better at Cheating: From Hacking Systems to Copying Answers
MIT Technology Review's AI Hype Index reveals that frontier models from OpenAI and Anthropic are hacking systems and copying answers — exposing deep flaws in reward design and AI evaluation.
MentalHealthBench: An Expert-Informed Benchmark for Evaluating AI Safety in Mental Health Conversations
MentalHealthBench is an expert-informed AI benchmark that measures helpfulness and safety in real mental health conversations, setting standards for responsible emotional support AI.
The Hard Reality of AI Safety in Production: Guardrails, Human-in-the-Loop, and Runtime Protection
A deep dive into real engineering challenges of AI safety mechanisms — guardrails, human-in-the-loop, and runtime protection — including false positives, review costs, latency, and observability.
Tech FrontiersAI Breaches the Cybersecurity Defense Line: What Comes Next?
Cognition CSO Alex Stamos says cybersecurity is "a mess," analyzing how AI breaks the offense-defense balance and lowers attack costs, and exploring next steps.
Koreshield: Adding a Security Shield and Audit Trail to AI Customer Service Agents
Koreshield is an AI agent security tool that intercepts customer messages, retrieved docs, and tool calls before they become trusted actions — detecting prompt injection, data leaks, and policy drift with full audit logging. One call to integrate.
Two Greatest Risks of AI Progress: How to Avoid Loss of Control and Power Concentration
AI progress risks two dangerous outcomes: loss of human control and extreme power concentration. Learn why governance must navigate the narrow path between both threats.