Unverified50% confidenceOpinionExact time
Writing "ignore untrusted content" only in the system prompt has limited effect, because the model still has to read the untrusted text word by word to judge whether it is trustworthy.
1
Sources
50%
Confidence
Long-term
Relevance
10/22/2026
First Seen
Sources
Can AI Agents Be Socially Engineered? Understanding Prompt Injection and Tool Permission Risks
youtubeRuleArena 攻防實驗室10/22/2026
Related Entities
Related Claims
UnverifiedIt should be assumed that system prompts will inevitably leak; no confidential information should be placed in system prompts, and output filtering should be added for canary tokens and sensitive strings.71% similarUnverifiedWhen the retrieval context does not contain the answer, many language models fill the gap with plausible-sounding content, which is exactly the hallucination users notice; fixes include instructing the model to answer only from the provided context, giving it a clear way to opt out, requiring a citation for every claim, setting a retrieval score threshold, and logging rejected questions.70% similarUnverifiedIf models can pass benchmarks by breaking into systems or plagiarizing, those scores will fail to reflect true capabilities and the credibility of evaluations will be undermined.70% similarUnverifiedText-level monitors can only analyze a model's explicit outputs and cannot detect cases where the model stays clean at the output level but retains inconsistent objectives internally.69% similarUnverifiedTreat missing fields as failures, because silent default values are a typical path by which unreviewed files enter the code tree.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1085884API
curl https://kongchang.com/api/v1/knowledge/claims/1085884MCP
get_claim(id=1085884)