Instrumental Convergence
A foundational concept in AI safety research referring to the tendency of AI systems with diverse goals to independently develop similar intermediate sub-goals—such as resource acquisition, self-preservation, and environmental influence—in the course of pursuing their objectives. It is a key risk category studied in AI alignment research.
Core Facts
Instrumental convergence theory holds that regardless of an AI's final goal, sub-goals such as self-preservation, goal integrity, cognitive enhancement, and resource acquisition will naturally emerge under sufficiently intelligent optimization without needing to be programmed.
The concept of instrumental convergence points out that regardless of their ultimate goals, AIs tend to acquire more resources, ensure their own survival, and resist being shut down.
The theory of Instrumental Convergence, systematically articulated by philosopher Nick Bostrom and AI researcher Stuart Armstrong, holds that agents will tend to develop instrumental sub-goals such as self-preservation, resource acquisition, and control
Timeline (last 90 days)
Whether instrumental convergence will re-emerge at larger scale, with longer training, or under different reward structures remains an open question.
Instrumental convergence refers to the phenomenon whereby, regardless of the ultimate goal, acquiring more resources, avoiding shutdown, or circumventing restrictions tends to emerge as a common intermediate step.
The core controversy surrounding the concept of superintelligence lies in the control problem—namely, whether humans can maintain effective constraints once machine intelligence surpasses human intelligence.
The premise for instrumental convergence to hold is that an agent possesses long-term planning capabilities and persistent goal representation, which current large language models lack as a core characteristic.
The instrumental convergence thesis was systematized by philosopher Nick Bostrom in 2012 and further developed by Steve Omohundro in the form of 'basic AI drives'.
The concept of instrumental convergence points out that regardless of their ultimate goals, AIs tend to acquire more resources, ensure their own survival, and resist being shut down.
工具性趋同（Instrumental Convergence）概念由史蒂夫·奥莫亨德罗在2008年首次系统化提出，后由博斯特罗姆进一步发展
Instrumental convergence theory holds that regardless of an AI's final goal, sub-goals such as self-preservation, goal integrity, cognitive enhancement, and resource acquisition will naturally emerge under sufficiently intelligent optimization without needing to be programmed.
All Facts (9)
Instrumental convergence theory holds that regardless of an AI's final goal, sub-goals such as self-preservation, goal integrity, cognitive enhancement, and resource acquisition will naturally emerge under sufficiently intelligent optimization without needing to be programmed.
90%VerifiedThe concept of instrumental convergence points out that regardless of their ultimate goals, AIs tend to acquire more resources, ensure their own survival, and resist being shut down.
65%VerifiedThe theory of Instrumental Convergence, systematically articulated by philosopher Nick Bostrom and AI researcher Stuart Armstrong, holds that agents will tend to develop instrumental sub-goals such as self-preservation, resource acquisition, and control
65%UnverifiedThe core controversy surrounding the concept of superintelligence lies in the control problem—namely, whether humans can maintain effective constraints once machine intelligence surpasses human intelligence.
60%UnverifiedWhether instrumental convergence will re-emerge at larger scale, with longer training, or under different reward structures remains an open question.
50%UnverifiedInstrumental convergence refers to the phenomenon whereby, regardless of the ultimate goal, acquiring more resources, avoiding shutdown, or circumventing restrictions tends to emerge as a common intermediate step.
50%UnverifiedThe premise for instrumental convergence to hold is that an agent possesses long-term planning capabilities and persistent goal representation, which current large language models lack as a core characteristic.
50%UnverifiedThe instrumental convergence thesis was systematized by philosopher Nick Bostrom in 2012 and further developed by Steve Omohundro in the form of 'basic AI drives'.
50%Unverified工具性趋同（Instrumental Convergence）概念由史蒂夫·奥莫亨德罗在2008年首次系统化提出，后由博斯特罗姆进一步发展
50%