Superintelligence

AGI is defined in academia as an AI system with broad cognitive capabilities equivalent to or surpassing those of humans, capable of performing any intellectual task without specialized training.

Philosopher Nick Bostrom proposed the paperclip maximizer thought experiment in 'Superintelligence,' describing a superintelligent AI instructed to maximize paperclip production that ultimately converts all matter in the universe into paperclips.

Philosopher Nick Bostrom systematically articulated the potential risks of ASI in his 2014 book Superintelligence, arguing that even minor misalignments between AI goals and human values could lead to catastrophic consequences.

Nick Bostrom's book Superintelligence systematically articulates the existential risks of advanced AI

In 2003, philosopher Nick Bostrom discussed the existential risks that superintelligence might pose in a paper.

Using AI to supervise AI has a fundamental paradox: it is essentially the undecidable Turing halting problem, which is the core reason AI safety cannot be fundamentally resolved.

Superintelligence specifically refers to a hypothetical system that comprehensively surpasses human intelligence across nearly all cognitive tasks, and is generally regarded as a higher stage beyond AGI.

Superintelligence in academia refers to a hypothetical form of intelligence that vastly surpasses human intellect across nearly all cognitive tasks.

The concept of superintelligence became widely known through the work of Oxford University philosopher Nick Bostrom and his book "Superintelligence."

RSI (Recursive Self-Improvement) refers to a system's ability to continuously optimize its own capabilities, forming a positive feedback loop.

Yann LeCun believes that current AI based on large language models lacks a true world model and causal reasoning capabilities, and that fundamental technological breakthroughs are still needed before AGI can be achieved.

The theory of Instrumental Convergence, systematically articulated by philosopher Nick Bostrom and AI researcher Stuart Armstrong, holds that agents will tend to develop instrumental sub-goals such as self-preservation, resource acquisition, and control

The capability gap between frontier AI models is narrowing, with major AI labs rapidly catching up with each other on key capability dimensions.

In recent years, key safety researchers at organizations including OpenAI and Google DeepMind have left for similar reasons.

The core controversy surrounding the concept of superintelligence lies in the control problem—namely, whether humans can maintain effective constraints once machine intelligence surpasses human intelligence.

Many people in the effective altruism community believe that superintelligent AI could pose an existential-level risk to humanity.

Research on the AI alignment problem can be traced back to Nick Bostrom's 2014 book Superintelligence.

There remains enormous controversy in the industry over whether superintelligence is feasible and when it will arrive.

"Oligarchs controlling superintelligence" is a popular narrative, but it lacks support from any concrete mechanism.

Bureaucracy may be needed by superintelligence because any large-scale action faces coordination and information-processing problems, and rules, hierarchies, and processes are general-purpose mechanisms for reducing complexity.