AI Alignment and Global Governance: New Directions in Tech Safety from the Attention Conference

A Conversation on AI Safety and Governance

At the Attention Conference held in Montreal last week, a fireside chat sparked in-depth discussion on AI alignment and global governance. According to a Twitter user who shared the conversation, they and @camillefrancois explored the technical work being advanced by @LawZero_, focusing on solving the "misalignment" between AI systems and human values, as well as the potential risks this poses to society and democratic institutions.

The value of this conversation lies in the fact that it did not stop at abstract ethical appeals, but placed technical practice and institutional design side by side. LawZero_'s work points to a key reality: when the behavioral logic of AI systems deviates from human intent, post-hoc fixes alone are far from enough—alignment mechanisms need to be embedded in the early stages of model training and deployment.

Why Technical Alignment Has Become an Urgent Issue

AI alignment discusses how to ensure that the goals and behavior of AI systems conform to human intent and values. As model capabilities rapidly improve, the urgency of this issue has risen significantly. The more capable a system is, the harder it becomes to predict and correct the unintended consequences if its goals are set incorrectly.

The "emerging risks" that LawZero_ focuses on are not science fiction imaginings, but refer to the behavioral patterns that systems may develop that developers did not anticipate as model scale expands and autonomy increases. The characteristic of such risks is that they are difficult to fully cover through traditional testing methods and require continuous technical monitoring and dynamic adjustment mechanisms. The difficulty of alignment research lies precisely in the fact that it must ensure the controllability of the system without excessively limiting its usefulness.

AI Alignment is a core subfield of AI safety research, whose goal is to make the actual objective functions and behavioral strategies of AI systems consistent with the true intentions of human designers or society. The concept was first systematized by machine learning researcher Stuart Russell and others in discussions of "value alignment," emphasizing that AI cannot be constrained only by fixed rules, but must be made to understand and infer human intent. The difficulty of alignment is often summarized as "Goodhart's Law" and "reward hacking": when a model is optimized for some proxy metric, it may find shortcuts that deviate from the original intent. Common technical approaches include reinforcement learning from human feedback (RLHF), scalable oversight, interpretability research, and red teaming. Alignment is not synonymous with "AI safety": the former focuses on internal goal consistency of the system, while the latter also covers external risks such as misuse prevention, robustness, and privacy.

Global AI Governance Requires International Coordination

At the same conference, the speaker also participated with @camillefrancois and @mariaressa in a dedicated discussion on global AI governance. The discussion cited the key work of the UN Independent International Scientific Panel on AI, emphasizing two core propositions: the necessity of international coordination and the importance of digital sovereignty.

"The trajectory of AI development should be guided democratically, not driven solely by commercial pressure"—this view directly addresses the structural contradiction in current AI development. Commercial competition drives rapid technological iteration, but speed itself does not mean the direction is correct. The proposal of digital sovereignty responds to the reality of increasingly stark disparities among countries in data, computing power, and model capabilities: if governance frameworks are dominated only by a few technological powers, the interests and values of Global South countries will be difficult to reflect in AI systems.

The work of the UN scientific panel provides a knowledge base for such coordination. Unlike corporate-led governance initiatives, the international scientific panel attempts to assess AI risks in a transnational and interdisciplinary manner, providing a relatively neutral basis for policymaking. But the challenges are equally obvious: how scientific consensus can be translated into binding governance rules, and how to balance digital sovereignty with coordination among states, remain unresolved problems.

Digital sovereignty refers to the demand of states or political communities to exercise autonomous control over their own data, computing infrastructure, algorithmic models, and governance rules. This concept emerged in European digital policy discussions, emphasizing data localization, platform regulation, and domestic AI capacity building; in the context of developing countries, it refers more to avoiding being locked in by a few technological powers in terms of model capabilities and standard setting. Alongside digital sovereignty is the term "AI sovereignty," meaning that states want to train and deploy models within their own language, cultural, and legal frameworks. The controversy surrounding digital sovereignty lies in the fact that overemphasizing sovereignty may fragment the global model ecosystem and raise compliance costs, while complete laissez-faire may allow governance rules to be dominated by a few companies or countries. The UN Independent International Scientific Panel on AI is precisely trying to provide transnational scientific assessment amid this tension, but its recommendations are non-binding and must go through intergovernmental negotiations before they can be translated into rules.

Technology and Governance Advancing in Parallel

Looking at the fireside chat and the governance discussion together, this Attention Conference conveyed a clear signal: AI safety cannot be advanced unilaterally by the tech community or policy institutions alone. Technical alignment addresses the issue of behavioral consistency within systems, while global governance addresses the issue of rule distribution outside systems. Neither can be dispensed with.

For practitioners and researchers concerned with the field of AI safety, both LawZero_'s technical path and the UN scientific panel's governance framework are worth continued tracking. The former represents the advancement of alignment research at the practical level, while the latter tests whether the international community can establish effective coordination mechanisms in the AI era. This Montreal conversation at least brought these two issues together on the same table.

One-line Summary

Attention Conference focuses on AI alignment technology and global governance advancing in parallel.

Paragraph Summary

The article reviews discussions on AI safety and governance at the Attention Conference in Montreal. The fireside chat focused on LawZero_'s work on AI alignment technology, aimed at solving the "misalignment" between AI systems and human values and addressing emerging risks; the dedicated discussion cited the work of the UN Independent International Scientific Panel on AI, emphasizing international coordination and digital sovereignty, and arguing that AI development should be guided democratically rather than driven by commercial pressure. The article points out that technical alignment addresses behavioral consistency within systems, while global governance addresses external rule distribution; neither can be dispensed with. However, difficult issues remain to be resolved, including the controllability trade-offs in alignment research, the translation of scientific consensus into binding rules, and the balance between digital sovereignty and coordination among states.