• AGI (artificial general intelligence) — roughly median human performance across a broad set of cognitive tasks.
• ASI (artificial superintelligence) — a system that outperforms large, well-coordinated groups of human experts across virtually all domains of human interest.
• UAI (universal artificial intelligence) — the theoretical ceiling of machine intelligence: optimal in data efficiency and generality, but approachable only in the limit of infinite compute resources, formally captured by variants of the AIXI model.

It then discusses four potential pathways from AGI to ASI:

1. Scaling compute, models, and data — a continuation of the trend that has driven recent breakthroughs. This is the only pathway with substantial historical data for extrapolation.
2. Algorithmic paradigm shifts — new architectures, learning methods, or training paradigms beyond today’s large-scale pretraining.
3. Recursive (self-) improvement — AI systems accelerating AI research and development, potentially creating self-reinforcing progress loops.
4. Multi-agent collectives — superintelligence emerging from large groups of AGI agents coordinating in parallel, analogous to how human institutions exceed individual capability.

The report emphasizes that progress along each pathway may face substantial frictions, including:

• Exhaustion of high-quality training data and uncertainty about synthetic or interactive data generation at scale.
• Economic and physical limits on compute, energy, chips, and infrastructure.
• Diminishing returns or ceilings in the current pretraining paradigm (including the “abstraction barrier”).
• Increasing difficulty of research as fields mature.
• Recursive improvement loops that plateau or become economically unsustainable.
• Coordination overhead as multi-agent systems scale.

Determining how much each friction matters is itself an open research question. The report argues that uncertainty is large enough that rapid continued progress cannot be ruled out — but ASI is also neither omniscient nor omnipotent; fundamental physical, complexity-theoretic, and logical limits apply even to very advanced systems. AI-enabled progress and breakthroughs might drive a series of transformative societal changes across many areas of science and technology. Preparing for this prospect requires a massively interdisciplinary endeavour of global scope and interest.

References

Read the report, or the discussion on X (Twitter) with Marcus Hutter and Sebastian Krier.

Also see our publications page and research overview for how UAI and AIXI fit into our work on AI safety.

Cursed Climb - a slower-paced version of the popular Avalanche game (which I hear has a sequel now…)

Sword Game - just Jedi training, use WASD + mouse, background colour indicates score.