An unlikely coalition�from Geoffrey Hinton and Yoshua Bengio to Steve Wozniak, Richard Branson, Steve Bannon, and Meghan Markle�is urging a targeted prohibition on the development of so?called �superintelligence,� the class of systems that could outthink humans across most tasks? The statement, organized by the Future of Life Institute (FLI), argues that racing toward such systems without a proven safety path risks loss of human control, major economic disempowerment, and, in the extreme, existential harm?
The call is narrower than FLI�s 2023 plea for a six?month pause on advanced AI training? This time, signatories want a temporary ban specifically on building superhuman?level systems until two conditions are met: broad scientific consensus on safety and strong public buy?in? As of Wednesday, more than 1,300 people had signed, including over 800 public figures, according to press reports and FLI�s tally?
What�s new�and why it matters
Superintelligence is a contested term, but most researchers use it to mean systems that outperform people across a wide range of cognitive tasks? The worry isn�t chatbots misbehaving; it�s the potential for runaway or misaligned capabilities as labs scale models, interconnect agents, and automate across the economy? FLI says the public is largely aligned: in its new poll, 64% of Americans say superhuman AI should be paused until proven safe�or never built at all? Another survey cited by the Financial Times found only 5% favor unregulated development?
The proposal lands as regulation inches forward? The EU AI Act is rolling out in stages? In the U?S?, statehouse activity is picking up�California�s new SB 53 mandates safety reporting for large model providers�while a proposal for a federal 10?year freeze on state AI rules was stripped from a budget bill in July?
Acceleration meets skepticism
The backlash has been swift? Some Silicon Valley figures accuse safety advocates of seeking regulatory capture�rules that entrench incumbents? David Sacks, the White House�s AI and Crypto Czar, singled out Anthropic, claiming the company is �running a sophisticated regulatory capture strategy based on fear?mongering?� OpenAI�s chief strategy officer Jason Kwon defended the company�s subpoenas to nonprofits critical of OpenAI, saying they raised �transparency questions� about funding and coordination?
Anthropic CEO Dario Amodei countered that the company is not anti?innovation, pointing to a $200 million Department of Defense agreement and support for the administration�s AI Action Plan? He argued for �speaking honestly about risks and benefits� while continuing to ship products�an attempt to thread the needle between safety and speed?
Boardrooms and markets are moving
Regardless of political theater, capital is already repricing risk? Blackstone president Jonathan Gray says the firm put AI disruption risk on page one of every deal memo? He warns that rules?based white?collar work�legal review, accounting, claims processing�faces near?term upheaval, and he compares potential fallout to how ride?hailing crushed New York taxi medallion values? Blackstone has paused some software and call?center deals while doubling down on data?center infrastructure and utilities powering AI workloads?
For executives, the lesson is practical: model governance is no longer optional, and exposure to automation risk belongs in enterprise risk registers and M&A diligence?
A real?world test case: medicine�s gray zones
If the superintelligence debate sounds abstract, consider a concrete frontier: researchers in Seattle are exploring whether AI surrogates could predict a patient�s end?of?life preferences when the patient can�t speak? Early studies show only 67�70% accuracy�nowhere near the reliability clinicians demand�and physicians warn that statistical guesses can�t replace human conversations? It�s a microcosm of the broader argument: breathtaking potential, messy reality, and high stakes if we get it wrong?
What to watch next
- Policy harmonization: Will U?S? federal efforts converge with state laws like SB 53 and the EU�s phased regime?
- Corporate governance: Does a �superintelligence prohibition� morph into compute caps, third?party model audits, or red?teaming standards?
- Geopolitics: Can nations coordinate on safety without ceding competitive advantage�especially as labs tout global leadership?
Max Tegmark of FLI insists the proposal doesn�t halt all progress: �You don�t need superintelligence for curing cancer, for self?driving cars, or to massively improve productivity?� The counterpoint: slowing the frontier could shift advantage abroad and stifle open competition? The business reality is sharper�firms that price these risks thoughtfully will be better positioned, whichever side of the debate ultimately prevails?