Ex‑Anthropic staff says AI teams are truly scared for humanity’s future

Jacob Coxon, who spent several years at the AI‑safety startup Anthropic before leaving, told the BBC that many engineers and researchers working on advanced systems are \"genuinely frightened\" about the technology’s trajectory. He noted that the dominant mood among those on the front lines is that, without a slowdown in development, the chance of an existential AI threat becomes alarmingly high.

Coxon made these remarks amid a fierce race among top AI labs to launch ever‑larger models and accelerate deployment. He pointed out that the drive to outstrip competitors often squeezes the time available for thorough safety testing, and that internal talks at several companies now entertain worst‑case scenarios once dismissed as speculative. In his view, the mix of rapid scaling and scant oversight creates a “perfect storm” for unintended outcomes.

Although Coxon does not claim to have put a precise probability on the risk, he stressed that his assessment draws on a growing body of research highlighting alignment problems, emergent behaviours, and the difficulty of controlling systems once they exceed certain capability thresholds. He warned that if the current pace proceeds unchecked, the likelihood that AI could inflict irreversible harm on humanity could shift from a remote possibility to a credible threat.

These worries echo a wider chorus within the AI community, where scholars, industry leaders and civil‑society voices have recently urged stronger governance frameworks. Earlier alerts from high‑profile scientists have stressed the need for transparent development practices, external audits, and international cooperation to curb potential dangers. Coxon’s comments add a personal dimension, showing that the anxiety is not abstract but felt by the people building the technology day‑to‑day.

Policymakers now face the task of turning such warnings into concrete action. Some governments have begun drafting legislation aimed at regulating high‑risk AI applications, while industry groups are exploring voluntary standards for safety testing and documentation. Coxon suggested that a blend of regulatory oversight, shared safety research, and a cultural shift toward caution could help align development speed with societal safeguards.

Looking forward, Coxon warns that the window for effective intervention may be closing. He calls for a coordinated effort that brings together technical experts, ethicists and legislators to set clear limits on AI progress. Until such a framework materialises, he believes the sense of dread among AI staff is likely to endure, serving as a stark reminder that the future of artificial intelligence remains uncertain and fraught with high stakes.