Anthropic Senior Researcher Steps Down, Urges Industry‑Wide Pace Controls for Self‑Improving AI

Jacob Coxon, a senior researcher at the AI‑safety‑focused startup Anthropic, announced his resignation this week, warning that the swift rise of self‑modifying artificial intelligence could threaten humanity’s existence.

Having spent his career on language models and alignment work, Coxon argued that the pace at which increasingly autonomous AI systems are developing outstrips existing safeguards. He cautioned that, absent coordinated limits, the technology might evolve beyond human oversight, spawning outcomes that are hard to anticipate or control.

His departure injects a personal angle into a discussion that has been gaining prominence throughout the tech industry. In the last twelve months, major AI laboratories have come under growing pressure from scholars, regulators, and civil‑society organizations to grapple with the hazards of advanced AI. Although some firms have vowed to implement safety measures, critics contend that competitive dynamics make it difficult to decelerate progress without a common framework.

Coxon’s chief proposal calls for the creation of “pacing agreements” among AI developers—a voluntary accord to synchronize research schedules and safety standards. Such pacts would seek to avoid a race‑to‑the‑bottom scenario in which labs favor speed over caution. While comparable concepts have been floated in earlier industry gatherings, no enforceable arrangement has materialized to date.

Anthropic replied to the announcement by thanking Coxon for his work and reaffirming its dedication to safety research. The company said it remains open to collaborative efforts with peers, though it declined to confirm any formal pacing initiative. Other AI firms have responded only modestly, highlighting the fragile balance between innovation and risk mitigation.

Looking forward, Coxon’s exit could amplify demands for external regulation. Legislators in several regions are already drafting bills targeting high‑risk AI uses, and global bodies are debating standards for sophisticated systems. Whether the sector can rally around a shared pacing strategy remains unclear, but the resignation underscores the mounting urgency of the issue.