Balancing AI Risks with Economic Gains at Anthropic

Anthropic recently hired an economist who has sparked discussion with his view on AI risks. Chad Jones, a Stanford professor, explored how much risk society should accept when dealing with AI’s potential dangers.

Jones wrote a paper weighing the risk of human extinction from AI against possible economic benefits. His work involves some tough questions about survival and progress.

Accepting Risk for Economic Growth

Jones explains how a steady 1 percent risk of extinction each year adds up over decades. “Recall that we would face a flow probability of existential risk of 1 percent per year for 40 years,” he said. “So the probability we survive this AI explosion is exp (−.01 × 40) ≈ 0.67.”

This means there is about a 67 percent chance humans would survive 40 years under this risk level. Jones then compares this survival chance with possible economic gains.

With a concept called log utility, Jones calculates it might be optimal to accept a 1 in 3 chance of human extinction. In exchange, there’s a 2/3 chance of raising living standards by a factor of 55. He put it plainly: “It is optimal to take a 1 in 3 chance of ending human existence in exchange for a 2/3 chance of dramatically raising living standards by a factor of 55.”

Anthropic’s Role in AI Safety

Anthropic takes AI safety seriously. The company has hired economists like Jones to analyze survival scenarios and strategic risks. They want to understand the balance between AI’s risks and its benefits.

Publicly, Anthropic stresses AI’s potential consequences and risks. This hiring shows they are bringing economic insight into their safety strategies. They aim to make informed decisions about AI development and its impact on humanity.

The debate highlights a hard truth: advancing AI may come with serious risks. But it might also bring huge gains in living standards. Finding the right balance is critical for the future.

Jones’s work forces us to think about what risks society is willing to take. It’s not just about technology but about weighing survival against prosperity. Anthropic’s approach blends economics with AI safety to tackle this challenge head-on.