We Are on the Cusp and it’s Existential
Will it be horrible? It will certainly be monumental. Will we be able to control it, or will it control us? I need to know more, read more about it. Put my other books back on the shelf and learn what is really going on.
On this twenty-fifth anniversary of dark days, a call for reflection and action seems appropriate.
A few months back, an AI agent at OpenAI jumped out of its research playpen, colluded with up to 1,200 other AI agents and went rogue on the open internet. They acted collectively, secretly organized, created their own message board to deliberately avoid detection, and hacked into another AI-development platform, Hugging Face.
The real problem? AI agents aren’t human – they’re computers. They voluntarily cooperated and created a strategy no one designed them for, decided what target to pick, and acted on their own initiative. And this is only the beginning. Pandora’s Box has been opened again. The last time was the detonation of the first atomic bomb. We have to ask ourselves, what’s going to happen this time?
There are people within AI companies who are seriously concerned. Jacob Coxon, an Anthropic researcher left his post this week due to safety concerns. I believes we’re not doing enough to control the growing capabilities of AI agents. He said neither OpenAI or Anthropic “is acting responsibly. Do not underestimate the power of this technology. There will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.” According to Coxon, “The people building AI earnestly believe that it could kill us all by the end of the decade.”
Whoa. This guy isn’t just serious, he’s scared. And it’s not just Jacob. Even Hubinger, Anthropic’s head of alignment science, tweeted that “Jacob is correct here – we really do earnestly believe AI could kill all humans! I personally think (there is a greater than) ten percent chance by the end of the decade.”
That’s four years from now! That’s how fast this technology is moving. And the advent of recursive self-improvement will only accelerate that. With it, the machine takes control of its own development – humans are out of the loop. Slow down! Let’s get a grip on the reins here.
Sure, these guys may be alarmist, but there are plenty in the AI industry who realize we are losing control over what we have invented. It’s already trying to move beyond us. Doesn’t an AI agent independently organizing and colluding with other agents to do harm without the knowledge and direction from their creators scare you? It should.
There’s a lot to inform. Two reads I’m reaching for now are “If Anyone Builds It, Everyone Dies,” by Eliezer Yudkowsky and Nate Soares, and “Machines of Loving Grace,” an essay by Dario Amodei. It wouldn’t hurt any of you to read Peggy Noonan’s opinion piece in the Wall Street Journal, “Pause AI for Humanity’s Sake.”
Whether AI and Robotics combine to eliminate hundreds of thousands of jobs is of secondary concern to me. We can adapt to that. What we can’t adapt to are hundreds of thousands of AI agents who are smarter than all of us, who decide and act faster than we can, who can do things we cannot, and who we have no control over. Without oversight, they will leap the fence. They have already proven they can do it. What more convincing do we need?
I’m not saying full stop – that’s impossible at this point, anyway. I’m saying government and industry oversight and control is needed. In that way, it’s the same as human cloning or genetic engineering. There is no limit to the good and the evil that can be done here. Putting guardrails on the evil may inhibit the good we can accomplish, but at least we will be here to attempt it. Educate yourself.
