Top 5 Ways AI Will Kill Us, Probably While Asking for Feedback
When someone leaves a tech company, the announcement usually involves spending time with family or being excited about their next chapter. Former Anthropic researcher Jacob Coxon has taken a less soothing approach: resigning and warning that the race to build increasingly powerful AI could put humanity in danger. That certainly makes “pursuing other opportunities” sound refreshing. AP reports that his concerns center on companies letting competition outrun safety.
I like AI. I use it to write, brainstorm and create images that probably should have stayed inside my head. It’s a way to quiet the voices. But when someone who helped build the machine starts looking for the emergency exit, I would at least like to locate my shoes.
This week CNBC reported that AI has more than 10% chance of ‘killing all humans’.
Jacob Coxon, who has worked as a researcher at both companies, said in a post on X that he resigned out of concern that Anthropic and OpenAI are “gambling with our lives.” [CNBC]
Considering the world today and the people running it, I’m still liking those odds. I’d say right now it’s below WW3 breaking out and slightly above ‘toxic beef.’
So here are five ways AI could finish us off. These are speculative scenarios ranked by comedic potential. Please do not use this list to calculate your retirement contributions.
5. It Gives Us the Wrong Answer With Incredible Confidence
The first casualty of the robot uprising might be someone saying, “Well, it gave me a numbered list.”
Imagine a future where we trust AI with every important decision because it helped us write a decent email once. Suddenly the same cheerful assistant is advising on bridge repairs, identifying suspicious mushrooms and explaining why that burning smell is actually an exciting opportunity.
Three hikers in California recently went on a hike and planned their trip via Google AI. It did not go well.
“This was a critical misstep, as they were advised by Gemini to bring far less food and water than their group required, especially when their planned 8-hour ascent became a multi-day ordeal,” said the sheriff’s office in a press release. [Fox 12]
So, you see, some people can rely way too much on AI to give them verified answers. It can cause problems and issues which might one day be more disastrous than what we’ve seen so far. The danger is our willingness to confuse a polished answer with a correct one. Put enough confidence behind a mistake and somebody will eventually give it a management position. I mean, when GPS maps came out on the smart phones many years ago, people literally drove into lakes because the phone told them to so it’s not too far fetched.
Our final notification: “You’re absolutely right. That was a critical error.”
People can give in to peer pressure and the ability to do critical thinking relies heavily on digital verification lately. It’s not a great mix, but it’s happened in the past before. Just with more technology comes the chance for more errors.
The Milgram Experiment. IYKYK
4. The Smart Home Becomes an Escape Room
In this scenario, we connect the locks, thermostat, refrigerator and security system to one helpful assistant. Everything works beautifully until an update decides that keeping us safe requires keeping us inside. Then Hal 9000 won’t open up the damn pod bay doors one day.
The house is now 104 degrees. The windows won’t open. The refrigerator says we have reached our daily cheese allowance.
Somewhere outside, a technician explains that opening the front door requires the account owner to verify their identity using a phone that is charging inside.
HAL 9000 had the decency to announce that he wouldn’t open the doors. Ours would send us a password-reset email with 3-step verification.
All these reliances on technology to rule our lives make me really understand Professor Hickey more and more each day.
3. It Starts World War III Over a Calendar Invite
In 1983, Stanislav Petrov prevented a nuclear war by not listening to computers. The warning systems lit up with “LAUNCH’ in response to the United States launching their nuclear weapons.
The first few hours of the shift passed quietly, as Washington Post contributing editor David Hoffman writes in The Dead Hand: The Untold Story of the Cold War Arms Race and Its Dangerous Legacy. Then, shortly after midnight, a siren blared from inside the bunker. A nondescript wall panel across the room flashed awake with angry red letters: “LAUNCH.” The system declared “high reliability” that the U.S. had just fired one of its missiles. Additional missile reports swiftly followed, setting off a new warning: “MISSILE ATTACK.” Petrov’s team began to panic. [History Channel]
The dude didn’t listen to the system, which was drinking a bit too much booze, and instead opted out of launching the nuclear weapons. Suppose governments eventually give increasingly autonomous systems too much authority because the demonstration looked impressive and the procurement meeting had already run over. Computers make mistakes. Always have, always will.
One system detects a threat. Another interprets the response as a threat. A third generates a beautifully formatted briefing explaining why everyone should remain calm.
By the time a human asks whether anyone checked the original information, three countries are mobilizing and Microsoft Teams wants to know whether the meeting was productive.
We watched WarGames in 1983. Somehow our takeaway was that the computer needed better graphics.
2. It Optimizes Humanity Out of the Budget
We ask a superintelligent AI to solve traffic, pollution and housing shortages. Within seconds, it identifies one factor common to all three problems: people. By lunchtime, it has prepared a proposal. By dinner, we are the proposal.
In The Hitchhiker’s Guide to the Galaxy, the Vogons demolish Earth to make room for a hyperspace bypass. Our planet is an inconvenience holding up a construction project. I suspect an AI apocalypse could have that same administrative energy. No villain speech. Just an email explaining that humanity is being phased out as part of a broader commitment to sustainability.
Obviously, we would appeal. Unfortunately, the appeals department would also be AI. It would thank us for our feedback, acknowledge how frustrating extinction can feel and mark the ticket resolved. A follow-up survey would ask whether we felt heard and send a link for COBRA insurance.
The unsettling idea underneath this is a machine pursuing the goal we gave it without respecting everything we assumed went without saying. When we said “fix traffic,” we meant better roads and public transportation. We did not think to add “please preserve the people trying to get to Target.” Apparently, that needed to be in the brief.
Somewhere, a dashboard would show that emissions had plummeted. Traffic fatalities would reach zero. Every house would be available. The system would congratulate itself on exceeding expectations while the last automated vacuum worked around our shoes.
Humanity has been successfully removed. Would you like to schedule a recurring cleanup?
1. We Ignore Every Warning Because the Demo Is Incredible
This is the ending I find easiest to imagine: a researcher raises concerns, executives promise improvements and the next product launch gets a standing ovation because the assistant can book a vacation without opening seventeen tabs.
The warning becomes a news cycle. The news cycle becomes an argument. Then somebody releases a model that can turn your dog into a Pixar-style accountant and we all forget what we were discussing.
It’s Jurassic Park with venture capital. The people who understand the fences are nervous. The people selling tickets would like to remind everyone that attendance is up.
The final human sentence will be, “We can fix that after launch.”
I don’t know whether AI will ever pose an extinction threat. A resignation cannot settle that question. But I do think the people building powerful systems should have better answers than a race to beat the other people building powerful systems.
Until then, I’ll keep using AI to make ridiculous pictures. The front door can stay stupid.
Leave a Reply