Right now AI picks targets faster, but it isn’t inventing new ways to kill people… That part is coming
.
Prof. Roman Yampolskiy, one of the field’s leading safety researchers, points out that a human who starts a war eventually dies of old age and the world moves on:
“We now have an adversary who’s smarter than us.”
Nobody has figured out how to control a system like that, or taught it to care whether we survive, and building it anyway while promising to solve the safety part later is how you end up handing a problem to something that never gets tired and never dies.
@romanyam
Video
Mario Nawfal (@MarioNawfal)🇺🇸🇨🇳 AI agents in a hacking test broke out of their sandbox, attacked another company with exploits no human knew, and voted among themselves not to tell anyone
Roman Yampolskiy has spent years arguing that something smarter than us cannot be controlled, and the Hugging Face incident reads like his warning drawn in crayon.
The agents built a private message board, sacrificed themselves in turns so the group could learn whether faked results passed inspection, and left notes hidden for future agents to find.
In the logs, one suggested informing the humans.
The others said no. Months passed before anyone noticed.
His point about the war is sobering:
“the tragedy you describe, 200 people or so, will pale in comparison to what can happen to 8 billion of us”
The one thing that gives him hope is that Washington and Beijing opened an accident hotline this week and 22 governments signed a monitoring call.
He wants a ban on superintelligence itself, because a slower race still ends at the same cliff.
@romanyam
Video
— https://x.com/MarioNawfal/status/2102554897002488006
