Arecent episode of 4 Cornersreviewed this and many other risks posed by developments in AI.
History shows such treaties are needed, and that they can work.
The lesson of nuclear weapons
Scientists are pretty good at warning of the dangers facing the planet.
Unfortunately, society is less good at paying attention.
Japan surrendered days later.
The second world war was over, and the Cold War began.
It’s free, every week, in your inbox.
The world still lives today under the threat of nuclear destruction.
On a dozen or so occasions since then, we have come within minutes of all-out nuclear war.
Asecret petitionwas sent to President Harry S. Truman in July 1945.
Only this time, the petition wasnt secret.
The world wasnt at war.
And the technologies werent being developed in secret.
Nevertheless, they pose a similar threat to global stability.
The media often like to call them killer robots.
Our open letter to the UN carried a stark warning.
If any major military power pushes ahead with AI weapon development, a global arms race is virtually inevitable.
The endpoint of such a technological trajectory is obvious: autonomous weapons will become the Kalashnikovs of tomorrow.
Strategically, autonomous weapons are a military dream.
They let a military scale its operations unhindered by manpower constraints.
One programmer can command hundreds of autonomous weapons.
An army can take on the riskiest of missions without endangering its own soldiers.
First and foremost, there is a strong moral argument against killer robots.
Beyond the moral arguments, there are many technical and legal reasons to be concerned about killer robots.
One of the strongest is that they will revolutionize warfare.
Autonomous weapons will be weapons of immense destruction.
You had to persuade this army to follow your orders.
You had to train them, feed them and pay them.
Now just one programmer could control hundreds of weapons.
In some ways, lethal autonomous weapons are even more troubling than nuclear weapons.
To build a nuclear bomb requires considerable technical sophistication.
As a result, nuclear weapons have not proliferated greatly.
Autonomous weapons require none of this, and if produced they will likely become cheap and plentiful.
They will be perfect weapons of terror.
Can you imagine how terrifying it will be to be chased by a swarm of autonomous drones?
They will be an ideal weapon with which to suppress a civilian population.
Unlike humans, they will not hesitate to commit atrocities, even genocide.
Time for a treaty
We stand at a crossroads on this issue.
It needs to be seen as morally unacceptable for machines to decide who lives and who dies.
In this way, we may be able to save ourselves and our children from this terrible future.
