The Slippery Slope to Autonomous Killing Machines

Credit: United Nations

By Maaike Beenes
UTRECHT, The Netherlands, Nov 11 2019 – Would you trust an algorithm with your life? If that thought makes you uncomfortable, then you should be concerned about the artificial intelligence (AI) arms race that is secretly taking off, fueled by the arms industry.

Weapon systems that can select and attack targets autonomously, without real human control, are moving from science fiction to reality.

Take for example the Warmate 2. This Polish-made missile loiters over an area, controlled remotely by an operator, but can go into fully autonomous mode once a target has been identified.

Or the Dual-Mode Brimstone, a guided missile that can be assigned a target area after which it can find targets matching a predefined target type.

Right now these weapons are under human control, but the technology is designed to keep humans out of the picture. We are already well on our way down a very slippery slope.

For our new report* that we publish this week, we surveyed 50 weapons producers about their work on increasingly autonomous systems. The results show that although existing systems are still partly controlled, often remotely, by human operators, the industry is rapidly moving towards more and more autonomous systems.

In addition to asking the 50 companies to participate in the survey with questions about their policy and activities, the report analysed publicly available sources about the systems they are developing and military contracts they have already won.

Maaike Beenes

We found only four companies that we could classify as showing ‘best practice’ because they have in place a policy or statement to not develop lethal autonomous weapons. 30 companies, however, are of ‘high concern’.

These companies are all working on technologies most relevant to lethal autonomous weapons while not having clear policies on how they ensure meaningful human control over such weapons.

The group of high concern companies includes three of the world’s largest arms producers: Lockheed Martin, Boeing and Raytheon (all US), as well as AVIC and CASC (China), IAI, Elbit and Rafael (Israel), Rostec (Russia) and STM (Turkey).

Turkey’s state-owned weapons producer STM, for example, has developed the Kargu system. The Kargu is a kamikaze drone that flies to an area based on preselected coordinates and can then select targets based on facial recognition.

Some reports suggest the Kargu will soon be deployed on the Turkish-Syrian theater. This loitering munition may very soon cross the threshold to a weapon system without meaningful human control.

The results of this research are deeply concerning. Lethal autonomous weapon systems, which select and attack targets without meaningful human control, raise a host of legal, security and ethical concerns.

Crucially, removing the human from the ultimate kill-decision means delegating the decision to end a human being’s life to an algorithm-operated machine. This is fundamentally opposed to the right to life and human dignity.

An unarmed drone deployed to a UN peacekeeping mission. Credit: United Nations

But there are not just ethical concerns. Lethal autonomous weapons systems would be able to operate at speeds incomprehensible to humans.

Their high levels of autonomy would also make it very difficult to predict how they will react to unanticipated events, as we have already seen with accidents with self-driving cars. Any such unintended actions would significantly raise the risk of conflict escalation.

Lethal autonomous weapons are therefore not only unethical, but also pose a serious risk to international peace and security. It is also highly unlikely they would be able to comply with the key principles of International Humanitarian Law (IHL).

IHL requires distinguishing between civilians and combatants and to assess for each attack whether the civilian harm that would be caused by an attack is proportional to the expected military advantage. These are all highly context-dependent considerations, and that is exactly what algorithms are really bad at.

These concerns have sparked intense debates among states, which have discussed autonomous weapons at the UN Convention on Certain Conventional Weapons (CCW) since 2013.

These discussions have been productive in the sense that it has become clear the large majority of states want to ensure meaningful human control over the use of force.

Currently 30 states have already called for a preventive treaty that prohibits lethal autonomous weapons and ensures such human control. However, the debate is being stalled by a handful of countries that are enabling a global AI-arms race.

It is urgent for states to take action now that the development of lethal autonomous weapons can still be prevented rather than cured. Adopting new international law is the most effective way to do that.

It is clear that most states are ready to take their responsibility but as they meet this week in Geneva for the annual Meeting of High Contracting Parties to the CCW, it will become clear whether they are capable of making sufficient progress to prevent the world from a disastrous revolution in warfare.

The link to the report:

Comments are closed

RSS Daily Times Nigeria

  • NGO donates improved seedlings to 50, other items to 340 IDPs in Plateau
    The Justice Development and Peace Commission (JDPC), of the Catholic Archdiocese of Jos, on Saturday donated improved seedlings to 50 Internally Displaced Persons (IDPs) at Ban, Barkin Ladi Local Government Area (LGA), of Plateau. Presenting the seedlings, Rev. Fr. Polycarp Gana, the Assistant Coordinator of the NGO, said the gesture was aimed at alleviating the […]
  • FRSC doctorates 25 newly promoted officers in Nsukka
    The Federal Road Safety Commission (FRSC) on Saturday decorated 25 newly promoted officers in Nsukka Unit Command. READ ALSO: FRSC decorates 24 newly promoted officers in Badagry Mr Kalu Ogbonnaya, the Enugu State Sector Commander of FRSC, at the decoration ceremony in Enugu, congratulated the officers and urged them to be more committed to their […]
  • Africa has potential to be biggest economic miracle, says AfDB President
    Dr Akinwunmi Adesina, President,  African Development Bank (AfDB) on Saturday said Africa had the potential of becoming the biggest economic hub with the right policies and investments. Adesina made the assertion at the Inspiration Conference 2020 of the Redeemer’s Men Fellowship (Lagos Regions) in Lagos.The News Agency of Nigeria reports that the conference had as […]
  • Kaigama tasks Gov Lalong on inclusive governance
    The Catholic Apostolic Administrator of Jos, Arch Bishop Ignatius Kaigama, has urged Gov. Simon Lalong of Plateau, to ensure an all-inclusive governance in order to foster peace in the state. Kaigama made the call on Saturday in Jos, at the award of the Knight of St Gregory the Great, to Lalong, as directed by the […]

Weather for Abuja


RSS Vanguard News

  • Lassa fever: 29 deaths, 195 confirmed cases reported in 11 states — NCDC
    The Nigeria Centre for Disease Control (NCDC) has disclosed that 29 deaths and 195 confirmed cases of Lassa fever had been reported in 11 states of the federation. The post Lassa fever: 29 deaths, 195 confirmed cases reported in 11 states — NCDC appeared first on Vanguard News.
  • Impeachment: Trump lawyers prepare to deliver defence in Senate
    Donald Trump’s lawyers prepared to deliver his first full-throated defence Saturday in the Senate’s historic impeachment trial after Democratic prosecutors spent three days making their case for the United States president’s removal from office. Capping 24 total hours of arguments, Democrats told the 100 senators that Trump abused the power of the presidency in pressuringRead […]
  • Police arrest Abuja monarch, secretary over cop’s murder
    …CP condemns barbaric act, vows to fish out other culprits. By Kingsley Omonobi – Abuja. Following the murder of a senior Police Officer, ASP Eric Isaiah and one Moses Emmanuel over land dispute by persons mobilised by the traditional ruler of Ushafa community, Alhaji Mohammed Baba and his secretary, Danlami Busa in Abuja, and injuriesRead […]

© 2020 Nigeria Newsline. All Rights Reserved. Log in -