The Slippery Slope to Autonomous Killing Machines

Credit: United Nations

By Maaike Beenes
UTRECHT, The Netherlands, Nov 11 2019 – Would you trust an algorithm with your life? If that thought makes you uncomfortable, then you should be concerned about the artificial intelligence (AI) arms race that is secretly taking off, fueled by the arms industry.

Weapon systems that can select and attack targets autonomously, without real human control, are moving from science fiction to reality.

Take for example the Warmate 2. This Polish-made missile loiters over an area, controlled remotely by an operator, but can go into fully autonomous mode once a target has been identified.

Or the Dual-Mode Brimstone, a guided missile that can be assigned a target area after which it can find targets matching a predefined target type.

Right now these weapons are under human control, but the technology is designed to keep humans out of the picture. We are already well on our way down a very slippery slope.

For our new report* that we publish this week, we surveyed 50 weapons producers about their work on increasingly autonomous systems. The results show that although existing systems are still partly controlled, often remotely, by human operators, the industry is rapidly moving towards more and more autonomous systems.

In addition to asking the 50 companies to participate in the survey with questions about their policy and activities, the report analysed publicly available sources about the systems they are developing and military contracts they have already won.

Maaike Beenes

We found only four companies that we could classify as showing ‘best practice’ because they have in place a policy or statement to not develop lethal autonomous weapons. 30 companies, however, are of ‘high concern’.

These companies are all working on technologies most relevant to lethal autonomous weapons while not having clear policies on how they ensure meaningful human control over such weapons.

The group of high concern companies includes three of the world’s largest arms producers: Lockheed Martin, Boeing and Raytheon (all US), as well as AVIC and CASC (China), IAI, Elbit and Rafael (Israel), Rostec (Russia) and STM (Turkey).

Turkey’s state-owned weapons producer STM, for example, has developed the Kargu system. The Kargu is a kamikaze drone that flies to an area based on preselected coordinates and can then select targets based on facial recognition.

Some reports suggest the Kargu will soon be deployed on the Turkish-Syrian theater. This loitering munition may very soon cross the threshold to a weapon system without meaningful human control.

The results of this research are deeply concerning. Lethal autonomous weapon systems, which select and attack targets without meaningful human control, raise a host of legal, security and ethical concerns.

Crucially, removing the human from the ultimate kill-decision means delegating the decision to end a human being’s life to an algorithm-operated machine. This is fundamentally opposed to the right to life and human dignity.

An unarmed drone deployed to a UN peacekeeping mission. Credit: United Nations

But there are not just ethical concerns. Lethal autonomous weapons systems would be able to operate at speeds incomprehensible to humans.

Their high levels of autonomy would also make it very difficult to predict how they will react to unanticipated events, as we have already seen with accidents with self-driving cars. Any such unintended actions would significantly raise the risk of conflict escalation.

Lethal autonomous weapons are therefore not only unethical, but also pose a serious risk to international peace and security. It is also highly unlikely they would be able to comply with the key principles of International Humanitarian Law (IHL).

IHL requires distinguishing between civilians and combatants and to assess for each attack whether the civilian harm that would be caused by an attack is proportional to the expected military advantage. These are all highly context-dependent considerations, and that is exactly what algorithms are really bad at.

These concerns have sparked intense debates among states, which have discussed autonomous weapons at the UN Convention on Certain Conventional Weapons (CCW) since 2013.

These discussions have been productive in the sense that it has become clear the large majority of states want to ensure meaningful human control over the use of force.

Currently 30 states have already called for a preventive treaty that prohibits lethal autonomous weapons and ensures such human control. However, the debate is being stalled by a handful of countries that are enabling a global AI-arms race.

It is urgent for states to take action now that the development of lethal autonomous weapons can still be prevented rather than cured. Adopting new international law is the most effective way to do that.

It is clear that most states are ready to take their responsibility but as they meet this week in Geneva for the annual Meeting of High Contracting Parties to the CCW, it will become clear whether they are capable of making sufficient progress to prevent the world from a disastrous revolution in warfare.

The link to the report:

Comments are closed

RSS Daily Times Nigeria

  • Daily Times Newspaper, Thursday, December 12, 2019
    Read Also: Buhari cares about senior citizens – PTAD boss The post Daily Times Newspaper, Thursday, December 12, 2019 appeared first on Daily Times Nigeria. Nigeria News from Nigeria Newspapers
  • Buhari cares about senior citizens – PTAD boss
    …Wants pensioners happy …Says pensioners to celebrate Christmas holidays in good mood President Muhammadu Buhari does not play with any issue concerning the welfare of senior citizens who have served the nation and retired as he believes that their services contributed a lot to the development and unity of the country. Executive Secretary of Pension […]
  • FG probes invasion of Justice Ojukwu’s courtroom
    The Federal Government has set up a committee to investigate the invasion of Justice Ifeoma Ojukwu’s courtroom by yet-to-be identified persons on December 6. The Attorney General of the Federation and Minister of Justice, Abubakar Malami (SAN), announced this on Wednesday in Abuja at the Federal High Court where he had gone to pay the […]
  • Senate to investigate PPPRA over N800bn road maintenance fund – Lawan
    President of the Senate, Ahmad Lawan, has said the Senate will investigate the Petroleum Products Pricing Regulatory Agency (PPPRA) over alleged non-remittance of N800 billion to the Federal Roads Maintenance Agency (FERMA) for the maintenance of roads across the country. Lawan stated this in his concluding remarks on a bill for an Act to Repeal […]

Weather for Abuja


RSS Vanguard News

  • Alleged video capture & killing of soldiers by Boko Haram is false – Army
    … says it’s the handiwork of IPOB By Kingsley Omonobi The Nigerian Army has described a trending video purporting that Boko Haram terrorists captured and killed several troops of Operation Lafiya Dole in the North East, as false and the handiwork of members of the outlawed Independent Peoples of Biafra. It said the video is […]
  • NUC Visiting Registrar tasks FCE registry staff on excellence
    THE Visiting Registrar/Scholar, National Universities Commission, NUC, Mr Olujimi Olukoya, has urged registry staff of the Federal College of Education, Technical, Yaba, Lagos to identify catalysts for excellence and imbibe them if they want to go far in their chosen fields. Continue reading NUC Visiting Registrar tasks FCE registry staff on excellence at Vanguard News.
  • FG should use National Health Act to fast track UHC — Anthropologist
    Mrs. Abieyuwa Abel, a Social Anthropologist, has called on the Federal Government to use the National Health Act (NHA), 2014, to fast track Universal Health Coverage (UHC) in Nigeria. Abel made the call in an interview with the News Agency of Nigeria (NAN), on Wednesday in Abuja. She was speaking ahead of the commemoration of […]

© 2019 Nigeria Newsline. All Rights Reserved. Log in -