You can read more on the importance of this letter to the current global effort of banning lethal autonomous weapon systems (LAWS) on the Campaign to Stop Killer Robots website.

ICRAC is committed to the peaceful uses of robotics and the regulation of robot weapons. We welcome this open letter, and numerous members of ICRAC have not only signed but also commented on this youngest development over the last couple of days in various media. Below is a selection of audio and video content in English, Italian and German language.

CNET – Ban autonomous weapons, urge AI experts including Hawking, Musk and Wozniak
(with ICRAC’s Noel Sharkey and Thomas Nash)
Release date: 27 Jul 2015

Huffpost Live – A.I. Experts Push For Military Robot Ban
(with ICRAC’s Heather Roff and Ian Kerr)
Hundreds of artificial intelligence experts and revered thinkers, including Stephen Hawking and Elon Musk, are calling for a global ban on military robots. We explore the issue and whether these autonomous weapons could lower the threshold for war.
Release date: 28 Jul 2015

CTV – Will the growth of killer robots set off a global arms race?
(with ICRAC’s Ian Kerr)
In this 5 minute interview with Canada AM host, Beverly Thompson, Ian Kerr discusses the the difference between semi-autonomous and autonomous weapons, the call to ban killer robots, why he is a signatory to the open letter, and why efficacy is not the only consideration in deciding whether to ban a dangerous use of technology.
Release date: 04 Aug 2015

BBC World Service – Fighting off ‘Killer Robots’
(with ICRAC’s Heather Roff)
More than 1,000 tech experts, scientists and researchers have written a letter warning about the dangers of autonomous weapons, warning that a ‘military AI race is a bad idea’. One signatory, Heather Roff Perkins from the University of Denver spoke to the BBC’s Dominic Laurie.
Release date: 28 Jul 2015

Killer robots: the coming arms race?
(with ICRAC’s Peter Asaro)

Are you worried about killer robots? Last week, some of the most prominent thinkers in science and technology signed an open letter that warned of the coming arms race should militaries pursue the development and deployment of artificially intelligent weaponry. The letter was written by The Campaign to Stop Killer Robots, an international coalition of NGOs, and was signed by almost 14,000 people, including Stephen Hawking, Elon Musk, and Steve Wozniak. Today, we discuss the threat of fully autonomous weapons and artificially-intelligent warfare with PETER ASARO, professor of media studies at The New School and spokesperson for The Campaign to Stop Killer Robots, roboticist and Georgia Tech professor RONALD ARKIN, as well as GEORGE ZARKADAKIS, writer and AI architect.

Release date: 6 Aug 2015

Ban Killer Robots Interview – Calgary Newstalk 770
(with ICRAC’s Ian Kerr)
In this 25 minute interview with Roger Kinkade and Rob Breakenridge of Newstalk770, CHQR Radio, Ian Kerr talks at length about autonomous weapons and the AI community’s call to ban “killer robots”. The conversation is about what a killer robot is, why they are likely to be developed, what the dangers are if we don’t ban them and a number of broader issues regarding the future of artificial intelligence.
Release date: 8 Aug 2015

Author information

Frank Sauer

UN Secretary General Ban Ki-moon addresses National Model UN conference in the General Assembly Room, 26 March 2015. Photo: NMUN.

In less than two weeks, diplomats from around the world will gather at the United Nations in Geneva to discuss potential global regulations on “lethal autonomous weapons systems” that would be able to select and attack targets without direct human control.

But last week, at the National Model UN conference in New York, attended by some 2,500 undergraduate students from all over the world, a simulation of the UN General Assembly passed three resolutions calling for states to take action to prevent the threat of these “killer robots” to security, human rights and humanitarian law.

Addressing the closing ceremony of the conference, UN Secretary General Ban Ki-moon told students he was “energized by this dynamic gathering” and its “serious discussions” on “cutting-edge issues on the international agenda”, such as “lethal autonomous weapons systems.”

“You are not just leaders of the future – you can start to lead right now,” he told them, “now is the time for your generation to build human solidarity around the world.”

The NMUN NY resolutions defined lethal autonomous robots as “weapons that can select and attack targets independently – without meaningful human input or control”, suggested all countries immediate adopt a national moratorium on such weapons, and urged the negotiation of an international ban through an additional Protocol VI at the Convention on Certain Conventional Weapons (NMUN NY 2015A/GA1-1-1).

Model UN is a simulation of diplomacy, negotiation and decisionmaking by international organizations. Students play the role of diplomats from Member States of the UN and discuss issues at the top of the global policymaking agenda. NMUN NY is one of the biggest undergraduate Model UN conferences in the world.

The students assigned to simulate the General Assembly First Committee – which deals with issues of disarmament and international security – spent several months learning about their countries’ policy positions, the General Assembly and the politics of killer robots. (See for example, their background guide). After debate and drafting in the First Committee, the resolutions were passed by students representing the full plenary body in the actual General Assembly Room at the UN in New York.

The resolutions also called attention to the “work and expertise” of civil society, particularly the Campaign to Stop Killer Robots, Human Rights Watch and the International Committee for Robot Arms Control (ICRAC) (NMUN NY 2015A/GA1-1 -1, GA1-1-2 and GA1-1-3).

In a briefing, Dr. Matthew Bolton, Model UN advisor for Pace University New York City and member of ICRAC, told students at the conference that when new weapons technologies are not adequately addressed by existing regulations, the Marten’s Clause in international law requires states to be guided by “the principle of humanity and the dictates of public conscience.”

“Avoid the temptation to think this simulation is a meaningless game,” said Bolton, “A statement of strong concern from you could be considered an expression of public conscience – a challenge to policymakers in the real world to take action against killer robots.”

Author information

mbolton

Matthew Bolton is professor of political science at Pace University in New York City. He is an expert on global peace and security policy, focusing on multilateral disarmament and arms control policymaking processes. He has a PhD in Government and Master's in Development Studies from the London School of Economics and a Master's from SUNY Environmental Science and Forestry. Since 2014, Bolton has worked on the UN and New York City advocacy of the International Campaign to Abolish Nuclear Weapons (ICAN), recipient of the 2017 Nobel Peace Prize. Bolton has published six books, including Political Minefields (I.B. Tauris) and Imagining Disarmament, Enchanting International Relations (Palgrave Pivot).

ICRAC welcomes the historic decision taken by nations to begin international discussions on how to address the challenges posed by fully autonomous weapons. The agreement marks the beginning of a process that the campaign believes should lead to an international ban on these weapons to ensure there will always be meaningful human control over targeting decisions and the use of violent force.

At 16:48 on Friday, 15 November 2013, at the United Nations in Geneva, states parties to the Convention on Conventional Weapons adopted a report containing a decision to convene on May 13-16, 2014 for their first meeting to discuss questions related to “lethal autonomous weapons systems” also known as fully autonomous weapons or “killer robots.” These weapons are at the beginning of their development, but technology is moving rapidly toward increasing autonomy.

“This is a very significant step forward for the International Committee for Robot Arms Control (ICRAC ),” said Professor Noel Sharkey, Chairman of ICRAC. “We are now on the first rung of the international ladder to fulfill our goal of stopping these morally obnoxious weapons from ever being deployed.”

ICRAC was formed in 2009 to initiate international discussion on autonomous weapons systems. It is made up of experts in robotic technology, artificial intelligence, computer science, international security and arms control, ethics and international law. It is a co-founder of the Campaign to Stop Killer Robots.

The Campaign to Stop Killer Robots believes that robotic weapons systems should not be making life and death decisions on the battlefield. That would be inherently wrong, morally and ethically. Fully autonomous weapons are likely to run afoul of international humanitarian law, and that there are serious technical, proliferation, societal, and other concerns that make a preemptive ban necessary.

“Law follows technology.  With robotic weapons, we have an rare opportunity to regulate a category of dangerous weapons before they are fully realized and the CCW is our best opportunity for regulation,” said Dave Akerson an ICRAC legal expert.

A total of 117 states are party to the Convention on Conventional Weapons, including nations known to be advanced in developing autonomous weapons systems: United States, China, Israel, Russia, South Korea, and United Kingdom. Adopted in 1980, this framework convention contains five protocols, including Protocol I prohibiting non-detectable fragments, Protocol III prohibiting the use of air-dropped incendiary weapons in populated areas, and Protocol IV, which preemptively banned blinding lasers.

“This is a momentous opportunity to get states on the record and behind a ban on fully autonomous offensive weapons,” said Heather Roff, an ICRAC philosopher. “If we can gain enough support, we might succeed in banning a technology before it actually harms innocent civilians.”

The agreement to begin work in the Convention on Conventional Weapons could lead to a future CCW Protocol VI prohibiting fully autonomous weapons.

ICRAC with the Campaign to Stop Killer Robots supports any action to urgently address fully autonomous weapons in any forum. The decision to begin work in the Convention on Conventional Weapons does not prevent work elsewhere, such as the Human Rights Council.

Since the topic was first discussed at the Human Rights Council on 30 May 2013, a total of 44 nations have spoken publicly on fully autonomous weapons since May: Algeria, Argentina, Australia, Austria, Belarus, Belgium, Brazil, Canada, China, Costa Rica, Cuba, Ecuador, Egypt, France, Germany, Ghana, Greece, Holy See, India, Indonesia, Iran, Ireland, Israel, Italy, Japan, Lithuania, Madagascar, Mexico, Morocco, Netherlands, New Zealand, Pakistan, Russia, Sierra Leone, Spain, South Africa, South Korea, Sweden, Switzerland, Turkey, Ukraine, United Kingdom, and United States. All nations that have spoken out have expressed interest and concern at the challenges and dangers posed by fully autonomous weapons.

Together with the Campaign to Stop Killer Robots, ICRAC urges nations to prepare for extensive and intensive work next year, both within the CCW and outside the CCW context.  We urge states to develop national policies, and to respond to the UN Special Rapporteur on Extrajudicial Executions’ call for national moratoria on fully autonomous weapons. We urge states to come back one year from now and agree to a new mandate to begin negotiations. The new process must be underscored by  a sense of urgency.

Peter Asaro, vice-chairman of ICRAC said “The actions of the CCW this week are a hopeful first step towards an international ban on autonomous weapons systems.’

Mathew Bolton delivered a statement on behalf ICRAC at the UN CCW meeting yesterday. As a group of experts we are prepared to help any nations with expert discussions of autonomous weapons systems and to help develop clear definitions for the language to be used in a treaty to ban them. Video footage of the statement, ICRAC’s first ever statement in an official diplomatic forum, is available here.

ICRAC recently coordinated the circulation of a “Scientists Call” to ban fully autonomous weapons systems, signed by more than 270 Computer Scientists, Engineers, Artificial Intelligence experts, Roboticists and professionals from related disciplines in 37 countries, saying: “given the limitations and unknown future risks of autonomous robot weapons technology, we call for a prohibition on their development and deployment. Decisions about the application of violent force must not be delegated to machines.”

Author information

mbolton

Matthew Bolton is professor of political science at Pace University in New York City. He is an expert on global peace and security policy, focusing on multilateral disarmament and arms control policymaking processes. He has a PhD in Government and Master's in Development Studies from the London School of Economics and a Master's from SUNY Environmental Science and Forestry. Since 2014, Bolton has worked on the UN and New York City advocacy of the International Campaign to Abolish Nuclear Weapons (ICAN), recipient of the 2017 Nobel Peace Prize. Bolton has published six books, including Political Minefields (I.B. Tauris) and Imagining Disarmament, Enchanting International Relations (Palgrave Pivot).

Over the last couple of months, lethal autonomous robots (LARs) went from a sidelined issue to not only a hotly and widely debated topic but most importantly an official item of United Nations (UN) arms control diplomacy. This post provides an overview over recent media coverage as well as events and statements given at the UN.

The First Secretary and representative from the France Permanent Mission to the UN, Anais Laigle, reported that LARs will be discussed at the Convention on Conventional Weapons (CCW) conference in November. NBC reported that Switzerland, France, and Egypt were interested in the regulation of LARs that the International Committee for Robot Arms Control (ICRAC) and Human Rights Watch (HRW) proposed. NBC reported that “representatives” of Austria, Germany, the United States, France, Brazil, Morocco, and Algeria also were interested in regulation. The NBC article also suggests that the discussion about LARs will be complicated by the secrecy of LAR programs, citing criticism from NYU Law human rights professor and ICRAC legal advisor Sarah Knuckey. She said:

“We’re not going to know what laws are going to be programmed [into the robots], and where they’re going to be used.”

The article also mentioned ICRAC’s opposition to such secrecy quoting Jody Williams who said:

“I don’t like my tax dollars being used on weapons that are not even discussed in the public domain… We have every right and every responsibility to have a public discussion as to where war is going.”

An article from Co.EXIST reported on the UN visit by activists, technologists, and others to raise awareness for a ban on LARs. The article cites Jody Williams’ warning that the governments are on a slippery slope towards developing LARs and that “in the loop” killer robots may “just mean the programmer” is “in the loop.” The article also mentions ICRAC chairman Noel Sharkey and his concerns about LARs, particularly their potential ability to distinguish targets, “proportionality-test,” and properly recognize the surrender process of combatants. The article also reports that “272 engineers, computer scientists and roboticists” recently co-signed the ICRAC’s letter to the UN to implement a ban.The article clarifies that ICRAC does not oppose non-autonomous weapons or peaceful autonomous robots.

Another article in the International Business Times reported on ICRAC’s call for “urgent international action” and HRW’s warning that LARs would be developed. HRW was quoted expressing concern that civilians were at risk by the development of LARs and stating that all killing decisions should be made by humans. The article further mentions the Pentagon’s downplaying of the development of LARs.

Computerworld reported that human rights organizations and representatives have been at the UN calling for a ban on LARs.  Like the Co.EXIST article, they quote HRW’s Mary Wareham saying that HRW does not oppose armed robots, but it wants to ensure that humans remain in control of kill decisions and expresses concern that the military is working in the direction towards greater autonomy. The article also articulates Wareham’s concern that there are currently only policies, not laws governing the use of LARs.

A recent article in TIME underscored a surge of activity and momentum in the effort to stop LARs. The article explains that although LARs are not currently in use, the X-47B and the South Korean robot sentry are signs of the increasing autonomy of armed robots. It also cites arguments from those opposing LARS, including the possibility of LAR malfunction, hacking, as well as moral issues.

The Voice of Russia outlet published a lengthy article outlining the concerns about LARs. It seems to claim that Pentagon analysts will have operational autonomous robots in 20 years and humans will become the most insignificant factor in their use by 30 years, the article says. It cites professor Chistoph Heyns, the UN’s Special Rapporteur on extrajudicial, summary or arbitrary executions, who voiced concern over the legal responsibility of LAR killing and his warning that their use will lead to “mechanical slaughter.”  The article explains that many there are many “experts” who desire regulation of LARs. The article draws a parallel to current drone use, arguing that targeting mistakes of drones in current use could apply to LARs. The author also argues that LARs will not be able to appropriately react to unexpected behavior. It also underscores three pro-LAR arguments. Namely, that they would make better decisions more quickly because they could process more information, would not be hampered by emotion, and could make decisions faster.

On 29 October, ten states spoke on fully autonomous weapons or lethal autonomous robots at UN General Assembly First Committee:
Costa Rica, Ecuador, Greece, Ireland, Japan,Netherlands, Pakistan, Switzerland, United Kingdom, the United States. For all except Pakistan, Switzerland, UK, and the US, this was the first time they had made a public statement on the topic. At the outset of the 2013 First Committee, Pakistan also expressed its concerns over the weapons, as did Austria, Egypt, and France. Relevant extracts from the statements follow.

Austria: Prevention and accountability for deliberate targeting of civilians during war, as well as disproportionate collateral casualties as a result of military action, are at the centre of our concern. Today, arms technology is undergoing rapid changes. The use of armed drones in conflict situations is increasing. In a not too distant future, fully autonomous weapons systems might become available. As a result, the implications of these developments on IHL require urgent engagement by relevant UN forums and further discussion with a view to ensure that these weapons will not be used in a way that violates universally recognized principles of IHL such as the proportionality of the use of force or the obligation to distinguish between civilians and combatants.

Costa Rica: Furthermore, we worry that many problems identified with the use of armed drones would be exacerbated by the trend toward increasing autonomy in robotic weapons. My delegation feels that we should begin international dialogue soon on the issue of lethal autonomous robotics, and calls for States to consider placing national moratoria on their development, production and use and discuss eventual prohibition.

Egypt: Egypt reiterates that technology should not overtake humanity. The potential or actual development of Lethal Autonomous Robotics raises many questions on their compliance with international humanitarian law, as well as issues of warfare ethics. Such issues need to be fully addressed. Regulations should be put into place before such systems (LARs) are to be developed and/or deployed.

France: We must look to the future and address its challenges. An important debate has emerged in recent months on the issue of Lethal Autonomous Robots (LARs). This is a key debate as it raises the fundamental question of the place of Man in the decision to use lethal force. It is also a difficult debate, as it highlights many ethical, legal and technical issues. It covers technologies which are not yet fully developed and which are dual-use. The terms of this debate need to be clarified. To be useful and allow progress, this discussion needs to be held in an appropriate disarmament forum, combining the necessary military, legal and technical expertise and all the States concerned.

Ecuador: My country believes that the international community should deepen the debate around Unmanned Aerial Vehicles and fully autonomous armed robots. The high number of victims indiscriminate use of drones in civilian areas has also caused serious ethical and legal questions that the development of new military technologies precluding participation and human responsibility in decision-making, is urgent a discussion would be on these new problems in the field of conventional weapons. – Google Translation

Greece: Greece remains firmly committed to the Convention on Certain Conventional Weapons (CCW) and its Protocols and continues to believe that the CCW remains the most appropriate forum for the discussion on a Protocol on Cluster munitions, as it includes both the most significant producers and users, and will thus be in a position to strike a delicate balance between military utility and humanitarian concerns. It is in this same forum that we believe that the topic of Lethal Autonomous Robotics (LARS) should be discussed considering that the CCW is in a unique position to gather the competent diplomatic, legal and military expertise to address this emerging issue.

Ireland: The same principles which provide the foundation for the Arms Trade Treaty must also be applied to all topics of debate in relation to conventional weapons. Whether with regard to anti-personnel landmines, cluster munitions, transparency measures, the environmental impact of weapons, or the use of incendiary weapons, to name a few, our focus must always be to ensure respect for international humanitarian law and human rights, including the rights of women. These same principles must also apply to weapons which will be developed in the future, such as fully autonomous weapons systems. Constructive engagement and debate is essential to ensure that our actions comply with the principles which underlie the United Nations and international law.

Japan: Japan recognizes growing interests, in the international community, in the issues regarding fully autonomous weapons. We think it useful to start discussion about basic elements related to those weapons, including their definition. CCW, where military, legal and other arms control experts are involved, could provide an appropriate venue to address these issues. Japan looks forward to discussing these issues with other interested States and civil society.

The Netherlands: The possible development of Lethal Autonomous Robot Systems raises many legal, ethical and policy questions. In the Netherlands we have started a discussion on this issue with involvement of the ministries of Foreign Affairs and. Defence, relevant partners of civil society and academia in order to get a better understanding of the developments in this field and the related problems. In answering the question about the legality of weapon systems we are guided by international law and in particular by International Humanitarian Law. While developing new weapon systems, states should remain within the boundaries of international law. We will participate actively in discussions on LARS and in that regard support the proposal of the CCW chair for an informal discussion on LARS in the framework of CCW.

Pakistan: Another disturbing trend is the development of new types of conventional weapons like the Lethal Autonomous Robots (LARs), and the use of armed drones which cause indiscriminate killing of civilians. The use of drones, especially outside the zone of conflict or the battlefield, not only poses a legal challenge but also has serious human rights and humanitarian implications. It needs to be stopped immediately. The use of drones needs to be brought under international regulation before it spirals out of control. Similarly, LARs, which would choose and fire on pre-programmed targets on their own without any human intervention, pose a fundamental challenge to the protection of civilians and the notion of affixation of responsibility. They could alter traditional warfare in unimaginable ways. Their development needs to be addressed at the relevant international fora including at the UN and the CCW Conference of State Parties. The states that currently possess and use such weapons cannot afford to be complacent that such capabilities will not proliferate over time and hence they too shall become vulnerable unless such weapons7 production is curtailed forthwith under an international regime.

Pakistan: Lethal Autonomous Robots (LARs) – that would chose and fire on pre-programmed targets on their own without any human intervention – pose a fundamental challenge to the protection of civilians and the notion of affixation of responsibility. … We recognize that consensus building will be a difficult task, but we take this opportunity to put forward some ideas that we feel are essential to promote greater global security: … Nine, The development and use of drones and Lethal Autonomous Robots (LARS) need to be checked and brought under international regulation. Besides the UNGA and its First Committee, the CCW Conference of State Parties also provides a forum to address these issues.

Switzerland: In conclusion, I would like to reiterate the importance of conventional arms in disarmament and international security. New technologies are changing warfare and challenges loom on the horizon. One emerging issue is that of “fully autonomous weapon systems” as highlighted in this year’s report of the Secretary-General’s Advisory Board on Disarmament Matters. We note with interest that the Secretary-General should consider commissioning a comprehensive study, involving UNlDlR and other research institutes and think tanks, in order to support the appropriate efforts. Switzerland is of the view that there is a need to understand, identify, and clarify the potential challenges associated with fully autonomous weapon systems and the relevant technology. Switzerland therefore recognizes the need for a structured intergovernmental dialogue in the existing forum of the Conventional Weapons Convention (CCW) on this issue. Switzerland stands ready to take an active part in the discussions.

United Kingdom: I am looking forward to returning to Geneva for the meeting of States Parties to the Convention on Certain Conventional Weapons and our discussions on lethal autonomous robotics. This is an important issue, and one that sits well within the expert remit of the CCW. I hope that we can bring the UK’s expertise and experience to bear.

United States: Mr. Chairman, the United States is a High Contracting Party to the Convention on Certain Conventional Weapons and all of its five Protocols. The United States attaches importance to the CCW as an instrument that has been able to bring together states with diverse national security concerns. We look forward to the annual meetings of High Contracting Parties in November and to establishing a program of work for 2014 that will allow CCW States to continue supporting the universalization of the CCW and the implementation of all its Protocols. During this past year, questions have arisen regarding the development and use of lethal fully autonomous weapons in forums such as the Human Rights Council. As the United States delegation to the Human Rights Council stated, we welcome discussion among states of the legal, policy, and technological implications associated with lethal fully autonomous weapons in an appropriate forum that has a primary focus on international humanitarian law issues, if the mandate is right. The United States believes the CCW is that forum. CCW High Contracting Parties include a broad range of States, including those that have incorporated or are considering incorporating automated and autonomous capabilities in weapon systems. The CCW can bring together those with technical, military, and international humanitarian law expertise, ensuring that all aspects of the issue can be considered. Accordingly, we support an informal, exploratory discussion of lethal fully autonomous weapons and are engaged with our fellow CCW High Contracting Parties in formulating an appropriate mandate that will facilitate these discussions.

As of today, a total of 28 states have spoken publicly on fully autonomous weapons: Algeria, Austria, Argentina, Belarus, Brazil, China, Costa Rica, Cuba, Ecuador, Egypt, France, Germany, Greece, Indonesia, Iran, Ireland, Japan, Mexico, Morocco, Netherlands, New Zealand, Pakistan, Russia, Sierra Leone, Sweden, Switzerland, United Kingdom, and the United States.”

Author information

Frank Sauer

ICRAC member Dr. Matthew Bolton, presenting a statement on disarmament at the UN General Assembly’s First Committee on Tuesday. Photo by Shant Alexander for Control Arms.

On behalf of global civil society organizations, International Committee for Robot Arms Control member Matthew Bolton calls for disarmament and arms control “driven by the needs and rights of people most affected by armed violence.” The Campaign to Stop Killer Robots also spoke, calling for fully autonomous weapons to “be prohibited through an international treaty, as well as through national laws and other measures.” To watch video footage of the NGO speeches, click here.

Dr. Matthew Bolton, a member of the International Committee for Robot Arms Control (ICRAC),  addressed the United Nations General Assembly First Committee Tuesday afternoon, on behalf of Article 36 and other international non-governmental organizations (NGOs) working on disarmament, peacebuilding and humanitarian issues.

“We call for an approach to disarmament that is driven by the needs and rights of people most affected by armed violence, not by the discretion of states and organizations most responsible for it,” said Dr. Bolton to representatives of the 193 UN member states, as well as UN agencies and NGOs. The First Committee has responsibility for disarmament and international security.

The NGO statement, read by Dr. Bolton and endorsed by 11 organizations, congratulated states on “some noteworthy progress” in recent international discussions on the elimination of nuclear weapons, the recent Security Council resolution on small arms and light weapons as well as the Arms Trade Treaty, signed by over 100 states since June.

Despite these developments in global policy making on controlling weapons, however, Dr. Bolton asserted that “now is not the time for resting on laurels.” The NGO statement identified numerous concerns, including the abuse of the consensus rule in disarmament forums, exclusion of meaningful civil society participation, lack of equal opportunities for women in decisionmaking and the marginalization of the voices of victims and survivors of armed violence.

“Creativity and new human-centered approaches must be a requirement for all states advocating nuclear disarmament, conventional arms control and reduced military expenditure,” said Dr. Bolton, reading the NGO statement. “We can and must replace stalemate and watered-down outcomes with alternatives that advance human security and social and economic justice.”

The Campaign to Stop Killer Robots also delivered a statement in the same session, calling for a prohibition on fully autonomous weapons.

“Our campaign believes that human control is essential to ensure the protection of civilians and to ensure compliance with international law,” said Mary Wareham of Human Rights Watch, delivering the statement on behalf of the campaign. “We seek a comprehensive and preemptive ban on weapons systems that would be able to select and attack targets without meaningful human intervention. These fully autonomous weapons or ‘lethal autonomous robots’ must be prohibited through an international treaty, as well as through national laws and other measures.”

Dr. Bolton is an expert on global disarmament policy and assistant professor of political science at Pace University. He is author of Foreign Aid and Landmine Clearance: Governance, Politics and Security in Afghanistan, Bosnia and Sudan (I.B. Tauris, 2010) and a forthcoming travelogue Political Minefields (I.B. Tauris, 2014). He has written widely on the politics of landmines, cluster munitions, the Arms Trade Treaty and fully autonomous military robotics (“killer robots”). He recently co-authored an ICRAC Working Paper on regulating robotic weapons with the Arms Trade Treaty.

ICRAC is an international committee of experts in robotics technology, robot ethics, international relations, international security, arms control, international humanitarian law, human rights law, and public campaigns, concerned about the pressing dangers that military robots pose to peace and international security and to civilians in war.

Author information

mbolton

Matthew Bolton is professor of political science at Pace University in New York City. He is an expert on global peace and security policy, focusing on multilateral disarmament and arms control policymaking processes. He has a PhD in Government and Master's in Development Studies from the London School of Economics and a Master's from SUNY Environmental Science and Forestry. Since 2014, Bolton has worked on the UN and New York City advocacy of the International Campaign to Abolish Nuclear Weapons (ICAN), recipient of the 2017 Nobel Peace Prize. Bolton has published six books, including Political Minefields (I.B. Tauris) and Imagining Disarmament, Enchanting International Relations (Palgrave Pivot).

CRUSER Robo-Ethics Continuing Education Series (RECES)
Robo-Ethics Panelist Debate (Part 3)

Author information

Frank Sauer

ICRAC members discuss the difficulties of compliance with International Humanitarian law using autonomous weapons as well as international security issues and the moral red line.

final video – robots from Amy Kohn on Vimeo.

Author information

Peter Asaro

Dr. Peter Asaro is a philosopher of science, technology and media. His work examines the interfaces between social relations, human minds and bodies, artificial intelligence and robotics, and digital media. His current research focuses on the social, cultural, political, legal and ethical dimensions of military robotics and UAV drones, from a perspective that combines media theory with science and technology studies. He has written widely-cited papers on lethal robotics from the perspective of just war theory and human rights. Dr. Asaro's research also examines agency and autonomy, liability and punishment, and privacy and surveillance as it applies to consumer robots, industrial automation, smart buildings, and autonomous vehicles. His research has been published in international peer reviewed journals and edited volumes, and he is currently writing a book that interrogates the intersections between military robotics, interface design practices, and social and ethical issues. Dr. Asaro has held research positions at the Center for Cultural Analysis at Rutgers University, the HUMlab of Umeå University in Sweden, and the Austrian Academy of Sciences in Vienna. He has also developed technologies in the areas of virtual reality, data visualization and sonification, human-computer interaction, computer-supported cooperative work, artificial intelligence, machine learning, robot vision, and neuromorphic robotics at the National Center for Supercomputer Applications (NCSA), the Beckman Institute for Advanced Science and Technology, and Iguana Robotics, Inc., and was involved in the design of the natural language interface for the Wolfram|Alpha computational knowledge engine (winner of the 2010 SXSW Web Interactive Award for Technical Achievement), for Wolfram Research. He is currently working on an Oral History of Robotics project that is funded by the IEEE Robotics and Automation Society and the National Endowment for the Humanities Office of Digital Humanities. Dr. Asaro received his PhD in the History, Philosophy and Sociology of Science from the University of Illinois at Urbana-Champaign, where he also earned a Master of Arts from the Department of Philosophy, and a Master of Computer Science from the Department of Computer Science.

For the ban, he notes that

The arguments against developing fully autonomous weapons, as they are called, range from moral (“they are evil”) to technical (“they will never be that smart”) to visceral (“they are creepy”).

“This is something people seem to feel at a very gut level is wrong,” says Stephen Goose, director of the arms division of Human Rights Watch, which has assumed a leading role in challenging the dehumanizing of warfare. “The ugh factor comes through really strong.”

He then discusses the three International Humanitarian issues with autonomous robot weapons (i) inability to conform to the principle of distinction; (ii) inability to conform to the principle of Proportionality and (iii) difficulties with accountability with mishaps or war crimes.

He brings out the usual suspect, Ron Arkin, to argue against a ban. Arkin still believes that robots could do better than human because they don’t have emotional responses. Others argue that that is one of the main problems. The funniest comment to Keller’s article was a response to Ron Arkin:

Professor Arkin argues that automation can also make war more humane.” This guy has obviously been a civilian all his life. Only a civilian would believe there is a humane way to kill another human being. Does he get out of the house on a regular basis?

But Arkin’s position in other respects does not now seem that removed from those calling for a ban. “He advocates a moratorium on deployment and a full-blown discussion of ways to keep humans in charge.” The human’s in charge is a subtle change in Arkin’s position that is greatly appreciated. It moves us some way toward the discussions that should be had.

However, without a ban on the development and research on these weapons systems, they are going to end up in the US arsenal. Other countries have not said that they will have a moratorium and so we can expect and arms race that the US will not be able to resist.

In fact in terms of a moratorium, Keller appears to have made an error of interpretation with regards to the recent Department of Defence directive (November 21 2012) ” Last November the Defense Department issued what amounts to a 10-year moratorium on developing them while it discusses the ethical implications and possible safeguards.”

ICRAC member Mark Gubrud picks up on this error in a comment after Keller’s piece:

The DoD Directive (3000.09) does not impose any moratorium. It says that the United States will develop and use autonomous weapons.

Although it draws a line at AW that kill humans autonomously, it does not forbid crossing the line; rather, it sets forth the procedure for doing so. Four sub-cabinet level signatures are required. Other than that, the rules for AW that kill humans are essentially the same as for AW that target materiel, which the Directive approves already.

The directive also approves for immediate development and use “semi-autonomous weapons” which may automatically acquire, track, identify and prioritize potential targets, cue a human operator to their presence, and upon approval, engage them, automatically determining the timing of when to fire.

So, a semi-autonomous weapon system might detect a group of persons, highlight their dim outlines on a screen, and say to the operator “target group identified.” The operator says “engage” and the machine kills them.

Such a system already has every capability needed for full lethal autonomy. It has only been programmed to request approval. One trivial software modification will fix that, if the system doesn’t already have a switch to throw it into full autonomous mode.

DoDD 3000.09 approves such systems for immediate development, acquistion and use.

There is no moratorium; it is a full-speed charge into the unknown.

Nonetheless, Keller is clearly on the right side of the issues and shows a clear understanding: ” It’s a squishy directive, likely to be cast aside in a minute if we learn that China has sold autonomous weapons to Iran”

Although Keller is not optimistic about the chance of us getting a ban on killer robots, he supports it and ICRAC appreciates him for that:

I don’t hold out a lot of hope for an enforceable ban on death-dealing robots, but I’d love to be proved wrong. If war is made to seem impersonal and safe, about as morally consequential as a video game, I worry that autonomous weapons deplete our humanity. As unsettling as the idea of robots’ becoming more like humans is the prospect that, in the process, we become more like robots.

It is well worth reading Bill Keller’s full story and the comments that come afterwards – Smart Robots.

Author information

nsharkey

Noel SharkeyPhD, DSc FIET, FBCS CITP FRIN FRSA is Professor of AI and Robotics and Professor of Public Engagement at the University of Sheffield and was an EPSRC Senior Media Fellow (2004-2010).

From the Observer:
“Killer robots must be stopped, say campaigners

A new global campaign to persuade nations to ban “killer robots” before they reach the production stage is to be launched in the UK by a group of academics, pressure groups and Nobel peace prize laureates.

Robot warfare and autonomous weapons, the next step from unmanned drones, are already being worked on by scientists and will be available within the decade, said Dr Noel Sharkey, a leading robotics and artificial intelligence expert and professor at Sheffield University. He believes that development of the weapons is taking place in an effectively unregulated environment, with little attention being paid to moral implications and international law.

The Stop the Killer Robots campaign will be launched in April at the House of Commons and includes many of the groups that successfully campaigned to have international action taken against cluster bombs and landmines. They hope to get a similar global treaty against autonomous weapons.”

Read the full story in the Observer

This was also picked up by the Independent and Wired Magazine

The UK Tabloids were also in on the act: The Sun, The Daily Mail and the Daily Mail again and The Daily Star

There are too many news reports around the world to list but here are two quality ones from Norway
and The Netherlands

I will try to get time to give a full media report later.

Author information

nsharkey

Noel SharkeyPhD, DSc FIET, FBCS CITP FRIN FRSA is Professor of AI and Robotics and Professor of Public Engagement at the University of Sheffield and was an EPSRC Senior Media Fellow (2004-2010).

Here is my response on the relationship between these two documents in the Guardian Newspaper, December 2012. The article has links to both the Human Rights Watch report and the Department of Defence directive.

Author information

nsharkey

Noel SharkeyPhD, DSc FIET, FBCS CITP FRIN FRSA is Professor of AI and Robotics and Professor of Public Engagement at the University of Sheffield and was an EPSRC Senior Media Fellow (2004-2010).