Google clarified it will warn humans when they are interacting with its new Duplex System, responding to an outcry around the artificial intelligence (AI)-based capability which is designed to mimic the human voice to perform certain tasks.

In a demo at its annual I/O developer conference, Google showed off Google Duplex, a new conversation capability it is working on which allows its Assistant to call businesses on behalf of users to handle tasks like making hair appointments and dinner reservations. However, while the demo showed Duplex using a human voice, it didn’t identify itself as a non-human, which sparked an outcry around the ethical concerns of such a system (see video below).

Google moved to ease the situation, stating the system will inform the person receiving the call from the system that they are talking to an AI-based software.

“We understand and value the discussion around Google Duplex – as we’ve said from the beginning, transparency in the technology is important,” the company told The Verge in a statement. “We are designing this feature with disclosure built in, and we’ll make sure the system is appropriately identified.”

Google said what it demonstrated last week was an “early technology demo” and it will move to incorporate feedback as it develops the system into a product.

The voice used in the demo was controlled by Google’s DeepMind WaveNet software, and was developed to be familiar with numerous conversations so it is able to understand human sounds and effectively mimic them.

Ultimately, the software is designed to allow people to have natural conversations and complete real-world tasks through technology.

The demo at I/O was described as “horrifying” by Zeynep Tufekci, an associate professor at the University of North Carolina and known technology critic, stating the idea of mimicking human speech was “horrible and obviously wrong”.