Dave Gershgorn* says leading AI experts are calling on Amazon to stop selling facial recognition to law enforcement until the bugs are ironed out.
More than two dozen artificial intelligence (AI) experts are calling for Amazon to stop selling facial recognition software to law enforcement until legislation and oversight are put in place.
The letter, published on 3 April and shared with Quartz, claims that Amazon’s General Manager of AI, Matthew Wood and Vice-President of Global Public Policy, Michael Punke misrepresented technical aspects of research that suggests Amazon’s facial recognition is less accurate on women and people of colour in a company blog post.
“Overall, we find Dr Wood and Mr Punke’s response to the peer-reviewed research findings disappointing,” the letter says.
“We hope that the company will instead thoroughly examine all of its products and question whether they should currently be used by police.”
The referenced research, called Gender Shades and published by MIT Media Lab’s Joy Buolamwini and Deborah Raji, found that Amazon’s Rekognition facial recognition software mistook women of colour for men 31 per cent of the time, while making no errors on classifying the gender of white men.
“There is a barrier between critiques of facial analysis technologies and the companies making them,” Morgan Klaus Scheuerman, an information-science doctoral candidate at UC Boulder and letter signee, told Quartz.
“It’s important for us to think about, for example, how gender is conceptualised and encoded into commercial facial analysis technologies that are available today.”
“Facial analysis technologies, when they are packaged up as cloud-based services, can be appropriated for malicious intent.”
“Even in ways that the companies supplying them aren’t even aware of,” he wrote.
“Amazon has a lot of influence in the field of facial analysis technology.”
“It is one of the major suppliers of these types of computer vision services and can help shape norms around its development and use.”
Those signing the petition also include a 2019 winner of computer science’s highest honour, the Turing Award, Yoshua Bengio, and former head scientist at Amazon Web Services, Anima Anandkumar, along with researchers from Google, Facebook, Microsoft, Alphabet’s DeepMind, Harvard University and the University of California.
“What we hope is for a broad dialogue,” Anandkumar told Quartz.
“How are we measuring these systems, and what are the different evaluation metrics we need to think about, and what impact is it going to have on society by putting it in the hands of law enforcement?”
She added that what convinced her to sign was the technical nature of the letter, and that having the scientific discussion about making facial recognition work is necessary before it can be legislated on effectively and put into real-world situations.
“We have to have the right version of regulations, one that doesn’t stop us pushing research forward, but at the same time that would be effective,” Anandkumar said.
“It’s a complex problem, but I’m optimistic that with more open dialogue that can happen.”
* Dave Gershgorn is the artificial intelligence writer for Quartz. He tweets at @davegershgorn.
This article first appeared at qz.com.
