Tom Simonite* says makers of facial-recognition technology are scrambling to adapt to a world where more and more people are covering their faces.
Facial-recognition algorithms from Los Angeles startup TrueFace are good enough that the US Air Force uses them to speed security checks at base entrances.
But TrueFace CEO, Shaun Moore says he’s facing a new question: How good is TrueFace’s technology when people are wearing face masks?
“It’s something we don’t know yet because it’s not been deployed in that environment,” Moore says.
His engineers are testing their technology on masked faces and are hurriedly gathering images to tune their machine-learning algorithms for pandemic times.
Facial recognition has become more widespread and accurate in recent years, as an artificial intelligence (AI) technology called deep learning made computers much better at interpreting images.
Governments and private companies use facial recognition to identify people at workplaces, schools, and airports, among other places, although some algorithms perform less well on women and people with darker skin tones.
Now the facial-recognition industry is trying to adapt to a world where many people keep their faces covered to avoid spreading disease.
Facial-recognition experts say that algorithms are generally less accurate when a face is obscured, whether by an obstacle, a camera angle, or a mask.
“When you have fewer than 100,000 people in the database, you will not feel the difference,” says Alexander Khanin, CEO and cofounder of VisionLabs, a startup based in Amsterdam.
With 1 million people, he says, accuracy will be noticeably reduced and the system may need adjustment, depending on how it’s being used.
Some vendors and users of facial recognition say the technology works well enough on masked faces.
“We can identify a person wearing a balaclava, or a medical mask and a hat covering the forehead,” says Artem Kuharenko, founder of NtechLab, a Russian company whose technology is deployed in Moscow.
He says the company has experience with face masks through contracts in South East Asia, where masks are worn to curb colds and flu.
US Customs and Border Protection (CBP), which uses facial recognition on travellers boarding international flights, says its technology can identify masked faces.
But Anil Jain, a Professor at Michigan State University who works on facial recognition and biometrics, says such claims can’t be easily verified.
“Companies can quote internal numbers, but we don’t have a trusted database or evaluation to check that yet,” Jain says.
“There’s no third-party validation.”
A US Government lab at the National Institute of Standards and Technology (NIST) that functions as the world’s arbiter on the accuracy of facial-recognition algorithms hopes to provide that external validation — but is being held up by the same pandemic that prompted the project.
Patrick Grother, a computer scientist who leads NIST’s testing program, says his group is to quantify how accurately algorithms identify people wearing masks.
NIST plans to digitally add masks to its existing stockpile of photos and test algorithms previously submitted to a test that involves checking whether one photo matches another.
Later, it will invite companies to submit new algorithms tuned for face masks.
Chinese and Russian companies tend to dominate NIST’s widely watched leaderboards for facial-recognition accuracy.
Lighter privacy rules and wider acceptance of surveillance make it easier for those companies to gather the data and operational experience needed to improve facial-recognition algorithms.
Early in March, China’s SenseTime, which became the world’s most valuable AI startup in part through providing facial recognition to companies and government agencies, said it had upgraded its product for controlling access to buildings and workplaces to work with face masks.
The software attends to facial features left uncovered, such as eyes, eyebrows, and the bridge of the nose, a spokesperson said.
The US restricted sales to SenseTime and other Chinese AI companies last year for allegedly supplying technology used to oppress Uyghur Muslims.
China’s facial-recognition vendors faced the challenge of identifying masked faces earlier, and more broadly, because the country is both the origin of the novel coronavirus and the world’s most developed market for facial recognition.
Reports from China of the systems’ effectiveness with masks are mixed.
One Beijing resident told WIRED she appreciated the convenience of not having to remove her mask to use Alipay, China’s leading mobile payments network, which has updated its facial-recognition system.
But Daniel Sun, a Gartner analyst also in Beijing, says he has had to step out of crowds to pull down his mask to use facial recognition for payments.
Still, he believes facial recognition will continue to grow in usage, perhaps helped by interest in more hygienic, touch-free transactions.
The Japanese conglomerate NEC, which provides the facial-recognition technology used at US airports, is cautious in discussing the tech’s capability against masked faces.
Benji Hutchinson, a vice president with NEC’s US division, says the company’s labs in Japan that develop its algorithms have always tested on face masks because they are commonly worn during flu seasons in Asia.
But the company has launched new rounds of testing now that masks are set to become the norm.
“Masks are nothing new to us, but that doesn’t mean it’s all perfect,” Hutchinson says.
He says the company is advising customers, such as CBP, to make their own decisions about the technology for now.
A CBP spokesperson said it is using facial recognition at more than two dozen US airports and the technology works with masks.
“CBP’s biometric facial-comparison technology can match travellers wearing masks to photos from their travel documents,” the spokesperson said.
CBP’s system checks faces of passengers at the departure gate against “faceprints” from photos the Department of Homeland Security has on file for people listed on that flight.
Although the Agency says passengers are always free to opt out, some people have found it difficult to do so.
CBP says that if its technology fails on a person wearing a face mask, they may keep it on while a person manually checks their passport.
Will Knight contributed reporting.
* Tom Simonite is a senior writer for WIRED in San Francisco. He tweets at @tsimonite.
This article first appeared at www.wired.com.
