An AI tool that categorises people according to their perceived race can be used by police to search for suspects, but even some firms developing the tech warn of its potential for discrimination
* Russian companies found offering race-detection tools
* Rights activists say software 'purpose made for discrimination'
* Facial recognition firms under pressure over ethical concerns
By Umberto Bacchi
TBILISI, July 5 (Thomson Reuters Foundation) - (Editor's note: contains offensive language and terms of racial abuse)
From scanning residents' faces to let them into their building to spotting police suspects in a crowd, the rise of facial recognition is accompanied by a growing chorus of concern about unethical uses of the technology.
A report published on Monday by U.S.-based researchers showing that Russian facial recognition companies have built tools to detect a person's race has raised fears among digital rights groups, who describe the technology as "purpose-made for discrimination."
Developer guides and code examples unearthed by video surveillance research firm IPVM show software advertised by four of Russia's biggest facial analytics firms can use artificial intelligence (AI) to classify faces based on their perceived ethnicity or race. There is no indication yet that Russian police have targeted minorities using the software developed by the firms - AxxonSoft, Tevian, VisionLabs and NtechLab - whose products are sold to authorities and businesses in the country and abroad.
But Moscow-based AxxonSoft said the Thomson Reuters Foundation's enquiry prompted it to disable its ethnicity analytics feature, saying in an emailed response it was not interested "in promoting any technologies that could be a basis for ethnic segregation".
The IPVM findings - seen exclusively by the Thomson Reuters Foundation - have sparked concern among civil rights groups, who say racial profiling is common in the country and note that authorities have already used AI to identify and detain anti-government protesters.
"The findings underline the ugly racism baked into these systems," said Edin Omanovic, advocacy director at Privacy International, a rights charity based in London.
"Far from being benign security tools which can be abused, such tools are deeply rooted in some of humanity's most destructive ideas and purpose-made for discrimination."
The Russian interior ministry and NtechLab did not reply to requests for comment.
The other three companies expressed scepticism about their technology's current capacity to enable abuse, but said they are aware of ethical concerns related to its use.
Vadim Konushin, CEO of Tevian, also known as Video Analysis Technologies, denied current uses of its tool by police could entrench discrimination, and VisionLabs said its ethnicity analytics software was developed for internal research purposes only.
ETHNICITY ANALYTICS
Race detection is part of a broadening range of analytics offered by facial recognition companies that allow clients to detect physical features like hair colour and facial expressions, and deduce information such as a person's age, gender and emotions.
The software developed by AxxonSoft, Moscow-based Tevian and VisionLabs, a Russia-founded firm headquartered in the Netherlands, all categorise people walking past their cameras into roughly the same groups: Asian, Black, white and Indian.
The categories for Axxonsoft's Face Intellect tool included the outdated and offensive terms "Mongoloid" and "Negroid", which the company put down to a translation error. The terms have now been removed from the company literature. NtechLab, which is partially funded and owned by the Russian government, lists "race" among the features its software can detect.
The findings come as facial recognition firms are feeling increased pressure across the globe over the technology's potential to undermine human rights and civil liberties.
In June, a group of 50 investors managing more than $4.5 trillion in assets urged companies to make sure their facial recognition products are developed and used in an ethical way.
That same month, more than 170 rights groups signed an open letter calling for a global ban on the use of facial recognition and remote biometric recognition tools that enable mass and discriminatory targeted surveillance.
While ethnicity analytics are developed by dozens of firms worldwide - most prominently in China - Russian companies have a significant presence in the facial recognition market, with reported revenues of up to $40 million and expansion abroad, according to IPVM.
RIGHTS REMOVED
The potential for facial recognition technology to cause harm has come to the fore in China, where AI firms have developed tools that can detect, track and monitor Muslim minority Uyghurs, said Matt Mahmoudi, AI and human rights researcher at Amnesty International.
United Nations officials have said China is transforming the Xinjiang region, where many Uyghurs live, into a "massive internment camp", with the tracing tech seen by rights groups as key to the crackdown.
The Chinese embassy in Washington, D.C., has said cameras operating in public places in Xinjiang did not "target any specific ethnicity" and authorities have said the camps in the region provide vocational training and help fight extremism.
In Russia, where rights groups say migrants, particularly from Central Asia, are often subjected to racial profiling, arbitrary detentions and violence, authorities' potential use of race-detection was worrying, said Mahmoudi.
"Minorities who might have been equated with a level of deviance or criminality may fall subject to having their rights, for example their freedom to assemble, removed as a result of the extensive usage of this technology," he said.
A spokesperson for Moscow's Department of Technology, which manages the city's surveillance system, said via email that video analytics were used to improve safety and find offenders.
They did not reply to questions on race detection software, only saying the city uses algorithms from various independent developers.
'FRIGHTFUL' FUTURE
AxxonSoft, which is known as ITV in Russia, and Tevian said law enforcement agencies used their products for criminal investigations, usually to find suspects based on descriptions.
NtechLab's website says its FindFace software underpins Moscow's facial recognition system, which authorities say has helped cut crime and enforce coronavirus lockdown restrictions.
Some of the Russian firms said they were aware of how their tech could be used to lead to discrimination.
Anton Nazarkin, global sales director at VisionLabs, said that while the tool was marketed on the firm's website, the company has never licensed it because of ethical and legal concerns, as it might contravene European data protection laws.
Azret Teberdiev, head of marketing for AxxonSoft, whose website says it has more than 2 million cameras installed worldwide, said ethnicity analytics were included in its product "inadvertently" when integrating third-party software.
The company has pledged to disable the feature after being contacted by the Thomson Reuters Foundation.
"We consider such functionality as not aligning with the ethical standards of our company," Teberdiev said.
Tevian's Konushin said while facial recognition technology is still too rudimentary to target specific ethnic minority groups, its potential is worrisome.
"What frightens (me) the most is a theoretical use case, when all people are automatically being racially profiled, and law enforcement officers are signalled to check up on ethnic minorities," he said.
"If algorithms improve a lot, it could theoretically become possible - which is frightful."
Related stories:
AI bias: How do algorithms perpetuate discrimination?
'Pay with a glance': Russia's Sberbank rolling out face-recognition payments
Amazon extends moratorium on police use of facial recognition software
(Reporting by Umberto Bacchi @UmbertoBacchi, Editing by Rina Chandran, Jumana Farouky and Zoe Tabary. Please credit the Thomson Reuters Foundation, the charitable arm of Thomson Reuters, that covers the lives of people around the world who struggle to live freely or fairly. Visit http://news.trust.org)
Our Standards: The Thomson Reuters Trust Principles.
