19 Sep 2021

Facial recognition systems tests fail to record accuracy for Māori

6:05 am on 19 September 2021

Tests of the main facial recognition (FR) system used to produce passports did not record how accurate it is on Māori.

Woman face recognition - biometric verification concept

Photo: 123rf

The Department of Internal Affairs ran tests four years ago on 400,000 people's passport photos to decide which system to buy.

Japanese company NEC won out - its system was accurate more than 99 percent of the time, and has since been upgraded.

Read the Biometric Data Solutions Vendor Performance Assessment (PDF, 4MB)

The 2017 report released after an appeal by RNZ to the Ombudsman, shows tests were run to see how accurate the tech was on people of various ages and across genders, but none for ethnicity.

It has been widely reported for at least five years that facial recognition can have a higher misidentification rate with minorities, and lead to unsafe prosecutions.

Internal Affairs says it was aware "of some limitations with accuracy when FR is applied across various ethnicities".

"We used New Zealand data for the testing which we ensured fully represented the New Zealand demographic.

"Use of FR in our production system has not indicated any material issues with ethnicity," it told RNZ.

The facial recognition algorithms are a "cornerstone" of automated passport processing.

An Internal Affairs' tester in Australia assessed the two systems for their ability to identify fraudsters with multiple identities in DIA's system; and to verify if a person applying for a renewed passport is the right person.

Read a summary of accuracy test comparison (PDF 116KB)

The NEC system scored tops, especially on photos of people under 16.

The system is run for the department by US company DXC.

"DXC did request that we explain that the facial recognition service is just one component of the wider and comprehensive identity validation and security control processes as part of the overall New Zealand passport system," DIA said.

The upgrades since 2017 meant the new system was faster, more secure and more accurate, it said.

Get the RNZ app

for ad-free news and current affairs