FRVT 1:N Identification

Starting from July 2023, the FRVT has been rebranded and split into FRTE and FATE. This evaluation track is now available as FRTE 1:N Identification. See our comments on our newest results there.

These comments provided by Neurotechnology are based on NIST FRVT 1:N Identification ongoing evaluation results published on March 14, 2023.

Neurotechnology algorithm has been ranked in the top 4% of most accurate results matching frontal and profile mugshots scenarios among 341 submissions by 93 different providers. The algorithm showed top results among border control supervised (Visa vs Border, Border vs Border ΔT ≥ 10 YRS) and unsupervised (Visa vs Kiosk) scenarios. Also, considering the template size, the algorithm showed the best results among all other submissions with the same template size.

Eight scenarios with six datasets were tested. Two experiments were performed in each scenario:

• Identification – intended to test civilian use cases. This experiment used a threshold set to limit the False Positive Identification Rate (FPIR) to 0.3%. FPIR is the proportion of non-mated searches producing one or more candidates above threshold. The False Negative Identification Rates (FNIR) is the proportion of mated searches failing to return the mate above threshold. The use of thresholding supports use of face recognition in making mostly automated decisions e.g. for access into facility.
• Investigation – intended to test law enforcement use cases. This experiment used a threshold set to zero FPIR, and the algorithm should return a fixed number of candidates. FNIR is the proportion of mated searches for which the algorithm does not place the correct candidate at rank 1. The use of face recognition without a threshold supports investigating uses where it is assumed and necessary that a human expert will have to review the candidates returned from each search. For mated-searches the human is tasked with finding the correct mate; for non-mated searches the reviewer must reject all the candidates.

Click to zoom

Identification experiments

1. Mugshot vs Mugshot, 12 million gallery
2. Mugshot vs Mugshot, 1.6 million gallery
3. Mugshot vs Webcam, 1.6 million gallery
4. Mugshot vs Profile, 1.6 million gallery
5. Visa vs Border, 1.6 million gallery
6. Visa vs Kiosk, 1.6 million gallery
7. Border vs Border ΔT ≥ 10 years, 1.6 million gallery
8. Mugshot vs Mughsot ΔT ≥ 12 years, 3 million gallery

Investigation experiments

9. Mugshot vs Mugshot, 12 million gallery
10. Mugshot vs Mugshot, 1.6 million gallery
11. Mugshot vs Webcam, 1.6 million gallery
12. Mugshot vs Profile, 1.6 million gallery
13. Visa vs Border, 1.6 million gallery
14. Visa vs Kiosk, 1.6 million gallery
15. Border vs Border ΔT ≥ 10 years, 1.6 million gallery
16. Mugshot vs Mughsot ΔT ≥ 12 years, 3 million gallery

Identification – Neurotechnology algorithm accuracy in this experiment was 0.65% FNIR at 0.3% FPIR. The most accurate contender showed 0.12% FNIR at the same FPIR.

Investigation – Neurotechnology algorithm accuracy in this experiment was 0.13% FNIR at Rank1. The most accurate contender showed 0.10% FNIR at Rank1.

Scenario overview

• 12 million images from the Mugshot dataset were used for persons' enrollment.
• In part of the cases more than one image corresponded a unique person.
• Each face template was compared against all other enrolled templates.
• The comparisons were fully zero-effort, meaning there was no pre-grouping by gender, age or other covariates.

Identification – Neurotechnology algorithm accuracy in this experiment was 0.26% FNIR at 0.3% FPIR. The most accurate contender showed 0.10% FNIR at the same FPIR.

Investigation – Neurotechnology algorithm accuracy in this experiment was 0.10% FNIR at Rank1. The most accurate contender showed 0.08% FNIR at Rank1.

Scenario overview

• 1.6 million images from the Mugshot dataset were used for persons' enrollment.
• In part of the cases more than one image corresponded a unique person.
• Each face template was compared against all other enrolled templates.
• The comparisons were fully zero-effort, meaning there was no pre-grouping by gender, age or other covariates.

Identification – Neurotechnology algorithm accuracy in this experiment was 1.82% FNIR at 0.3% FPIR. The most accurate contender showed 0.65% FNIR at the same FPIR.

Investigation – Neurotechnology algorithm accuracy in this experiment was 0.77% FNIR at Rank1. The most accurate contender showed 0.54% FNIR at Rank1.

Scenario overview

• Images from Mugshot and Webcam datasets were used for persons' enrollment. About 86% of the images were from the Mugshot dataset.
• 1.6 million images from the Mugshot dataset were used.
• In part of the cases more than one image corresponded a unique person.
• Each enrolled face template from the Webcam dataset was compared against all enrolled templates from the Mugshot dataset.
• The comparisons were fully zero-effort, meaning there was no pre-grouping by gender, age or other covariates.

Identification – Neurotechnology algorithm accuracy in this experiment was 16.00% FNIR at 0.3% FPIR. The most accurate contender showed 5.89% FNIR at the same FPIR.

Investigation – Neurotechnology algorithm accuracy in this experiment was 5.81% FNIR at Rank1. The most accurate contender showed 5.17% FNIR at Rank1.

Scenario overview

• Images from Mugshot and Profile datasets were used for persons' enrollment. About 86% of the images were from the Mugshot dataset.
• 1.6 million images from the Mugshot dataset were used.
• In part of the cases more than one image corresponded a unique person.
• Each enrolled face template from the Profile dataset was compared against all enrolled templates from the Mugshot dataset.
• The comparisons were fully zero-effort, meaning there was no pre-grouping by gender, age or other covariates.

Identification – Neurotechnology algorithm accuracy in this experiment was 0.72% FNIR at 0.3% FPIR. The most accurate contender showed 0.16% FNIR at the same FPIR.

Investigation – Neurotechnology algorithm accuracy in this experiment was 0.14% FNIR at Rank1. The most accurate contender showed 0.09% FNIR at Rank1.

Scenario overview

• Images from Visa and Border datasets were used for persons' enrollment.
• 1.6 million images from the Border dataset were used.
• In part of the cases more than one image corresponded a unique person.
• Each enrolled face template from the Border dataset was compared against all enrolled templates from the Visa dataset.
• The comparisons were fully zero-effort, meaning there was no pre-grouping by gender, age or other covariates.

Identification – Neurotechnology algorithm accuracy in this experiment was 24.12% FNIR at 0.3% FPIR. The most accurate contender showed 4.77% FNIR at the same FPIR.

Investigation – Neurotechnology algorithm accuracy in this experiment was 5.10% FNIR at Rank1. The most accurate contender showed 3.87% FNIR at Rank1.

Scenario overview

• Images from Visa and Kiosk datasets were used for persons' enrollment.
• 1.6 million images from the Visa dataset were used.
• In part of the cases more than one image corresponded a unique person.
• Each enrolled face template from the Kiosk dataset was compared against all enrolled templates from the Visa dataset.
• The comparisons were fully zero-effort, meaning there was no pre-grouping by gender, age or other covariates.

Identification – Neurotechnology algorithm accuracy in this experiment was 6.82% FNIR at 0.3% FPIR. The most accurate contender showed 1.23% FNIR at the same FPIR.

Investigation – Neurotechnology algorithm accuracy in this experiment was 1.22% FNIR at Rank1. The most accurate contender showed 0.45% FNIR at Rank1.

Scenario overview

• 1.6 million images from the Border dataset were used for persons' enrollment.
• In part of the cases more than one image corresponded a unique person.
• Age attribute was used to exclude template pairs, which represented same or different persons with age difference of less than 10 years at the capture moment, from the comparison.
• Each face template was compared against all other enrolled templates.

Identification – Neurotechnology algorithm accuracy in this experiment was 2.86% FNIR at 0.3% FPIR. The most accurate contender showed 0.20% FNIR at the same FPIR.

Investigation – Neurotechnology algorithm accuracy in this experiment was 0.39% FNIR at Rank1. The most accurate contender showed 0.23% FNIR at Rank1.

Scenario overview

• 3 million images from the Mugshot dataset were used for persons' enrollment.
• In part of the cases more than one image corresponded a unique person.
• Age attribute was used to exclude template pairs, which represented same or different persons with age difference of less than 12 years at the capture moment, from the comparison.
• Each face template was compared against all other enrolled templates.

These datasets were used for the experiments.