Optimizing face-matching with artificial intelligence (AI) through trust calibration

Ng, Yu Wa (2024) Optimizing face-matching with artificial intelligence (AI) through trust calibration. PhD thesis, University of Glasgow.

Abstract

Face-matching is an important task that is used as an identity verification method in many applied settings. Advances in Artificial Intelligence (AI) have meant that facial recognition systems are continuously improving in terms of accuracy and are increasingly incorporated into the workplace. The use of e-gate at the border is an example of this technology and illustrates the involvement of humans in the identity verification process. The overarching aim of this research was to identify and better understand factors that influence this human-AI interaction. There was a particular focus on understanding the role of trust, to investigate the possibility of trust calibration. Calibrated trust was expected to help facilitate the interaction and improve face-matching performance. This thesis details the experimental studies that were conducted to achieve this aim.

Chapter 3 presented the findings of Pilot Study 1, providing baseline results of using AI support in a face-matching context by comparing face-matching performance between different groups using AI support of high or low reliability or no AI support. Findings showed that using AI in the decision-making process improved accuracy, particularly when AI was reliable. When AI had limited reliability, this did not affect performance more than not using AI support. The impact of AI errors on human performance was also explored. Experiment 1, also in Chapter 3, used a similar experimental design and found that using AI with limited reliability introduced response bias showing that participants were more likely to believe that face pairs were of matched identities regardless of the truth.

Chapter 4 consisted of Pilot Study 2 and Experiment 2, both designed to examine the influence of presenting AI scores, a quantitative measure of (dis)similarity between two faces, in the face-matching decision process. Pilot Study 2 examined the influence of AI scores by comparing the performance of participants completing a face-matching task with or without AI scores. Findings showed AI support in the form of (dis)similarity scores influenced performance compared to using no AI. Experiment 2 showed that AI scores do not help calibrate trust when dissimilarity scores were presented alongside incorrect AI labels and there were no effects on face-matching decisions.

In Chapter 5, the final study examined the influence of face-matching expertise on face-matching performance with AI. The experiment recruited face-matching professionals and novices to take part in a face-matching test using AI support. Findings showed both face-matching professionals and novices experienced a decrease in trust and performance on trials where AI provided incorrect advice. The role of confidence in trust was also discussed.

In summary, the results of this research highlighted the impact of using AI on facematching performance and the role of trust in the use of facial recognition as a decision support system. The thesis concluded with recommendations on the use of AI in a facematching context and directions for future research are discussed to further the current understanding of human-AI collaboration in face-matching and calibrating trust to facilitate team collaboration.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Subjects: B Philosophy. Psychology. Religion > BF Psychology; Q Science > QA Mathematics > QA76 Computer software; T Technology > T Technology (General)
• Colleges/Schools: College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
• Funder's Name: Economic and Social Research Council (ESRC)
• Supervisor's Name: Pollick, Professor Frank and Scheepers, Dr. Christoph
• Date of Award: 2024
• Depositing User: Theses Team
• Unique ID: glathesis:2024-84686
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 12 Nov 2024 12:43
• Last Modified: 12 Nov 2024 12:51
• Thesis DOI: 10.5525/gla.thesis.84686
• URI: https://theses.gla.ac.uk/id/eprint/84686