Moving usable security research out of the lab: evaluating the use of VR studies for real-world authentication research

Mathis, Florian (2023) Moving usable security research out of the lab: evaluating the use of VR studies for real-world authentication research. PhD thesis, University of Glasgow and University of Edinburgh.

Abstract

Empirical evaluations of real-world research artefacts that derive results from observations and experiments are a core aspect of usable security research. Expert interviews as part of this thesis revealed that the costs associated with developing and maintaining physical research artefacts often amplify human-centred usability and security research challenges. On top of that, ethical and legal barriers often make usability and security research in the field infeasible. Researchers have begun simulating real-life conditions in the lab to contribute to ecological validity. However, studies of this type are still restricted to what can be replicated in physical laboratory settings. Furthermore, historically, user study subjects were mainly recruited from local areas only when evaluating hardware prototypes. The human-centred research communities have recognised and partially addressed these challenges using online studies such as surveys that allow for the recruitment of large and diverse samples as well as learning about user behaviour. However, human-centred security research involving hardware prototypes is often concerned with human factors and their impact on the prototypes’ usability and security, which cannot be studied using traditional online surveys.

To work towards addressing the current challenges and facilitating research in this space, this thesis explores if – and how – virtual reality (VR) studies can be used for real-world usability and security research. It first validates the feasibility and then demonstrates the use of VR studies for human-centred usability and security research through six empirical studies, including remote and lab VR studies as well as video prototypes as part of online surveys.

It was found that VR-based usability and security evaluations of authentication prototypes, where users provide touch, mid-air, and eye-gaze input, greatly match the findings from the original real-world evaluations. This thesis further investigated the effectiveness of VR studies by exploring three core topics in the authentication domain: First, the challenges around in-the-wild shoulder surfing studies were addressed. Two novel VR shoulder surfing methods were implemented to contribute towards realistic shoulder surfing research and explore the use of VR studies for security evaluations. This was found to allow researchers to provide a bridge over the methodological gap between lab and field studies. Second, the ethical and legal barriers when conducting in situ usability research on authentication systems were addressed. It was found that VR studies can represent plausible authentication environments and that a prototype’s in situ usability evaluation results deviate from traditional lab evaluations. Finally, this thesis contributes a novel evaluation method to remotely study interactive VR replicas of real-world prototypes, allowing researchers to move experiments that involve hardware prototypes out of physical laboratories and potentially increase a sample’s diversity and size.

The thesis concludes by discussing the implications of using VR studies for prototype usability and security evaluations. It lays the foundation for establishing VR studies as a powerful, well-evaluated research method and unfolds its methodological advantages and disadvantages.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
• Colleges/Schools: College of Science and Engineering > School of Computing Science
• Supervisor's Name: Khamis, Dr. Mohamed and Vaniea, Dr. Kami
• Date of Award: 2023
• Depositing User: Theses Team
• Unique ID: glathesis:2023-83723
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 10 Jul 2023 10:06
• Last Modified: 11 Jul 2023 12:45
• Thesis DOI: 10.5525/gla.thesis.83723
• URI: https://theses.gla.ac.uk/id/eprint/83723

Related URLs

• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Video
• Video
• Video
• Video
• Video
• Video
• Video
• Video
• Video
• Video
• Video
• Video