Evaluating the impact of physical activity apps and wearables: an interdisciplinary investigation of research designs and methods

McCallum, Claire H. (2019) Evaluating the impact of physical activity apps and wearables: an interdisciplinary investigation of research designs and methods. PhD thesis, University of Glasgow.

Abstract

Many smartphone apps and wearables have been developed to promote physical activity, however there are challenges in assessing their impact. Apps and wearables are rapidly evolving technologies and thousands of physical activity apps that are publicly available on app stores remain unevaluated. There are concerns that traditional “gold standard” evaluation approaches, such as randomised control trials (RCTs), may be too slow to keep up with these, and produce effectiveness results that do not reflect real world settings. Rapid research designs (such as single case designs; SCDs) and innovative data collection methods (in-device sensors, device-generated user logs) have been proposed to improve research efficiency, yet preliminary evidence suggests they are not widely used in mHealth.

This thesis reports three studies undertaken to investigate the use of rapid research designs and efficient methods for evaluating physical activity apps and wearables. First, a scoping review of the extent to which these approaches are employed by health and HCI researchers. Second, semi-structured interviews with researchers, data scientists and industry professionals to provide a deeper understanding of current evaluation practices. Third, the development and refinement of a methodological framework to support researchers in using SCDs in automated app store evaluations of physical activity apps.

The findings suggest rapid research designs are not often employed in evaluations of physical activity and other health behaviour change apps. Researchers feel they face opportunity barriers (e.g. risking not being funded or published) and do not have the necessary skills (e.g. in using device generated user logs). Industry professionals appear to lack the motivation and time to evaluate effectiveness. Trade-offs were perceived between the measurement accuracy of in-device sensors and other factors such as user burden.

Automated trials may speed up evaluations of physical activity apps and wearables, and the suggested data collection framework aims to support researchers in conducting rigorous effectiveness evaluations using app store- based SCDs. However, further work is needed to enable industry professionals to use the framework to evaluate their publicly-available apps.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Keywords: Digital health, evaluation, mobile health, physical activity, smartphones, research design.
• Subjects: H Social Sciences > H Social Sciences (General); T Technology > T Technology (General)
• Colleges/Schools: College of Social Sciences > School of Social and Political Sciences
• Supervisor's Name: Gray, Dr. Cindy M., Rooksby, Dr. John and Chalmers, Professor Matthew
• Date of Award: 2019
• Depositing User: Dr Claire H McCallum
• Unique ID: glathesis:2019-72978
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 30 May 2019 10:09
• Last Modified: 05 Mar 2020 21:46
• Thesis DOI: 10.5525/gla.thesis.72978
• URI: https://theses.gla.ac.uk/id/eprint/72978