Mobile learning evaluation: the development of tools and techniques for the evaluation of learning exploiting mobile devices through the analysis of automatically collected usage logs - an iterative approach

Trinder, Jonathan James (2012) Mobile learning evaluation: the development of tools and techniques for the evaluation of learning exploiting mobile devices through the analysis of automatically collected usage logs - an iterative approach. PhD thesis, University of Glasgow.

Abstract

There have been many claims as to the benefits of personal digital assistants (PDA) as tools in education, but little objective data concerning device usage patterns. The aim of this project was to overcome this deficiency by objectively investigating the use of mobile devices in teaching and learning, specifically in the process of formative assessment.

A bespoke PDA application was written, which recorded in detail when PDA applications were being used and overcame a number of technical barriers in securing this information for later analysis. This data, along with information on student access to the University Virtual Learning Environment (VLE) and final student examination results, formed the main objective datasets recovered in the project. Novel data analysis tools and methodologies were developed to mine the extensive, heterogeneous datasets obtained, and efficiently characterise how students used PDA applications. Finally, data analysis was performed on four cohorts of students: i) fourteen joint honours students in electronics and software engineering, where researchers learned that even for technologically adept students, simple issues (such as failure to replace batteries and resultant data loss) could significantly restrict useful research outputs; ii) five summer school students using quiz applications and question sets, where the first significant evidence of the educational benefits of using mobile devices was obtained; iii) students who failed to interact in any way with what they saw as obsolete devices, emphasising the importance of keeping student PDAs current with modern technology; iv) a full scale trial involving a 1st year cohort of BTechEd students, where the lessons learned in phases 1-3 were applied.

In the trials, technical and human-computer interface barriers to securing useful data were encountered and overcome, and guidelines for future good practice, of significant use to practitioners in the research area, determined. Patterns and modes of their PDA use - considering a range of factors including overall duration of use, use as a function of time of day or time of week, and the complexity of use (e.g. frequency of application switching within a usage session) - were obtained and correlated with exam results and access to the University VLE. A number of usage characteristics of successful and unsuccessful learners were extracted from this data. In addition to these results, novel student behaviour was observed, with volunteer students actively avoiding returning data despite stated interest in the project, lowered technical barriers, significant inducements, and guarantees of data anonymity. We suggest a number of social factors, including on the nature of peer group formation in student cohorts and the socially disruptive nature of new technology, as contributing to this effect and identify the area as worthy of future investigation.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Keywords: mobile,mlearning,evaluation,iterative,learning,PDA
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science; L Education > L Education (General)
• Colleges/Schools: College of Social Sciences > School of Education
• Supervisor's Name: Magill, Dr. J.V. and Roy, Dr. S.
• Date of Award: 2012
• Depositing User: Mr Jonathan Trinder
• Unique ID: glathesis:2012-3303
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 25 Apr 2012
• Last Modified: 10 Dec 2012 14:06
• URI: https://theses.gla.ac.uk/id/eprint/3303