Multimodal radar sensing for ambient assisted living

Li, Haobo (2021) Multimodal radar sensing for ambient assisted living. PhD thesis, University of Glasgow.

Full text available as:
[img] PDF
Download (9MB)


Data acquired from health and behavioural monitoring of daily life activities can be exploited to provide real-time medical and nursing service with affordable cost and higher efficiency. A variety of sensing technologies for this purpose have been developed and presented in the literature, for instance, wearable IMU (Inertial Measurement Unit) to measure acceleration and angular speed of the person, cameras to record the images or video sequence, PIR (Pyroelectric infrared) sensor to detect the presence of the person based on Pyroelectric Effect, and radar to estimate distance and radial velocity of the person.
Each sensing technology has pros and cons, and may not be optimal for the tasks. It is possible to leverage the strength of all these sensors through information fusion in a multimodal fashion. The fusion can take place at three different levels, namely, i) signal level where commensurate data are combined, ii) feature level where feature vectors of different sensors are concatenated and iii) decision level where confidence level or prediction label of classifiers are used to generate a new output. For each level, there are different fusion algorithms, the key challenge here is mainly on choosing the best existing fusion algorithm and developing novel fusion algorithms that more suitable for the current application.
The fundamental contribution of this thesis is therefore exploring possible information fusion between radar, primarily FMCW (Frequency Modulated Continuous Wave) radar, and wearable IMU, between distributed radar sensors, and between UWB impulse radar and pressure sensor array. The objective is to sense and classify daily activities patterns, gait styles and micro-gestures as well as producing early warnings of high-risk events such as falls. Initially, only “snapshot” activities (single activity within a short X-s measurement) have been collected and analysed for verifying the accuracy improvement due to information fusion. Then continuous activities (activities that are performed one after another with random duration and transitions) have been collected to simulate the real-world case scenario. To overcome the drawbacks of conventional sliding-window approach on continuous data, a Bi-LSTM (Bidirectional Long Short-Term Memory) network is proposed to identify the transitions of daily activities. Meanwhile, a hybrid fusion framework is presented to exploit the power of soft and hard fusion. Moreover, a trilateration-based signal level fusion method has been successfully applied on the range information of three UWB (Ultra-wideband) impulse radar and the results show comparable performance as using micro-Doppler signature, at the price of much less computation loads. For classifying ‘snapshot’ activities, fusion between radar and wearable shows approximately 12% accuracy improvement compared to using radar only, whereas for classifying continuous activities and gaits, our proposed hybrid fusion and trilateration-based signal level improves roughly 6.8% (before 89%, after 95.8%) and 7.3% (before 85.4%, after 92.7%), respectively.

Item Type: Thesis (PhD)
Qualification Level: Doctoral
Keywords: Multimodal sensing, radar sensing, machine learning, human activity recognition.
Subjects: T Technology > TK Electrical engineering. Electronics Nuclear engineering
Colleges/Schools: College of Science and Engineering > School of Engineering
Supervisor's Name: Fioranelli, Dr. Francesco and Lekernec, Dr. Julien and Heidari, Dr. Hadi and Gurbuz, Dr. Sevgi
Date of Award: 2021
Depositing User: Dr Haobo Li
Unique ID: glathesis:2021-82039
Copyright: Copyright of this thesis is held by the author.
Date Deposited: 03 Mar 2021 09:23
Last Modified: 05 Dec 2022 10:00
Thesis DOI: 10.5525/gla.thesis.82039

Actions (login required)

View Item View Item


Downloads per month over past year