Contactless AI-enabled hybrid sensing for cognitive impairment

Hameed, Hira (2024) Contactless AI-enabled hybrid sensing for cognitive impairment. PhD thesis, University of Glasgow.

Abstract

This thesis covers various aspects of Multi Model (MM) hearing impairments including human speech, sign language, behavior analysis, and facial expressions, which facilitate the deaf communities. Recent research using wearable, audio, and visual technologies for monitoring cognitive impairments in deaf individuals has its benefits but also presents certain limitations. For instance, while wearable devices provide body-mounted monitoring, their constant use can be uncomfortable, and there is a risk that deaf individuals might forget to wear them. Moreover, these devices need regular removal for recharging. In audio noise, even individuals with regular hearing may struggle to clearly hear someone’s voice. Camera-based visual information raises privacy concerns, and legal implications might restrict its broad usage in public and private areas due to issues like filming without consent, which is illegal in many countries. This thesis explores the use of Radio Frequency (RF) signals to sense human speech, sign recognition, behavior identification, and facial expressions using Wi-Fi, radar, and Radio Frequency Identification (RFID) signals. RF sensing provides an exciting opportunity for next-generation MM hearing aid devices. The RF-based hearing aid just requires Tx and Rx on a single chip. Additionally, RF signal in the form of Wi-Fi is currently present in many homes. People move around Wi-Fi signals, signal propagation is affected. Channel State Information (CSI) in Wi-Fi describes how a signal propagates from the transmitter to the receiver. In this thesis, the data collected in the form of CSI, micro-doppler, and Received Signal Strength Indicator (RSSI) signals are fed into Machine Learning (ML) and Deep Learning (DL) techniques for classification purposes. The proposed techniques successfully differentiate various activities, such as speech recognition, sign language recognition, and behavior analysis by using head movements, and facial expressions to understand the expressions of individuals when communicating with deaf people. These techniques utilise RF signals to individually differentiate each activity and achieve over 90% test accuracy. This thesis serves as a proof of concept for contactless MM hearing aid systems that can assist deaf people with different perspectives to live independently without the need to wear monitoring devices, audio, and visual devices.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Subjects: T Technology > T Technology (General); T Technology > TA Engineering (General). Civil engineering (General); T Technology > TK Electrical engineering. Electronics Nuclear engineering
• Colleges/Schools: College of Science and Engineering > School of Engineering
• Supervisor's Name: Abbasi, Professor Qammer H.
• Date of Award: 2024
• Depositing User: Theses Team
• Unique ID: glathesis:2024-84179
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 28 Mar 2024 13:49
• Last Modified: 28 Mar 2024 13:51
• Thesis DOI: 10.5525/gla.thesis.84179
• URI: https://theses.gla.ac.uk/id/eprint/84179

Related URLs

• Data DOI
• Data DOI
• Data DOI
• Article DOI
• Article DOI
• Article DOI
• Enlighten Publications Record
• Article DOI
• Article DOI
• Conference proceeding