Facial expressions modulate the interpretation of spoken quantifiers in communication

Wu, Yichen (2023) Facial expressions modulate the interpretation of spoken quantifiers in communication. MSc(R) thesis, University of Glasgow.

Abstract

Face-to-face communication is multimodal and involves different information channels between interacting individuals, including speech, hand/arm gestures, vocalizations, and facial expressions. However, due to methodological challenges, most studies of multimodal communication have focused primarily on (1) hand/arm gestures while neglecting facial expressions and (2) the production of these gestures rather than their perception and interpretation. Consequently, it remains unknown how facial expressions contribute to multimodal communication, including how they interact with speech. We addressed this knowledge gap by investigating whether and how facial expressions influence the interpretation of spoken utterances of vague quantifiers (e.g., many or several). In each trial, participants viewed different faces identities who each uttered a sentence using a vague quantifier—e.g., ‘Of these, several are cows’— while displaying one of two facial expressions—opening or closing— or neutral. We expected that the opening facial expression would lead to larger number responses and the closing facial expression to lead to smaller number responses, analogous to hand gestures. We therefore examined whether participant number estimations shift in these expected directions. Results show that facial expressions modulated the participants’ number responses as expected. Specifically, four out of ten participants’ responses increased with opening facial expressions and decreased with closing facial expressions. This result suggests that facial expressions can represent quantities in a similar way as iconic hand gestures. Further, two participants increased their responses regardless of facial expression type, two participants increased their responses only for opening facial expression, and two participants showed no significant effect. Together, these results suggest that facial expressions serve several pragmatic functions in communication by operating as iconic gestures or as emphasizers. To our knowledge, this is the first study showing that facial expressions can influence the interpretation of vague quantifiers. Our results lay a foundation for future work examining how people interpret multimodal signals in daily conversation.

• Item Type: Thesis (MSc(R))
• Qualification Level: Masters
• Subjects: B Philosophy. Psychology. Religion > BF Psychology
• Colleges/Schools: College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
• Supervisor's Name: Jack, Professor Rachael
• Date of Award: 2023
• Depositing User: Theses Team
• Unique ID: glathesis:2023-83411
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 09 Feb 2023 08:52
• Last Modified: 09 Feb 2023 08:55
• Thesis DOI: 10.5525/gla.thesis.83411
• URI: https://theses.gla.ac.uk/id/eprint/83411