The processing of orthography in visual word recognition and its sensitivity to top-down modulation

Taylor, Jack E. (2022) The processing of orthography in visual word recognition and its sensitivity to top-down modulation. PhD thesis, University of Glasgow.

Abstract

A vital component of visual word recognition is the decoding of orthography, the rules by which language is transcribed from and to visual script. Literate humans demonstrate considerable consistency in the timing and localisation of orthographic processing in the brain, with an early occipitotemporal response showing robust sensitivity to orthographic information as early as 150 - 200 ms post-stimulus. It has been proposed that, consistent with mechanisms involved in other visual perceptual processes, orthographic processing is sensitive to higher-level information provided via top-down inputs. In this thesis, I investigate the degree to which early orthographic processing is modulated by higher-level expectations for word forms over unpredicted word forms that vary in their predictability. I focus on the N1 event-related potential component observed in electroencephalography (EEG). Peaking around 170 ms, this component has shown consistent sensitivity to orthographic information.

I present evidence from two EEG experiments probing the effect of predictions on orthographic processing. In the first of these experiments, I examine the interaction between task (lexical decision, semantic categorisation) and stimulus (category-relevant words, category-irrelevant words, pseudowords, nonwords). I replicate findings of sensitivity to orthography in the N1, and, consistent with previous research, find evidence for a general effect of task on processing during the N1. However, I observe a lack of selective sensitivity for category-relevant word forms in the semantic categorisation task, where such a finding would advocate category-level top-down modulation of the N1. I argue that a sensitivity to higher-level predictions in orthographic processing would require a transcoding of information from semantic to orthographic representations, which would be necessarily computationally lossy and entail a loss of specificity in predictions. As a result, selective sensitivity to predicted word forms may only be expected when predictions are more targeted, such that they maximise the specificity of any predictions for orthographic input.

In the second EEG experiment I show that, indeed, when predictions are more targeted, for specific word forms, an effect of prediction is observed in the N1. I employ a picture-word verification paradigm to induce participants to generate strong predictions for upcoming words. I show an interaction between picture-word congruency and predictability, where predictability negatively predicts N1 amplitudes for picture-congruent words, and positively predicts N1 amplitudes for picture-incongruent words. I argue that these findings are inconsistent with typical predictive coding accounts, in which predicted orthographic information is "explained away" such that activity scales with prediction error, but support an account in which top-down modulation results in a "sharpened" sensitivity to predicted orthographic features, such that activity scales with prediction congruency. I suggest that the development and testing of computational models of orthographic processing can better delineate the specific mechanisms by which top-down contributions influence orthographic processing.

A vital component of any model of orthographic processing is a description of orthography and orthographic similarity. I argue that orthographic similarity is particularly relevant to descriptions of how top-down modulation influences orthographic processing - whether responses are "explained away" or "sharpened", the degree to which predictions modulate neural activity associated with orthographic processing should correlate with the similarity between the predicted and presented word form. Orthographic Levenshtein distance, the current gold-standard measure of orthographic similarity in alphabetic orthographies, by default overlooks sub-character complexities. In work in this thesis, I develop and validate a sub-character measure of orthographic similarity, showing that its performance in predicting behavioural and neural correlates of visual word recognition, including the N1 component in EEG, can elucidate and better explain sensitivity to sub-character features of orthography.

I additionally describe and validate methodological approaches that can improve experimental design and statistical inference in the research area. Specifically, I describe an R package I developed, LexOPS, that provides a formalisation of an item-matching approach that is flexible and reproducible. Such item-wise matching of factorial conditions is a key component of experimental design in visual word recognition research, as well as in other areas of psychological science. I also describe a formalised distribution-wise approach to matching that can be integrated with the item-wise approach implemented in LexOPS. I apply the item-wise and distribution-wise approaches to matching in stimulus design of the experiments reported in this thesis. Another key component of psychological research that I examine is the norming of items on subjective Likert ratings. Work in this thesis demonstrates, via Monte Carlo simulations, that a statistical approach that appropriately accounts for the hierarchical and ordinal nature of rating norming studies’ data, drawing inferences from cumulative-link mixed-effects models, can more accurately and meaningfully summarise rating norms. I demonstrate the improvements conferred by this approach on existing datasets, including normed ratings of perceived orthographic similarity.

This thesis combines multiple complementary approaches to provide insight into the processing of orthography in visual word recognition, and the degree to which such processing may be sensitive to top-down contributions. I provide in-depth experimental evidence and methodological developments that can inform and equip future research and computational models of orthographic processing in the brain.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Colleges/Schools: College of Medical Veterinary and Life Sciences > School of Psychology & Neuroscience
• Supervisor's Name: Sereno, Dr. Sara C. and Rousselet, Dr. Guillaume
• Date of Award: 2022
• Depositing User: Theses Team
• Unique ID: glathesis:2022-83081
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 15 Aug 2022 11:01
• Last Modified: 15 Aug 2022 11:01
• Thesis DOI: 10.5525/gla.thesis.83081
• URI: https://theses.gla.ac.uk/id/eprint/83081

Related URLs

• Article DOI