Learning and reversal in the sub-cortical limbic system: a computational model

Thompson, Adedoyin Maria (2010) Learning and reversal in the sub-cortical limbic system: a computational model. PhD thesis, University of Glasgow.

Abstract

The basal ganglia are a group of nuclei that signal to and from the cerebral
cortex. They play an important role in cognition and in the initiation
and regulation of normal motor activity.
A range of characteristic motor diseases such as Parkinson's and Huntington's
have been associated with the degeneration and lesioning of the
dopaminergic neurons that target these regions.
The study of dopaminergic activity has numerous benefits from understanding how and what
effects neurodegenerative diseases have on behavior to determining
how the brain responds and adapts to rewards.
The study is also useful
in understanding what motivates
agents to select actions and do the things that they do.

The striatum is a major input structure of the
basal ganglia and is a target structure of dopaminergic neurons which originate from the
mid brain. These dopaminergic neurons release dopamine which
is known to exert modulatory influences on the striatal projections.
Action selection and
control are involved in the dorsal regions of the striatum while the dopaminergic
projections to the ventral striatum are involved in reward based learning
and motivation.

There are many computational models of the dorsolateral
striatum and the basal ganglia nuclei which have been proposed
as neural substrates for prediction, control and action selection.
However, there are relatively few models which aim to describe the role of the
ventral striatal nucleus accumbens and its core and shell sub divisions in motivation
and reward related learning.
This thesis presents a systems level computational
model of the sub-cortical nuclei of the limbic system which
focusses in particular, on the nucleus accumbens shell and core circuitry.

It is proposed that the nucleus accumbens core plays a role in enabling
reward driven motor behaviour by acquiring stimulus-response
associations which are used to invigorate responding.
The nucleus accumbens shell mediates the facilitation of highly rewarding behaviours
as well as behavioural switching.

In this model, learning is achieved by implementing
isotropic sequence order learning and a third factor (ISO-3) that
triggers learning at relevant moments. This third factor is modelled by
phasic dopaminergic activity which enables long term potentiation
to occur during the acquisition of stimulus-reward associations.
When a stimulus no longer predicts reward, tonic dopaminergic activity
is generated. This enables long term depression.
Weak depression has been simulated in the core so that stimulus-response
associations which are used to enable instrumental response
are not rapidly abolished. However, comparatively strong depression is implemented
in the shell so that information about the reward is quickly updated.
The shell influences the facilitation of highly rewarding behaviours
enabled by the core through a shell-ventral pallido-medio dorsal pathway.
This pathway functions as a feed-forward switching mechanism and enables
behavioural flexibility.

The model presented here, is capable of acquiring associations between stimuli and
rewards and simulating reversal learning.
In contrast to earlier work, the reversal is modelled by the
attenuation of the previously learned behaviour. This allows for
the reinstatement of behaviour to recur quickly as observed in
animals.
The model will be tested in both open- and closed-loop experiments
and compared against animal experiments.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Keywords: Limbic system, Learning, reversal learning, dopamine, nucleus accumbens
• Subjects: B Philosophy. Psychology. Religion > BF Psychology; Q Science > Q Science (General)
• Colleges/Schools: College of Science and Engineering > School of Engineering
• Supervisor's Name: Porr, Dr. Bernd and John, Prof. O'Reilly
• Date of Award: 2010
• Depositing User: Miss Adedoyin Maria Thompson
• Unique ID: glathesis:2010-1760
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 11 Jun 2010
• Last Modified: 10 Dec 2012 13:46
• URI: https://theses.gla.ac.uk/id/eprint/1760