CORTEX FOR DECISION MAKING Sandrine Duverne Paris, Dec 02, 2019 - - PowerPoint PPT Presentation
Master of Integrative Biology Neuroscience Specialization UE 5NB04 INFORMATION PROCESSING IN THE PREFRONTAL CORTEX FOR DECISION MAKING Sandrine Duverne Paris, Dec 02, 2019 Decision making A SERIES OF INTERWEAVED CHOICES IN EVER-CHANGING
Master of Integrative Biology Neuroscience Specialization UE 5NB04
Sandrine Duverne
Paris, Dec 02, 2019
A SERIES OF INTERWEAVED CHOICES IN EVER-CHANGING ENVIRONMENT
Decision making refers to a set of cognitive processes that results in the selection of a belief or a course of action among several alternative possibilities.
Decisions are : either automatic/uncontrolled or voluntary/controlled. either irrational/intuitive/emotional or rational/thoughtful/effortful Based on both knowledge (subjective values, preferences, beliefs) and information in the immediate environment (context, instructions, rewards). dynamic and temporally intermixed.
The prefrontal cortex
A very large part of the brain that plays a key role in high- level cognitive processes involved in goal-driven behaviors
macaques and 1.2-fold greater than in chimpanzees
2.4-fold greater than in macaques and 1.7-fold greater than in chimpanzees (Donahue et al., 2018)
Human cognitive uniqueness should therefore focus less on the frontal lobes in isolation and more on distributed neural networks.
(Barton & Venditti, 2013)
MODULAR APPROACH localization
Goal-directed choices can be subdivided into a set of discrete component processes with serial and localized neural implementation.
CIRCUIT-BASED APPROACH distribution
Goal-directed choices emerge from repeated computations that are distributed across many brain regions that perform similar computations.
Hunt & Hayden, Nature Rev Neurosci, 2017
Temporal resolution EEG, MEG Spatial resolution fMRI, PET scans Connectivity modelling static functional connectivity dynamic functional connectivity
An old hypothesis recycled to study the organization of cognitive functions in the brain.
Fodor, 1983 Gall & Surzheim , 1810
Modules are : Domain specific: dedicated to a specific type of process or computation Encapsulated: each module refer to
Associated with a specific neural structure Their output converge and are centralized in a central system
The mind is composed of composed of neural structures (modules) with distinct evolutionarily-developed functions.
Fodor, The Modularity of Mind, 1983
A series of component processes : evaluation of options comparison of option values selection of an appropriate action plan monitoring of the outcome of the choice
Each component process is assumed to correspond to discrete neural computations implemented in distinct neural modules .
Hunt & Hayden, Nature Rev Neurosci, 2017
Lateral PFC Control system Ventral PFC Valuation system Dorso-Medial PFC Motivational system
Each PFC axis
distinct dimensions of information processing
COGNITIVE CONTROL IN LATERAL PFC
Hierarchical processing of information
Attention
attention
attention
attention Memory
memory
memory
memory Problem solving
resolution
Set of processes that allow information processing and behavior to vary adaptively from moment to moment depending on current goals and environmental constrains
The frontal and posterior parts of the brain are two networks that are hierarchically organized:
PFC functions are hierarchically organized in an anterior-posterior axis to process abstract to more concrete information.
Fuster, Trends NeuroSci, 1997
Task sets : a configuration of cognitive processes that is actively maintained for subsequent task performance Dorsolateral PFC functions are organized to selectively represent, update and implement a specific task Competition between neural representations of task sets during task switching Facilitate task performance by establishing context-dependent rules
Sakai, Ann Rev Neurosci, 2003 Collins & Frank, PLoS Comput Biol, 2016
Abstraction
subsets of actions are linked with contecxtual cues to form task sets
associations
Temporal processing
ensemble of task sets are maintained and temporally extended
episode (frequencies of cues)
performed but can be quickly reactivated
Koechlin & Summerfield, Trends Cog Sci, 2007
Cognitive control in lateral PFC
set based on contextual cues
sets during episodes with repeated cues
VALUATION SYSTEM IN VENTRAL PFC
Reward-based choices
Reward
dopaminergic system with direct projection from the ventral striatum Subjective values
and reward- probabilities associated with stimuli
across stimuli Social cognition
representation
Set of processes that allow information processing and behavior to vary adaptively from moment to moment depending on available rewards in the environment.
Daw et al., Nature 2006 4-arm bandit choice task
Markov Decision Process to maximize expected gain by choosing between competing choices associated with stockastic rewards
Reward-related activations
vmPFC activations linearly increase with rewads and choices
Doya, HFSPJ 2007 Dopaminergic loops Reinforcement learning and dopaminergic loops
MODEL-FREE DECISION IN STRIATUM
temporal-difference learning
stored value representations
from experience MODEL-BASED DECISION IN PFC
transitions
relationships is used to anticipate long- run consequences of candidate actions
estimated by iteratively searching through a decision tree
Daw et al, Nature Neurosci 2005
Glascher et al., Neuron 2010
Decision tree paradigm sequential two-choice Markov decision State and reward prediction error in striatum vs lateral PFC State prediction correct signal in striatum and vmPFC State-based > standard RL
Hampton et al., J Neurosci 2006
Valuation system in ventral PFC
behavioral choices: combines model-free striatal signals and model-based lateral PFC signals
MOTIVATIONAL CONTROL IN DORSO-MEDIAL PFC
A key interface between reward and control systems
Cognitive control
control
cognitive control Reward-based decisions
values
Effort
tradre-off
Mediating role of the dorsal anterior cingulate cortex (dACC) as building the links between rewards and actions
Error monitoring
van Schie et al, 2004
Conflict detection
Carter & van Veen, 2008
Hierarchical motivation
Kouneiher et al, 2009 Holroyd & Coles, 2002
Volatility of reward
ACC learns the value of information in an uncertain world
Behrens et al., Nature Neurosci 2007
Foraging
ACC encodes the average value of the foraging environment and cost of foraging
Kolling et al., Science, 2012
ACC activations scale with choice difficulty Shenhav et al., Nature Neurosci 2014 ACC activations signal the need to switch and depart from the default choice Donoso et al., Science 2014
Motivational control in Dorso- medial PFC
valuation system
control
choice
SUMMARY OF THE MODULAR APPROACH
Strenghts and limits
connected components support the emergence of adaptive behavior and cognition
evolve under changing or competitive selection criteria
STRENGHTS RESOLUTION LIMITS
modules, which contain sub-sub-modules (the homunculus fallacy of infinite regress)
large-scale networks
pathologies)
BRAIN DYNAMICS FOR DECISION MAKING
Highlights of large-scale brain circuits: Functional connectivity: parallel and antagonistic pathways complicate circuit neuronal dynamics shape the activity
neuromodulation reconfigures circuit properties circuits interact to generate behavior
Neural circuits interconnect to
information and form large scale brain networks.
Bargmann & Marder, Nature Methods, 2013
comprises multiple distributed networks of distinct connectivity profiles that are embedded within largely parallel, interdigitated circuits.
networks reveals how a distributed network might serve as a bridge between sensory and motor networks
Yeo et al., J Neurophysiol 2011
Three overarching principles : Decisions (and values) are formed in a distributed fashion across many brain regions. The distributed networks are highly recurrent in nature. The networks are organized into functional and temporal hierarchies.
Choices are formed in a distributed fashion across many brain regions that act in concert and perform similar computations.
Hunt & Hayden, Nature Rev Neurosci, 2017
COGNITIVE CONTROL BEYOND LATERAL PFC
Sensorimotor transformations in the fronto- parietal network
Dual attention system
Goal-directed (top-down) attention for selection of stimuli and responses
Stimulus-driven (bottom-up) attention for detection of relevant stimuli 'circuit breaker' of the dorsal system
Corbetta & Schulman, Nature Reviews Neurosci 2002
SPL) are related to visuospatial attention and eye movements
familiarity or perceived memory strength
Dynamic competition between dorsal and ventral parietal regions contributes to reorienting retrieval in episodic memory.
Sestieri et al., Nature Rev Neurosci 2017
from the continuous accumulation of noisy information until one
Evidence accumulation and motor intention are coupled: network-level interactions organize environmental sampling into rhythmic cycles (Fiebelkorn et al., Neuron 2018)
the process of accumulating evidence about the relatedness between probes and items in the memory set Evidence accumulation and motor intention are NOT coupled: IPS tracks perceived oldness, regardless of whether the response is correct or incorrect
Sestieri et al., Nature Rev Neurosci 2017
Cognitive control beyond lateral PFC
Two frontoparietal circuits of cognitive control:
evidence accumulation
evidence accumulation
VALUATION SYSTEM BEYOND VENTRAL PFC
Linking past and future memories in the default mode circuit
Large-scale network with a set of hubs densively connected with surounding regions Task-negative functional activations
independent thoughts, spontaneous activity dynamic, monitoring external environment broadly for unexpected events
the DMN
Raichle et al., PNAS 2001 ; Grecius e al., PNAS 2003
Interacting subsystems
prior experiences
use of this information for self-relevant mental simulations.
facilitate the retrieval and integration of relevant information, stored in their modality-specific cortical areas. Buckner, Ann N Y Acad Sci 2008
Rapid adjustments in control may occur via flexible task-dependent shifts in frontoparietal coupling patterns.
positive coupling change with task demand Functional connectivity varies with top-down signals of task-relevant
Sensorimotor processing facilitate moment-to-moment interactions with the environment
nodes when attention is directed toward one’s own thoughts and away from perceptual inputs abstract “top-level management” of thought exerts a general constraint according to task-relevant material
Dixon et al., PNAS 2018
Valuation system beyond ventral PFC DM network as an internal attention system:
relevant information
frontoparietal coupling
MOTIVATIONAL CONTROL BEYOND DORSO-MEDIAL PFC
A key interface between the default mode and control systems
Pyramidal cells van Economo cells
Eisenberger, N., 2012
The salience network is involved in identifying biologically and cognitively relevant events to guide flexible behavior.
affective tasks involving pain, empathy, disgust, and introspective processes, external and internal signals
valuation systems
signals from the AI and dACC to the motor system
Switching mechanisms in the ‘spotlight of attention’
control regions to maintain cognitive set and manipulate information in working memory
network to keep attention focused on task- relevant goals Menon, Brain Mapping: An Encyclopedic Reference 2015
Beaty et al., PNAS 2018
The creative connectome
High-creative network: set of frontal and parietal regions typically associated with default, salience, and executive brain systems. The creative brain is marked by a tendency to selectively and simultaneously engage these large-scale circuits to a greater degree than the less creative brain. Default and executive network coupling is mediated by salience network and supports creative idea production.
Motivational control beyond Dorso- medial PFC
Detection of salient stimuli to guide flexible behaviors throught two properties:
and DMN
creative idea generation
SUMMARY OF THE CIRCUIT- BASED APPROACH
Stenghts and limits
qualitatively distinct signals exchanged within a functional circuit
sequences of signals embedded in a single functional circuit
are distributed across several regions that support adaptive behavior
algorithms, in which noisy estimates of value are integrated sequentially over time
STRENGHTS LIMITS
HOW TO LINK THE MODULAR AND CIRCUIT- BASED APPROACHES
MODULAR APPROACH
functional data are sampled in a controlled environment
behavioral and functional data aim at testing hypotheses
CIRCUIT-BASED APPROACH
constrained by assumptions on anatomical connectivity
analyses: Analyses of behavioral and functional data are either free of or constrained by hypothesis testing
MODULAR APPROACH
behavioral responses: no one-to-one mapping of input-output in functional activations and behaviors
modules of a system: how to account for different developmental trajectories in life and for commonalities and differences across pathologies
CIRCUIT-BASED APPROACH
dynamics: how a system’s output results from the interactions of its components (causality)
that are likely to change in different modulatory states.
SDUVERNE@CREACOG.COM