Language & Brain - - PowerPoint PPT Presentation

Language & Brain

Why bother? What could we learn?

• something about how language works
• something about how the brain works
• nothing (interdisciplinary cross-sterilization)

Chomsky 1959

Franz Josef Gall 1758-1828

Ha Hand s sca canner

Phrenology (bad idea) Organology (good idea)

Two enduring ideas deriving from Gall

• Faculty psychology

The mind has a ‘parts list.’

• Experience-dependent plasticity

Using the parts changes their neuronal realization.

The wave of the past “Intuitive psychological organology ”

Phineas Gage, 1848 Before: responsible, well- mannered, well-liked, efficient worker, pious After: capricious, impulsive, irreverent, hypersexual Damage involved VMPFC

Paul Broca 1861, 1865

Leborgne’s (Tan’s) brain

Geschwind 1985

Broca’s Area “production” “syntax” Wernicke’s Are “reception” “semantics” Visual system (van Essen) Auditory system (Hackett)

The visual and auditory systems are highly articulated. Is there any a priori reason to believe that language will be an order of magnitude simpler, captured by two brain areas?

Functional anatomy of speech perception. Hickok & Poeppel 2007

Non-invasive recording from human brain (Functional brain imaging)

Positron emission tomography (PET) Functional magnetic resonance imaging (fMRI) Electro- encephalography (EEG)

Excellent spatial resolution (<1 mm) Limited temporal resolution (~1sec) Limited spatial resolution (<1 cm) Excellent temporal resolution (<1msec) Hemodynamic techniques Electro-magnetic techniques

Magneto- encephalography (MEG)

• D. Poeppel , A. Braun et al.

Language is not monolithic

Phonetics/phonology

sound structure

Morphology

word structure

Lexical semantics

word meaning

Syntax

sentence structure

Prosody

sentence melody

Compositional semantics

sentence meaning

Discourse

larger meaning scale

language-o-topy

The wave of the past “Intuitive psychological organology ” The wave of the present “Cognitive psychological organology”

syntax phonology semantics

Linguistics Neuroscience Fundamental elements of representation distinctive feature dendrites, spines syllable neuron morpheme cell-assembly/ensemble noun phrase population clause cortical column Fundamental operations on primitives concatenation long-term potentiation linearization receptive field phrase-structure generation

• scillation

semantic composition synchronization

? ?

Is there a future? Problems for interdisciplinarity and unification I

There is an absence of ‘linking hypotheses’ by which we explore how brain mechanisms form the basis for linguistic computation. Aligning the alphabets or primitives or atoms is a formidable challenge.

Is there a future? Problems for interdisciplinarity and unification II Ontological Incommensurability Problem (neurolinguistics in principle): The units of linguistic computation and the units of neurobio- logical computation are incommensurable. Therefore, an attempt at reduction makes no sense. Why are there no linking hypotheses? Granularity Mismatch Problem (neurolinguistics in practice): Linguistic and neuroimaging studies of language operate with objects of different granularity. linguistics

• -- fine-grained distinctions

neuroscience

• -- broader conceptual distinctions

Neuroscience cannot succeed in seeking “syntax” (or “phonology”) because syntax etc. are not monolithic but have many parts.

Poeppel & Embick, 2005

Is there a future? Problems for interdisciplinarity and unification III Linguistics Neuroscience distinctive feature dendrites, spines morpheme cell-assembly/ensemble noun phrase population clause cortical column concatenation long-term potentiation linearization receptive field phrase-structure generation

• scillation

semantic composition synchronization fractionate into generic formal operations segmentation concatenation comparison recursion identify basis for generic formal operations segmentation concatenation comparison recursion

?

Desiderata for a model bridging neuronal mechanisms and linguistic representation x---y y x z concatenation constituency recursion Neurobiological mechanisms that can form the basis of elemental steps involved in most linguistic computation: Is there a future? Problems for interdisciplinarity and unification IV This is the granularity - and level of abstractness - of operations that can profitably be studied in animal research as well, doing away with questions such as “are humans different or better or higher, or not” and turning to the typical questions such as: “How does this work?”

Putative primitives - the view on irreducible representations and operations from semantics (Pietroski) and syntax (Hornstein)

• Variables, a way to link variables
• One-place predicates, thematic roles
• Operation with the power of conjunction and existential closure
• Concatenation (a-directional)
• Labeling: concatenate turns into one of its constituents
• Some mechanism (copy) to deal with positional specificity of variables

‘Unification Problem’ Marr’s computational approach permits development of linking hypotheses computational algorithmic implementational

How much can be gained by focusing only on localization by way of imaging? Not much, at this point - it is the ‘homework problem’, that is, an important but ultimately uninteresting step from the point of view of explanation. Can we achieve unification by working on localization? No! We need explicit linking hypotheses between well characterized brain mechanisms and linguistic computation. WRONG QUESTION: where are syntax/phonology/ semantics mediated? RIGHT QUESTION: what kind of computations in the brain form the basis of linguistic representations and operations? Is there a future? Problems for interdisciplinarity and unification V

xx yy xxx yy zzzz zzz ppppp pppp qqqq qqq

• ojjjoo
• ooojj
• oooojjjj

There is localisation, but what is localized is tissue that executes specific computations, such as, say, addition (xxx) or subtraction (zzz) or division (qq),

• ver representations (data structures) of certain types.

sss sss

The cognitive faculties (the “parts”

• f the human cognome) are not

monolithic but composed of multiple computational subroutines.

The wave of the past “Intuitive psychological organology ” The wave of the present “Cognitive psychological organology”

syntax phonology semantics

The wave of the future “Computational organology”

recursion constituency sequencing linearization

Localization of generic computational subroutines

÷÷÷ ÷÷÷

[+ cons, -son] [-cons, +son] [+ cons, -son] x x x

c t a

LAR/PHAR LAR/PHAR LAR/PHAR

[-cont] [-cont]

PLACE PLACE PLACE GLOT

[-voice]

DORSAL

[-ATR]

DORSAL CORONAL

[-back, -high, +low]]

GLOT

[-voice] [+ant]

?

(a) (b) (c) (d)

phonological primal sketch

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci Lau et al. 2008, Nat Rev Neurosci

Newton, Principia Our brain Our brain, really Our intuition

Axial View Sagittal View

Neuromagnetic activity is recorded from the whole head (160 channels)

MEG: From signals to magnetic field maps to sources

Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex

Huan Luo & David Poeppel

Single-unit responses robustly encode conspecific vocalizations

Machens et al., Nat. Neurosci. 2003 Grasshopper (peripheral auditory neurons) Narayan et al., J. Neurophys. 2006 Zebra Finch (Field L)

“Many natural sounds including vocal communication sounds display striking time-varying structure over multiple time scales” “We demonstrate the existence of distinct time scales for temporal resolution and temporal integration and explain how they arise from cortical neural responses to complex dynamic sounds.” [~10 ms and ~ 500 ms]

Design: evaluate coherence across single trials elicited by sentences

Luo & Poeppel, Neuron, 2007

Luo & Poeppel, Neuron, 2007

Theta phase

Materials: Smith, Delgutte, and Oxenham, Nature, 2002

Luo & Poeppel, Neuron, 2007

Theta phase has the sensitivity to discriminate based on single trials

Theta phase tracking displays the specificity to discriminate sentences

Luo & Poeppel, Neuron, 2007

Classification analysis

Luo & Poeppel, Neuron, 2007

A ~ 200 ms window analyzes the input signal -- The syllable as primitive

Endogenous cortical rhythms determine cerebral specialisation for speech perception and production

Giraud et al. 2007, Neuron

Distribution of intrinsic cortical rhythms

Combined EEG/fMRI recordings

N=12 + 8 subjects at rest (twice 20 min.) EEG, 32 channels, continuous acquisition. fMRI, 1.5 T and 3 T Siemens, sparse acquisition No auditory input beyond the MRI scanner noise. Analyses from central electrodes to observe temporal asymmetries with minimal lateralization bias.

Giraud et al. 2007, Neuron

3–6 Hz

50 100 150 200 250 300 5 50 100 150 200 250 300 5

time [scan number]

Continuous EEG recording (20 min.) Sparse fMRI acquisition (20 min.)

Regressors

FFT

power [1010 uV2]

3s

gap 1s fMRI 3s gap 1s fMRI 3s gap 1s Segments of EEG trace Sequence of fMRI acquisition Raw data Estimated data (raw data convolved with hemodynamic function)

Combined EEG/fMRI recordings: Approach

28 – 40 Hz

Giraud et al. 2007, Neuron

Mouth Tongue

3-6 Hz EEG band 28-40 Hz EEG band

Heschl Experiment 1 (1.5T) Experiment 2 (3T)

Group results: topography of theta and gamma in premotor cortex

Giraud et al. 2007, Neuron

Motor constraints on speech (Frame/Content theory, MacNeilage and Davis, Current Opinion Neurobiol. 2001)

Mechanical properties of the speech apparatus (e.g.,spontaneous

• scillation frequency of the jaw) determine rhythmic properties of spoken

language (e.g. syllabic rate - theta rhythm)

Giraud et al. 2007, Neuron

Speech analysis

Functional interactions between perceptual and motor speech systems: internal forward model at time scales ‘of interest’

Motor output Sensory feedback

Efference copies Fine articulatory tuning

Three messages

• 1. Language is not monolithic.

(Even subroutines of language comprehension, such as speech perception, are highly complex.) The constituent elementary computations are likely mediated by an array of cortical areas.

• 2. MEG is a useful -- and

underutilized - tool to investigate a range of issues in cognitive neuroscience. The data can provide an interesting bridge to questions of neural coding.

• 3. The phase of low frequency

responses (e.g. theta) provides a sensitive (trial by trial) neurophysiological index of online processing and can be used to assess the ‘temporal granularity’ of perceptual analysis (sliding temporal window).

UQAM -- Origins of Language, 6/24/10 David Poeppel NYU Psychology and Neural Science david.poeppel@nyu.edu http://psych.nyu.edu/clash/poeppellab.html http://talkingbrains.blogspot.com/