Brain Asymmetry and the Processing of Native, Second, and Artificial - - PowerPoint PPT Presentation

Brain Asymmetry and the Processing of Native, Second, and Artificial Languages

Chuansheng Chen Department of Psychology and Social Behavior University of California, Irvine

UCSD Cognitive Sciences 200, 11/13/2006

An International Collaborative Research Team

• Prof. Qi Dong, BNU

Dr . G ui Xue, UC LA

Li u Li , Li Ti an, Xue Feng Ji ang Ti ng, M ei Lei l ei , He Q i nghua, BNU G uo Yi , Hust on U. Zhao Li bo, U. I owa

Dr . Kewei C hen, ASU Dr . Yapeng W ang, BNU Dr . Xi nl i n Zhou, BNU Dr . Hongchuan Zhang, UC SD Pr of . Zhen Ji n, B 306 Pr of . Danl i ng Peng,

BNU

Zhang Lei , Zeng Yawei

Zhen Dong, Li Ke, B 306

Outline

 I. Left-hemisphere dominance in language processing

– Two likely exceptions: Chinese, second languages

 II. Special features of Chinese language and the right

hemisphere

– Reading (Study 1) and tone processing (Study 2)

 III. Right brain and second-lang. (English) processing

– Working memory (Study 3) and language switching (Study 4)

 IV. Brain asymmetry and artificial language learning:

– Visual words and phonological learning (Studies 5-8)

 V. Overall conclusions and implications for the

understanding of neural basis of language learning

– Specifically, language-specificity, neural compensation, and individual differences.

• I. Hemispheric Lateralization of Language

 About 95% of the total (Western) population

shows left hemispheric specialization for language:

– Almost all right-handed people – About 1/3 of the left-handed people

 Left dominance may have resulted from the

right-handed gestural system of communication that preceded the origin of language.

Two likely exceptions

 Chinese language, because of its

special features

 Second language, because of

compensation?

• II. What is special about Chinese

language?

 Pictographic Origins  Tonal  Addressed phonology

商 代 甲 骨 文 上 的 象 形 文 字 House Crossing Woods Eye Cup set Tongue Hand Elephant

鼠 牛 虎 兔 龙 蛇 马 羊 猴 鸡 狗 猪

Mouse Ox Tiger Rabbit Dragon Snake Horse Ram Monkey Chicken Dog Pig

Neural Substrates for Reading

Fiez & Petersen, 1998

 Left fusiform (labeled

as Visual Word Form Area (VWFA))

– Words – Pseudowords (e.g., Joki)

 Right fusiform

– Faces – Pictures of objects

Hemispheric specialization: Words on the left, faces/pictures on the right

Tarkiainen et al., 2002

What about (pictographic) Chinese then?

Stress vs. Tonal Languages

 Two phonological units:

– Segmental units (vowels, consonants) – Suprasegmental units (pitch, tone, stress)

 English is a stress language: Stress does not

provide much lexical information. Tomato: toMAto, but no TOmato, tomaTO. (with a few exceptions such as, CONtent, conTENT).

In contrast

 Chinese is a tonal language.  For example, 20 characters are pronounced

“ma”, with 4-6 characters for each of the four tones.

 Thus, tone is essential to the lexical

processing.

 “Tones, like consonants, are listed in the

lexicon as unit phonemes.” (Packard, 1986)

 Separate neural systems for segmental

and suprasegmental units? If separate, does the processing of Chinese tones rely more on right hemisphere for tone/music processing?

Addressed vs. assembled phonology

 Alphabetic languages typically have

assembled phonology, but some (e.g., Italian,called shallow orthographies, Paulesu et

al.) are easier to assemble than others (e.g.,

English, called deep orthographies).

 Some logographic languages (i.e., Korean)

use assembled phonology.

 Chinese mostly uses addressed phonology

•  Assembled phonology: One can read/sound it
• ut a word without understanding it.

 Addressed phonology: One can understand a

word without being able to sound it out.

Ability to read aloud words

lower grade level grade level beyond grade level Chinese American

Lee, Uttal, Chen, 1995

The addressed vs. assembled phonology, however, is not expected to vary by hemispheres .

So is there evidence of a special role of the right hemisphere in the processing of Chinese?

Past Research on Chinese

 Hemifield Experiments

– Some found right hemispheric dominance

(Cheng & Yang, 1986, Brain Lang.; Tzeng et al., 1979, Nature)

– Some found left hemispheric dominance

(Besner et al., 1982, Brit. J. Psych.)

– Some found bilaterality (Fang, 1997, J. Exp Psych;

Leong et al., 1997, Brain Lang.)

– Gender differences (with the foveal splitting method,

Hsiao & Shillcock, 2005) – There is a need to localize possible hemispheric asymmetry

Past Research (Cont’d)

 ERP and fMRI Experiments

– All 20 studies found left hemispheric dominance in the frontal region – But with bilaterality or rightward lateralization in the occipital and occipito-temporal regions.

 Direct comparisons between the two

hemispheres are needed to pinpoint the location of Chinese-specific processing

Study 1: Reading Chinese Characters

 Question: Which is Chinese-specific area in

the visual cortex?

 Hypothesis: Bilaterality or rightward laterality

• ccurs at the primary visual cortex, but whole

word processing is left lateralized.

 Rationale: The visual cortex is hierarchically

• rganized:

– Primary visual cortext (BA 17 and 18) visuospatial tasks visual features of words – Higher-level visual cortex (BA 19 and 37) object processing whole word recognition – Chinese is no longer a pictographic language.

Xue, Dong, Chen, Jin, Chen, Zeng, Reiman, 2005, Cog. Brain Res.

Hierarchical organization of visual cortex

Source: Riesenhuber & Piggo, 1999

Tasks

a “black [hei(1)]” and “white [bai(2)]” for the semantic task b “ticket [piao(4)] “ and “jump [tiao(4)]” for the phonological task.

Semantic Task vs. Fixation

Phonological Task vs. Fixation

Voxel-wise comparison

Original Flipped Subtraction

Brain Asymmetry: Semantic Tasks

Brain Asymmetry: Phonological Task

Summary of Study 1

 Our hypothesis was confirmed: significant

leftward asymmetry in the fusiform region (like alphabetical languages) but bilaterality in the primary visual cortex (unlike alphabetical languages) for Chinese processing.

 The right hemisphere appears to be important

for the initial visual processing of the complex spatial features of Chinese characters.

Study 2: Chinese Tone processing

 Question: Where in the brain does Chinese

tone processing (suprasegmental) vary from vowel processing (segmental)?

 Hypothesis: Right hemisphere, possibly areas

in the inferior frontal gyrus

 Rationale: Tone processing resembles music

processing, which is right lateralized.

 Previous research showed mixed results,

perhaps due to single task and lack of direct comparisons between the two hemispheres.

Liu, Peng, Ding, Jin, Zhang, Li, Chen., 2005, NeuroImage

We used an adaptation paradigm (i.e., keeping the tone constant but varying the vowel, or keeping the vowel constant but varying the tone) and two tasks (pinyin and Chinese characters) to study the neural overlap or dissociation between tones and vowels.

Vowels Tones

Clear leftward laterality, but…

Differences between tones and vowels

Crosshair marks right inferior frontal gyrus.

Conclusion of Study 2

 Left dominance in both vowel and tone

processing.

 Neural dissociation in tones and vowels for

Chinese, suggesting special neural bases (especially right IFG)for tonal language processing (also see Gandour et al. Wang et al.).

Summarizing Studies 1 and 2 and studies by others

 In general leftward lateralization for Chinese

processing (both visual words and sounds).

 Chinese tones and initial visual processing

showed right-hemisphere involvement (IFG and BA 17 and 18)

 Thus, in all there are three main differences

between neural bases of Chinese and English processing: Right primary visual cortex, right inferior frontal gyrus, and left dorsal lateral frontal region (see next slide)

Based on meta images From: Bolger et al. (2005). Cross-cultural effect on the brain revisited. Hum. Brain Mapp

Green circles: occipitotemporal boundary: All languages [VWFA] Blue circles: dorsal inferior frontal area: all languages [motor/speech] Yellow circles: ventral inferior frontal region (more lateral for Japanese): all languages [speech] Red circles: superior posterior temporal and inferior parietal region: English and Kana, but not Chinese and Kanji [graphophoneme conversion] Pink circles: Dorsal lateral frontal region: Chinese (and Kanji) [addressed phonology, not assembled phonology] Brown circle: Right primary visual cortex and right IFG: Chinese

rOTB

Summary data

rIFG

III: Right Brain and Second- Language Processing

 There is much evidence that the right

brain is involved in second language processing, especially at the early stages (e.g., nonfluent bilinguals).

 But why is it important? Compensation?

Study 3: Second language processing

 Working memory task:

Xue, Dong, Jin, Chen, 2004, NeuroImage

Red: Chinese Green: English

Conclusions: Study 3

 Similar working memory systems for native

and nonfluent second languages

 Greater activation in both hemispheres, but

especially the right hemisphere for the nonfluent second language.

 Suggesting compensation?

– Maybe, but with fluency confounded with L1 and L2, it is not clear whether it is compensation or language differences (L2 in the right hemisphere). A training paradigm is needed.

 Other cognitive functions such as those

involved in inhibiting L1? (especially the prefrontal areas for executive functions?)

Study 4: Language switching

 ER design.  Naming drawings of objects.  Conditions:

– Switching: ChineseEng; EngChinese – Non-switching: Chi Chi; Eng  Eng.

Wang, Xue, Chen, Dong, Xue. in press, NeuroImage.

Brain Region BA Coordinates z p x y z Frontal _Mid_R BA10 36 59 5 3.47 0.000 Frontal _Mid_R BA 46 39 50 9 3.57 0.000 Frontal _Mid_Orb_R BA 46 45 52 3.01 0.001 Frontal _Mid_Orb_R BA 11 3 43

• 12

3.45 0.000 Cingulum_Ant_R BA 32 6 50 17 3.19 0.001 Occipital_Sup_R BA 18 18

• 89

27 3.36 0.000 Precentral _R BA 6 39 2 50 3.97 0.000 Frontal _Sup_Medial_L 29 46 3.72 0.000 Parietal_Sup_L BA 2

• 33
• 47

60 3.12 0.001 supraMarginal _ L BA 40

• 62
• 36

35 3.15 0.001 Angular_L BA 39

• 42
• 56

39 3.09 0.001 Temporal_Mid_L BA 21

• 62
• 29

1 3.71 0.000

Brain regions activated by forward switching (Chinese English) relative to backward switching (English  Chinese).

Brain Region BA Coordinates z p x y z Occipital_ Inf_ R BA19 33

• 85
• 3

4.17 0.000 Lingual_R BA17 3

• 73
• 4

3.46 0.000 Cerebellum_L BA37

• 24
• 48
• 23

4.21 0.000 Cerebellum_L BA19

• 30
• 62
• 20

3.24 0.001 Fusiform_L BA37

• 36
• 59
• 10

3.27 0.001 Precentral_L BA6

• 33

5 36 3.29 0.001

Brain regions activated by backward switching relative to forward switching.

Study 4: Conclusions

 When switching between Chinese and

English, executive control areas are especially important. Much right hemisphere action is due to that reason. Perhaps that explains to some extent the role of right hemisphere for second language processing

 This study also confirmed some language-

specific areas: Right occipital for switching to Chinese reading and left temporo-parietal region for English (assembled phonology)

IV: Brain asymmetry and learning an artificial language

 To study the effects of language experience

• n the neural bases of language processing--

neural plasticity, especially the right hemisphere.

 More specifically, to examine whether the role

• f the right hemisphere is compensatory

 Finally, to explore whether the left-right shift

could be neurofunctional predictors of language learning efficiency

Logographic artificial language (LAL) :

Korean Hangul characters (in different fonts) Mismatched sounds (those different from Chinese and English) Arbitrary meanings (Chinese translation, picture)

Training

Study 5: Fusiform and visual word learning

Xue, Chen, Jin, Dong, 2006a. J. Cog. Neuro. Xue, Chen, Jin, Dong, 2006b. NeuroImage

Pre-Training Results: Fusiform Area

Asymmetry Index: Chinese: .55 Korean: .21

(-1 = 100% right lateralization; 0 = bilaterality 1 = 100% left lateralization)

Preliminary conclusions: Evidence of compensation (both right and left hemisphere) Second-language is more bilateral However, two challenges to these preliminary conclusions…..

Challenge #1

 If the right hemisphere plays only a

compensatory role, post-training results should no longer show activation in the right hemisphere (provided that training is effective behaviorally as well as neurally).

Training Effects: Behavioral Results

70% 75% 80% 85% 90% 95% 100%

Before After

Correct Ratio

Chinese Korean

400 450 500 550 600 650 700 750 800

Before After Reaction time (ms)

Chinese Korean

Training effects at the neural level

It appears that the right hemisphere may not simply play a compensatory role when processing a new language. Why bilateral then?

Challenge #2: Individual Results

It appears that bilaterality was

• nly at the group
• level. Most

individuals were not bilateral.

Furthermore, individual differences in AI were stable across training sessions.

r =. 778; p < .005 If right hemisphere had played a compensatory role, training should have resulted in leftward laterality.

Leftward lateralization facilitated learning

Pretraining AI Post-training AI Post-training performance Also: A caution against interpreting correlation between brain activation and learning as evidence of brain plasticity.

Regression Analysis: Predicting post-training performance

Note: RT = reaction time, AI = asymmetry index, R2 = .75 Predictors B (s.e.) β t p Pre-training RT .12 (.12) .17 .97 .36 Pre-training AI

• 100.3 (22.4)
• .79
• 4.47

.002

Summary of Study 5

 The right hemisphere perhaps plays a

compensatory role (as does the left hemisphere) initially, but its sustained activation may have more to do with individual differences in learning a new language.

 Individual differences in the brain asymmetry

during the initial processing of second language predicted later learning. In this case, the use of the left fusiform seems to facilitate the learning

• f visual forms of words.

Studies 6 - 8: Does Sex Matter?

 Much evidence that in language

processing males show more left laterality, whereas females show more bilaterality (e.g., Hsiao & Shillcock, 2006; Kansaku et

al., 2000; Shaywitz et al., 1995).

 Does that affect the learning of a new

language?

Chen, Xue, Dong, Jin, Li, Xue, Zhao, Guo, 2006. Neuropsychologia

• 1
• 0.8
• 0.6
• 0.4
• 0.2

D6 D7 D8 D9 D10 !"##$%&'(")*+"$,,(+($)'- . /

Male Female

Similarly, in phonological training:

He, Li, Xue, Chen, Dong, under review Mei, Chen, Xue, et al. in preparation

Individual differences in left middle temporal gyrus predicted listening comprehension

Six months later

Dong, Mei, Xue, Chen, under review

• V. Overall Conclusions and

implications

 The processing of Chinese shows

mainly left-hemispheric dominance, just as that of alphabetic languages. Two regions of the right hemisphere play a special role in Chinese processing: the primary visual cortex for visual words and the inferior frontal gyrus for Chinese tones.

Overall Conclusions cont’d

 The sustained right-hemisphere involvement

in second-language processing may have less to do with compensation, more to do with language switching and with individual tendencies to use one side of the brain when learning a new language.

 Native-language-tuned brain facilitates the

acquisition of a new language. Because of sex differences in native language tuning (left laterality for males, and bilaterality for females), brain asymmetry differentially predicted learning for males and females.

Thank you for your comments & suggestions