Neurological Divide: An fMRI Study of Prose and Code Writing Ryan - - PowerPoint PPT Presentation

▶
neurological divide an fmri study of prose and code
SMART_READER_LITE
LIVE PREVIEW

Neurological Divide: An fMRI Study of Prose and Code Writing Ryan - - PowerPoint PPT Presentation

Feb 04, 2024 344 likes •736 views

Neurological Divide: An fMRI Study of Prose and Code Writing Ryan Krueger 1 , Yu Huang 1 , Xinyu Liu 2 , Tyler Santander 3 , Westley Weimer 1 , Kevin Leach 1 1 University of Michigan 2 Georgia Institute of Technology 3 University of California,

slide-1
SLIDE 1

Neurological Divide: An fMRI Study of Prose and Code Writing

Ryan Krueger1, Yu Huang1, Xinyu Liu2, Tyler Santander3, Westley Weimer1, Kevin Leach1

1University of Michigan

2Georgia Institute of Technology 3University of California, Santa Barbara

July 10, 2020

slide-2
SLIDE 2

Yu Huang @ ICSE2020

Thank You to the Collaborators!

2

  • Dr. Westley Weimer

Yu Huang is going on the Job Market this year! yhhy@umich.edu

Ryan Krueger Xinyu Liu

  • Dr. Kevin Leach
  • Dr. Tyler Santander
slide-3
SLIDE 3

Yu Huang @ ICSE2020

Motivation

3

Dijkstra might be right. However, readers may take it in a different way and become really concerned…

slide-4
SLIDE 4

Yu Huang @ ICSE2020

Motivation

4

slide-5
SLIDE 5

Yu Huang @ ICSE2020

Motivation

5

Suggested Answer:

slide-6
SLIDE 6

Yu Huang @ ICSE2020

Motivation

  • Objectively understanding the subjective cognitive process
  • Medical imaging: fMRI

6

slide-7
SLIDE 7

Yu Huang @ ICSE2020

Motivation

  • Objectively understanding the subjective cognitive process
  • Medical imaging: fMRI

7

PROSE READING

NeuroImg’16 NeuroImg’12 NeuroImg’15 Neurology’01 NeuroImg’06 JCogSci’10

slide-8
SLIDE 8

Yu Huang @ ICSE2020

Motivation

  • Objectively understanding the subjective cognitive process
  • Medical imaging: fMRI

8

CODE PROSE READING

BrainImg’18 ISSRE’16 ICPC’18 ICSE’17 FSE’17 ICSE’14 NeuroImg’16 NeuroImg’12 NeuroImg’15 Neurology’01 NeuroImg’06 JCogSci’10

slide-9
SLIDE 9

Yu Huang @ ICSE2020

Motivation

  • Objectively understanding the subjective cognitive process
  • Medical imaging: fMRI

9

CODE PROSE READING WRITING

BrainImg’18 ISSRE’16 ICPC’18 ICSE’17 FSE’17 ICSE’14 JWR’2008 WrComm’2000 LangLearn’1989 TESOL’1992 CogBrR’2001 HBM’2013 NeuroImg’2006 NeuroImg’16 NeuroImg’12 NeuroImg’15 Neurology’01 NeuroImg’06 JCogSci’10

slide-10
SLIDE 10

Yu Huang @ ICSE2020

Motivation

  • Objectively understanding the subjective cognitive process
  • Medical imaging: fMRI

10

CODE PROSE READING WRITING

BrainImg’18 ISSRE’16 ICPC’18 ICSE’17 FSE’17 ICSE’14 JWR’2008 WrComm’2000 LangLearn’1989 TESOL’1992 CogBrR’2001 HBM’2013 NeuroImg’2006 NeuroImg’16 NeuroImg’12 NeuroImg’15 Neurology’01 NeuroImg’06 JCogSci’10

slide-11
SLIDE 11

Yu Huang @ ICSE2020

High-level Question

11

  • Are code writing and prose writing similar neural

activities? Is being good at writing associated with being a good software developer?

slide-12
SLIDE 12

Yu Huang @ ICSE2020

Outline

  • Motivation
  • High-level question
  • Challenges
  • Experimental design
  • Results
  • Conclusions

12

slide-13
SLIDE 13

Yu Huang @ ICSE2020

13

  • Physics
  • Magnetic interference

Challenges

slide-14
SLIDE 14

Yu Huang @ ICSE2020

14

  • Physics
  • Magnetic interference
  • Solution

○ Employ an fMRI-safe bespoke keyboard

Challenges

slide-15
SLIDE 15

Yu Huang @ ICSE2020

15

  • Physics
  • Magnetic interference
  • Solution

○ Employ an fMRI-safe bespoke keyboard

Challenges

slide-16
SLIDE 16

Yu Huang @ ICSE2020

16

  • Physics
  • Design
  • Contrast setup
  • Solution:

○ Two-by-two contrast task design

Challenges

slide-17
SLIDE 17

Yu Huang @ ICSE2020

17

  • Two-by-two contrast task design

○ Code writing vs. Prose writing ○ Fill in the blank (FITB) vs. Long response (LR)

Experimental Design

slide-18
SLIDE 18

Yu Huang @ ICSE2020

18

  • Two-by-two contrast task design

Experimental Design

CODE PROSE FITB LR

slide-19
SLIDE 19

Yu Huang @ ICSE2020

19

  • Two-by-two contrast task design

Experimental Design

CODE PROSE FITB LR

Low-level

slide-20
SLIDE 20

Yu Huang @ ICSE2020

20

  • Two-by-two contrast task design

Experimental Design

CODE PROSE FITB LR

Low-level High-level

slide-21
SLIDE 21

Yu Huang @ ICSE2020

21

  • Two-by-two contrast task design

Experimental Design

CODE PROSE FITB LR

Low-level High-level

slide-22
SLIDE 22

Yu Huang @ ICSE2020

22

  • Two-by-two contrast task design

○ Code writing vs. Prose writing ○ Fill in the blank (FITB) vs. Long response (LR)

  • Source

○ Code: Turing’s Craft

Experimental Design

slide-23
SLIDE 23

Yu Huang @ ICSE2020

23

  • Two-by-two contrast task design

○ Code writing vs. Prose writing ○ Fill in the blank (FITB) vs. Long response (LR)

  • Source

○ Code: Turing’s Craft ○ Prose: Scholastic Assessment Test (SAT)

Experimental Design

slide-24
SLIDE 24

Yu Huang @ ICSE2020

24

  • Recruitment

○ 30 participants

  • 20 male vs. 10 female
  • 27 undergraduate vs. 3 graduate
  • Tasks

○ Four randomized blocks

  • Code FITB: 17
  • Code LR: 9
  • Prose FITB: 17
  • Prose LR: 9

Results

slide-25
SLIDE 25

Yu Huang @ ICSE2020

25

  • Data analysis: we need to be careful

○ Spurious correlation or false discovery from multiple comparisons

Results

slide-26
SLIDE 26

Yu Huang @ ICSE2020

26

  • Data analysis: we need to be careful

○ Spurious correlation or false discovery from multiple comparisons ○ Three steps

Results

slide-27
SLIDE 27

Yu Huang @ ICSE2020

27

  • Data analysis: we need to be careful

○ Spurious correlation or false discovery from multiple comparisons ○ Three steps

Results

Preprocessing

slide-28
SLIDE 28

Yu Huang @ ICSE2020

28

  • Data analysis: we need to be careful

○ Spurious correlation or false discovery from multiple comparisons ○ Three steps

Results

Preprocessing First-level Analysis

slide-29
SLIDE 29

Yu Huang @ ICSE2020

29

  • Data analysis: we need to be careful

○ Spurious correlation or false discovery from multiple comparisons ○ Three steps

Results

Preprocessing First-level Analysis Contrast & Group-level analysis

slide-30
SLIDE 30

Yu Huang @ ICSE2020

30

  • RQ1: Do self reports claim code writing is like prose writing?
  • RQ2: Does the brain treat code writing like prose writing?
  • RQ3: What low-level features explain code and prose writing?
  • RQ4: What high-level features explain code and prose writing?

Results

slide-31
SLIDE 31

Yu Huang @ ICSE2020

31

  • RQ1: Do self reports claim code writing is like prose

writing? ○ 38.5% reported similarity between prose and code writing

Results

slide-32
SLIDE 32

Yu Huang @ ICSE2020

32

  • RQ2: Does the brain treat code writing like prose writing?

Results

slide-33
SLIDE 33

Yu Huang @ ICSE2020

33

  • RQ2: Does the brain treat code writing like prose writing?

○ Significant and widely-distributed difference in neural activity

  • More than 10 brain regions (Broadmann Areas)

Results

Code > Prose

slide-34
SLIDE 34

Yu Huang @ ICSE2020

34

  • RQ3: What low-level features explain code and prose writing?
  • RQ4: What high-level features explain code and prose writing?

Results

slide-35
SLIDE 35

Yu Huang @ ICSE2020

35

  • RQ3: What low-level features explain code and prose writing?

○

Low-level: code writing requires more in parts of the brain associated with top-down control, planning, and categorization

  • RQ4: What high-level features explain code and prose writing?

Results

Code FITB > Prose FITB

slide-36
SLIDE 36

Yu Huang @ ICSE2020

36

  • RQ3: What low-level features explain code and prose writing?

○

Low-level: code writing requires more in parts of the brain associated with top-down control, planning, and categorization

  • RQ4: What high-level features explain code and prose writing?

○ High-level: prose writing requires more in parts of the brain associated with language; code writing involves more in attention, memory, planing, and spatial ability.

Results

Code FITB > Prose FITB Code LR > Prose LR

slide-37
SLIDE 37

Yu Huang @ ICSE2020

Summary

  • First fMRI study of code writing
  • Bespoke fMRI-safe QWERTY keyboard
  • Controlled, contrast-based experiment
  • Main result: All analysis of all code writing tasks against prose writing tasks

showed distinct neural mechanisms

  • At a more granular level:
  • Code FITB > Prose FITB: top-down control, planning, categorization
  • Code LR > Prose LR: code involves more of the right hemisphere (spatial ability, planning)

prose involves more canonical left hemisphere (language production)

  • Discussion
  • Pedagogy; Workforce retraining; Encouraging more diverse participation in computer science

37