Computational modeling of reading Danie niela Rotelli lli - - PowerPoint PPT Presentation

▶
computational modeling of reading
SMART_READER_LITE
LIVE PREVIEW

Computational modeling of reading Danie niela Rotelli lli - - PowerPoint PPT Presentation

May 19, 2023 562 likes •952 views

Computational modeling of reading Danie niela Rotelli lli Computer Science PhD Student Universit di Pisa Outline How do we read? The dyslexic reader The project 7 How do we read? Visual cortex The human lobes 9 Fovea The

slide-1
SLIDE 1

Computational modeling of reading

Danie niela Rotelli lli Computer Science PhD Student Università di Pisa

slide-2
SLIDE 2
slide-3
SLIDE 3
slide-4
SLIDE 4
slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7

Outline

  • How do we read?
  • The dyslexic reader
  • The project

7

slide-8
SLIDE 8

How do we read?

slide-9
SLIDE 9

9

The human lobes Visual cortex

slide-10
SLIDE 10

10

The visual cortex Fovea

slide-11
SLIDE 11

11

Written word processing

slide-12
SLIDE 12

12

Objects recognition system

Objects contours:

  • erased junctions
  • spared junctions
  • three-dimensional objects

Neuronal hierarchy

slide-13
SLIDE 13

13

The neuronal hierarchy

slide-14
SLIDE 14

Visual normalisation Ortographic units (letters, graphemes) Lexical access Ortographic lexicon Semantic lexicon Lexical conversion Written input

knight

[k] [n] [i] [g h t] [knight]

Spelling to sound conversion Phonologicallexicon Phonological units (phonemes, syllables) Spoken output

[ˈnaɪt] ['n] [a] [ɪ] [t]

Access to pronunciation Speech production

14

Word reading process

slide-15
SLIDE 15

The dyslexic reader

slide-16
SLIDE 16

16

The dyslexic brain Normal readers Dyslexic readers

slide-17
SLIDE 17

17

Reading and symmetry

slide-18
SLIDE 18

The project

slide-19
SLIDE 19

19

Goals BETTER TEACHING PRACTICES QUALITY OF MEDICAL SERVICES WAITING LISTS AND SCREENING OVERHEADS

slide-20
SLIDE 20

20

Background In Italy, about 21% of fifteen-year-old students have poor literacy skills.

Literacyskil ills ls:

"understanding, evaluating, using and engaging with written text to participate in society, achieve one’s goals and develop one’s knowledge and potential" In Italy, the proportion

  • f

adults performing at or below level 1 (out of 5) is 28%, much higher than the EU-17 average (16.4%). Level 1: "read a brief text on familiar topics and locate a single piece of specific information"

slide-21
SLIDE 21

1) Assessing ing readin ding ski kills lls

  • laborious and time-consuming
  • separate monitoring interlocked abilities:

Motivation ✓ word rendering​ ✓ linguistic comprehension

21

2) Data elic licit itatio ion, storage and nd ana naly lysis is

  • lack of adequatetechnologicalsupport for

natural reading in ecological context:

✓ limited ecological validity ✓ artificially-restrained settings

CLOSE /kləʊs/ /kləʊz/

slide-22
SLIDE 22

22

Key requirements NATURAL READING CONDITIONS READING TEXT NARRATIVES TEXT UNDERSTANDING

Reading ding assessment: ✓ efficient ✓ replicable and reliable ✓ customisable and flexible ✓ scalable ✓ ecological ✓ multi-dimensional ✓ non invasive

MINIMALLY CONSTRAINING DEVICES

slide-23
SLIDE 23

Re Reading Ef Efficiency Model

The proposed solution

23

ReadLet

"The ability to fully understand connected texts by minimising reading time"

slide-24
SLIDE 24
slide-25
SLIDE 25

25

The protocol - Step 1 Required information about the reader and the task being administered

slide-26
SLIDE 26

26

The protocol - Step 2 Read (either silently or aloud) while sliding the finger across the written text

slide-27
SLIDE 27

27

The protocol - Step 3 Multiple-answer questions ontext content

✓ retrieving the general content ✓ retrieving word meaning from context ✓ identifying specific information in the text ✓ retrieving information from syntactic structure ✓ forming mental representations

slide-28
SLIDE 28

28

Data acquisition

Time-alig ligne ned d recorded d data: ✓ voice recording ✓ finger sliding time ✓ time of reading ✓ time of question answering ✓ correct answers

slide-29
SLIDE 29

29

Data acquisition

slide-30
SLIDE 30

30

Data acquisition

slide-31
SLIDE 31

31

Preliminary testing ✓ touchscreen time-stamped data ✓ microphone audio data ✓ time-stamped data of questionnaire filling 400 children 200 Tuscany, Italy (Italian) 200 Ticino canton, Switzerland (Italian) 15 Fez, Morocco (French and Arabic) 50% F/M 3rd -5th grade level 1300 trials 1/3 listening (3 pages) + questionnaire (15 questions) 1/3 reading aloud 1/3 silent reading (3 pages) + questionnaire (15 questions) 15 stories and questionnaires 9 Italian 3 French 3 Arabic

slide-32
SLIDE 32

32

Finger touch processing

slide-33
SLIDE 33

CHILDIT2

Speech processing

33

≠

slide-34
SLIDE 34

34

The architecture

slide-35
SLIDE 35

People

slide-36
SLIDE 36

36

People

slide-37
SLIDE 37

Thank you

daniela.rotelli@phd.unipi.it

slide-38
SLIDE 38

References

  • Dehaene, S., (2009). Les neuronesde la lecture.
  • Dehaene, S., et al., (2010). How Learning to Read Changesthe Cortical Networks for Vision and

Language.

  • Ferro, M., Pirrelli, V., et al. , (2010). Reading as active sensing: a computational model of gaze

planning in word recognition.

  • Ferro, M., Marzi, C., Pirrelli, V., et al., (2018). ReadLet: Reading for Understanding.
  • Gage, N., Baars, B. (2018). Fundamentals of cognitive neuroscience.
  • Comphys Lab - PISA ILC-CNR, (2019). PresentazioneComputer modeling of reading efficiency.
  • Images: Pixabay (kerttu, Tumisu, MabelAmber, VictoriaBorodinova ), Flickr.com (Pete Ashton -

Umberto Boccioni - States of Mind I The Farewells)

38