The Browser and the Brain Jenna Zeigen NationJS December 1, 2017 - - PowerPoint PPT Presentation

▶
the browser and the brain
SMART_READER_LITE
LIVE PREVIEW

The Browser and the Brain Jenna Zeigen NationJS December 1, 2017 - - PowerPoint PPT Presentation

Mar 26, 2024 289 likes •928 views

The Browser and the Brain Jenna Zeigen NationJS December 1, 2017 Jenna Zeigen @zeigenvector zeigenvector jenna.is/at-nationjs Jenna Zeigen @zeigenvector Jenna Zeigen @zeigenvector Gee, Brain. What is Jenna going to do in

slide-1
SLIDE 1

Jenna Zeigen • @zeigenvector

The Browser and the Brain

Jenna Zeigen • NationJS • December 1, 2017

slide-2
SLIDE 2

Jenna Zeigen • @zeigenvector

zeigenvector jenna.is/at-nationjs

slide-3
SLIDE 3

Jenna Zeigen • @zeigenvector

slide-4
SLIDE 4

Jenna Zeigen • @zeigenvector

Gee, Brain. What is Jenna going to do in her talk today? The same thing she does every time, Browser! Talk about cognition and computers!

slide-5
SLIDE 5

Jenna Zeigen • @zeigenvector

  • 1. Understanding
  • 2. Visualizing
  • 3. Task Management
slide-6
SLIDE 6

Jenna Zeigen • @zeigenvector

Browser, are you processing the same way I’m processing? I don’t think so, Brain, but let’s dig into it!

slide-7
SLIDE 7

Jenna Zeigen • @zeigenvector

We made computers so we know all the

  • answers. We do science on humans to

find out the answers.

slide-8
SLIDE 8

Jenna Zeigen • @zeigenvector

Understanding

slide-9
SLIDE 9

Jenna Zeigen • @zeigenvector

How does the browser process HTML, CSS, and JavaScript, and how does the human mind process natural languages?

slide-10
SLIDE 10

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

Most programming languages have a vocabulary described using regular expressions and a syntax described by a context-free grammar

https://www.html5rocks.com/en/tutorials/internals/howbrowserswork https://en.wikipedia.org/wiki/Context-free_grammar

slide-11
SLIDE 11

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

A parser’s job is to take a document and break it into a structure the code can use

https://www.html5rocks.com/en/tutorials/internals/howbrowserswork

slide-12
SLIDE 12

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

Parsing can be separated into two parts — lexical and syntactic analysis— which are performed by a lexer and parser, respectively

https://www.html5rocks.com/en/tutorials/internals/howbrowserswork

slide-13
SLIDE 13

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

HTML isn’t a context-free language and therefore can’t be parsed by a regular parser

https://www.html5rocks.com/en/tutorials/internals/howbrowserswork

slide-14
SLIDE 14

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

CSS is a context-free language and therefore easier to parse.

https://www.html5rocks.com/en/tutorials/internals/howbrowserswork

slide-15
SLIDE 15

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

Both HTML and CSS parsers end up creating a tree representing the language it parsed, the DOM and CSSOM trees

https://www.html5rocks.com/en/tutorials/internals/howbrowserswork

slide-16
SLIDE 16

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

JavaScript is also context-free and can use a regular parser, but browsers complicate things in order to optimize

https://www.youtube.com/watch?v=Fg7niTmNNLg http://www.ecma-international.org/ecma-262/#sec-notational-conventions

slide-17
SLIDE 17

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

V8 uses two parsers—eager and lazy — to eventually create an abstract syntax tree and scope structure

https://www.youtube.com/watch?v=Fg7niTmNNLg

slide-18
SLIDE 18

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

The AST and scope structures get turned into low-level code, which then gets executed

https://www.youtube.com/watch?v=Fg7niTmNNLg

slide-19
SLIDE 19

Jenna Zeigen • @zeigenvector

HTML, CSS, & JS

The bytecode also gets fed to the

  • ptimizing compiler which spits out

machine code that then gets executed

https://www.youtube.com/watch?v=p-iiEDtpy6I

slide-20
SLIDE 20

Jenna Zeigen • @zeigenvector

Natural Language

Human languages have a lexicon and syntax that cannot be described by a context-free grammar

slide-21
SLIDE 21

Jenna Zeigen • @zeigenvector

Natural Language

Humans language contains a ton of ambiguity

slide-22
SLIDE 22

Jenna Zeigen • @zeigenvector

Natural Language

Step 1: To understand speech, humans break the unbroken speech stream into words

slide-23
SLIDE 23

Jenna Zeigen • @zeigenvector

Natural Language

Step 2: Our minds match the sounds to words in the mental lexicon

slide-24
SLIDE 24

Jenna Zeigen • @zeigenvector

Natural Language

Step 3: Once we access a word, we have access to its meaning and its syntactic and thematic roles

slide-25
SLIDE 25

Jenna Zeigen • @zeigenvector

Natural Language

Step 4: We then parse the sentence. But we’re not 100% sure how…

slide-26
SLIDE 26

Jenna Zeigen • @zeigenvector

Natural Language

Modular View: the phases involved

  • ccur separately in different modules

https://en.wikipedia.org/wiki/Sentence_processing

slide-27
SLIDE 27

Jenna Zeigen • @zeigenvector

Natural Language

Interactive View: all available information can be used at once in parsing the sentence

https://en.wikipedia.org/wiki/Sentence_processing

slide-28
SLIDE 28

Jenna Zeigen • @zeigenvector

Natural Language

Does this happen in serial or in parallel?

https://en.wikipedia.org/wiki/Sentence_processing

slide-29
SLIDE 29

Jenna Zeigen • @zeigenvector

Natural Language

Humans are forgiving of syntax errors

slide-30
SLIDE 30

Jenna Zeigen • @zeigenvector

Natural Language

Receptive language processing occurs in the dominant hemisphere of the brain, in Wernicke’s area.

https://en.wikipedia.org/wiki/Language_processing_in_the_brain

slide-31
SLIDE 31

Jenna Zeigen • @zeigenvector

Visualizing

slide-32
SLIDE 32

Jenna Zeigen • @zeigenvector

How does the human visual system paint a picture of our world, and how does the browser render pixels to the screen?

slide-33
SLIDE 33

Jenna Zeigen • @zeigenvector

Human Vision

Step 1: Light goes into the eye via the cornea and lens

https://en.wikipedia.org/wiki/Visual_system

slide-34
SLIDE 34

Jenna Zeigen • @zeigenvector

Step 2: The retina turns the light into neural signals using rods and cones

https://en.wikipedia.org/wiki/Visual_system https://en.wikipedia.org/wiki/Flicker_fusion_threshold

Human Vision

slide-35
SLIDE 35

Jenna Zeigen • @zeigenvector

Human Vision

Step 3: The neural signals get sent via the optic nerve to the brain

https://en.wikipedia.org/wiki/Visual_system

slide-36
SLIDE 36

Jenna Zeigen • @zeigenvector

Human Vision

Step 4: Signals from both eyes reach the optic chiasm, are combined, split by visual field, and sent to the opposite side of the brain

https://en.wikipedia.org/wiki/Visual_system

slide-37
SLIDE 37

Jenna Zeigen • @zeigenvector

Human Vision

https://en.wikipedia.org/wiki/Visual_system https://commons.wikimedia.org/wiki/File:1204_Optic_Nerve_vs_Optic_Tract.jpg

slide-38
SLIDE 38

Jenna Zeigen • @zeigenvector

Human Vision

Step 5: Most signals get sent to the lateral geniculate nuclei

Blake, R., & Sekuler, R. (2006)

slide-39
SLIDE 39

Jenna Zeigen • @zeigenvector

Human Vision

Step 6: Signals then get sent to the primary visual cortex

Blake, R., & Sekuler, R. (2006)

slide-40
SLIDE 40

Jenna Zeigen • @zeigenvector

Human Vision

Step 7: Signals gets sent to higher visual processing centers that help us actually perceive what we are seeing

Blake, R., & Sekuler, R. (2006)

slide-41
SLIDE 41

Jenna Zeigen • @zeigenvector

Rendering

Step 1: HTML and CSS are parsed into DOM and CSSOM trees, respectively

https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction

slide-42
SLIDE 42

Jenna Zeigen • @zeigenvector

Rendering

Step 2: The DOM and CSSOM trees are combined to form the render tree

https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction

slide-43
SLIDE 43

Jenna Zeigen • @zeigenvector

Rendering

Step 3: The browser traverses the render tree, calculating the location and size of all elements

https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction

slide-44
SLIDE 44

Jenna Zeigen • @zeigenvector

Rendering

Step 4: The browser again traverses the render tree, creating bitmaps for each layer

https://www.youtube.com/watch?v=gqc88qWuiI4

slide-45
SLIDE 45

Jenna Zeigen • @zeigenvector

Rendering

Step 5: Bitmaps are sent to the GPU for compositing

https://www.youtube.com/watch?v=gqc88qWuiI4 https://www.html5rocks.com/en/tutorials/speed/layers/

slide-46
SLIDE 46

Jenna Zeigen • @zeigenvector

Rendering

Step 6: Do it all again, maybe 60 times a second

https://www.html5rocks.com/en/tutorials/speed/layers/

slide-47
SLIDE 47

Jenna Zeigen • @zeigenvector

Task Management

slide-48
SLIDE 48

Jenna Zeigen • @zeigenvector

Can the human mind multitask? Can the browser?

slide-49
SLIDE 49

Jenna Zeigen • @zeigenvector

Human Attention

  • attention as a filter
  • attention as a spotlight
  • attention as control
slide-50
SLIDE 50

Jenna Zeigen • @zeigenvector

Human Attention

BLUE PURPLE RED GREEN PURPLE GREEN

slide-51
SLIDE 51

Jenna Zeigen • @zeigenvector

Human Attention

slide-52
SLIDE 52

Jenna Zeigen • @zeigenvector

Human Attention

BLUE PURPLE RED GREEN PURPLE GREEN

slide-53
SLIDE 53

Jenna Zeigen • @zeigenvector

Human Attention

BLUE PURPLE RED GREEN PURPLE GREEN

slide-54
SLIDE 54

Jenna Zeigen • @zeigenvector

Human Attention

  • attention as a filter
  • attention as a spotlight
  • attention as control
slide-55
SLIDE 55

Jenna Zeigen • @zeigenvector

Human Attention

  • attention as a filter
  • attention as a spotlight
  • attention as control
  • attention as threads!
slide-56
SLIDE 56

Jenna Zeigen • @zeigenvector

Human Attention

Humans are pretty bad at multitasking:

  • inattentional blindness
  • dichotic listening task
  • shadowing

(Simons, 1999; Cherry, 1953; Triesman, 1964; Allport et al., 1972 )

slide-57
SLIDE 57

Jenna Zeigen • @zeigenvector

Human Attention

(Simons, 1999; Cherry, 1953; Triesman, 1964; Allport et al., 1972 )

“These are the words you aren’t supposed to be listening to.” “These are the words you need to repeat back.” “These are the words you need to repeat back.”

slide-58
SLIDE 58

Jenna Zeigen • @zeigenvector

“Words these are the aren’t supposed to you be to listening.”

Human Attention

“These are the words you need to repeat back.” “These are the words you need to repeat back.”

(Simons, 1999; Cherry, 1953; Triesman, 1964; Allport et al., 1972 )

slide-59
SLIDE 59

Jenna Zeigen • @zeigenvector

Browser “Attention”

JavaScript itself doesn’t multitask.

  • single-threaded
  • non-blocking
  • asynchronous
slide-60
SLIDE 60

Jenna Zeigen • @zeigenvector

Browser “Attention”

Within the browser, JavaScript can call

  • ut to APIs, making the browser itself

able to effectively multitask!

slide-61
SLIDE 61

Jenna Zeigen • @zeigenvector

Browser, are you processing the same way I’m processing? Guess not, Brain! But I hope you learned something!

slide-62
SLIDE 62

Jenna Zeigen • @zeigenvector

Resources

Books

  • Blake, R., & Sekuler, R. & (2006). Perception (5th ed.). Boston: McGraw-Hill.
  • Harley, T. A. (2008). Psychology of Language: From Data to Theory (3rd ed.). New York:

Psychology Press.

  • Kellogg, R. T. (2007). Fundamentals of cognitive psychology. Thousand Oaks, CA: SAGE.

Websites

  • https://www.html5rocks.com/en/tutorials/internals/howbrowserswork
  • https://www.html5rocks.com/en/tutorials/speed/layers/
  • https://developers.google.com/web/fundamentals/performance/critical-rendering-path/

render-tree-construction

  • Marja Hölttä: Parsing JavaScript - better lazy than eager? (Video)
  • Franziska Hinkelmann: JavaScript engines - how do they even? (Video)
  • Chelsea Derrick: True Grit: Debugging CSS & Render Performance (Video)
  • www.ecma-international.org/ecma-262/
  • https://en.wikipedia.org/wiki/Visual_system
  • https://en.wikipedia.org/wiki/Lateral_geniculate_nucleus
  • https://en.wikipedia.org/wiki/Language_processing_in_the_brain
  • https://en.wikipedia.org/wiki/Sentence_processing
  • http://www.imdb.com/title/tt0112123/quotes
slide-63
SLIDE 63

Jenna Zeigen • @zeigenvector

jenna.is/at-nationjs @zeigenvector

Thanks!