Rethinking HCI with Neural Interfaces @CTRLlabsco Adam Berenzweig, - - PowerPoint PPT Presentation

▶
rethinking hci with neural interfaces
SMART_READER_LITE
LIVE PREVIEW

Rethinking HCI with Neural Interfaces @CTRLlabsco Adam Berenzweig, - - PowerPoint PPT Presentation

Aug 20, 2023 373 likes •814 views

Rethinking HCI with Neural Interfaces @CTRLlabsco Adam Berenzweig, Director of R&D, CTRL-labs @madadam | adam@ctrl-labs.com 1. A Brief History of User Interface Paradigm Shifts 2. Introduction to Neural Interfaces 3. Designing for Neural

slide-1
SLIDE 1

Rethinking HCI with Neural Interfaces

@CTRLlabsco

Adam Berenzweig, Director of R&D, CTRL-labs @madadam | adam@ctrl-labs.com

slide-2
SLIDE 2
slide-3
SLIDE 3
slide-4
SLIDE 4
  • 1. A Brief History of User Interface Paradigm Shifts
  • 2. Introduction to Neural Interfaces
  • 3. Designing for Neural Interfaces
slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8
slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13
slide-14
SLIDE 14
  • 1. A Brief History of User Interface Paradigm Shifts
  • 2. Introduction to Neural Interfaces
  • 3. Designing for Neural Interfaces
slide-15
SLIDE 15

Human

  • utput

30 bits/s Human input 10,000,000 bits/s

slide-16
SLIDE 16
slide-17
SLIDE 17

Cortical BMI is hard

slide-18
SLIDE 18
slide-19
SLIDE 19

Surface EMG neural interface.

Biophysical signal to control signal.

Neural activity EMG Control

slide-20
SLIDE 20

In the future you will connect your nervous system to everything.

slide-21
SLIDE 21
slide-22
SLIDE 22
  • 1. A Brief History of User Interface Paradigm Shifts
  • 2. Introduction to Neural Interfaces
  • 3. Designing for Neural Interfaces
slide-23
SLIDE 23

Neural Interaction Design Challenges 1. Robustness & Reliability 2. Novel Interactions 3. Ergonomics a. Touchless Control vs. Spatial Computing 4. Learnability & Customization a. Fun to learn, and it learns you. 5. Fit to Environment 6. Props

slide-24
SLIDE 24
slide-25
SLIDE 25

Neural Interaction Design Challenges 1. 3- and 6-DOF navigation 2. Activation, “wake-word” & mode switching 3. Simultaneous continuous & discrete control 4. Hand-object interaction in XR. 5. Text Input!!

slide-26
SLIDE 26

Ergonomics & Comfort 1. Smaller is better -- motionless is best? 2. Anatomically informed 3. Variation is essential for comfort. 4. Force: brief or light.

slide-27
SLIDE 27
slide-28
SLIDE 28

Hand-Object Interaction in XR 1. Grasping, manipulating. 2. Spatial Computing vs. Touchless Control, i.e. direct vs indirect. 3. Haptics & feedback. Use your arm, or use The Force?

slide-29
SLIDE 29

Novel Interactions Co-evolution of software and controllers. Control these with an xbox controller… or your hand?

slide-30
SLIDE 30

Novel Interactions Co-evolution of software and controllers. Control these with an xbox controller… or your hand?

slide-31
SLIDE 31

Text Input!! 1. One-handed vs two 2. QWERTY-or-not 3. Visual, audio feedback 4. Swipe, chords, …? We can do better than this...

slide-32
SLIDE 32

Props 1. Constrain hand shape & muscle activation 2. Something to push against 3. Feels cool 4. Optional scaffolding?

slide-33
SLIDE 33
slide-34
SLIDE 34

https://www.reddit.com/r/gaming/comments/84wjsk/inverted_mouse/

slide-35
SLIDE 35
slide-36
SLIDE 36
slide-37
SLIDE 37
slide-38
SLIDE 38

2019 2020

…

slide-39
SLIDE 39

Hardware SDK & API

CTRL-kit developer platform.

@CTRLlabsco ctrl-labs.com

slide-40
SLIDE 40

Questions?

@CTRLlabsco @madadam ctrl-labs.com adam@ctrl-labs.com

slide-41
SLIDE 41

[Extra slides]

slide-42
SLIDE 42

Neurocontrol: EMG → motor neuron activity

slide-43
SLIDE 43

Tool / object control Spoken language

The richest signals are at the forearm.