VRPS YC: Lab VRPS YC: Lab VRPS YC: Lab VR/ Games to Motivate - - PDF document

▶
vrps yc lab vrps yc lab vrps yc lab
SMART_READER_LITE
LIVE PREVIEW

VRPS YC: Lab VRPS YC: Lab VRPS YC: Lab VR/ Games to Motivate - - PDF document

Aug 15, 2023 217 likes •637 views

Virtual Iraq: Virtual Iraq: Virtual Reality Cognitive Performance Assessment Test Virtual Reality Cognitive Performance Assessment Test Thomas D. Parsons, PhD Research Scientist, Neuropsychologist Assistant Research Professor Co-director of

slide-1
SLIDE 1

Virtual Iraq: Virtual Iraq:

Virtual Reality Cognitive Performance Assessment Test Virtual Reality Cognitive Performance Assessment Test

Thomas D. Parsons, PhD

Research Scientist, Neuropsychologist Assistant Research Professor Co-director of VRPSYCH Lab

Institute for Creative Technologies University of Southern California tparsons@usc.edu

2

VR/ Games to Motivate Motor Rehabilitation VR/ Games to Motivate Motor Rehabilitation

VR Classroom VR Classroom

Funded by Funded by

Virtual Iraq Virtual Iraq

Virtual Patients Virtual Patients Virtual Patients

VRPS YC: Lab

Virtual Reality & S

  • cial Neuroscience

VRPS YC: Lab VRPS YC: Lab

Virtual Reality & S

  • cial Neuroscience

Virtual Reality & S

  • cial Neuroscience

Virtual Classroom Virtual Classroom Virtual Classroom

slide-2
SLIDE 2

3

VRPSYCH Lab: Collaborators VRPSYCH Lab: Collaborators

  • Institute for Creative Technologies

– – Rizzo, Pair, Gratch, Marsala, Hill, Swartout, Morie Rizzo, Pair, Gratch, Marsala, Hill, Swartout, Morie

  • Brain and Creativity Institute

– – Antonio and Hanna Damasio Antonio and Hanna Damasio

  • Child Psychiatry

– Pato, Pataki, Sugar

  • Integrated Media Systems Center

– – Sawchuck, Yeh Sawchuck, Yeh

  • Annenberg School for Communication/Journalism

– – McLaughlin McLaughlin

  • School of Biokinesiology and Physical Therapy

– – Winstein, Flynn Winstein, Flynn

  • Department of Psychology

– – Davison, Arbib Davison, Arbib

  • Children’s Hospital Los Angeles

– – Gold, Katz Gold, Katz

  • School of Gerontology, Neurology

– – Zilenski, Chui Zilenski, Chui

4

VRPSYCH Lab: Collaborators VRPSYCH Lab: Collaborators

  • Center for Study of Human Operator Performance
  • University of Minnesota
  • UCSF Dynamic Multimodal Imaging Lab
  • UCSD & VR Medical Center
  • Yale University
  • Kessler Medical Rehab Center
  • Virtually Better
  • LA Children’s Hospital
  • St. John’s University
  • North Carolina A&T State University
  • Utah Neurodevelopment Center
  • Columbia University
  • Emory University
slide-3
SLIDE 3

5

ARL Neuroscience STI ARL Neuroscience STI

Other Academia

Integrate State-of-the-Art in Neuroscience Advance State-of-the-Art in Neuroscience

6

Social Neuroscience: Integrated Interdisciplinary Focus Emotion Modelling & Regulation

  • Social psychological topics
  • Attitude change
  • Stereotyping

Neuroscience methods

  • Neuropsychological Assessment
  • Psychophysiological
  • Neuroimaging

VRPSYCH Lab VRPSYCH Lab

slide-4
SLIDE 4

7

Social Neuroscience: VRPSYCH Lab Projects Emotion Models & Regulation

  • Virtual Standardized Patients
  • Justin/Justina
  • Startle eye-blink and bias
  • Panoramic (Anger Management)
  • VR Exposure Therapy

Neuroscience methods

  • Visuospatial Processes
  • VRClassroom/Office/Apartment
  • VEGS
  • VRCPAT (NP Assess)
  • Psychophys Correlates

8

The S tudy of Brain/ Behavior Relationships The S tudy of Brain/ Behavior Relationships

VRPSYCH Lab VRPSYCH Lab

slide-5
SLIDE 5

9

Scent Scent System System

  • Gunpowder
  • Cordite
  • Body Odor
  • Garbage
  • Burning

Rubber

  • Diesel Fuel
  • Iraqi Spices

Night Vision Night Vision HMD Rig HMD Rig… … Bass S haker Bass S haker Platform Platform

Multisensory Multisensory Simulations Simulations Physiology Physiology Assessment Assessment Neuropsyc Neuropsyc Assessment Assessment

S tandard Tests S tandard Tests VR Tests VR Tests

Stimulus Stimulus Delivery Delivery State State Trait Trait

Biopac S ystem Biopac S ystem

10

  • Standardized tests used in a neuropsychological evaluation

typically assess functioning in the following areas:

– Attention

  • Learning and Memory

– Executive functions

  • Visual-spatial functions

– Language functions

  • Sensory-Perceptual functions

– Processing Speed

  • Motor functions
  • Academic skill development and emotional functioning are

typically assessed as well.

Neuropsychological Assessment Neuropsychological Assessment

slide-6
SLIDE 6

11

  • Advantages:

– Cognitive functioning

  • Broad Spectrum of brain

functioning

– Standardized procedures

  • Psychometrically sound
  • Reliable scoring methods

Neuropsychological Assessment Neuropsychological Assessment

12

  • Disadvantages:

–Outmoded Tests

  • using methods developed

60-100 years ago!

–Ecological Validity

  • Relevance to real world

functioning

Neuropsychological Assessment Neuropsychological Assessment

slide-7
SLIDE 7

13

Binet (1905) Binet (1905) Drawing a Design from Memory Drawing a Design from Memory

Neuropsychological Tests (1905) Neuropsychological Tests (1905)

14

Wechsler Memory Scale: Wechsler Memory Scale: Visual Reproduction Subtest Visual Reproduction Subtest

Neuropsychological Tests (Now) Neuropsychological Tests (Now)

slide-8
SLIDE 8

15

The Manikin Test: Developed by Pitner- Patterson, 1917

Neuropsychological Tests (1917) Neuropsychological Tests (1917)

16

Now the “Object Assembly” Subtest on the WAIS, WISC

Neuropsychological Tests (Now) Neuropsychological Tests (Now)

slide-9
SLIDE 9

17

Kohs Blocks (1915)

Neuropsychological Tests (1915) Neuropsychological Tests (1915)

18

Now the “Block Design” Subtest on the WAIS, WISC, etc.

Neuropsychological Tests (Now) Neuropsychological Tests (Now)

slide-10
SLIDE 10

19

Are we st ill limit ed t o using met hods developed 60-100 years ago!

Neuropsychological Tests Neuropsychological Tests

20

Manikin Test Mathematical Processing Tower Puzzle

Computerized Neuropsych Tests Computerized Neuropsych Tests

ANAM

slide-11
SLIDE 11

21

Manikin Test Mathematical Processing Tower Puzzle

Computerized Neuropsych Tests Computerized Neuropsych Tests

Slide on stimulus complexity

  • SIMPLE = Numbers in middle of screen
  • COMPLEX = Numbers presented

randomly throughout the screen

22

Manikin Test Mathematical Processing Tower Puzzle

Computerized Neuropsych Tests Computerized Neuropsych Tests

Attention: Simple Presentation

slide-12
SLIDE 12

23

Manikin Test Mathematical Processing Tower Puzzle

Computerized Neuropsych Tests Computerized Neuropsych Tests

Attention: Simple Presentation

24

Manikin Test Mathematical Processing Tower Puzzle

Computerized Neuropsych Tests Computerized Neuropsych Tests

Attention: Simple Presentation

slide-13
SLIDE 13

25

Manikin Test Mathematical Processing Tower Puzzle

Computerized Neuropsych Tests Computerized Neuropsych Tests

Attention: Complex Presentation

26

Manikin Test Mathematical Processing Tower Puzzle

Computerized Neuropsych Tests Computerized Neuropsych Tests

Attention: Complex Presentation

slide-14
SLIDE 14

27

Manikin Test Mathematical Processing Tower Puzzle

Computerized Neuropsych Tests Computerized Neuropsych Tests

Attention: Complex Presentation

28

Relevance to Real World Functioning

Neuropsychological Tests Neuropsychological Tests

slide-15
SLIDE 15

29

  • 1. Apple
  • 2. Car
  • 3. Pear
  • 4. Banana

Neuropsychological Tests Neuropsychological Tests

Relevance to Real World Functioning

Virtual Iraq Virtual Iraq

slide-16
SLIDE 16

31

Virtual Iraq: VRET & VRCPAT Virtual Iraq: VRET & VRCPAT

The Vision… The Vision…

  • Re c yc le F

SW

  • Highly c o ntro llable
  • Co nte xt re le vant
  • Psyc ho me tric ally so und
  • Co nstruc t Valid
  • E

c o lo gic ally Valid

32

Full Spectrum VR Exposure Full Spectrum VR Exposure Therapy for Iraq War PTSD Therapy for Iraq War PTSD

Funded by Funded by

slide-17
SLIDE 17

Virtual Reality Cognitive Performance Virtual Reality Cognitive Performance Assessment Test Assessment Test (VRCPAT) (VRCPAT)

34

Military Relevance Military Relevance

slide-18
SLIDE 18

VRCPAT: Mixed Reality Scenario VRCPAT: Mixed Reality Scenario

(currently being developed) (currently being developed)

36

VRCPAT: Mixed Reality VRCPAT: Mixed Reality

slide-19
SLIDE 19

37

FlatWorld based Urban Terrain Module: Ft. Sill Oklahoma

VRCPAT: Mixed Reality VRCPAT: Mixed Reality

38

Reaction Time Reaction Time Memory Memory Attn Attn Executive Functions Executive Functions

VRCPAT: Mixed Reality VRCPAT: Mixed Reality

FlatWorld based VRCPAT

slide-20
SLIDE 20

39

Executive Functions Executive Functions

VRCPAT: Mixed Reality VRCPAT: Mixed Reality

FlatWorld based VRCPAT

VRCPAT: HMD Scenario VRCPAT: HMD Scenario

slide-21
SLIDE 21

41

Currently assessed:

  • Attention
  • Learning and Retrieval (Memory)
  • Information processing speed

To be developed:

  • Language/Verbal Skills
  • Executive Function
  • Visuospatial perception

Neurocognitive Domains Neurocognitive Domains

ENCODING ENCODING (getting the information in) (getting the information in) CONSOLIDATION CONSOLIDATION (transferring information into long (transferring information into long-

  • term store)

term store) RETRIEVAL RETRIEVAL (getting the information out) (getting the information out)

MEMORY MODULE MEMORY MODULE

Virtual Reality Cognitive Performance Virtual Reality Cognitive Performance Assessment Test (VRCPAT) Assessment Test (VRCPAT)

slide-22
SLIDE 22

43

Study 1: Memory Module Study 1: Memory Module

Parsons, T.D., and Rizzo, A.A. (2008). Initial Validation of a Virtual Environment for Assessment of Memory Functioning: Virtual Reality Cognitive Performance Assessment Test. Cyberpsychology and Behavior, 11, 17-25.

44

  • Neuropsychological battery

– Pencil/paper tests: including Self- reports and Demographics

  • Virtual Reality (VRCPAT)

– Learning Phase: Trials 1–3 (Similar to HVLT & BVMT-R) – VR Acclimation Phase – VRCPAT Immersion Phase

  • Capture images of 10 items (from

Acquisition Phase: Trials 1-3)

  • 2 images at each of the 5 Zones
  • Time limit = 1 min. for each Zone
  • Actual immersion is around 15 min.

Study 1: Memory Module Study 1: Memory Module

slide-23
SLIDE 23

45

  • Sample:

– 70 healthy subjects (50% female) – Recruited from undergraduate and graduate schools.

  • Comparable in age, education, ethnicity, sex, self-reported symptoms
  • f depression
  • Age Range:

– 21 to 36 (mean = 24.97, SD = 3.78)

  • Education Range:

– 13 to 20 years (mean = 16.13, SD = 1.69)

  • Inclusion/Exclusion Criteria:

– Excluded persons with significant psychiatric history – Excluded persons with significant neurologic history

  • USC’s Institutional Review Board

Study 1: Memory Module Study 1: Memory Module

46

Learning

  • HVLT Trials 1–3
  • BVMT-R Trials 1–3

Memory

  • HVLT Retention and Recognition
  • BVMT-R Retention and Recognition

Executive Functions

  • TMT B
  • Stroop Interference

Attentional Processing

  • WAIS-III Digit Span Forward
  • WAIS-III Digit Span Backward

Processing Speed

  • TMT A
  • Digit Coding

Verbal Fluency

  • Category Fluency
  • Letter Fluency

Study 1: Memory Module Study 1: Memory Module

slide-24
SLIDE 24

47

  • 1. intact wooden barrel with US Army stencil
  • 2. man with blue shirt (white stripes)
  • 3. tanned Caucasian American soldier with moustache
  • 4. camel with an M brand on rear left side
  • 5. large red shipping container numbered 7668
  • 6. sealed wooden crate with Iraqi flag on side corner
  • 7. injured brown and white dog on its side
  • 8. blue car with bullet holes in the windshield
  • 9. sign above door with English words “Internet Homeland”
  • 10. Robed man holding cell phone by graffiti image of Saddam on wall

Study 1: Memory Module Study 1: Memory Module

48

  • 1. intact wooden barrel with US Army stencil
  • 2. man with blue shirt (white stripes)
  • 3. tanned Caucasian American soldier with moustache
  • 4. camel with an M brand on rear left side
  • 5. large red shipping container numbered 7668
  • 6. sealed wooden crate with Iraqi flag on side corner
  • 7. injured brown and white dog on its side
  • 8. blue car with bullet holes in the windshield
  • 9. sign above door with English words “Internet Homeland”
  • 10. Robed man holding cell phone by graffiti image of Saddam on wall

Study 1: Memory Module Study 1: Memory Module

slide-25
SLIDE 25

49

  • Demonstrate:

– HMD, earphones, and controller – Adjust for proper fit

  • Acclimatation:

– Participant navigates the Virtual Environment

Once instructions are clear, participant begins scenario

Study 1: Memory Module Study 1: Memory Module

50

  • Capture images of 10 items

– from Acquisition Phase – Trials 1-3

  • 2 targets at each of 5 Zones

Study 1: Memory Module Study 1: Memory Module

slide-26
SLIDE 26

51

Study 1: Memory Module Study 1: Memory Module

52

Study 1: Memory Module Study 1: Memory Module

slide-27
SLIDE 27

53

Zone Targets Zone Targets

Sealed wooden crate w/ Iraqi flag on side corner Blue car WITHOUT bullet holes in the windshield Man with RED (instead of blue) shirt

Zone Distractors Zone Distractors

54

Convergent Validity Convergent Validity

Theory Theory Memory Memory construct construct

Verbal Verbal Learning Learning Verbal Verbal Memory Memory

Visual Visual

Learning Learning

Visual Visual

Memory Memory

we we theorized theorized that the VRCPAT that the VRCPAT will reflect memory will reflect memory

Observation Observation

VRCPAT & NP Test Correlations Learning HVLT Trials 1–3 0.58 BVMT-R Trials 1–3 0.75 Mem ory HVLT Retention 0.50 BVMT-R Retention 0.40

Neuropsychogical tests Neuropsychogical tests (HVLT; (HVLT; BVMT

BVMT) reflect

) reflect a memory construct a memory construct

the correlations provide evidence that the items all converge

  • n the same construct
slide-28
SLIDE 28

55 Learning HVLT Trials 1–3 0.58 BVMT-R Trials 1–3 0.75 Mem ory HVLT Retention 0.50 BVMT-R Retention 0.40

Theory Theory Observation Observation Memory Memory Construct Construct HVLT HVLT BVMT BVMT Other Other Domains Domains Exec Exec Attn Attn PS PS VF VF

Executive Functions Trail Making Test: Part B ns Stroop Interference ns Attention Digit Span Forward ns Digit Span Backward ns Processing Speed Trail Making Test: Part A ns Digit Symbol Coding ns Verbal Fluency Animals ns Letter Fluency ns

VRCPAT & NP Test Correlations

theorize that we have two distinguishable constructs

correlations provide evidence that items discriminate

Discriminant Validity Discriminant Validity

Focused attention Focused attention (respond discretely to specific stimuli) (respond discretely to specific stimuli) Sustained attention Sustained attention (maintain consistent behavioral response) (maintain consistent behavioral response) Selective attention Selective attention (freedom from distractibility) (freedom from distractibility) Alternating attention Alternating attention (mental flexibility; set shifting) (mental flexibility; set shifting) Divided attention Divided attention (simultaneous response to multiple demands) (simultaneous response to multiple demands)

ATTENTION MODULE ATTENTION MODULE

Virtual Reality Cognitive Performance Virtual Reality Cognitive Performance Assessment Test (VRCPAT) Assessment Test (VRCPAT)

slide-29
SLIDE 29

57

  • Fixed position in the virtual city test

– Requires user to man checkpoint

  • Exploratory Path

– well marked to minimize navigational cognitive load

  • Humvee scenario

– User will be either driving or riding as a passenger in a simulated Humvee

Attention Module Attention Module

58

Fixed Position: Checkpoint Fixed Position: Checkpoint

slide-30
SLIDE 30

59

Humvee: Simple Humvee: Simple (Low Intensity) (Low Intensity)

60

Humvee: Simple Humvee: Simple (High Intensity) (High Intensity)

slide-31
SLIDE 31

61

Humvee: Complex Humvee: Complex (Low Intensity) (Low Intensity)

62

Humvee: Complex Humvee: Complex (High Intensity) (High Intensity)

slide-32
SLIDE 32

63

  • VRCPAT and Traditional NP Tests
  • Sample:

– 16 healthy subjects (50% female) – Recruited from undergraduate and graduate schools

  • Comparable in age, education, ethnicity, sex, self-

reported symptoms of depression

– Age Range: mean = 26.71, SD = 4.49 – Education Range: mean = 15.50, SD = 2.54

  • Inclusion/Exclusion Criteria:

– Excluded persons with significant psychiatric history – Excluded persons with significant neurologic history

  • USC’s Institutional Review Board

Study 2: Attention Module Study 2: Attention Module

64

  • Neuropsychological battery

– Pencil/paper tests: – including Self-reports and Demographics

  • Virtual Reality (VRCPAT)

– Attention Modules

  • Psychophysiological Metrics

– EEG, EMG, ECG, respiration, HR, GSR, etc.)

Study 2: Attention Module Study 2: Attention Module

slide-33
SLIDE 33

65

Humvee Scenario

66

Study 2: Preliminary Results Study 2: Preliminary Results

*

* Significant, p<.05.

*

results… Increased Complexity Decreased Attention VRCPAT Attention

slide-34
SLIDE 34

67

Study 2: Preliminary Results Study 2: Preliminary Results

*

* Significant, p<.05.

results… Increased Intensity Decreased Attention VRCPAT Attention

Collaboration with. . .

  • Automated

Neuropsychological Assessment Metrics (ANAM)

  • University of Oklahoma

Center for the Study of Human Operator Performance (C SHOP) Robert E. Schlegel Kirby Gilliland

Research Question: What can be measured easily and effectively with the ANAM and how does VRCPAT supplement this measurement?

slide-35
SLIDE 35

69

Envirodine Scent Envirodine Scent System System

  • Gunpowder
  • Cordite
  • Body Odor
  • Garbage
  • Burning

Rubber

  • Diesel Fuel
  • Iraqi Spices

And Night Vision And Night Vision HMD Rig HMD Rig… … Bass S haker Bass S haker Platform Platform

Multisensory Multisensory

Biopac S ystem Biopac S ystem

Neuropsychophysiology Neuropsychophysiology

70

Do High Immersion experiences impact people more than Low Immersion experiences

  • Startle Eye-Blink

– Measures: Negativity Emotional State

  • Heart Rate

– Measures: Intensity of Emotional State

  • High Immersion, to justify its costs, should impact Emotional State

– ↑Startle Eye-Blink – ↑Heart Rate

Study 3: Immersion Study 3: Immersion

slide-36
SLIDE 36

71

Study 3: Preliminary Results Study 3: Preliminary Results

*

* Significant, p<.05.

results… Higher Immersion, More Negative Response Startle Eye-blink

72

Study 3: Preliminary Results Study 3: Preliminary Results

*

* Significant, p<.05.

results… Higher Immersion, Stronger Emotional State Heart Rate BPM

slide-37
SLIDE 37

73

Study 4: Immersion: Cadets Study 4: Immersion: Cadets

74

Study 4: Preliminary Results Study 4: Preliminary Results

results… Low Immersion Cadet less negative (ns) Startle Eye-blink

slide-38
SLIDE 38

75

Study 4: Preliminary Results Study 4: Preliminary Results

results… High Immersion Cadets same as civilians Startle Eye-blink

76

Study 4: Preliminary Results Study 4: Preliminary Results

results… Low Immersion Cadets less intense Heart Rate BPM

* Significant, p<.05.

*

slide-39
SLIDE 39

77

Study 4: Preliminary Results Study 4: Preliminary Results

results… High Immersion Cadets same as civilians Heart Rate BPM

78

Future Work: Intelligent Feedback System Future Work: Intelligent Feedback System

slide-40
SLIDE 40

79