A novel EEG-based spelling system using N100 and P300 Hikaru Sato - - PowerPoint PPT Presentation

A novel EEG-based spelling system using N100 and P300

Hikaru Sato and Yoshikazu Washizawa

The University of Electro-Communications, Tokyo, Japan.

September 2, 2014

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 1 / 30

Table of Contents

1

Motivation

Brain computer interfaces Event-related potentials Previous studies (P300-speller) and technical issues

2

Proposed method

N100 ERP components New design of stimulus

3

Experiment and Result

4

Summary and future plans

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 2 / 30

Introduction

What is BCI?

Brain Computer Interface

⊲ Without physical movement ⊲ By only brain signal ⊲ Mental task

Support human by using brain signal

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 3 / 30

Introduction

ERP; Event-related potential

ERP

⊲ Caused by cognition and mental task ⊲ ERP components; N100, N200, P300

P300

⊲ Peak is roughly 300ms ∼ 500 ms ⊲ Observed in the oddball task

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 4 / 30

Introduction

ERP; Event-related potential

0.2 0.4 0.6 −6 −4 −2 2 4 6 x 10

−6

P300 waveform Time [s] Amplitude [V]

Averaged brain signal of Subject A for each 16 channel in the proposed system

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 4 / 30

Previous work

P300-speller

(Farwell et al,1988)

Feature components

⊲ P300 → Detect the target row and column

Stimuli

⊲ 12 images (6 rows and 6 columns) ⊲ Flash row and column in random order

How to input

⊲ Respond the row and column which include the target

character

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 5 / 30

Previous work

How to detect the target character “Row” × “Column” ⇒ “Target character”

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 6 / 30

Previous work

Technical issues of P300-speller

Require at least 12 flashes to input one character

⊲ Restrict the improvement of typing speed ⊲ Make the user fatigue

At least one character flashes twice in a row

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 7 / 30

Proposed method

Proposed method: Purpose and Approach

⋆ Purpose

⊲ Increase Information transfer rate and typing speed ⊲ Same stimulus never flashes twice in a row ⊲ Reduce the user’s fatigue

⋆ Approach

⊲ Uniquely designed stimulus images ⊲ P300 ⇒ Detect the target stimulus image ⊲ N100 ⇒ Detect the user’s gazing position

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 8 / 30

Proposed method

Characteristics of visual N100

Elicited without user’s cognitive process Peak is around 150ms after the stimulation Amplitude is larger for intended location than for non-intended location

0.1 0.2 0.3 −3 −2 −1 1 2 3x 10

−6

N100 waveform Time [s] Amplitude [V]

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 9 / 30

Proposed method

Detection of gazing position using N100

Detect the gazing position by N100 response

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 10 / 30

Proposed method

Peoposed method

Feature components

⊲ P300 →Detect the target image ⊲ N100 →Detect the gazing position

Stimuli

⊲ 9 images (4 characters and 2 blanks) ⊲ Flash in random order

How to input

⊲ Gaze at the position presenting the target character

and respond the target character.

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 11 / 30

Proposed method

How to detect the gazing position

Gaze at position No. 1.

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 12 / 30

Proposed method

How to detect the target character “Image” × “Position” ⇒ “Target character”

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 13 / 30

Proposed method

Advantages of the proposed method

Reduce the number of stimulus images

⊲ 12 in P300-speller → 9 in proposed method ⊲ Increase typing speed ⊲ Reduce the user’s fatigue

Every character is never presented twice in a row

⊲ Easy to conduct mental task ⊲ Only one response in a loop (Two in P300-speller)

Extend the distance between characters

⊲ Prevent the user from responding neighbors of target

by mistake

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 14 / 30

Experimental condition

Experimental condition

Method P300-speller Proposed method Stimulus images Total number of flashes 24 (12 × 2 loops) 18 (9 × 2 loops ) Flash duration 125 ms + 62.5 ms Trial duration 4.5 sec 3.375 sec

Sampling rate : 512 Hz EEG electrodes : 16 channels Subjects : 10 healthy males Trials : 40 trials

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 15 / 30

Analysis

Feature extraction range

P300：125ms - 625ms N100：100ms - 250ms

0.2 0.4 0.6 −6 −4 −2 2 4 6 x 10

−6

Averaged waveform (Cz) Time [s] Amplitude [V] P300 non−P300 0.2 0.4 0.6 −6 −4 −2 2 4 6 x 10

−6

Averaged waveform (FCz) Time [s] Amplitude [V] N100 non−N100

Range for P300 Range for N100

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 16 / 30

Analysis

Classification and Evaluation

Preprocessing

⊲ 1-13Hz bandpass Butterworth filter ⊲ 49-51Hz bandstop Butterworth filter ⊲ Downsampling: 512Hz → 64Hz

Classification method

⊲ Linear Support Vector Machine (Linear SVM)

Evaluation

⊲ Five-fold cross-validation ⊲ Information transfer rate (ITR) [bit/sec] is given by ITR = 1 T { log2 N + P log2 P + (1 − P) log2 1 − P N − 1 }

(1)

T: Input time P: Accuracy N: The number of character , subject to P > 1/N

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 17 / 30

Result

Averaged signal waveform of P300

0.2 0.4 0.6 −6 −4 −2 2 4 6 x 10

−6

Time [s] Amplitude [V] Averaged waveform (Cz) (P300−speller) P300 non−P300

P300-speller

0.2 0.4 0.6 −6 −4 −2 2 4 6 x 10

−6

Time [s] Amplitude [V] Averaged waveform (Cz) (Proposed) P300 non−P300

Proposed mthod Averaged waveform of Subject A

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 18 / 30

Result

Averaged signal waveform of N100

0.2 0.4 0.6 −6 −4 −2 2 4 6 x 10

−6

Time [s] Amplitude [V] Averaged waveform (FCz)(Proposed) N100 non−N100

Averaged waveform of Subject A

Red line : Gaze at characters Blue line : Gaze at blanks

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 19 / 30

Result

Detection performance of ERP components

Averaged classification accuracy and standard deviation of 10 subjects

P300 [%] N100 [%] P300-speller 80.9 ± 7.4 – Proposed 74.0 ± 18.4 88.5 ± 10.4 P300 classification

P300-speller Proposed The number of training samples 64 32 The number of classes 6 9

N100 detection performance

⊲ The number of averaging

N100: 6 times P300: 2 times

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 20 / 30

Result

Performance of methods

Averaged classification accuracy and ITR of 10 subjects

Character [%] ITR [bit/sec] P300-speller 67.8 ± 15.6 0.60 ± 0.22 Proposed 70.3 ± 17.1 0.85 ± 0.34 ITR performance (Proposed > P300-speller)

⊲ Higher accuracy of detection character ⊲ Proposed method takes shorter time to type

P300-speller : 4.5 sec. Proposed : 3.375 sec.

The significance of ITR was confirmed by the Welch’s t-test at level α = 0.05.

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 21 / 30

Conclusion

Conclusion

Spelling BCI using N100 and P300

⊲ 9 stimulus images : 4 characters and 2 blanks ⊲ P300 : To detect the target image ⊲ N100 : To detect the user’s gazing position ⊲ Shorter time to type character

Experiment shows

⊲ Higher detection performance of N100 ⊲ Improvement of ITR

Future work

⊲ The comparison to other spelling BCI ⊲ The optimal selection of electrode positions

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 22 / 30

Thank you

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 23 / 30

Appendix

Appendix

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 24 / 30

Appendix

Appendix

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 25 / 30

Appendix

Appendix

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 26 / 30

Appendix

Appendix

Place at parietal area uniformly

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 27 / 30

Appendix Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 28 / 30

Appendix Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 29 / 30

References

References

• L. A. Farwell and E. Donchin. Talking off the top of your

head: Toward a mental prosthesis utilizing event-related brain potentials. Electroenceph. Clin. Neurophysiol., 70:510 ―523, 1988.

• E. K. Vogel and S. J. Luck. The visual N1 component as an

index of a discrimination process. Psychophysiology, 37:190 ―203, 2000.

Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 30 / 30