Relationship between Motion Vectors of Vection-Induced Image and - - PowerPoint PPT Presentation

▶
relationship between motion vectors of vection induced
SMART_READER_LITE
LIVE PREVIEW

Relationship between Motion Vectors of Vection-Induced Image and - - PowerPoint PPT Presentation

May 24, 2023 332 likes •571 views

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks Tohru Kiryu 1 , Yoko Nanbo 1 , Naoki Kobayasi 2 , and Takehiko Bando 3 1 Graduate School of Science and Technology, Niigata University,

slide-1
SLIDE 1

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks

Tohru Kiryu1, Yoko Nanbo1, Naoki Kobayasi2, and Takehiko Bando3

1Graduate School of Science and Technology, Niigata University, Niigata, Japan 2NTT Cyber Space Laboratories, Yokosuka, Japan

3Graduate School of Medical and Dental Sciences, Niigata University, Niigata, Japan

slide-2
SLIDE 2

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Evaluation of Stress by Biosignals Engineering Side Evaluation of Image Quality by SN Ratio Medical Side However… Relationship with image has not been evaluated However, … Influence of image on human has not yet been discussed

Relationship between Image and Biosignals?

Background Background

slide-3
SLIDE 3

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Overview Overview

30

  • 30

motion[pixel] right left

distant-range

Global Motion Vector

Motion of Camera Control

10

  • 40

zoom, pan, and tilt

RR Interval

20 0.6 0.8 70 motion[pixel] 20 70 20 70 [sec] [sec]

Local Motion Vector

Motion at Each Local Section

Correlation? Correlation?

slide-4
SLIDE 4

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Purpose Purpose

  • 1. Feature extraction from Multi-Biosignals that

might contain useful information. → PCA Score

  • 2. Quantitative representation of Image by Motion
  • Vectors. → Time-Frequency Representation
  • 3. Relationship between the Structure of Image

and Biosignals. → Correlation

  • 3. Searching the Structure of Image that affects
  • human. → Simulation by CG
  • 4. Design of Total Evaluation System

→ Contents Search Engine etc.

slide-5
SLIDE 5

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

BP ECG RR Int. LPF Extraction of ANA Info.

PCA Feature Extraction，Arrangement

Resp.

Time-Series of PCA Score

Biosignals Image

Motion Vector Feature Extraction

Video Image

Time-Series of Motion Vector at each Block Arrangement

Correlation

Approach Approach

slide-6
SLIDE 6

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

  • ECG：chest-lead
  • Blood Pressure：tonometry method.
  • Respiration: tube type sensor,

at the chest & abdominal regions.

  • Perspiration：capsule type sensor,

at the thenar of left hand.

  • Blood Flow：laser doppler,

at the thenar of left hand.

School of Medicine, Niigata University (Nov.13-14,1999)

80-inch Screen 2 [m]

Experiments Experiments

vehicle and bodily sensation video (Mountain Bike)

Subjects

  • Five healthy elderly subjects (from 52 to 71 yrs. old)
  • Six elderly subjects with mild hypertension or

diabetes mellitus (5 males and 1 female from 50 to 71 yrs. old) Biosignals

slide-7
SLIDE 7

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Multivariate Multivariate Biosignals Biosignals

Mountain bike Racing car Car race diving Bike race

Respiration R-R interval HF LF/HF Persipiration Blood flow Head motion(R/L) Head motion(A/P) Head motion(U/D) speech

5min.

slide-8
SLIDE 8

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

3 33 2 32 1 31 3 3 23 2 22 1 21 2 3 13 2 12 1 11 1

x x x Z x x x Z x x x Z ϕ ϕ ϕ ϕ ϕ ϕ ϕ ϕ ϕ + + = + + = + + =

PCA Score

1

Z

3

x

1

x

2

x

3

Z

2

Z

Direction ratio

13 12 11

: : ϕ ϕ ϕ

：eigen vector

ϕ

RR Int. BP

time[sec]

Resp. 1st PCA Score (z1) 2nd PCA Score (z2)

time[sec]

3rd PCA Score (z3)

20 70 0.6 0.8 20 70 20 70

4

  • 4

20 70 20 70 20 70

0.8 1.2

  • 0.2

0.2

4 –4 4

  • 4

Subject A

Arrangement by PCA Arrangement by PCA

slide-9
SLIDE 9

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Rest Image-stress

Subject A

0.8 0.1 0.4 0.6 0.2 0.7 0.5 0.3 20 30 40 50 60 70 time[sec] Frequency[Hz] 0.8 0.1 0.4 0.6 0.2 0.7 0.5 0.3 0 10 20 30 40 50 time[sec] Frequency[Hz] 0.8 0.1 0.4 0.6 0.2 0.7 0.5 0.3 0 10 20 30 40 50 time[sec] Frequency[Hz] 0.8 0.1 0.4 0.6 0.2 0.7 0.5 0.3 20 30 40 50 60 70 time[sec] Frequency[Hz]

z1 z2 z2 z1

TFRs TFRs of PCA Score Time-Series

  • f PCA Score Time-Series
slide-10
SLIDE 10

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Frequency Component around 0.1Hz Frequency Component around 0.1Hz

Resting Image Task Subject BP Z2 BP Z2 A ○ ○ B ○ ○ ○ ○ C ○ ○ ○ D NA NA NA NA E NA NA NA NA F NA NA NA NA O ○ ○ P ○ ○ ○ Q ○ ○ R ○ ○ ○ S ○ ○ ○

Influence on human was expected ・frequency component around 0.1Hz appeared. ・increase of perspiration ・report of ill-condition

elderly subjects with mild hypertension or diabetes mellitus healthy elderly subjects

slide-11
SLIDE 11

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Motion Vector

Image

1-frame 30 [frame/sec] Static image

・compression by the correlation in space (space frequency, DCT) ・compression by the correlation between frames → Motion Vector ・compression by the deviation of appearance for sign (entropy cording) Image Compression

Correlation to Biosignals?

Feature Extraction of Image Feature Extraction of Image

slide-12
SLIDE 12

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Local Motion & Global Motion Local Motion & Global Motion

30

  • 30

motion[pixel]

right left distant-range

Local Motion Vector Global Motion Vector

Motion at each Local Section Motion of Camera Control

10

  • 10
  • 20
  • 30

pan [pixel]

  • 40

(l-1)th frame l-th frame Block Matching Method

camera screen tilt pan x x y y

b1,1

slide-13
SLIDE 13

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

20 30 40 50 60 70 time[sec] 30

  • 30

20 30 40 50 60 70 time[sec]

motion[pixel]

right left

distant-range close-range

30

  • 30

30

  • 30

30

  • 30

30

  • 30

30

  • 30

motion[pixel] motion[pixel] motion[pixel] motion[pixel] motion[pixel]

Time-Series of LMV Time-Series of LMV

Local Motion Vector

middle-range

down u p

b1,1 b3,3 b5,3

slide-14
SLIDE 14

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

30

  • 30

20 30 40 50 60 70 time[sec] 30

  • 30

20 30 40 50 60 70 time[sec]

motion[pixel] motion[pixel] 1

  • 1

1

  • 1

相 関 係 数 Right/Left LMV

No-correlation

Up/down LMV

Significant Correlation

BP

z2

Correlation between LMV & Correlation between LMV & Biosignals Biosignals

No-correlation

BP

z2

Correlation coefficient Correlation coefficient

middle-range distant-range

slide-15
SLIDE 15

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Influences of Motion Vectors Influences of Motion Vectors

Local Motion Vectors（distant/near） Right/left Up/down Global Motion Vectors （zoom/pan/tilt） Subject RRInt. BP Z2 RRInt. BP Z2 RRInt. BP Z2 A ○/○ ○/× ○/× △/× ×/× ×/× ○/○/○ ○/○/○ ○/○/○ B ×/○ △/○ ○/○ ×/× △/× ○/× ○/○/○ ○/○/○ ○/○/○ C △/× ○/△ △/○ ○/× ○/△ ×/△ ×/○/○ ○/○/○ ○/○/○ D,E,F NA NA NA NA NA NA NA NA NA O △/△ ○/△ ×/× △/○ ○/○ ×/△ △/△/△ ○/○/○ ○/×/○ P △/○ △/× ○/△ △/○ △/○ ○/○ ○/○/△ △/○/△ ○/○/△ Q ×/△ △/○ △/× △/○ △/○ ×/× △/×/○ ○/△/○ ○/×/△ R ○/× ×/△ ×/△ ○/○ △/○ △/○ △/○/○ ○/△/○ △/△/○ S ×/× △/△ ×/△ △/○ ○/○ △/△ ×/×/○ △/△/○ ○/△/○

elderly subjects with mild hypertension or diabetes mellitus healthy elderly subjects

slide-16
SLIDE 16

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

TFRs TFRs of Motion Vectors

  • f Motion Vectors

30

  • 30

20 30 40 50 60 70

time [sec]

20 30 40 50 60 70

time [sec]

motion[pixel]

zoom

0.16

  • 0.16

motion 1 5 10 15

frequency [Hz]

30 40 40 50 1 5 10 15

frequency [Hz]

Features in the intervals

motion vector varied from quick vibration to slow motion Local Motion Vector Global Motion Vector

3,3

b

slide-17
SLIDE 17

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Summary Summary

Next Step

  • Does the local motion vector represent the stressor of

image quantitatively?

  • Which factors? Content or Structure of Image?

・In the Vection-Induced Image, time-varying behavior (quick vibration of local motion vector (around 10Hz), then

slow vibration (around 5Hz))of image components induced

unpleasant feelings for human． ・Arrangement of Biosignals & Time-Frequency Representation were effective

slide-18
SLIDE 18

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

vection-induced video image

Searching for Factors from Image Searching for Factors from Image that Influence on Human that Influence on Human

Simulation by CG

Actuality image of descending wild field by moutain bike

  • video camera was attached at the top of the mountain bike handle
  • - several events consist of quick motions after temporally motion pauses

Concrete CG Image Abstract CG Image

Similar Content Similar Motion Vector

Local Motion Vector Global Motion Vector

Synthesis of CG image by Local Motion and Global Motion separately

slide-19
SLIDE 19

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

Differences in HRV Differences in HRV

0.6 0.7 0.8 0.9 1 1.1 0.6 0.7 0.8 0.9 1 1.1 20 30 40 50 60 70 [sec] RR interval R U/D D L L L D LD R 0.3 0.4 0.5 0.6 0.7 0.8

  • 25 -20 -15 -10
  • 5

5 10 15 20 25 U D L

  • subj. K; 1st trial
  • subj. A: healthy subject
  • subj. O: mild hypertension
  • subj. K; 6th trial
  • subj. S; 6th
  • subj. S; 1st trial

local time [sec] local time RR interval second corner

Actual Cycling Exercise Watching Mountain Bike Riding Image

slide-20
SLIDE 20

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

( )

( ) ( )

∑ ∑ + + − =

= = − ≤ ≤ − N i N j l l N n m N j i y x

n j m i f j i f Min v v

1 1 1 2 , 2 ,

, , arg ,

2 2

：position of pixel ：frame number

l

：searching range

) , ( n m

：pixel value（intensity, color）

( )

j i fl ,

[ ]

T j i y x j i

v v

, ,

, = v

) ( arg

,

á Min

n m

： that minimizes á

Block Matching Method

) , ( n m ) , ( j i

x

v

y

v

：horizontal component ：vertical component

Block (N×N[pixel])

frame

y

x

) 1 ( − l frame

x

y

l

m

n

Estimation of Local Motion Vector Estimation of Local Motion Vector

slide-21
SLIDE 21

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

[ ]

      ∑ ∑ = =

α′ α = α β′ β = β β α , , ,

, v b average b b

T j i y x j i

j j i i × = β′ + − × = β × = α′ + − × = α ) 5 / ] 4 / 288 ([ , 1 ) 1 ( ) 5 / ] 4 / 288 ([ ) 5 / ] 4 / 365 ([ , 1 ) 1 ( ) 5 / ] 4 / 365 ([

1 , 1

b

1 , 2

b

2 , 1

b

5 , 5

b

Whole image was divided into 25 ( 5×5 ) sections. Motion vector was estimated in the block of pixels and then averaged in each section to estimate the averaged motion vector.

Image size 365× × × ×288[pixel] ・ ・ ・ ・Block size 8× × × ×8[pixel] ・ ・ ・ ・shift interval 4[pixel] ・ ・ ・ ・motion vector (365/4)× × × ×(288/4)

Local Motion Vector at each Section Local Motion Vector at each Section

slide-22
SLIDE 22

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

TFRs TFRs of Local Motion Vector

  • f Local Motion Vector

20 30 40 5 0 60 70 time [sec]

  • 30

30 [pixel]

  • 30

30

Right/Left LMV Up/Down LMV

[Hz] 2 4 6 8 10 12 14 2 4 6 8 10 12 14

  • 30

30

  • 30

30

Right/Left LMV Up/Down LMV

2 4 6 8 10 12 14 2 4 6 8 10 12 14 [Hz] [pixel] [Hz] [pixel] 20 30 40 5 0 60 70 time [sec] 20 30 40 5 0 60 70 time [sec] 20 30 40 5 0 60 70 time [sec] 20 30 40 5 0 60 70 time [sec] 20 30 40 5 0 60 70 time [sec] 20 30 40 5 0 60 70 time [sec] 20 30 40 5 0 60 70 time [sec] [Hz] [pixel]

distant-range middle-range

slide-23
SLIDE 23

Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ，BSI2002, Como, Italy， ， ， ，June 27, 2002

TFRs TFRs of Global Motion Vector

  • f Global Motion Vector

1.1 1.0 0.9

zoom

0.8

time [sec]

14 12 10 8 6 4 2 10

  • 10
  • 20
  • 30

pan

  • 40

14 12 10 8 6 4 2 14 12 10 8 6 4 2

frequency

10

  • 10
  • 20
  • 30

tilt

  • 40

20 30 40 50 60 70 20 30 40 50 60 70 20 30 40 50 60 70 20 30 40 50 60 70 20 30 40 50 60 70 20 30 40 50 60 70 [pixel] [pixel] [Hz] [Hz] [Hz] time [sec]