Dance Movement Patterns Recognition (Part II) Jess Snchez Morales - - PowerPoint PPT Presentation

▶
dance movement patterns recognition part ii
SMART_READER_LITE
LIVE PREVIEW

Dance Movement Patterns Recognition (Part II) Jess Snchez Morales - - PowerPoint PPT Presentation

Oct 20, 2022 478 likes •803 views

Dance Movement Patterns Recognition (Part II) Jess Snchez Morales EEMCS - HMI Contents Contents Goals Goals HMM HMM Recognizing Simple Steps Recognizing Simple Steps Recognizing Complex Patterns

slide-1
SLIDE 1

EEMCS - HMI

Dance Movement Patterns Recognition (Part II)

Jesús Sánchez Morales

slide-2
SLIDE 2

EEMCS - HMI

Contents Contents

  • Goals

Goals

  • HMM

HMM

  • Recognizing Simple Steps

Recognizing Simple Steps

  • Recognizing Complex Patterns

Recognizing Complex Patterns

  • Auto Generation of Complex Patterns Graphs

Auto Generation of Complex Patterns Graphs

  • Test Bench

Test Bench

  • Conclusions

Conclusions

2/32

slide-3
SLIDE 3

EEMCS - HMI

Contents Contents

  • Goals

Goals

  • HMM

HMM

  • Recognizing Simple Steps

Recognizing Simple Steps

  • Recognizing Complex Patterns

Recognizing Complex Patterns

  • Auto Generation of Complex Patterns Graphs

Auto Generation of Complex Patterns Graphs

  • Test Bench

Test Bench

  • Conclusions

Conclusions

3/32

slide-4
SLIDE 4

EEMCS - HMI

Goals Goals

  • Recognizing simple user

Recognizing simple user’ ’s movements s movements

  • Recognizing complex patterns

Recognizing complex patterns

  • Auto generation of reference patterns

Auto generation of reference patterns

  • Pattern searching during the dance without

Pattern searching during the dance without any reference any reference

4/32

slide-5
SLIDE 5

EEMCS - HMI

Contents Contents

  • Goals

Goals

  • HMM

HMM

  • Recognizing Simple Steps

Recognizing Simple Steps

  • Recognizing Complex Patterns

Recognizing Complex Patterns

  • Auto Generation of Complex Patterns Graphs

Auto Generation of Complex Patterns Graphs

  • Test Bench

Test Bench

  • Conclusions

Conclusions

5/32

slide-6
SLIDE 6

EEMCS - HMI

Hidden Markov Model (HMM) Hidden Markov Model (HMM)

  • Highly used in Speech Recognition

Highly used in Speech Recognition

  • We adapted it for our problem

We adapted it for our problem

  • Data sequences to analyze

Data sequences to analyze

  • HMM graphs for each recognition

HMM graphs for each recognition

  • Viterbi

Viterbi’ ’s s algorithm algorithm

Initial State (1-x1)% State 1 State 2 Final State x1% State 3 x2% x3% x4% x5% (1-x2)% (1-x3)% (1-x4)% (1-x5)%

Stop move 1 stop move 2

6/32

slide-7
SLIDE 7

EEMCS - HMI

Contents Contents

  • Goals

Goals

  • HMM

HMM

  • Recognizing Simple Steps

Recognizing Simple Steps

  • Recognizing Complex Patterns

Recognizing Complex Patterns

  • Auto Generation of Complex Patterns Graphs

Auto Generation of Complex Patterns Graphs

  • Test Bench

Test Bench

  • Conclusions

Conclusions

7/32

slide-8
SLIDE 8

EEMCS - HMI

Recognizing Simple Steps Recognizing Simple Steps

  • What we understand as being a simple

What we understand as being a simple step? step?

– – Right, left, down, up, jump and twister Right, left, down, up, jump and twister

  • How we divided these recognitions

How we divided these recognitions

– – Horizontal Horizontal – – Vertical Vertical – – Twister Twister

8/32

slide-9
SLIDE 9

EEMCS - HMI

Recognizing simple steps (Horizontal) Recognizing simple steps (Horizontal)

  • Recognized steps

Recognized steps

– – Left step Left step – – Right step Right step

  • Used data

Used data

– – Horizonal Horizonal variation of the centre of mass between variation of the centre of mass between frames frames – – User radius User radius

9/32

slide-10
SLIDE 10

EEMCS - HMI

Recognizing simple steps (Horizontal) Recognizing simple steps (Horizontal)

  • Building a sequence and launching a

Building a sequence and launching a thread thread

  • Markov graph

Markov graph

10/32

slide-11
SLIDE 11

EEMCS - HMI

Recognizing simple steps (Vertical) Recognizing simple steps (Vertical)

  • Recognized steps

Recognized steps

– – Up Up – – Down Down – – Jump Jump

  • Used data

Used data

– – Vertical position of the centre of mass Vertical position of the centre of mass – – Vertical average position of the centre of mass Vertical average position of the centre of mass – – User radius User radius

11/32

slide-12
SLIDE 12

EEMCS - HMI

Recognizing simple steps (Vertical) Recognizing simple steps (Vertical)

  • Building a sequence and launching a

Building a sequence and launching a thread thread

  • Markov graph

Markov graph

12/32

slide-13
SLIDE 13

EEMCS - HMI

Recognizing simple steps (Twister) Recognizing simple steps (Twister)

  • Recognized steps

Recognized steps

– – Twister Twister

  • Used data

Used data

– – User radius User radius – – Average radius Average radius – – Horizontal and vertical position of the centre of mass Horizontal and vertical position of the centre of mass .

13/32

slide-14
SLIDE 14

EEMCS - HMI

Recognizing simple steps (Twister) Recognizing simple steps (Twister)

  • Building a sequence and launching a

Building a sequence and launching a thread thread

  • Markov graph

Markov graph

14/32

slide-15
SLIDE 15

EEMCS - HMI

Contents Contents

  • Goals

Goals

  • HMM

HMM

  • Recognizing Simple Steps

Recognizing Simple Steps

  • Recognizing Complex Patterns

Recognizing Complex Patterns

  • Auto Generation of Complex Patterns Graphs

Auto Generation of Complex Patterns Graphs

  • Test Bench

Test Bench

  • Conclusions

Conclusions

15/32

slide-16
SLIDE 16

EEMCS - HMI

Recognizing Complex Patterns Recognizing Complex Patterns

  • What we understand as being complex

What we understand as being complex pattern? pattern?

– – Combinations of simple steps Combinations of simple steps

Left Step + Right Step + Left Step + Jump + Twister + Left Step + Right Step + Left Step + Jump + Twister +… …

  • What we receive from the simple step

What we receive from the simple step recognition recognition

– – Step code Step code – – Duration Duration – – Step time Step time Left Step Code: 11 Duration: 3 frames Time: second 54

16/32

slide-17
SLIDE 17

EEMCS - HMI

Recognizing Complex Patterns Recognizing Complex Patterns

  • Building a sequence and launching a thread

Building a sequence and launching a thread

  • Markov Graph

Markov Graph

.

17/32

slide-18
SLIDE 18

EEMCS - HMI

Contents Contents

  • Goals

Goals

  • HMM

HMM

  • To Recognize Simple Steps

To Recognize Simple Steps

  • To Recognize Complex Patterns

To Recognize Complex Patterns

  • Auto Generation of Complex Patterns Graphs

Auto Generation of Complex Patterns Graphs

  • Test Bench

Test Bench

  • Conclusions

Conclusions

18/32

slide-19
SLIDE 19

EEMCS - HMI

Auto Generation of Complex Patterns Auto Generation of Complex Patterns Graphs Graphs

  • Easy way of building graphs for

Easy way of building graphs for complex patterns recognition complex patterns recognition

  • Included in the pattern recognition

Included in the pattern recognition

  • Saved as a text file

Saved as a text file

19/32

slide-20
SLIDE 20

EEMCS - HMI

Auto Generation of Complex Patterns Auto Generation of Complex Patterns Graphs Graphs

  • We receive the sequence of simple

We receive the sequence of simple steps steps

  • We build the graph for the recognition

We build the graph for the recognition

20/32

slide-21
SLIDE 21

EEMCS - HMI

Contents Contents

  • Goals

Goals

  • HMM

HMM

  • Recognizing Simple Steps

Recognizing Simple Steps

  • Recognizing Complex Patterns

Recognizing Complex Patterns

  • Auto Generation of Complex Patterns Graphs

Auto Generation of Complex Patterns Graphs

  • Test Bench

Test Bench

  • Conclusions

Conclusions

21/32

slide-22
SLIDE 22

EEMCS - HMI

Test Bench Test Bench

  • Test setup

Test setup

– – Good external conditions Good external conditions – – Real time tests Real time tests -

  • 210 steps

210 steps – – Possible results for each test Possible results for each test

  • Well detected

Well detected

  • Wrong detected

Wrong detected

  • Not detected

Not detected

22/32

slide-23
SLIDE 23

EEMCS - HMI

Test Bench Test Bench

  • Test organization

Test organization

– – Horizontal recognition Horizontal recognition – – Vertical recognition Vertical recognition – – Twister recognition Twister recognition – – Complex pattern recognition Complex pattern recognition

23/32

slide-24
SLIDE 24

EEMCS - HMI

Test Bench (Horizontal recognition) Test Bench (Horizontal recognition)

  • High success rate

High success rate

  • No wrong detections

No wrong detections

  • In case of fast dance some steps get

In case of fast dance some steps get lost lost

100,00% 0,00% 0,00% 100,00% 0,00% 0,00%

80,77% 19,23% 0,00%

Individual steps Various steps (slow) Various steps (fast)

Well detected No detected Wrong detected

24/32

slide-25
SLIDE 25

EEMCS - HMI

Test Bench (Vertical recognition) Test Bench (Vertical recognition)

  • Very good individual step recognition

Very good individual step recognition

  • Slow dance tests: some recognition problems

Slow dance tests: some recognition problems

  • Fast dance tests: recognition performance

Fast dance tests: recognition performance decreases decreases

Well detected No detected Wrong detected

81,82% 0,00% 18,18%

81,25% 6,25% 12,50%

Individual steps Various steps (slow) Various steps (fast)

53,85% 30,77% 15,38%

25/32

slide-26
SLIDE 26

EEMCS - HMI

Test Bench (Twister recognition) Test Bench (Twister recognition)

  • High recognition performance for steps

High recognition performance for steps separately taken separately taken

  • Slow dance tests: start having major

Slow dance tests: start having major detection problems detection problems

  • Fast dance tests: many steps are mixed

Fast dance tests: many steps are mixed

Well detected No detected Wrong detected

83,33% 16,67% 0,00%

66,67% 16,67% 16,67% 57,14% 28,57% 14,29%

Individual steps Various steps (slow) Various steps (fast)

26/32

slide-27
SLIDE 27

EEMCS - HMI

Test Bench (Complex pattern) Test Bench (Complex pattern)

  • Perfect recognition performance for steps

Perfect recognition performance for steps separately taken separately taken

  • No concrete pattern to analyze

No concrete pattern to analyze

  • Inherit problems

Inherit problems

– – Wrong simple step recognition Wrong simple step recognition – – Too slow simple step recognition Too slow simple step recognition – – Wrong received order Wrong received order

27/32

slide-28
SLIDE 28

EEMCS - HMI

Contents Contents

  • Goals

Goals

  • HMM

HMM

  • Recognizing Simple Steps

Recognizing Simple Steps

  • Recognizing Complex Patterns

Recognizing Complex Patterns

  • Auto Generation of Complex Patterns Graphs

Auto Generation of Complex Patterns Graphs

  • Test Bench

Test Bench

  • Conclusions

Conclusions

28/32

slide-29
SLIDE 29

EEMCS - HMI

Conclusions (Difficulties encountered) Conclusions (Difficulties encountered)

  • Clothe variations

Clothe variations

  • Unknown frame rate

Unknown frame rate

  • Similarities in the vertical movements

Similarities in the vertical movements

  • User radius variation due to arms movements

User radius variation due to arms movements

  • Failed complex pattern recognition due to wrong order in simple

Failed complex pattern recognition due to wrong order in simple movements recognition movements recognition

29/32

slide-30
SLIDE 30

EEMCS - HMI

Conclusions (Possible improvements) Conclusions (Possible improvements)

  • Visual detection

Visual detection

  • Use of the data from the

Use of the data from the “ “dance dance revolution dance dance revolution pad pad” ”

  • Relating the beat detector with the user recognized

Relating the beat detector with the user recognized steps steps

  • Detection of other simple movements

Detection of other simple movements

  • Development of a learning algorithm to improve the

Development of a learning algorithm to improve the HMM graphs HMM graphs

30/32

slide-31
SLIDE 31

EEMCS - HMI

Conclusions (Reached goals) Conclusions (Reached goals)

  • We have found a good technique to recognize body movements

We have found a good technique to recognize body movements

  • In some cases the results have not been as good as we hoped

In some cases the results have not been as good as we hoped but we think that can be improved but we think that can be improved

  • This technique is also valid to detect more complex patterns

This technique is also valid to detect more complex patterns

  • We easily generate complex pattern graphs

We easily generate complex pattern graphs

  • It has not been possible to search patterns without reference

It has not been possible to search patterns without reference

31/32

slide-32
SLIDE 32

EEMCS - HMI

Thank you very much ! Thank you very much !

32/32