The Dynamic Image Segmentation for Sign Language Training Simulator - - PowerPoint PPT Presentation

▶
the dynamic image segmentation for sign language training
SMART_READER_LITE
LIVE PREVIEW

The Dynamic Image Segmentation for Sign Language Training Simulator - - PowerPoint PPT Presentation

Sep 15, 2023 299 likes •428 views

The Dynamic Image Segmentation for Sign Language Training Simulator Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 1 Gesture (a) Start of the gesture (b) Intermediate frame (c) Intermediate frame (d)

slide-1
SLIDE 1

The Dynamic Image Segmentation for Sign Language Training Simulator

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 1

slide-2
SLIDE 2

Gesture

(a) Start of the gesture (b) Intermediate frame (c) Intermediate frame (d) Final frame

Figure 1: Gesture “What for”

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 2

slide-3
SLIDE 3

Dactyl

(a) Starting dactyl (b) Final dactyl

Figure 2: Dactyls describing gesture “What for”

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 3

slide-4
SLIDE 4

Dactyl Matching

Figure 3: Recognition of Fingertips

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 4

slide-5
SLIDE 5

Colour Space in Image Clustering with SOM

(a) Original (b) RGB (c) HSL (d) CIELab

Figure 4: Clustering of an image represented in different colour spaces

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 5

slide-6
SLIDE 6

Training Data Composition

A =          (L1

1, a1 1, b1 1)T

(L1

2, a1 2, b1 2)T

(L1

3, a1 3, b1 3)T

(L1

4, a1 4, b1 4)T

(L2

1, a2 1, b2 1)T

(L2

2, a2 2, b2 2)T

(L2

3, a2 3, b2 3)T

(L2

4, a2 4, b2 4)T

(L3

1, a3 1, b3 1)T

(L3

2, a3 2, b3 2)T

(L3

3, a3 3, b3 3)T

(L3

4, a3 4, b3 4)T

(L4

1, a4 1, b4 1)T

(L4

2, a4 2, b4 2)T

(L4

3, a4 3, b4 3)T

(L4

4, a4 4, b4 4)T

         S1 =    (L1

1, a1 1, b1 1)T

(L1

2, a1 2, b1 2)T

(L2

1, a2 1, b2 1)T

(L2

2, a2 2, b2 2)T

  

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 6

slide-7
SLIDE 7

Cluster Interpretation

  • Image pixels represented by topologically close neurons should be-

long to the same cluster and therefore segment.

  • The colour or marker used for a segment representation is irrelevant

as long as each segment is associated with a different one. Rj ← xj + yj × λ; Gj ← xj + yj × λ; Bj ← xj + yj × λ; (1)

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 7

slide-8
SLIDE 8

Application Results

(a) Frame 25 original (b) Frame 25 segmented (c) Frame 35 original (d) Frame 35 segmented

Figure 5: Frames 25 and 35

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 8

slide-9
SLIDE 9

Application Results

(a) Frame 45 original (b) Frame 45 segmented (c) Frame 65 original (d) Frame 65 segmented

Figure 6: Frames 45 and 65

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 9

slide-10
SLIDE 10

Other Applications

(a) Original (b) Segmented (c) Original (d) Segmented

Figure 7: Cloth Segmentation for Image Search

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 10

slide-11
SLIDE 11

Alternative Approach

(a) Sample 1 (b) Sample 2

Figure 8: Cloth Segmentation for Image Search

Oles Hodych, Kostiantyn Hushchyn, Iouri Nikolski, Volodymyr Pasichnyk, Yuri Shcherbyna 11