Attention Mechanism Exploits Temporal Contexts: Real-time 3D Human - - PowerPoint PPT Presentation

Attention Mechanism Exploits Temporal Contexts: Real-time 3D Human Pose Reconstruction

Code is available at: (https://github.com/lrxjason/Attention3DHumanPose)

Attentional Mechanism

Attention Temporal Attention

(weights on tensors)

Kernel Attention

(weights on channels)

convolution unit

Attention Layer 0 Attention Layer 1 Attention Layer 2 Attention Layer 3

Attention Layer 0

⨂ w0 ⨂ w1 ⨂ w2 ⨂ w3 ⨂ w4 ⨂ w5 ⨂ w6 ⨂ w7

(1) (1) (1) (1) (1). (1) (1) (1)

Attention Layer 1

⨂ w0 ⨂ w1 ⨂ w2 ⨂ w3 ⨂ w4 ⨂ w5

Attention Layer 2

⨂ w0 ⨂ w1 ⨂ w2 ⨂ w3 ⨂ w4

Attention Layer 3

⨂ w0 ⨂ w1 ⨂ w2 ⨂ w3 ⨂ w4 ⨂ w5 ⨂ w6 ⨂ w7

(1) (1) (1) (1) (1). (1) (1) (1)

⨂ w0 ⨂ w1 ⨂ w2 ⨂ w3 ⨂ w4 ⨂ w5 ⨂ w0 ⨂ w1 ⨂ w2 ⨂ w3 ⨂ w4

Attention Layer 0

𝛊t

(1)

𝛊t

(2)

𝛊t

(3)

The Multi-scale Dilated Convolution Structure

Input To increase receptive field

… …

More layers

Attention Layer

⨂ ⨂ ⨂ ⨂ ⨂ ⨂ ⨂ ⨂

x y z Level 0 Level 1 Level 2 Level 3

… … … …

Output Tensors from each layer

Level 0 Level 1 Level 3 Level 2

𝛊t

(2) (1)

𝛊t

y z Output Input

layer 0 layer 1 layer 2 layer 3 layer 4 level 0 level 1 level 2

• 1. Side-by-side comparison with state-of-the-art

Quantitative Evaluation:

Motion Retargeting Views

Quantitative Evaluation:

• 2. Joint-wise MPJPE: comparison with

state-of-the-art

Quantitative Evaluation:

• 3. Frame-wise MPJPE: comparison with

state-of-the-art

Quantitative Evaluation:

• 4. Results on wild videos

Qualitative Evaluation:

• 5. Real-time performance using the causal model

Qualitative Evaluation:

Thank you for watching

Code is available at: (https://github.com/lrxjason/Attention3DHumanPose)