Attention Mechanism Exploits Temporal Contexts: Real-time 3D Human - PowerPoint PPT Presentation

Attention Mechanism Exploits Temporal Contexts: Real-time 3D Human Pose Reconstruction Code is available at: (https://github.com/lrxjason/Attention3DHumanPose)

Attentional Mechanism

Temporal Attention (weights on tensors) Attention Kernel Attention (weights on channels)

convolution unit

Attention Layer 0 Attention Layer 1 Attention Layer 2 Attention Layer 3

Attention Layer 0 Attention Layer 1 (1) (1) (1) (1) (1). (1) (1) (1) ⨂ w 1 ⨂ w 0 ⨂ w 2 ⨂ w 3 ⨂ w 4 ⨂ w 5 ⨂ w 6 ⨂ w 7 Attention Layer 2 ⨂ w 0 ⨂ w 1 ⨂ w 2 ⨂ w 3 ⨂ w 4 ⨂ w 5 Attention Layer 3 ⨂ w 0 ⨂ w 1 ⨂ w 2 ⨂ w 3 ⨂ w 4

Attention Layer 0 (1) 𝛊 t (1) (1) (1) (1) (1). (1) (1) (1) ⨂ w 1 ⨂ w 0 ⨂ w 2 ⨂ w 3 ⨂ w 4 ⨂ w 5 ⨂ w 6 ⨂ w 7 𝛊 t (2) ⨂ w 0 ⨂ w 1 ⨂ w 2 ⨂ w 3 ⨂ w 4 ⨂ w 5 𝛊 t (3) ⨂ w 0 ⨂ w 1 ⨂ w 2 ⨂ w 3 ⨂ w 4

The Multi-scale Dilated Convolution Structure

To increase receptive field Input More layers … …

x z y Attention Layer Level 1 𝛊 t (1) Level 2 ⨂ Level 3 ⨂ ⨂ ⨂ (2) 𝛊 t … … ⨂ ⨂ ⨂ ⨂ Level 0 Level 3 Level 2 Level 0 Level 1 Output Tensors from each layer … …

y layer 0 layer 1 layer 2 layer 3 layer 4 z Input Output level 0 level 1 level 2

Quantitative Evaluation: 1. Side-by-side comparison with state-of-the-art

Quantitative Evaluation: Motion Retargeting Views

Quantitative Evaluation: 2. Joint-wise MPJPE: comparison with state-of-the-art

Quantitative Evaluation: 3. Frame-wise MPJPE: comparison with state-of-the-art

Qualitative Evaluation: 4. Results on wild videos

Qualitative Evaluation: 5. Real-time performance using the causal model

Thank you for watching Code is available at: (https://github.com/lrxjason/Attention3DHumanPose)