S3VAE: Self-Supervised Sequential VAE for Representation - PowerPoint PPT Presentation

Sep 01, 2022 •303 likes •416 views

S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation Disentangled Representation Learning: Framework Encoder VAE Objectives: Decoder LSTM in the latent space Self-Supervised Signal (1):

S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation
Disentangled Representation Learning: Framework ● Encoder VAE Objectives: ● Decoder ● LSTM in the latent space
Self-Supervised Signal (1): Static Consistency Constraint ● To encourage the appearance representation to exclude any dynamic information. ● Triplet Loss: Anchor Shuffle temporal order Positive Negative
Self-Supervised Signal (2): Dynamic Factor Prediction ● To encourage the motion representation to carry adequate and correct time-dependent information of each timestep ● Optical flow provides the location of motion ○ Grid the optical flow map with indices ● Landmarks provides the subtle motion on facial expression The input frame and optical flow ○ Distances between upper and lower eyelips and distances between lips The three distances on faces
Self-Supervised Signal (3): Mutual Information ● To encourage the information in and to be mutually exclusive. ● To minimize the mutual information between and
Experiments: Representation Swapping ● Swap the appearance and motion representation of two given videos Video A Video B Video A Video B
Experiments: Representation Swapping Real Video Synthesized Video
Experiments: Manipulating video generation(Dsprite) Fix appearance representation Fix motion representation
Experiments: Manipulating video generation (MUG) Fix appearance representation Fix motion representation
Experiments: Quantitatively performance comparison ● Baseline: our sequential VAE without self-supervision ● Baseline-sv: our sequential VAE with supervision of ground truth labels ● Full model: our sequential VAE with self-supervision

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Supervised Sequential Classification Under Budget Constraints Kirill Trapeznikov and Venkatesh

Supervised Sequential Classification Under Budget Constraints Kirill Trapeznikov and Venkatesh Saligrama Boston University May 1st, 2013 1 / 50 Overview Introduce sequential decision problem Myopic approach: relies on current uncertainty to

828 views • 78 slides

Sequential Supervised Learning Sequential Supervised Learning Many Application Problems Require

Sequential Supervised Learning Sequential Supervised Learning Many Application Problems Require Many Application Problems Require Sequential Learning Sequential Learning Part- -of of- -speech Tagging speech Tagging Part Information

924 views • 60 slides

Variational Sequential Labelers for Semi-Supervised Learning Mingda Chen, Qingming Tang, Karen

Variational Sequential Labelers for Semi-Supervised Learning Mingda Chen, Qingming Tang, Karen Livescu, Kevin Gimpel Sequence Labeling Part-of-Speech (POS) Tagging determiner noun verb determiner adjective noun

740 views • 41 slides

Vision and Language Representation Learning Self Supervised Pretraining and Multi-Task Learning

Vision and Language Representation Learning Self Supervised Pretraining and Multi-Task Learning Jiasen Lu April 21, 2020 1 1 Vision and Language Visual Question Answering Image Captioning Visual Commonsense Reasoning Refer Expression 2

1.13k views • 53 slides

Towards an efficient representation for epistemic planning Supervised by Alexandre Niveau

Context of the work Theory Work Conclusion, perspectives Towards an efficient representation for epistemic planning Supervised by Alexandre Niveau Sebastien Gamblin University of Caen December 6, 2018 1/24 Context of the work Theory Work

289 views • 27 slides

ECO 199 B GAMES OF STRATEGY Spring Term 2004 B February 24 SEQUENTIAL AND SIMULTANEOUS GAMES

ECO 199 B GAMES OF STRATEGY Spring Term 2004 B February 24 SEQUENTIAL AND SIMULTANEOUS GAMES CLASSIFICATION Games Sequential Simultaneous Representation Tree Matrix Equilibrium Rollback Nash concept (subgame perfect) Pure Mixed

214 views • 5 slides

Sequential Files : Outline ! Overview ! Ordered vs. Unordered ! Physical sequential Files !

Sequential Files : Outline ! Overview ! Ordered vs. Unordered ! Physical sequential Files ! Physical Linked Sequential Files ! COBOL rasitjutrakul Sequential Files " Logically the records are stored consecutively Physical sequential file

335 views • 22 slides

Disentangled Representation Learning 2020.5.21 Seung-Hoon Na Jeonbuk National University

Disentangled Representation Learning 2020.5.21 Seung-Hoon Na Jeonbuk National University Contents Generative models Supervised disentangled representation Unsupervised disentangled representation Adversarial disentangled

1.85k views • 156 slides

Random Sampling Florian Schoppmann August 24, 2010 Non-Sequential Sequential Sequential with

Random Sampling Florian Schoppmann August 24, 2010 Non-Sequential Sequential Sequential with Reservoir Sequential With Reservoir and Replacement . . . . . . . . . . . . Sampling Algorithms Input: List [ 1 . . . N ] Length of

555 views • 13 slides

Sequential Circuits Combinational circuits : current input output Sequential circuit :

14:332:231 DIGITAL LOGIC DESIGN Ivan Marsic, Rutgers University Electrical & Computer Engineering Fall 2013 Lecture #15: Sequential Circuits: Latches Sequential Circuits Combinational circuits : current input output Sequential

483 views • 15 slides

A Compiler Representation for Incremental Parallelization Christoph Angerer and Thomas Gross ETH

A Compiler Representation for Incremental Parallelization Christoph Angerer and Thomas Gross ETH Zurich Parallel Continuation Passing Style Unified Intermediate Representation for: Fully sequential code Parallel code Advanced

582 views • 56 slides

1 Greedy Sequential Covering Example Greedy Sequential Covering Example Y Y + + + + + +

Learning Rules If-then rules in logic are a standard representation of knowledge that have proven useful in expert-systems and other AI systems CS 391L: Machine Learning: In propositional logic a set of rules for a concept is equivalent

396 views • 14 slides

Sequential team form and its simplification using graphical models Aditya Mahajan and Sekhar

Sequential team form and its simplification using graphical models Aditya Mahajan and Sekhar Tatikonda Yale University Allerton, September 30, 2009 Outline Sequential team Team form Simplification of team form Representation of team form as

941 views • 64 slides

Hardware Design with VHDL Sequential Stmts ECE 443 Sequential Statements This slide set covers

Hardware Design with VHDL Sequential Stmts ECE 443 Sequential Statements This slide set covers the sequential statements and the VHDL process (do NOT con- fuse with sequential circuits ) Sequential statements are executed in sequence and allow a

709 views • 38 slides

Chapter 5 Synchronous Sequential Logic 5-1 Outline ! Sequential Circuits ! Latches ! Flip-Flops

Chapter 5 Synchronous Sequential Logic 5-1 Outline ! Sequential Circuits ! Latches ! Flip-Flops ! Analysis of Clocked Sequential Circuits ! State Reduction and Assignment ! Design Procedure 5-2 1 Sequential Circuits ! Consist of a

595 views • 30 slides

Module 11 Introduction to Reinforcement Learning CS 886 Sequential Decision Making and

Module 11 Introduction to Reinforcement Learning CS 886 Sequential Decision Making and Reinforcement Learning University of Waterloo Machine Learning Supervised Learning Teacher tells learner what to remember Reinforcement Learning

401 views • 29 slides

Beyond Object Recognition in 2D Georgia Gkioxari Object Recognition in 2D The World is 3D

Beyond Object Recognition in 2D Georgia Gkioxari Object Recognition in 2D The World is 3D Whelan et al., Reconstructing Scenes with Mirror and Glass Surfaces, SIGGRAPH 2018 Motion is Important for Recognition Johansson, Biological Motion

1.04k views • 41 slides

Traffic analysis and modelling 1 Service classification Services may be classified according

Traffic analysis and modelling 1 Service classification Services may be classified according to many criteria. For example, loosely classify according to nature of transmitted information: voice services computer communications, and

407 views • 21 slides

System Buses Chapter 5 S. Dandamudi Outline Introduction Bus arbitration Dynamic

System Buses Chapter 5 S. Dandamudi Outline Introduction Bus arbitration Dynamic bus arbitration Bus design issues Implementation Bus width Centralized Bus type Distributed Bus operations Example

736 views • 41 slides

Learning Character-Agnostic Motion for Motion Retargeting in 2D Kfir Aberman, Rundi Wu, Dani

Learning Character-Agnostic Motion for Motion Retargeting in 2D Kfir Aberman, Rundi Wu, Dani Lischinski, Baoquan Chen, Daniel Cohen-Or Outline - Motivation - Approach - Results - Application Outline - Motivation - Approach - Results -

468 views • 33 slides

Human Focused Action Localization in Video Alexander Klaeser 1 , Marcin Marszalek 2 , Cordelia

Human Focused Action Localization in Video Alexander Klaeser 1 , Marcin Marszalek 2 , Cordelia Schmid 1 , Andrew Zisserman 2 1 LEAR, INRIA Grenoble 2 Visual Geometry Group, University of Oxford Workshop on Sign, Gesture, Activity ECCV 2010 The

469 views • 26 slides

https://www.tamus.edu/system/total-texas-am-university-system-enrollment 1 Agenda for Tuesday,

https://www.tamus.edu/system/total-texas-am-university-system-enrollment 1 Agenda for Tuesday, November 10, 2020 Provosts Presentation Minutes Approval (6 OCT 2020) Reports New Business Announcements 2 3 Mi Minut

446 views • 26 slides

Capturing full body motion Antoine Kaufmann antoinek@student.ethz.ch April 9, 2013 Distributed

Capturing full body motion Antoine Kaufmann antoinek@student.ethz.ch April 9, 2013 Distributed Systems Seminar 1 What is Motion Capture? April 9, 2013 Distributed Systems Seminar 2 What is Motion Capture? Motion capture is the process of

830 views • 61 slides

Semantic segmentation Image classification Object detection Semantic segmentation Evolution

Accel : A Corrective Fusion Network for Efficient Semantic Segmentation on Video Samvit Jain , Xin Wang , Joseph Gonzalez RISE Lab, UC Berkeley Semantic segmentation Image classification Object detection Semantic segmentation Evolution

857 views • 13 slides