vi video ob eo object ject segm segmen enta tati tion on
play

Vi Video Ob eo Object ject Segm Segmen enta tati tion on - PowerPoint PPT Presentation

Sep 12, 2022 •520 likes •807 views

Vi Video Ob eo Object ject Segm Segmen enta tati tion on CV3DST | Prof. Leal-Taix 1 Vi Video deo Objec ject Seg egmen entat ation on Lectures 2-3 Lectures 4-5 Object Detection Object Tracking Lectures 7-8 This lecture

  1. Vi Video Ob eo Object ject Segm Segmen enta tati tion on CV3DST | Prof. Leal-Taixé 1

  2. Vi Video deo Objec ject Seg egmen entat ation on Lectures 2-3 Lectures 4-5 Object Detection Object Tracking Lectures 7-8 This lecture Object Segmentation Video Object Segmentation CV3DST | Prof. Leal-Taixé 2

  3. Vi Video deo Objec ject Seg egmen entat ation on • Goal: Generate accurate and temporally consistent pixel masks for objects in a video sequence. CV3DST | Prof. Leal-Taixé 3

  4. VO VOS: som ome e chal allen enges es • Strong viewpoint/appearance changes CV3DST | Prof. Leal-Taixé 4

  5. VO VOS: som ome e chal allen enges es • Strong viewpoint/appearance changes • Occlusions CV3DST | Prof. Leal-Taixé 5

  6. VO VOS: som ome e chal allen enges es • Strong viewpoint/appearance changes • Occlusions • Scale changes CV3DST | Prof. Leal-Taixé 6

  7. VOS: som VO ome e chal allen enges es • Strong viewpoint/appearance changes • Occlusions Hard to make • Scale changes assumptions about • Illumination object’s appearance • Shape Hard to make • … assumptions about object’s motion CV3DST | Prof. Leal-Taixé 7

  8. VO VOS: tas asks Semi-supervised Unsupervised (zero- (one-shot) video shot) video object object segmentation segmentation We get the first frame We have to find the ground truth mask, we know objects as well as their what object to segment masks CV3DST | Prof. Leal-Taixé 8

  9. VO VOS: tas asks Motion segmentation, salient object detection.. Semi-supervised Unsupervised (zero- (one-shot) video shot) video object object segmentation segmentation We get the first frame We have to find the ground truth mask, we know objects as well as their what object to segment masks CV3DST | Prof. Leal-Taixé 9

  10. VO VOS: tas asks This lecture Semi-supervised Unsupervised (zero- (one-shot) video shot) video object object segmentation segmentation We get the first frame We have to find the ground truth mask, we know objects as well as their what object to segment masks CV3DST | Prof. Leal-Taixé 10

  11. Supe Superv rvised Video Obj bject Se Segm gment ntation Given: First-frame ground truth Goal: Complete video segmentation Task formulation • – Given: segmentation mask of target object(s) in the first frame – Goal: pixel-accurate segmentation of the entire video – Currently a major testing ground for segmentation-based tracking CV3DST | Prof. Leal-Taixé 11

  12. VO VOS Dat atas aset ets • Remember that large-scale datasets are needed for learning-based methods DAVIS 2016 DAVIS 2017 YouTube-VOS 2018 (30/20, single objects, (60/90, multiple (3471/982, multiple first frames) objects, first frames) objects, first frame where object appears) https://davischallenge.org https://youtube-vos.org CV3DST | Prof. Leal-Taixé 12

  13. Bef Befor ore e we e get et star arted… ed… • Pixel-wise output • If we talk about pixel-wise outputs and motion, there is a concept in Computer Vision that we need to know first CV3DST | Prof. Leal-Taixé 13

  14. Optical l flo low CV3DST | Prof. Leal-Taixé 14

  15. Opt Optica cal l flo flow • Input: 2 consecutive images (e.g. from a video) • Output: displacement of every pixel from image A to image B • Results in the “perceived” 2D motion, not the real motion of the object CV3DST | Prof. Leal-Taixé 15

  16. Opt Optica cal l flo flow CV3DST | Prof. Leal-Taixé 16

  17. Opt Optica cal l flo flow CV3DST | Prof. Leal-Taixé 17

  18. Opt Optica cal l flo flow with CNNs NNs • End-to-end supervised learning of optical flow P. Fischer et al. „FlowNet: Learning Optical Flow With Convolutional Networks“. ICCV 2015 CV3DST | Prof. Leal-Taixé 18

  19. Opt Optica cal l flo flow with CNNs NNs P. Fischer et al. „FlowNet: Learning Optical Flow With Convolutional Networks“. ICCV 2015 CV3DST | Prof. Leal-Taixé 19

  20. Fl FlowNet: a : arc rchit itecture ure 1 1 • Stack both images à input is now 2 x RGB = 6 channels CV3DST | Prof. Leal-Taixé 20

  21. Fl FlowNet: a : arc rchit itecture ure 2 2 • Siamese architecture CV3DST | Prof. Leal-Taixé 21

  22. Fl FlowNet: a : arc rchit itecture ure 2 2 • Two key design choices How to combine the information from both images? CV3DST | Prof. Leal-Taixé 22

  23. Cor Correl elation ion layer er • Multiplies a feature vector with another feature vector Fixed operation. No learnable weights! CV3DST | Prof. Leal-Taixé 23

  24. Cor Correl elation ion layer er • The matching score represents how correlated these two feature vectors are CV3DST | Prof. Leal-Taixé 24

  25. Cor Correl elation ion layer er • Hint for anyone interested in 3D reconstruction: Useful for finding image correspondences A Find a transformation from image A to image B B I. Rocco et al. “Convolutional neural network architecture for geometric matching. CVPR 2017. CV3DST | Prof. Leal-Taixé 25

  26. Fl FlowNet : a : arc rchit itecture ure 2 2 • Two key design choices How to obtain high- quality results? How to combine the information from both images? CV3DST | Prof. Leal-Taixé 26

  27. Ca Can we e do o VOS wit ith OF? • Indeed! • Better if we focus on the flow of the object • We can improve segmentation and OF iteratively (no DL yet) Y.H. Tsai et al. “Video Segmentation via Object Flow“. CVPR 2016 CV3DST | Prof. Leal-Taixé 29

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Vi Video Ob eo Object ject Segm Segmen enta tati tion on CV3DST | Prof. Leal-Taix 1

Vi Video Ob eo Object ject Segm Segmen enta tati tion on CV3DST | Prof. Leal-Taix 1

Vi Video Ob eo Object ject Segm Segmen enta tati tion on CV3DST | Prof. Leal-Taix 1 Vi Video deo Objec ject Seg egmen entat ation on Object Detection Object Tracking This lecture Object Segmentation Video Object

1.48k views • 79 slides
Sem Semanti tic c segm segmen enta tati tion on CV3DST | Prof. Leal-Taix 1 Ta Task d

Sem Semanti tic c segm segmen enta tati tion on CV3DST | Prof. Leal-Taix 1 Ta Task d

Sem Semanti tic c segm segmen enta tati tion on CV3DST | Prof. Leal-Taix 1 Ta Task d defin init itio ion: s semantic ic s segm gmentatio ion Classify the main object in the image. CAT , GRASS, TREE, SKY No objects, just

1.29k views • 95 slides
Ins Instanc nce segm segmen enta tati tion on CV3DST | Prof. Leal-Taix 1 Se Semanti

Ins Instanc nce segm segmen enta tati tion on CV3DST | Prof. Leal-Taix 1 Se Semanti

Ins Instanc nce segm segmen enta tati tion on CV3DST | Prof. Leal-Taix 1 Se Semanti ntic c segmenta ntati tion Label every pixel, including the background (sky, grass, road) Do not differentiate between the pixels coming from

1.29k views • 106 slides
Debt bt Refina inancing ncing Invest estor Presen enta tati tion on Outline 1 Executi

Debt bt Refina inancing ncing Invest estor Presen enta tati tion on Outline 1 Executi

December 2019 Consolid solidati ating ng Gains ns in Energy rgy Sector or Debt bt Refina inancing ncing Invest estor Presen enta tati tion on Outline 1 Executi cutive e Summa mary 2 The Transaction saction Structure cture

414 views • 17 slides
Focu cusing sing on De Deli liver ery Invest estor Presen enta tati tion on Novem

Focu cusing sing on De Deli liver ery Invest estor Presen enta tati tion on Novem

Focu cusing sing on De Deli liver ery Invest estor Presen enta tati tion on Novem ember er 2012 Certain of the statements made in this Presentation may contain forward-looking statements within the meaning of the United States Private

999 views • 53 slides
Focu cusing sing on De Deli liver ery Invest estor Presen enta tati tion on Januar ary

Focu cusing sing on De Deli liver ery Invest estor Presen enta tati tion on Januar ary

Focu cusing sing on De Deli liver ery Invest estor Presen enta tati tion on Januar ary 9, 2013 Certain of the statements made in this Presentation may contain forward-looking statements within the meaning of the United States Private

815 views • 45 slides
Sylv lvania ia I Int nter ercha hange S e Study dy City ty Council P Presen enta tati

Sylv lvania ia I Int nter ercha hange S e Study dy City ty Council P Presen enta tati

Sylv lvania ia I Int nter ercha hange S e Study dy City ty Council P Presen enta tati tion June 1 18, , 2018 PID 1 105889 05889 LUC-US2 S23-11 11.75 .75 Agenda 1. Project Team 2. Study Area 3. ODOT Process 4. Project

503 views • 23 slides
De Deli livering ering Sustainable stainable Val alue ue Invest estor Presen enta tati

De Deli livering ering Sustainable stainable Val alue ue Invest estor Presen enta tati

De Deli livering ering Sustainable stainable Val alue ue Invest estor Presen enta tati tion on Augus ust t 2012 Certain of the statements made in this Presentation may contain forward-looking statements within the meaning of the

762 views • 48 slides
ts of In Init itia iatio tion: The Th e Sa Sacr cramen aments BAPTIS PTISM EUCHA

ts of In Init itia iatio tion: The Th e Sa Sacr cramen aments BAPTIS PTISM EUCHA

ts of In Init itia iatio tion: The Th e Sa Sacr cramen aments BAPTIS PTISM EUCHA CHARIS IST CONF ONFIR IRMATION MATION Th This is pre presen enta tati tion on is is wh what at I ca I call ll the signature talk

441 views • 41 slides
Feb ebruar uary 2 2020 0 PetroTal H l Highlights A full-cycle, sustainable E&P enhanced

Feb ebruar uary 2 2020 0 PetroTal H l Highlights A full-cycle, sustainable E&P enhanced

Inves estor P Presen enta tati tion December 2 2019 Oil-Focus used ed, B , Balanced ed Po Portfolio io Pur Pure e Pl Play Delivering both steady productio ion growth and high impact explo loratio ion Feb ebruar uary 2 2020

503 views • 25 slides
TONIGHTS AGENDA Open n house se 5:00 to 5:30 Presen enta tatio tion 5:30 to 6:00

TONIGHTS AGENDA Open n house se 5:00 to 5:30 Presen enta tatio tion 5:30 to 6:00

TONIGHTS AGENDA Open n house se 5:00 to 5:30 Presen enta tatio tion 5:30 to 6:00 Workshop shop 6:00 to 7:30 Questio stions ns and Answer ers 7:30 to 8:00 1 Resolution Copper Project and Land Exchange EIS ALTERNATIVE

596 views • 28 slides
In Investor or Pr Presenta tati tion on. We simply grow better. The The Sup Supreme

In Investor or Pr Presenta tati tion on. We simply grow better. The The Sup Supreme

In Investor or Pr Presenta tati tion on. We simply grow better. The The Sup Supreme Canna nnabis Company ny T S X : F I R E 1 reasonable in the circumstances. However, the Company accessed under the Companys profile on the System

531 views • 29 slides
Disaster Recovery Waivers From Pla n to Im p lem enta tion: Lessons Lea rned Suggestions to

Disaster Recovery Waivers From Pla n to Im p lem enta tion: Lessons Lea rned Suggestions to

Departm ent of Housing and Urban Developm ent Disaster Recovery Waivers From Pla n to Im p lem enta tion: Lessons Lea rned Suggestions to Other Disaster Recovery Grantees Ask for everything even if you dont think youll need it.

477 views • 21 slides
OF 23 23 A PRIL IL 200 2008 C HA HAL M ARTI ELEC ECTED EGMEN ENTS TS O TIN 1 P RES TO M OUNT

OF 23 23 A PRIL IL 200 2008 C HA HAL M ARTI ELEC ECTED EGMEN ENTS TS O TIN 1 P RES TO M OUNT

T RANSCRIPT O PT OF S EL TED S EG OF 23 23 A PRIL IL 200 2008 C HA HAL M ARTI ELEC ECTED EGMEN ENTS TS O TIN 1 P RES TO M OUNT T V ER ERNON C ITY TY C OUNCIL ESEN ENTA TATI TION TO 2 3 [All times minutes, seconds until the 1-hour mark,

332 views • 11 slides
Holy Redeemer Roman Catholic Church The he P Presen senta tati tion on o of the L the Lor

Holy Redeemer Roman Catholic Church The he P Presen senta tati tion on o of the L the Lor

Holy Redeemer Roman Catholic Church The he P Presen senta tati tion on o of the L the Lor ord February 2, 2020 - Year A OFFICE HOURS: Monday to Friday 8:30 AM - 3:30 PM 1887 Bancroft Drive Sudbury, ON P3B 1S7 Phone: 705-566-8330

228 views • 8 slides
NM DPP Collaborative: A Partnership Prese senta tati tion by b Susan A Affholt lter

NM DPP Collaborative: A Partnership Prese senta tati tion by b Susan A Affholt lter

NM DPP Collaborative: A Partnership Prese senta tati tion by b Susan A Affholt lter er, H Healt lth Departmen D ment o of Northwest M Michiga gan Kim Chan C andl dler, D Distri rict H Health D Depart artment

316 views • 28 slides
A Bayesian Approach to A Bayesian Approach to Unsupervised One- Unsupervised One -Shot Shot

A Bayesian Approach to A Bayesian Approach to Unsupervised One- Unsupervised One -Shot Shot

A Bayesian Approach to A Bayesian Approach to Unsupervised One- Unsupervised One -Shot Shot Learning of Object Learning of Object Categories Categories Li Li Fei Fei- -Fei Fei Rob Rob Fegus Fegus Pietro Pietro Perona Perona

192 views • 4 slides
One-Shot Learning: Language Acquisition for Machine SS16 Computational Linguistics for

One-Shot Learning: Language Acquisition for Machine SS16 Computational Linguistics for

One-Shot Learning: Language Acquisition for Machine SS16 Computational Linguistics for Low-Resource Languages Mayumi Ohta July 6, 2016 Institute for Computational Linguistics Heidelberg University Table of contents 1. Introduction 2.

985 views • 70 slides
Intro to Unity Shaders CM163 Lab 1 Rendering Pipeline Vertex Shader - Program that transforms

Intro to Unity Shaders CM163 Lab 1 Rendering Pipeline Vertex Shader - Program that transforms

Intro to Unity Shaders CM163 Lab 1 Rendering Pipeline Vertex Shader - Program that transforms vertices in someway Rasterizer - Turn the transformed vertices to pixels on the screen Fragment Shader - Program that process the pixels Creating

498 views • 27 slides
Plan for Lexical Analysis with Jlex and One Pass Code Gen Structure of the MeggyJava Compiler

Plan for Lexical Analysis with Jlex and One Pass Code Gen Structure of the MeggyJava Compiler

Plan for Lexical Analysis with Jlex and One Pass Code Gen Structure of the MeggyJava Compiler Analysis Synthesis Overview of the MeggyJava Assignments character stream PA2:Lexer/scanner in MJPA2.jar lexical analysis code gen Expressing

203 views • 9 slides
. One-to-one and Onto Functions . . Definition . Let f A B denote a function from A to

. One-to-one and Onto Functions . . Definition . Let f A B denote a function from A to

. One-to-one and Onto Functions . . Definition . Let f A B denote a function from A to B . 1 The function f is one-to-one if a x a implies that . . f a x f a . . . 2 The function f is onto if, for every b >

201 views • 4 slides
Todays Agenda Upcoming Homework Section 3.2: Inverse Functions and Logarithms Lindsey

Todays Agenda Upcoming Homework Section 3.2: Inverse Functions and Logarithms Lindsey

Todays Agenda Upcoming Homework Section 3.2: Inverse Functions and Logarithms Lindsey K. Gamard, ASU SoMSS MAT 265: Calculus for Engineers I Wed., 7 October 2015 1 / 10 Upcoming Homework WeBWorK HW #12 Section 3.1 due 10/9

413 views • 10 slides
JUST THE MATHS SLIDES NUMBER 3.3 TRIGONOMETRY 3 (Approximations & inverse functions)

JUST THE MATHS SLIDES NUMBER 3.3 TRIGONOMETRY 3 (Approximations & inverse functions)

JUST THE MATHS SLIDES NUMBER 3.3 TRIGONOMETRY 3 (Approximations & inverse functions) by A.J.Hobson 3.3.1 Approximations for trigonometric functions 3.3.2 Inverse trigonometric functions UNIT 3.3 - TRIGONOMETRY 3 APPROXIMATIONS

418 views • 6 slides
Basic Recursion Dr. Mattox Beckman University of Illinois at Urbana-Champaign Department of

Basic Recursion Dr. Mattox Beckman University of Illinois at Urbana-Champaign Department of

Objectives Function Calls Example Recursions Lists References Basic Recursion Dr. Mattox Beckman University of Illinois at Urbana-Champaign Department of Computer Science Objectives Function Calls Example Recursions Lists References

672 views • 22 slides

More recommend