Emer Emergenc ence o e of O f Obj bjec ect S Seg egmen - - PowerPoint PPT Presentation

▶
emer emergenc ence o e of o f obj bjec ect s seg egmen
SMART_READER_LITE
LIVE PREVIEW

Emer Emergenc ence o e of O f Obj bjec ect S Seg egmen - - PowerPoint PPT Presentation

Sep 22, 2023 150 likes •518 views

Emer Emergenc ence o e of O f Obj bjec ect S Seg egmen mentatio ion n in P in Per ertur urbed Gener bed Generativ ive M e Models dels Adam Bielski, Paolo Favaro University of Bern Ob Object se segm gmentation Segmentation

slide-1
SLIDE 1

Emer Emergenc ence o e of O f Obj bjec ect S Seg egmen mentatio ion n in P in Per ertur urbed Gener bed Generativ ive M e Models dels

Adam Bielski, Paolo Favaro University of Bern

slide-2
SLIDE 2

Ob Object se segm gmentation

Segmentation model

2

slide-3
SLIDE 3

Ob Object se segm gmentation

Segmentation model Annotated data

3

slide-4
SLIDE 4

Ob Object se segm gmentation

Segmentation model Annotated data

  • Human annotated
  • Expensive
  • Expertise knowledge

4

slide-5
SLIDE 5

Un Unsuper ervis vised ed objec ject t seg egmen entatio tion

Segmentation model Annotated data

  • Human annotated
  • Expensive
  • Expertise knowledge

Can we do it without labels?

5

slide-6
SLIDE 6

Un Unsuper ervis vised ed ob

  • bject se

segm gmentation

G E

Image Mask

6

slide-7
SLIDE 7

Le Learn rning to to ge generate la layer ered ed sc scenes wi with GA GAN

G E

Image

𝑨~𝒪(0, 𝐽) Background Foreground Mask

7

slide-8
SLIDE 8

Le Learn rning to to ge generate la layer ered ed sc scenes wi with GA GAN

G E

Image

𝑨~𝒪(0, 𝐽) Background Foreground Mask = + ⊙ ⊙

Background 1 - Mask Foreground Mask Composite image

Image composition

8

slide-9
SLIDE 9

Le Learn rning to to ge generate la layer ered ed sc scenes wi with GA GAN

D Real / Fake G E

Image

𝑨~𝒪(0, 𝐽) Background Foreground Mask Unlabeled real images = + ⊙ ⊙

Background 1 - Mask Foreground Mask Composite image

Image composition

9

slide-10
SLIDE 10

De Degenerate s solu lutio ions

Background Foreground Mask Composite image

10

slide-11
SLIDE 11

De Degenerate s solu lutio ions

Background Foreground Mask Composite image

11

slide-12
SLIDE 12

De Degenerate s solu lutio ions

Background Foreground Mask Composite image

12

slide-13
SLIDE 13

De Degenerate s solu lutio ions

Background Foreground Mask Composite image

13

slide-14
SLIDE 14

Wha What is is a c a correct ect par partit itio ion? n?

14

slide-15
SLIDE 15

Wha What is is a c a correct ect par partit itio ion? n?

15

slide-16
SLIDE 16

Ra Rando ndom m shi hifts

16

slide-17
SLIDE 17

Ra Rando ndom m shi hifts

17

slide-18
SLIDE 18

Ra Rando ndom m shi hifts

18

slide-19
SLIDE 19

Ra Rando ndom m shi hifts

19

slide-20
SLIDE 20

Ra Rando ndom m shi hifts

Valid partition ⟹ valid scene after a small shift Invalid partition ⟹ invalid scene after a small shift

20

slide-21
SLIDE 21

Av Avoiding degenerate solutions

Background Foreground Mask Composite image Composite image w/shift

21

slide-22
SLIDE 22

Av Avoiding degenerate solutions

Background Foreground Mask Composite image Composite image w/shift

22

slide-23
SLIDE 23

Av Avoiding degenerate solutions

Background Foreground Mask Composite image Composite image w/shift

23

slide-24
SLIDE 24

Av Avoiding degenerate solutions

Background Foreground Mask Composite image Composite image w/shift

24

slide-25
SLIDE 25

Av Avoiding degenerate solutions

Background Foreground Mask Composite image Composite image w/shift Add a loss term to avoid empty masks

25

slide-26
SLIDE 26

Av Avoiding degenerate solutions

Background Foreground Mask Composite image Composite image w/shift Add a loss term to avoid empty masks

26

slide-27
SLIDE 27

Image composition = + ⊙ ⊙

Background 1 - Mask Foreground Mask Composite image

Le Learn rning wi with per pertur urba bations ns

G E

Image

𝑨~𝒪(0, 𝐽) Background Foreground Mask Minimum mask loss

27

slide-28
SLIDE 28

Image composition = + ⊙ ⊙

Background 1 – Mask shifted Foreground shifted Mask shifted Composite image (fake)

Le Learn rning wi with per pertur urba bations ns

D Real / Fake G E

Image

𝑨~𝒪(0, 𝐽) Background Foreground Mask Minimum mask loss Real image Random shift 𝑞~𝒱

28

slide-29
SLIDE 29

Ge Generatio tion results lts

Backgond Foegond Mak Foegond /mak Comoie image Backgond Foegond Mak Foegond /mak Comoie image

LSUN Car LSUN Bird LSUN Chair LSUN Horse

29

slide-30
SLIDE 30

Image composition = + ⊙ ⊙

Background 1 - Mask Foreground Mask Composite image

Le Learn rning to to se segm gment

G Background Foreground Mask

30

slide-31
SLIDE 31

Image composition = + ⊙ ⊙

Background 1 - Mask Foreground Mask Composite image

Le Learn rning to to se segm gment

G Background Foreground Mask Frozen

31

slide-32
SLIDE 32

Image composition = + ⊙ ⊙

Background 1 - Mask Foreground Mask Composite image

Le Learn rning to to se segm gment

G E

Image

Background Foreground Mask Frozen 𝑀1 𝑀𝑝𝑡𝑡

32

slide-33
SLIDE 33

Image composition = + ⊙ ⊙

Background 1 - Mask Foreground Mask Composite image

Le Learn rning to to se segm gment

G E

Image

Background Foreground Mask Frozen 𝑀1 𝑀𝑝𝑡𝑡

33

slide-34
SLIDE 34

Se Segme mentation

  • n r

results

Real image O mehod Mak R-CNN Real image Our method Ground truth

Real image Our results Approx. ground truth Real image Our results Ground truth

34

slide-35
SLIDE 35

Po Poster #60 East Exhibition Hall B + C

35