sample synthesis method for few-shot object recognition Eli - - PowerPoint PPT Presentation

▶
sample synthesis method
SMART_READER_LITE
LIVE PREVIEW

sample synthesis method for few-shot object recognition Eli - - PowerPoint PPT Presentation

Dec 29, 2022 37 likes •160 views

-encoder: An effective sample synthesis method for few-shot object recognition Eli Schwartz*, Leonid Karlinsky*, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogerio Feris, Raja Giryes, Alex M. Bronstein IBM Research AI |

slide-1
SLIDE 1

IBM Research AI | Computer Vision and Augmented Reality

Δ-encoder: An effective sample synthesis method for few-shot object recognition

Eli Schwartz*, Leonid Karlinsky*, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogerio Feris, Raja Giryes, Alex M. Bronstein

slide-2
SLIDE 2

IBM Research AI | Computer Vision and Augmented Reality

Who’s that dog?

slide-3
SLIDE 3

IBM Research AI | Computer Vision and Augmented Reality

  • The model is a variant of

an auto-encoder operating in feature space

Encoder Decoder

𝑎 Key idea – training

slide-4
SLIDE 4

IBM Research AI | Computer Vision and Augmented Reality

  • The model is a variant of

an auto-encoder operating in feature space

  • The network learns to

encode the delta between the reference and the target image

Encoder Decoder

𝑎

target reference delta

Key idea – training

slide-5
SLIDE 5

IBM Research AI | Computer Vision and Augmented Reality

  • The model is a variant of

an auto-encoder operating in feature space

  • The network learns to

encode the delta between the reference and the target image

  • This delta is used to recover

the target image as a (non-linear) combination of the reference and the delta

Encoder Decoder

𝑎

target reference recovered target delta

Key idea – training

slide-6
SLIDE 6

IBM Research AI | Computer Vision and Augmented Reality

Encoder Decoder

𝑎 Key idea – synthesizing

slide-7
SLIDE 7

IBM Research AI | Computer Vision and Augmented Reality

  • At test time we sample

encoded deltas from random training image pairs

Encoder Decoder

𝑎

sampled target sampled reference sampled delta

Key idea – synthesizing

slide-8
SLIDE 8

IBM Research AI | Computer Vision and Augmented Reality

  • At test time we sample

encoded deltas from random training image pairs

  • The sampled deltas are used

to create samples for new classes by combining them with the new class reference examples

  • These samples are used to train

a classifier for the new category

Encoder Decoder

𝑎

sampled target sampled reference sampled delta new class reference synthesized new class example

Key idea – synthesizing

slide-9
SLIDE 9

IBM Research AI | Computer Vision and Augmented Reality

mini niIm ImageN geNet et: 58.5 (previous SOA)  59.9 (ours) CIFAR AR-10 100: 63.4 (previous SOA)  66.7 (ours) Caltec ech-256 56: 63.8 (previous SOA)  73.2 (ours) CUB: 69.6 (previous SOA)  69.8 (ours)

50 55 60 65 70 75 minImageNet CIFAR-100 Caltech-256 CUB

  • ne-shot classification benchmarks
  • urs

previous state-of-the-art

Few-shot classification experiments

slide-10
SLIDE 10

IBM Research AI | Computer Vision and Augmented Reality

mini niIm ImageN geNet et: 58.5 (previous SOA)  59.9 (ours) CIFAR AR-10 100: 63.4 (previous SOA)  66.7 (ours) Caltec ech-256 56: 63.8 (previous SOA)  73.2 (ours) CUB: 69.6 (previous SOA)  69.8 (ours)

50 55 60 65 70 75 minImageNet CIFAR-100 Caltech-256 CUB

  • ne-shot classification benchmarks
  • urs

previous state-of-the-art

Few-shot classification experiments

slide-11
SLIDE 11

IBM Research AI | Computer Vision and Augmented Reality

Real vs synthetic examples ablation study

slide-12
SLIDE 12

IBM Research AI | Computer Vision and Augmented Reality