text adaptive generative adversarial networks
play

Text-Adaptive Generative Adversarial Networks: Manipulating Images - PowerPoint PPT Presentation

Mar 12, 2024 •33 likes •193 views

Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language Seonghyeon Nam, Yunji Kim, Seon Joo Kim Dept. of Computer Science, Yonsei University Seoul, South Korea Manipulating Images with Natural Language Icons

  1. Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language Seonghyeon Nam, Yunji Kim, Seon Joo Kim Dept. of Computer Science, Yonsei University Seoul, South Korea

  2. Manipulating Images with Natural Language Icons made by Freepik from www.flaticon.com

  3. Manipulating Images with Natural Language This small bird has a blue crown and white belly . Icons made by Freepik from www.flaticon.com

  4. Manipulating Images with Natural Language This small bird has a blue Processing... crown and white belly . Here it is. Icons made by Freepik from www.flaticon.com

  5. Related Work ● Existing methods rely heavily on sentence embedding vectors ● They fail to preserve text-irrelevant contents (e.g. background) ● Coarse multi-modal modeling is not enough for the disentanglement Original [Reed et al., [Dong et al., Ours 2016] 2017]

  6. Contribution ● Our key idea is word-level local discriminators for fine-grained training ● Our method effectively changes visual attributes while preserving text-irrelevant contents Original [Reed et al., [Dong et al., Ours 2016] 2017]

  7. Overview of TAGAN This flower has petals that are yellow and are very stringy .

  8. Generator This flower has petals that are yellow and are very stringy . To preserve original contents, we add a reconstruction loss:

  9. Discriminator The discriminator consists of 1. Unconditional discriminator → Make image realistic 2. Text-adaptive discriminator → Make image match the text This flower has petals that are yellow and are very stringy .

  10. Text-Adaptive Discriminator 1. Compute local discriminator scores Image text image text Encoder Global Local Average v Discriminator Pooling Text w Encoder

  11. Text-Adaptive Discriminator 1. Compute local discriminator scores 2. Compute text/image attentions : softmax weight for word i : softmax weight for word i , and image feature level j

  12. Text-Adaptive Discriminator 1. Compute local discriminator scores 2. Compute text/image attentions : softmax weight for word i : softmax weight for word i , and image feature level j 3. Aggregate the scores with attentions

  13. Manipulation Results on CUB-200

  14. Manipulation Results on Oxford-102 Gazania Wikipedia

  15. Qualitative Comparison Original [Dong et al., 2017] [Xu et al., 2018] Ours

  16. Conclusion ● We propose a Text-Adaptive Generative Adversarial Network (TAGAN) ● Our method disentangles and manipulates fine-grained visual attributes ● Our method outperforms existing methods on CUB-200 and Oxford-102 Please visit our poster (#126) for more information https://github.com/woozzu/tagan

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Generative Adversarial Networks Aaron Mishkin UBC MLRG 2018W2 1 Generative Adversial Networks

Generative Adversarial Networks Aaron Mishkin UBC MLRG 2018W2 1 Generative Adversial Networks

Generative Adversarial Networks Aaron Mishkin UBC MLRG 2018W2 1 Generative Adversial Networks Two imaginary celebrities that were dreamed up by a random number generator. https://research.nvidia.com/publication/2017-10

665 views • 39 slides
StackGAN Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks

StackGAN Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks

StackGAN Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks The Problem: 2-Stage Network Stage 1. Generates 64x64 images Structural information Low detail Stage 2. Requires

884 views • 16 slides
Adversarial Training Attacks on Deep Networks and Generative Adversarial Networks Erkut Erdem

Adversarial Training Attacks on Deep Networks and Generative Adversarial Networks Erkut Erdem

images from Geris Game (Pixar, 1997) Adversarial Training Attacks on Deep Networks and Generative Adversarial Networks Erkut Erdem Aykut Erdem Levent Karacan Computer Vision Lab, Hacettepe University Outline Part 1: Attacks on

1.11k views • 72 slides
Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative

Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative

Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks Generative Adversarial Text to Image Synthesis

614 views • 27 slides
Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative

Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative

Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks Generative Adversarial Text to Image Synthesis

666 views • 39 slides
Conditional Generative Adversarial Networks (and a brief look at image-to-image translation)

Conditional Generative Adversarial Networks (and a brief look at image-to-image translation)

Conditional Generative Adversarial Networks (and a brief look at image-to-image translation) Final Presentation Peter Bromley Generative Models What is generative modeling? Data: Samples from high dimensional probability distribution p real

621 views • 25 slides
Generative Adversarial Networks presented by Ian Goodfellow presentation co-developed with Aaron

Generative Adversarial Networks presented by Ian Goodfellow presentation co-developed with Aaron

Generative Adversarial Networks presented by Ian Goodfellow presentation co-developed with Aaron Courville 1 In todays talk Generative Adversarial Networks Goodfellow et al., NIPS 2014 Conditional Generative

2.84k views • 40 slides
LAB MEETING: A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning

LAB MEETING: A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning

LAB MEETING: A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning and Energy-Based models Suwon Suh POSTECH MLG Feb, 13, 2017 Goal Understanding Basic Models 1) Generative Adversarial Networks (GAN) 2)

722 views • 12 slides
Generative Adversarial Networks, Wasserstein Distance, and Adversarial Loss Zhiyu Min Alibaba

Generative Adversarial Networks, Wasserstein Distance, and Adversarial Loss Zhiyu Min Alibaba

Generative Adversarial Networks, Wasserstein Distance, and Adversarial Loss Zhiyu Min Alibaba AliMe X-Lab Outline GAN Definition and formulation Saddle point optimization Vanishing gradient Alternative objective for Generator

516 views • 25 slides
GAN-based Photo Video Synthesis Summary of Generative Adversarial Nets Lei Zhang What is

GAN-based Photo Video Synthesis Summary of Generative Adversarial Nets Lei Zhang What is

GAN-based Photo Video Synthesis Summary of Generative Adversarial Nets Lei Zhang What is Generative Adversarial Networks (GAN)? Generative - creating new data that depends on the choice of the training set Adversarial - competitive

894 views • 9 slides
Robust Estimation and Generative Adversarial Networks Weizhi ZHU Hong Kong University of Science

Robust Estimation and Generative Adversarial Networks Weizhi ZHU Hong Kong University of Science

Robust Estimation and Generative Adversarial Networks Weizhi ZHU Hong Kong University of Science and Technology wzhuai@ust.hk April 3, 2019 Robust Estimation and Generative Adversarial Nets [GLYZ18] Generative Adversarial Nets for Robust

252 views • 24 slides
Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference

Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference

Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference San Jose, California 2016-04-05 Generative Modeling Have training examples: x p train ( x ) Want a model that can draw samples: x p

317 views • 16 slides
Generative Adversarial Networks (GANs) Prof. Seungchul Lee Industrial AI Lab. Source 1

Generative Adversarial Networks (GANs) Prof. Seungchul Lee Industrial AI Lab. Source 1

Generative Adversarial Networks (GANs) Prof. Seungchul Lee Industrial AI Lab. Source 1 GAN (Generative Adversarial Network) by YouTube: https://www.youtube.com/watch?v=odpjk7_tGY0 Slides:

2.44k views • 49 slides
Generative Adversarial Networks Wanyue Li

Generative Adversarial Networks Wanyue Li

Generating Fundus Fluorescence Angiography Images from Structure Fundus Images Using Generative Adversarial Networks Wanyue Li (wanyueli93@126.com) University of Science and Technology of China (USTC)

144 views • 10 slides
Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at

Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at

Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at Berkeley Artificial Intelligence Lab, 2016-08-31 Generative Modeling Density estimation Sample generation Training examples Model samples

897 views • 37 slides
CSC321 Lecture 19: Generative Adversarial Networks Roger Grosse Roger Grosse CSC321 Lecture 19:

CSC321 Lecture 19: Generative Adversarial Networks Roger Grosse Roger Grosse CSC321 Lecture 19:

CSC321 Lecture 19: Generative Adversarial Networks Roger Grosse Roger Grosse CSC321 Lecture 19: Generative Adversarial Networks 1 / 25 Overview In generative modeling, wed like to train a network that models a distribution, such as a

920 views • 25 slides
First we will introduce some necessary background. 2 For example, VGG16 can correctly classify

First we will introduce some necessary background. 2 For example, VGG16 can correctly classify

First we will introduce some necessary background. 2 For example, VGG16 can correctly classify the left image as giant panda. By contrast, after introducing some subtle noises, the adversarial image can fool the neural networks. JSMA

301 views • 28 slides
Objec(ves Web Server Socket programming in Java Project 1 Open up a terminal Sept 11,

Objec(ves Web Server Socket programming in Java Project 1 Open up a terminal Sept 11,

Objec(ves Web Server Socket programming in Java Project 1 Open up a terminal Sept 11, 2017 Sprenkle - CSCI 325 1 Administra(on Background on Web Server project Abstrac(ng away networking Back to regularly scheduled

689 views • 35 slides
Distributed Control Lab - A component-based application Overview Architecture Experiments

Distributed Control Lab - A component-based application Overview Architecture Experiments

Distributed Control Lab - A component-based application Overview Architecture Experiments Outline Motivation The Distributed Control Lab (DCL) Architecture Foucault's Pendulum Details Hardware / Software Architecture

794 views • 48 slides
Verification of Protocol Implementations by Typechecking Cdric Fournet Microsoft Research w

Verification of Protocol Implementations by Typechecking Cdric Fournet Microsoft Research w

Formal/Computational Verification of Protocol Implementations by Typechecking Cdric Fournet Microsoft Research w ith Karthik Bhargavan, Andy Gordon, http://research.microsoft.com/~fournet http://msr-inria.inria.fr/projects/sec

1.28k views • 101 slides
qcow2 why (not)? Max Reitz <mreitz@redhat.com> Kevin Wolf <kwolf@redhat.com> KVM

qcow2 why (not)? Max Reitz <mreitz@redhat.com> Kevin Wolf <kwolf@redhat.com> KVM

qcow2 why (not)? Max Reitz <mreitz@redhat.com> Kevin Wolf <kwolf@redhat.com> KVM Forum 2015 Choosing between raw and qcow2 Traditional answer: Performance? raw! Features? qcow2! But what if you need both? A car analogy

700 views • 45 slides
A cool way to publish manga in Japan OGATA Katsuhiro (Denno MAVO LLC.) 2018/09/19 W3C Workshop

A cool way to publish manga in Japan OGATA Katsuhiro (Denno MAVO LLC.) 2018/09/19 W3C Workshop

A cool way to publish manga in Japan OGATA Katsuhiro (Denno MAVO LLC.) 2018/09/19 W3C Workshop on Digital Publication Layout and

516 views • 18 slides
Texture Mapping Texture (images) lecture 16 Texture mapping Aliasing (and anti-aliasing)

Texture Mapping Texture (images) lecture 16 Texture mapping Aliasing (and anti-aliasing)

Texture Mapping Texture (images) lecture 16 Texture mapping Aliasing (and anti-aliasing) Adding texture improves realism. Q: Why do we need texture mapping ? A: Because objects look fake and boring without it. recall mapping for

176 views • 6 slides
Texture and other Mappings Texture Mapping Texture Mapping Bump Mapping Bump Mapping

Texture and other Mappings Texture Mapping Texture Mapping Bump Mapping Bump Mapping

Texture and other Mappings Texture Mapping Texture Mapping Bump Mapping Bump Mapping Displacement Mapping Displacement Mapping Environment Mapping Environment Mapping Angel Chapter 7 Last Time - Shading Flat shading: Inexpensive to

629 views • 26 slides

More recommend