GAN Fashion Photo Shoot: Garment to Model Images Using Conditional - - PowerPoint PPT Presentation

Costa Colbert

GAN Fashion Photo Shoot: Garment to Model Images Using Conditional GANs.

Costa M. Colbert, Chief Scientist MAD Street Den Inc.

Studies show higher purchase rates when clothing is shown on human figures.

Li Live e model el photogr graphy is expen ensive

• Brands and retailers on average spend $100-500 per
• look. Lower per-look prices do not include hair, makeup,

and styling

• Shooting capacity is limited
• 35-40 looks per day with hair & makeup
• 60-70 looks per day without hair & makeup
• Bulk of the cost includes:
• Models’ time (at least $1,200 day rate)
• Photographer’s time
• Digital tech & post production
• Hair, makeup, styling
• Cost does not usually include:
• Pulling samples
• Transporting samples to photo studio
• Photo studio & equipment
• Time to cast models & hire photographers & stylists
• Time of internal teams involved in a photo shoot process
• Reshoots due to items not selling with a current image (3-5% of

items)

GA GANs to the e res escue. e...

If quality and control of images is sufficient...

GA GANs to the e res escue. e...

Garment images Generated images

GA GANs to the e res escue. e...

A few examples…

Catalog photo Garment image Generated image Generated image

A few examples…

Catalog photo Garment image Generated image Generated image

A few examples…

Catalog photo Garment image Generated image Generated image

Varying pose

Gen Gener erative e Adver ersarial Network

Real / Fake Discriminator Training Dataset providing real samples x Samples z from prior distribution e.g. N(0,1) Generator z G(z) approximates a sample from x D(x) decides if sample is from x

Also use L1 reconstruction loss term: abs(G(z)-x)

Generator CNN Discriminator CNN

Co Conditional Ge Generative Adversarial Network

Real or Fake ? G(z) approximates a sample from x Training Dataset providing real samples x ~ X Samples z from prior distribution e.g., garments, pose, other labels D(x, garment) decides if sample is from x, also requiring correct garment

Discriminator CNN

Co Conditional GA GAN - di disc scri rimi mina nator

Input is Model Image concatenated to Garment Image (Patch GAN, Isola et al. 2016) Real/Fake is determined by

• bserving

patches of limited extent. 6 CNN layers Convolution Instance Normalization Dropout

Hmm.., maybe that global discriminator term wasn’t such a bad idea..

Co Conditional GA GAN - ge generator

Latent Vector 4x3x512 Encoder Decoder Pose Garment Image Encoder 6 CNN layers Convolution Instance Normalization Adam Optimizer Fashion Model Image Decoder 6 CNN layers Deconvolution/Unpool Dropout Instance Normalization GTX1080ti 2-4 GB

Pose Interpolation

GA GAN gen ener erator - la latent t ve vector

Latent Vector 512x4x3 Encoder Decoder

la latent t vect ctor

• r - In

Inter erpolation

Enc Dec Enc Dec

X1 X2 LV2 LV1 X1garment X2garment

la latent t vect ctor

• r - In

Inter erpolation

Enc Dec Enc Dec

X1garment X2garment X1 F(LV1,LV2) LV2

Dec

XFi,n LV1 Fi,n(x,y) = x + (y-x)*i/n X2

Latent Variable Interpolation Shoes Neckline Hemline

Latent Variable Interpolation Hemline

Latent Variable Interpolation Color Background Note sleeves.

la latent t vect ctor

• r – mo

modi dify y value ues

Enc Dec

X1garment X1 F(LV1,i)

Dec

XF,

i

LV1 Latent Vector 512x4x3

Principal Component Analysis (PCA)

PCA is a dimension-reduction tool that can reduce a large set

• f variables to a small set that still contains most of the

information in the large set. PCA transforms a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables (principal components). PCA determines the new dimensions on the basis of variance.

Principal Component Analysis (PCA)

Use PCA to go from 512x4x3 (~6k) dimensions to 100. LV6k PCA LV100 Choose an entry, scale by +/- 10 PCA LV6k Inv(PCA) LV100

Dec

XF,

i

PCA Latent Variable Interpolation Skin color Model build

PCA Latent Variable Interpolation Shoes

Conclusions and Future Work

Conditional GAN’s are well-suited for image generation in well-defined domains. Good enough for the casual observer not to notice. GAN’s have many “moving parts,” but we are getting better at using them. More work needed on accessories, choosing specific shoes, handbags, etc. Requires more thought on implementing conditioning labels.

A big thanks to Preferred Networks

Thank you!!

l support@madstreetden.com