Learning Data Collection • 2000 portraits from Flickr with large variation • Keywords… • Different Age, gender, pose, hairstyle, background… • Different camera type… • Data example 88
89 89
Data Labeling • Apply closed-form matting and robust matting • Gradually refine the input trimap • Choose the best one from closed-form or robust matting • User interface • Ground truth example 90
91 91
Learn Automatic Matting 92
Our Method Trimap labeling • Input: RGB image • Output: trimap • Network: Fine tuned from FCN 93
Our Method Image Matting Layer • Input: trimap • Output: alpha matte • Novel-designed structure 94
Our Method Image Matting Layer • Feed-Forward: 𝑛𝑗𝑜 𝜇𝐵 𝑈 𝐶𝐵 + 𝜇 𝐵 − 1 𝑈 𝐺(𝐵 − 1) + 𝐵 𝑈 𝑀𝐵 • Back-Forward: 𝜖𝑔 𝜖𝐶 = −𝜇𝐸 −1 𝑒𝑗𝑏(𝐸 −1 𝐺) 𝜖𝐺 = 𝜖𝑔 𝜖𝑔 𝜖𝐶 + 𝐸 −1 𝜖𝑔 𝜖𝜇 = −𝜇𝐸 −1 𝑒𝑗𝑏 𝐺 + 𝐶 𝐸 −1 𝐺 95
Our Method Image Matting Layer • Loss function: 𝑢 | 𝐵 𝑗 − 𝐵 𝑗 𝑢 |, 𝑀(𝐵, 𝐵 𝑢 ) = 𝑥 𝐵 𝑗 𝑗 𝑢 = −𝑚𝑝(𝑞(𝐵 = 𝐵 𝑗 𝑢 )) 𝑥 𝐵 𝑗 96
Model Training • Data augmentation • 4 scales {0.6,0.8,1.2,1.5} • 4 rotations {-45,-22,22,45} degree • Gamma value {0.5,0.8,1.2,1.5} • Network initialization • Fine tuned from FCN-8s Model [J. Long, 2015] 97
Experiments • Running Time • Training time: 20k iterations, one day on Titan X GPU • Testing Time: 0.6s for 600×800 color image. • Comparisons • Graph-cut • FCN Baseline: direct FCN segmentation followed by closed-form matting 98
Results Input Graph-cut FCN Ours 99
Results Input Graph-cut FCN Ours 100
Recommend
More recommend
