SLIDE 1
Day 3 Lecture 12
Saliency
Elisa Sayrol
- Prof. Xavier Giró, Prof. Kevin McGuinness
Student: Junting Pan
2
Saliency Prof. Xavier Gir, Prof. Kevin McGuinness Student: Junting - - PowerPoint PPT Presentation
Saliency Prof. Xavier Gir, Prof. Kevin McGuinness Student: Junting - - PowerPoint PPT Presentation
Day 3 Lecture 12 Saliency Prof. Xavier Gir, Prof. Kevin McGuinness Student: Junting Pan Elisa Sayrol Saliency 2 Saliency W hat have you seen ? 3 Saliency Lighthouse 4 Saliency Lighthouse House 5 Saliency Lighthouse House Rocks 6
SLIDE 2
SLIDE 3
3
What have you seen?
Saliency
SLIDE 4
4
Lighthouse
Saliency
SLIDE 5
5
House
Lighthouse
Saliency
SLIDE 6
6
Rocks
House
Lighthouse
Saliency
SLIDE 7
7
Saliency
SLIDE 8
Saliency Map
Original Image Ground Truth Saliency Map (Eye-Fixation Map) The Goal is to obtain the Saliency Map of an Image. Regression problem, not Classification
SLIDE 9
9
Eye Tracker Mouse Click
Data Bases: Groundtruth generation
SLIDE 10
10
DataBases
Other databases: http://saliency.mit.edu/datasets.html
TRAIN VALIDATION TEST SALICON [Jiang’15] 10,000 5,000 5,000 iSun [Xu’15] 6,000 926 2,000 CAT2000 [Borji’15] 2,000
- 2,000
MIT300 [Judd’12] 300
- Pascal-S
850
SLIDE 11
11
Upsample + filter 2D map
96x96
2340=48x48
Architectures: Junting Net (Shallow Network)
SLIDE 12
Loss function Mean Square Error (MSE) Weight initialization Gaussian distribution Learning rate 0.03 to 0.0001 Mini batch size 128 Training time 7h (SALICON) / 4h (iSUN) Acceleration SGD+ nesterov momentum (0.9) Regularisation Maxout norm GPU NVidia GTX 980
Architectures: Junting Net (Shallow Network)
Shallow and Deep Convolutional Networks for Saliency Prediction Junting Pan, Kevin McGuinness, Elisa Sayrol, Noel O'Connor, Xavier Giro-i-Nieto, CVPR 2016
Winner of the LSUN Challenge 2015!!
SLIDE 13
Architectures: SalNet (Deep Network)
Loss function Mean Square Error (MSE) Weight initialization First 3 layers pre-trained with VGG, the rest of the layers random distribution Learning rate 0,01(halved every 100 iterations) Mini batch size 2 images for 24.000 iterations Training time 15h Acceleration SGD+ nesterov momentum (0.9) Regularisation L2 weight GPU NVidia GTX Titan
Shallow and Deep Convolutional Networks for Saliency Prediction Junting Pan, Kevin McGuinness, Elisa Sayrol, Noel O'Connor, Xavier Giro-i-Nieto, CVPR 2016
SLIDE 14
Quality Results
SLIDE 15
15
Results from CVPR LSUN Challenge 2015 (iSUN Database)
Architectures: Junting Net (Shallow Network) Winner of the LSUN Challenge 2015!!
SLIDE 16
Quantitative Results
Metrics: Saliency and Human Fixations: State-of-the-art and Study of Comparison Metrics Nicolas Riche, Matthieu Duvinage, Matei Mancas, Bernard Gosselin and Thierry Dutoit, iccv 2013
SLIDE 17
Similar to VGG_16
Architectures: Saliency Unified ( Very Deep Network)
Saliency Unified: A Deep Architecture for simultaneous Eye Fixation Prediction and Salient Object Segmentation Srinivas S S Kruthiventi, Vennela Gudisa, Jaley H Dholakiya and R. Venkatesh Babu, CVPR 2016
SLIDE 18
Quantitative Results
Saliency Unified: A Deep Architecture for simultaneous Eye Fixation Prediction and Salient Object Segmentation Srinivas S S Kruthiventi, Vennela Gudisa, Jaley H Dholakiya and R. Venkatesh Babu, CVPR 2016