SLIDE 1
Deep Networks for Computer Vision at Google
Chuck Rosenberg
ImageNet ILSVRC Workshop September 12, 2014
Deep Networks for Computer Vision at Google Chuck Rosenberg - - PowerPoint PPT Presentation
Deep Networks for Computer Vision at Google Chuck Rosenberg - - PowerPoint PPT Presentation
Deep Networks for Computer Vision at Google Chuck Rosenberg ImageNet ILSVRC Workshop September 12, 2014 Quick Intro Private Photo Search and Public Image Search Teams Google Photos Our work: Pixels Knowledge Search by Image
SLIDE 2
SLIDE 3
Applications
Google Image Search Search by Image
SLIDE 4
Applications - Photo Search
SLIDE 5
Applications - Auto Curation
SLIDE 6
Google Photos - Auto Awesome
SLIDE 7
More Image Understanding at Google
YouTube Google Shopping StreetView / Maps Self-Driving Cars Robotics Much more... Advertising
SLIDE 8
Understanding is about extracting Knowledge
SLIDE 9
Image Understanding: Pixels → Entities
Single-word entities go way beyond simple objects! My photos of …
- bjects: “dog”
fine-grained objects: “Husky” scenes: “beach”, “sunset” actions: “kitesurfing” ,“kiss” emotions: “happiness”, “laughter” events: “birthday”, “basketball game” abstract concepts: “love”, “zen”
SLIDE 10
Deep Neural Network
The Deep and now Deeper Hammer
Pixels Target output
“ImageNet Classification with Deep Convolutional Neural Networks”, Krizhevsky, Sutskever, Hinton, NIPS 2012
Deep learning infrastructure by the Google Brain team
SLIDE 11
Personal Photos - Example Annotations
Christmas tree Red Christmas decoration Christmas Crowd Cheering People Stadium Play Meal Cake Child Hummingbird Macro photography Reflection Red
SLIDE 12
More Example Network Annotations
SLIDE 13
More Example Network Annotations
SLIDE 14
Google Network Stats
- Training Data
○ ImageNet 1K ~1M images ○ X-Net 100’s of millions of images
- Label Set
○ Image 1K ○ X-Net ~10’s of thousands of labels
- Ground Truth Issues
○ Incomplete Training Data ○ Noisy Training Data
SLIDE 15
Challenge: Incomplete label ground-truth
Problem increasingly serious as we add more types of entities and fine- grained categories: “Airedale Terrier” but not “Terrier” “Dog” “Animal” or “Pet” “Cute” or “Curb” “Grass” “Street” ...
SLIDE 16
Challenge: Noisy data
“Tortoise” “Tortoise Shell Sunglasses” “Random noise”
SLIDE 17
Image Understanding: Localization
dog human running
Mountain sky grass road
- bject detection
scene parsing pose estimation
SLIDE 18
Sample detections
ImageNet Pascal VOC
SLIDE 19
Google Confidential and Proprietary
Training Embeddings Using Triplets
- Training data consists of triplets: an anchor image, positive image, and
negative image.
- Loss function:
... Deep Neural Net L2 Triplet Loss
E M B E D D I N G
Anchor Positive Negative
[1] “Learning Fine-grained Image Similarity with Deep Ranking”, Wang, Song, Leung, Rosenberg, Wang, Philbin, Chen, Wu, CVPR 2014
[1] Triplets
SLIDE 20
Google Confidential and Proprietary
Embedding Results
SLIDE 21
Google Confidential and Proprietary
Embedding Results
SLIDE 22
Google Confidential and Proprietary
Embedding Results
SLIDE 23
Google Confidential and Proprietary
SLIDE 24
Some Take Aways
What Works
- ImageNet - of course! =)
- More data leads to better performance
- Deeper and bigger networks lead to better performance
- Networks handle many diverse problems very well
What Needs Work
- More insight into the “Black Box” - diagnosis and understanding
- Understand and improve and training data efficiency
- Efficient means of collecting more training data
- Better ways to deal with noisy training data
SLIDE 25
Thanks to the teams...
- Image Understanding Team
- Google Photos Team
- Google Brain Team
- Google Research
- Our great interns
And We’re Hiring! I’m: chuck@google.com
SLIDE 26