Deep Learning in the Connected Kitchen or Launching a Computer - PowerPoint PPT Presentation

Deep Learning in the Connected Kitchen or “Launching a Computer Vision program in a new vertical” Hristo Bojinov, CTO

Company Vision

The Problem Food ↔ People disconnect Not-so-smart “smart kitchen” Food info not available, not actionable

What We Do Food, personalization, technology “Give food a voice” ( ⇒ Computer Vision is essential) Icons made by Madebyoliver, Popcorn Arts, Freepik from www.flaticon.com are licensed by CC 3.0 BY

Computer Vision at Innit Helps us understand users ❖ Inventory, behaviors, multi-sensor fusion, market analytics ❖ And, build a delightful user experience Applications in storage and processing ❖ Recognize and act on food state ❖ Visible light, depth, IR

Program Logistics Multi-site program (HQ, academia) Food Recognition service (AWS) ❖ G2 instance backend (blend of CPU and GPU workload) ❖ Frontend orchestrates auto and manual processing ❖ Service API for 3rd party use

CV Tech: Food Recognition System

CV Tech: Food Recognition System Data is King!

CV Tech: Object Detection Stage

CV Tech: Object Detection Stage DetectNet ➔ Easy setup and initial training ➔ Python layers, “low resolution” Faster-RCNN ➔ Multi-phase training/tuning ➔ High resolution & recall 😁 DeepMask & SharpMask

CV Tech: Object Detection Stage

CV Tech: Classification Stage

CV Tech: Classification Stage Controlled scene layout ⇒ precision In-house data collection and tools Command-line → DIGITS AlexNet → VGG

CV Tech: Product DB Image Retrieval

CV Tech: Product DB Image Retrieval ❖ Exact product (or attribute) matching ❖ KAZE descriptors (GPU acceleration WIP) ➢ Current need to balance CPU/GPU ➢ Order-of-magnitude acceleration ❖ Hierarchical analysis in the pipeline

CV Research: Training on Synthetic Sets

CV Research: Text Extraction

In a nutshell... ❖ Focus on differentiated capabilities, in the food space ❖ Tie in with all stages of human ↔ food interaction ❖ Fusion of images & other “sensors” ❖ GPU tech a strong enabler

Takeaways ❖ Objectives → domain constraints ( good! ) ❖ Sources of initial training+test data; build tools ❖ Hardware (local experiments OK, cloud for serving) ❖ Software (don’t get tied to a framework; abstract away)

We are hiring! 🚁 hristo@innit.com

About Innit ❖ Inform and elevate the interaction between people and food ❖ 4+ years in the making, substantial funding, IP & tech ❖ Pirch SOHO, ShopWell About the Speaker ❖ Embedded & Security ❖ Android, Computer Vision ❖ Computer technology at Innit

Deep Learning in the Connected Kitchen or Launching a Computer - PowerPoint PPT Presentation

Deep Learning in the Connected Kitchen or Launching a Computer Vision program in a new vertical Hristo Bojinov, CTO Company Vision The Problem Food People disconnect Not-so-smart smart kitchen Food info not available, not

On Connected Sublevel Sets in Deep Learning Quynh Nguyen Department of Mathematics and Computer

Better Living through Deep Learning and Edge Computing Peggy James Smart cities and big data

MDF and Oak Kitchen 1 Constructed from MDF with solid oak worktops MDF and Oak Kitchen 2 Kitchen

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: State of the Art (2020) Deep Learning Lecture Series https://deeplearning.mit.edu

MAR ARKET KET POTENT TENTIAL IAL FOR GE GENE NE EDI DITING TING Dr. Roger Johnson RCJ

Megan Lamson, Hawaii Wildlife Fund May 2017 megan@wildhawaii.org Over 242 tons of marine

Digital Communication Solutions C a b l e a n d C o n n e c t i v i t y C o m p a n y

Leadership Dennis Savaiano, Purdue University Director and Research Coordinator Marci

Radisson Blu Royal Hotel Brussels Collaboration 2019 How to get to Brussels? Brussels, the

College of Engineering & Applied Science Strategic Planning Information Session EAS

Ownership Problems 23.1 Million parcel in Cadaster Record 32,5 Million parcel in

VIGC EVOLUTIONS IN FOOD PACKAGING PRINTING Brussels, Belgium November 18, 2015 LOW MIGRATION

Deep Learning in the Connected Kitchen or Launching a Computer - PowerPoint PPT Presentation

Deep Learning in the Connected Kitchen or Launching a Computer Vision program in a new vertical Hristo Bojinov, CTO Company Vision The Problem Food People disconnect Not-so-smart smart kitchen Food info not available, not

On Connected Sublevel Sets in Deep Learning Quynh Nguyen Department of Mathematics and Computer

Better Living through Deep Learning and Edge Computing Peggy James Smart cities and big data

MDF and Oak Kitchen 1 Constructed from MDF with solid oak worktops MDF and Oak Kitchen 2 Kitchen

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: State of the Art (2020) Deep Learning Lecture Series https://deeplearning.mit.edu

MAR ARKET KET POTENT TENTIAL IAL FOR GE GENE NE EDI DITING TING Dr. Roger Johnson RCJ

Megan Lamson, Hawaii Wildlife Fund May 2017 megan@wildhawaii.org Over 242 tons of marine

Digital Communication Solutions C a b l e a n d C o n n e c t i v i t y C o m p a n y

Leadership Dennis Savaiano, Purdue University Director and Research Coordinator Marci

Radisson Blu Royal Hotel Brussels Collaboration 2019 How to get to Brussels? Brussels, the

College of Engineering &amp; Applied Science Strategic Planning Information Session EAS

Ownership Problems 23.1 Million parcel in Cadaster Record 32,5 Million parcel in

VIGC EVOLUTIONS IN FOOD PACKAGING PRINTING Brussels, Belgium November 18, 2015 LOW MIGRATION

College of Engineering & Applied Science Strategic Planning Information Session EAS