Why is Computer Vision on a Mobile Device Different? Instructor - Simon Lucey 16-623 - Designing Computer Vision Apps
Source: http://www.slashgear.com/iphone-7-potential-wanes-as-android-n-starts-to-tango-20440932/
Today • Course Logistics • Philosophy to Mobile Computer Vision R&D
About this Course • Team Me Eric Huang (Instructor) (TA) • Office hours: by appointment (use Piazza). • 16623.courses.cs.cmu.edu has ALL information. • Questions: Please use Piazza. • Finding a project partner: Please use Piazza.
Assignments • There will be 4 assignments - (5 + 15 + 15 + 15)% • Each assignment will relate to the topics of the previous lectures, but ALSO take us closer to the task of building our OWN augmented reality app. • Assignment 0 will be released on Thursday September 1st. • Assignment 0 is due Friday September 14th. • See course website (16623.courses.cs.cmu.edu) for full schedule.
Assignments • Goal is that every assignment takes you a step closer to building your OWN augmented reality app. • Assignments are designed to take us (step-by-step) towards an augmented reality app.
Final Project • Final project is worth 50% of final grade. • 5% for project proposal. • 45% for final project report and presentation. • Teams 1-2 (if it is something big we could discuss 3). • Topic: efficient or novel implementation of CV algorithm on a mobile device. • See 16623.courses.cs.cmu.edu/ideas for project ideas. • Until November 6th, • think about a topic • find a partner.
Project Ideas See more ideas at 16623.courses.cs.cmu.edu/ideas
Background Material • Most other parts of course cannot be found in books. • I post all slides, and notes in the course on the course website. • If you are completely new to OpenCV and Xcode you should consider getting this book too (link to Amazon.) • Good beginners guide to using OpenCV in Xcode, so you can build up additional experience during the course.
Resources • You will need access to a MAC. • If you do not have a MAC, do not panic CMU has ample MAC clusters on campus. • See:- https://www.cmu.edu/computing/clusters/facilities/index.html • We have iPADs for everyone in the class so that is cool (yay!!!) so everyone should have an iOS device.
If you have a MAC • Please ensure your MAC has the latest version of El Capitan. • Please ensure your iOS device has the latest version 9.3. • This will make life easy for you (less headaches for me).
Class Participation • I’ll start on time. • It is important to attend. • I will use part slides, part tutorial, part on board. • Do ask questions. • Use Piazza or come to my office (by appointment only).
see 16623.courses.cs.cmu.edu
see 16623.courses.cs.cmu.edu
see 16623.courses.cs.cmu.edu
Today • Course Logistics • Philosophy to Mobile Computer Vision R&D
MATLAB OpenCV
Why is Mobile CV Different? 22
Why is Mobile CV Different? 23
Connection to Robotics….. 24
Balancing Power versus Perception 25
Balancing Power versus Perception 26
Low Power Image Recognition Challenge 27 Taken from: http://lpirc.net/
Low Power Image Recognition Challenge • “Many mobile systems (smartphones, electronic glass, autonomous robots) can capture images. These systems use batteries and energy conservation is essential. This challenge aims to discover the best technology in both image recognition and energy conservation. Winners will be evaluated based on both high recognition accuracy and low power usage.” 28
?
Moore’s Law 30
Moore’s Law is Dead?? 31 Source: “Moore's Law Is Dead! (But Not In Mobile)” ReadWrite, April 2015.
Not in Mobile!!! 32 Source: “Moore's Law Is Dead! (But Not In Mobile)” ReadWrite, April 2015.
2014 2010
Ideal Von Neumann Processor • each cycle, CPU takes data from registers, does an operation, and puts the result back • load/store operations (memory ←→ registers) also take one cycle • CPU can do different operations each cycle output of one operation can be input to next ✲ time op1 ✲ ✲ ✲ op2 ✲ ✲ ✲ op3 ✲ ✲ ✲ • CPU’s haven’t been this simple for a long time! 34 Taken from http://people.maths.ox.ac.uk/gilesm/cuda/lecs/lec0.pdf
CPU clock is stuck!!!! • CPU clock stuck at about 3GHz since 2006 due to high power consumption (up to 130W per chip) • chip circuitry still doubling every 18-24 months • ⇒ more on-chip memory and MMU (memory management units) • ⇒ specialised hardware (e.g. multimedia, encryption) ⇒ multi-core (multiple CPU’s on one chip) • peak performance of chip still doubling every 18-24 months 35 Taken from http://people.maths.ox.ac.uk/gilesm/cuda/lecs/lec0.pdf
ASICs for Low Energy • Application Specific Integrated Circuits (ASIC) • ASICs are perfect for targeting a specific application domain. • Inherently low-power as they are “frozen in silicon” for a specific application domain (e.g. graphics cards, ethernet cards, DSPs, etc.). • Drawbacks, • incredibly expensive to develop. • time consuming and resource-intensive to develop. • Positives, • Extremely energy efficient. 36
Example: Adding Numbers 1 - 10 1 + 1 = 2 1 + 2 = 3 ……… 10 + 10 = 20 37
System on a Chip (SoC) • SoCs attempt to find balance between energy and programmability. • Designed with emphasis on low power consumption. • SOC shares the same system bus with CPU, GPU and DSP. • Therefore has much lower memory bandwidth. • Useful for computer vision algorithm design as one can switch between CPU and GPU with little memory overhead. • Not possible on conventional architecture. • More on this later….. (Taken from K. Cheng, Y. Wang “Using Mobile GPU for General-Purpose Computing – A Case Study of Face Recognition on Smartphones”) 38
Battle of Two Platforms 39
Recommend
More recommend
