Why is Computer Vision on a Mobile Device Different? Instructor - - - PowerPoint PPT Presentation

Why is Computer Vision

• n a Mobile Device

Different?

Instructor - Simon Lucey

16-623 - Designing Computer Vision Apps

Source: http://www.slashgear.com/iphone-7-potential-wanes-as-android-n-starts-to-tango-20440932/

Today

• Course Logistics
• Philosophy to Mobile Computer Vision R&D

About this Course

• Team
• Office hours: by appointment (use Piazza).
• 16623.courses.cs.cmu.edu has ALL information.
• Questions: Please use Piazza.
• Finding a project partner: Please use Piazza.

Me

(Instructor) (TA)

Eric Huang

Assignments

• There will be 4 assignments - (5 + 15 + 15 + 15)%
• Each assignment will relate to the topics of the previous

lectures, but ALSO take us closer to the task of building our OWN augmented reality app.

• Assignment 0 will be released on Thursday September 1st.
• Assignment 0 is due Friday September 14th.
• See course website (16623.courses.cs.cmu.edu) for full

schedule.

Assignments

• Goal is that every assignment takes you a step closer to

building your OWN augmented reality app.

• Assignments are designed to take us (step-by-step) towards

an augmented reality app.

Final Project

• Final project is worth 50% of final grade.
• 5% for project proposal.
• 45% for final project report and presentation.
• Teams 1-2 (if it is something big we could discuss 3).
• Topic: efficient or novel implementation of CV algorithm on a

mobile device.

• See 16623.courses.cs.cmu.edu/ideas for project ideas.
• Until November 6th,
• think about a topic
• find a partner.

Project Ideas

See more ideas at 16623.courses.cs.cmu.edu/ideas

Background Material

• Most other parts of course cannot be

found in books.

• I post all slides, and notes in the course
• n the course website.
• If you are completely new to OpenCV and

Xcode you should consider getting this book too (link to Amazon.)

• Good beginners guide to using OpenCV in

Xcode, so you can build up additional experience during the course.

Resources

• You will need access to a MAC.
• If you do not have a MAC, do not panic CMU has ample

MAC clusters on campus.

• See:- https://www.cmu.edu/computing/clusters/facilities/index.html
• We have iPADs for everyone in the class so that is cool

(yay!!!) so everyone should have an iOS device.

If you have a MAC

• Please ensure your MAC has the latest version of El Capitan.
• Please ensure your iOS device has the latest version 9.3.
• This will make life easy for you (less headaches for me).

Class Participation

• I’ll start on time.
• It is important to attend.
• I will use part slides, part tutorial, part on board.
• Do ask questions.
• Use Piazza or come to my office (by appointment only).

see 16623.courses.cs.cmu.edu

see 16623.courses.cs.cmu.edu

see 16623.courses.cs.cmu.edu

Today

• Course Logistics
• Philosophy to Mobile Computer Vision R&D

OpenCV MATLAB

Why is Mobile CV Different?

22

Why is Mobile CV Different?

23

Connection to Robotics…..

24

Balancing Power versus Perception

25

Balancing Power versus Perception

26

Low Power Image Recognition Challenge

27

Taken from: http://lpirc.net/

Low Power Image Recognition Challenge

• “Many mobile systems (smartphones, electronic glass,

autonomous robots) can capture images. These systems use batteries and energy conservation is essential. This challenge aims to discover the best technology in both image recognition and energy conservation. Winners will be evaluated based on both high recognition accuracy and low power usage.”

28

?

Moore’s Law

30

Moore’s Law is Dead??

31

Source: “Moore's Law Is Dead! (But Not In Mobile)” ReadWrite, April 2015.

Not in Mobile!!!

32

Source: “Moore's Law Is Dead! (But Not In Mobile)” ReadWrite, April 2015.

2010 2014

Ideal Von Neumann Processor

• each cycle, CPU takes data from registers, does an
• peration, and puts the result back
• load/store operations (memory ←→ registers) also take one

cycle

• CPU can do different operations each cycle output of one
• peration can be input to next
• CPU’s haven’t been this simple for a long time!

34

✲

time

• p1

✲ ✲ ✲

• p2

✲ ✲ ✲

• p3

✲ ✲ ✲

Taken from http://people.maths.ox.ac.uk/gilesm/cuda/lecs/lec0.pdf

CPU clock is stuck!!!!

• CPU clock stuck at about 3GHz since 2006 due to high

power consumption (up to 130W per chip)

• chip circuitry still doubling every 18-24 months
• ⇒ more on-chip memory and MMU (memory management

units)

• ⇒ specialised hardware (e.g. multimedia, encryption) ⇒

multi-core (multiple CPU’s on one chip)

• peak performance of chip still doubling every 18-24 months

35

Taken from http://people.maths.ox.ac.uk/gilesm/cuda/lecs/lec0.pdf

ASICs for Low Energy

• Application Specific Integrated Circuits (ASIC)
• ASICs are perfect for targeting a specific application domain.
• Inherently low-power as they are “frozen in silicon” for a

specific application domain (e.g. graphics cards, ethernet cards, DSPs, etc.).

• Drawbacks,
• incredibly expensive to develop.
• time consuming and resource-intensive to develop.
• Positives,
• Extremely energy efficient.

36

Example: Adding Numbers 1 - 10

37

1 + 1 = 2 1 + 2 = 3

………

10 + 10 = 20

System on a Chip (SoC)

• SoCs attempt to find balance between energy and

programmability.

• Designed with emphasis on low power consumption.
• SOC shares the same system bus with CPU, GPU and DSP.
• Therefore has much lower memory bandwidth.
• Useful for computer vision algorithm design as one can

switch between CPU and GPU with little memory overhead.

• Not possible on conventional architecture.
• More on this later…..

38

(Taken from K. Cheng, Y. Wang “Using Mobile GPU for General-Purpose Computing – A Case Study of Face Recognition on Smartphones”)

Battle of Two Platforms

39