Introduction to Machine Learning Jia-Bin Huang Virginia Tech - - PowerPoint PPT Presentation

▶
introduction to machine learning
SMART_READER_LITE
LIVE PREVIEW

Introduction to Machine Learning Jia-Bin Huang Virginia Tech - - PowerPoint PPT Presentation

Feb 23, 2023 335 likes •1.04k views

Introduction to Machine Learning Jia-Bin Huang Virginia Tech Spring 2019 ECE-5424G / CS-5824 Todays class Introduction A little about us A little about you Machine learning What is machine learning? Types of machine

slide-1
SLIDE 1

Introduction to Machine Learning

Jia-Bin Huang Virginia Tech

Spring 2019

ECE-5424G / CS-5824

slide-2
SLIDE 2

Today’s class

  • Introduction
  • A little about us
  • A little about you
  • Machine learning
  • What is machine learning?
  • Types of machine learning
  • Example applications
  • Course logistics
slide-3
SLIDE 3

About me

  • Born and raised in Taiwan
slide-4
SLIDE 4

National Chiao-Tung University B.S. in EE UIUC Ph.D. in ECE 2016 Microsoft Research Research Intern Disney Research Research Intern

slide-5
SLIDE 5

National Chiao-Tung University B.S. in EE UIUC Ph.D. in ECE 2016 Microsoft Research Research Intern Disney Research Research Intern

slide-6
SLIDE 6

Im Image Completion [S

[SIGGRAPH14]

  • Revealing unseen pixels
slide-7
SLIDE 7

Vid ideo Completion [S

[SIGGRAPH Asia ia16]

  • Revealing temporally coherent pixels
slide-8
SLIDE 8

Facebook F8 Keynote Talk 2017 Adobe Max 2017

slide-9
SLIDE 9

Im Image super-resolution [C

[CVPR15]

  • Revealing unseen high frequency details
slide-10
SLIDE 10

Detecting migrating birds [CVPR16] Object tracking [ICCV15] Multi-face tracking [ECCV16]

Vis isual Tracking

  • Locatin

ing movin ing objects across vid ideo frames

slide-11
SLIDE 11

Weakly supervised localization [CVPR16] Unsupervised feature learning [ICCV17]

Learning with weak labels

slide-12
SLIDE 12
slide-13
SLIDE 13
slide-14
SLIDE 14
slide-15
SLIDE 15

Teaching Assistant: Chen Gao

  • 1st year PhD student in ECE, VT
  • Email: chengao@vt.edu
  • Web: https://gaochen315.github.io/
  • Office hour:
  • TBD
  • Research:
slide-16
SLIDE 16

Teaching Assistant: Shih-Yang Su

  • 1st year PhD student in ECE, VT
  • Email: chengao@vt.edu
  • Web: https://lemonatsu.github.io/
  • Office hour:
  • TBD
  • Research:
slide-17
SLIDE 17

A little about you

  • Find two persons near you
  • Introduce yourself
  • Name?
  • Department?
  • Why taking this class?
  • One interesting fact?
  • Introduce your neighbors to the class!
slide-18
SLIDE 18

What this course is about? Learning to Teach Machine to Learn

slide-19
SLIDE 19

Let’s chat!

  • What is machine learning?
  • What applications?

Discuss with your neighbor

slide-20
SLIDE 20

What is machine learning?

  • Field of study that

gives computers the ability to learn without being explicitly programmed

Arthur Samuel (1959)

slide-21
SLIDE 21

What is machine learning?

  • A computer program is said to learn from

experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

Tom Mitchell (1998)

slide-22
SLIDE 22

A computer program is said to learn from experie ience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

Designing a spam filter

  • Classifying emails as spam or not spam
  • Watching you label emails as spam or not spam
  • The number (or fraction) of emails correctly

classified as spam/not spam

Slide credit: Andrew Ng

slide-23
SLIDE 23

A computer program is said to learn from experie ience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

Designing a spam filter

  • Classifying emails as spam or not spam

Tasks T

  • Watching you label emails as spam or not spam

Experience E

  • The number (or fraction) of emails correctly

classified as spam/not spam Performance measure P

Slide credit: Andrew Ng

slide-24
SLIDE 24

Types of machine learning algorithms

  • Supervised learning
  • Training data includes desired outputs
  • Unsupervised learning
  • Training data does not include desired outputs
  • Weakly or Semi-supervised learning
  • Training data includes a few desired outputs
  • Reinforcement learning
  • Rewards from sequence of actions

Slide credit: Dhruv Batra

slide-25
SLIDE 25

Machine learning algorithms

Supervised Learning Unsupervised Learning Discrete Classification Clustering Continuous Regression Dimensionality reduction

slide-26
SLIDE 26

Machine learning algorithms

Supervised Learning Unsupervised Learning Discrete Classification Clustering Continuous Regression Dimensionality reduction

slide-27
SLIDE 27

Breast cancer (malignant, benign)

Malignant? 0 (No) 1 (Yes) Tumor Size

Classification problem Discrete valued output e.g., 0 or 1 Multi-class classification e.g., 0 or 1 or 2 or 3

Tumor Size

Slide credit: Andrew Ng

slide-28
SLIDE 28

Multiple features

  • Clump thickness
  • Uniformity of cell size
  • Uniformity of cell shape
  • …

Tumor Size Age

?

Slide credit: Andrew Ng

slide-29
SLIDE 29

Image classification

slide-30
SLIDE 30

Spotting eye disease

  • Recognize 50

sight-threatening eye diseases

  • As accurately as world-

leading expert doctors

Clinically applicable deep learning for diagnosis and referral in retinal disease, Nature Medicine, 2018 https://www.youtube.com/watch?v=MCI0xEGvHx8

slide-31
SLIDE 31

Face recognition

Facebook auto-tagging

slide-32
SLIDE 32

Machine Translation

https://www.youtube.com/watch?v=WeByuOD8k1c

slide-33
SLIDE 33

Speech Recognition

Slide Credit: Carlos Guestrin

slide-34
SLIDE 34

Speech recognition

http://youtu.be/Nu-nlQqFCKg?t=7m30s

slide-35
SLIDE 35

Predicting aftershock patterns

Deep learning of aftershock patterns following large earthquakes, Nature, 2018

Credit: Aflo/REX/Shutterstock

slide-36
SLIDE 36

Machine learning algorithms

Supervised Learning Unsupervised Learning Discrete Classification Clustering Continuous Regression Dimensionality reduction

slide-37
SLIDE 37

Housing price prediction

Price ($) in 1000’s 500 1000 1500 2000 2500 100 200 300 400

Regression problem Continuous valued

  • utput (price)

Size in feet^2

Slide credit: Andrew Ng

slide-38
SLIDE 38

Stock market

Slide credit: Dhruv Batra

slide-39
SLIDE 39

Weather prediction

Temperature

Slide credit: Carlos Guestrin

slide-40
SLIDE 40

Human pose estimation

DensePose, CVPR 2018

slide-41
SLIDE 41

Facial landmark alignment

Snapchat filter https://www.youtube.com/watch?v=Pc2aJxnmzh0

slide-42
SLIDE 42

Machine learning algorithms

Supervised Learning Unsupervised Learning Discrete Classification Clustering Continuous Regression Dimensionality reduction

slide-43
SLIDE 43

Supervised Learning

𝑦1 𝑦2 𝑦1 𝑦2

Unsupervised Learning

slide-44
SLIDE 44

Google news

slide-45
SLIDE 45

Clustering DNA microarray data

build groups of genes with related expression patterns (also known as coexpressed genes)

Source: Su-In Lee et al.

slide-46
SLIDE 46

Slide credit: Andrew Ng

slide-47
SLIDE 47

Machine learning algorithms

Supervised Learning Unsupervised Learning Discrete Classification Clustering Continuous Regression Dimensionality reduction

slide-48
SLIDE 48

Dimensionality reduction

𝑦1 𝑦2

slide-49
SLIDE 49

3D face modeling

A morphable model for the synthesis of 3D faces, SIGGRAPH 1999

slide-50
SLIDE 50

Shape modeling

SMPL: Skinned multi-person linear model, SIGGRAPH Asia 2015

slide-51
SLIDE 51

Cocktail party problem

slide-52
SLIDE 52

Source: https://hbr.org/2016/11/the-competitive-landscape-for-machine-intelligence

slide-53
SLIDE 53

Course Overview

slide-54
SLIDE 54

General information

  • Course title: Advanced Machine Learning
  • Not really… this is an introductory machine learning course
  • ECE-5424 / CS-5824
  • Mon and Wed 2:30 PM – 3:45 PM
  • Surge Space Building 118C
  • Office hours - Jia-Bin
  • Mon 3:45 – 4:45 PM
  • Office hours - Chen, Shih-Yang
  • TBD. Survey on Piazza/Canvas
slide-55
SLIDE 55

Useful links

  • Course webpage: http://bit.ly/vt-machine-learning-spring-2019
  • Download lecture slides
  • Piazza discussion forum: https://piazza.com/class/jr6vbmqyvwy3wk
  • All communications go through piazza. No emails please.
  • HW submission: https://canvas.vt.edu/
  • Start early!
  • Anonymous course feedback:

https://goo.gl/forms/nSz66NogxKXnXLBD2

slide-56
SLIDE 56

Textbooks (optional)

slide-57
SLIDE 57

Course work

  • Homework assignments (50%)
  • Six main homework assignments + HW0
  • Late policy: Up to six free late days. After that, a penalty of 10% per day.
  • Midterm exam (10%)
  • Final exam (15%)
  • Final project (25%)
  • Proposal, project report, and spotlight video
  • Work in a team of 2-3 students

Grading [0-60] F, [60-62] D-, [63-66] D, [67-69] D+, [70-72] C-, [73-76] C, [77-79] C+, [80-82] B-, [83-86] B, [87-89] B+, [90-92] A-, [93-100] A

slide-58
SLIDE 58

Request

  • Homework extension request
  • Only for medical/family emergency (please send me email with doctor’s note)
  • No “I have an interview this week”, “I have a midterm exam”, “I am busy

recently.”

  • Homework regrade request
  • One week after the grade release date
  • Final grading change request
  • No “I need to get an B+ to graduate”, “Can I can a grade upgrade?”
slide-59
SLIDE 59

Academic Integrity

  • Can discuss HW with peers, but cannot copy and/or share code
  • Carefully document any sources within HW hand-in
  • Do not use code from Internet unless you have permission
  • If you’re not sure, ask
  • Do not use your published work as your final project
  • Plagiarism. Zero tolerance. We are required to report it to the university.
slide-60
SLIDE 60

Course enrollment

  • Classroom capacity 140
  • (70 ECE session + 70 CS session)
  • A long waiting list
  • Drop the class if you are not able to commit your time
  • Policy: no force-add students to a full class.
  • Sit in
  • Please leave room for students who registered the class
slide-61
SLIDE 61

Prerequisites

  • Linear algebra, basic calculus
  • Review: http://cs229.stanford.edu/section/cs229-linalg.pdf
  • Probability and statistics
  • Review: https://see.stanford.edu/materials/aimlcs229/cs229-prob.pdf
  • Python (NumPy)
  • http://web.stanford.edu/class/cs224n/readings/python-review.pdf
  • Review: Python review session by TAs
slide-62
SLIDE 62

Course topics

  • Supervised learning
  • Linear regression, logistic regression, SVM, deep neural network, ensemble

methods

  • Unsupervised learning
  • K-means, PCA, EM, GMM
  • Anomaly detection, recommender systems
  • Generative models, sequence predictions, reinforcement learning
slide-63
SLIDE 63

Office Hours

Source: PhD Comics Movie 2

slide-64
SLIDE 64

What to expect from this course

  • Broad coverage
  • Focus is on the fundamental, rather than specific systems.
  • Not about teaching you to use toolbox
  • Background to delve deeper into any machine learning related topics
  • Practical experience
  • Lots of work, tough material, fast pace, but lots of learning too!
slide-65
SLIDE 65
slide-66
SLIDE 66

Other related courses at Virginia Tech

  • Introductory courses:
  • Introduction to Machine Learning
  • Introduction to Artificial Intelligence
  • Computer Graphics
  • Advanced courses:
  • Deep Learning
  • Probabilistic Graphical Models and Large-Scale Learning
  • Advanced Computer Vision
  • Fundamentals:
  • ECE 5734 Convex Optimization
  • STAT 5444 Bayesian Statistics
  • STAT 4714 Prob and Stat for EE
slide-67
SLIDE 67

Goals and Expectations

  • My goal:
  • maximize the learning effectiveness of your time
  • What I expect from you
  • Attend and participate, when possible
  • No screens please (tablet, phone, laptop, etc)
  • Start assignments well before deadline
  • Tell me what’s working and suggest improvements

Anonymous feedback form

slide-68
SLIDE 68

Things to remember

  • Machine learning is awesome!
  • To-Do
  • Check out the review material

(linear algebra, probability, Python)

  • Start working on HW 0
  • Next class: k-NN classifier
  • Questions?

Supervised Learning Unsupervised Learning Discrete Classification Clustering Continuou s Regression Dimensionality reduction