CSCI 4520: Introduction to Machine Learning Spring 2020 Mehdi - - PowerPoint PPT Presentation

CSCI 4520: Introduction to Machine Learning

Mehdi Allahyari Georgia Southern University

(some slides from Pedro Domingos, University of Washington)

Spring 2020

1

Mehdi Allahyari

n Call me Dr. Yari n PhD from UGA n Machine Learning, Semantic Web, Topic Modeling,

Knowledge Graph

2

Course Information

3

CSCI 4520 - Introduction to Machine Learning

Spring 2020 – CRN: 17186

Instructor: Dr. Mehdi Allahyari <mallahyari@georgiasouthern.edu> Office: CEIT 2321 Course Webpage: https://sci2lab.github.io/mehdi/teaching/cs4520 Lecture Times Tuesday Thursday CRN 17186 12:30 PM – 01:45 PM 12:30 PM – 01:45 PM CEIT 2207 CEIT 2207 Office Hours Monday Wednesday 11:00 AM – 12:00 PM 11:00 AM – 12:00 PM

• r by appointment

Tuesday Thursday 11:00 AM – 12:00 PM 11:00 AM – 12:00 PM CEIT 2321 CEIT 2321

Text Books ( Highly Recommended)

4

Other Great Text Books

5

Resources

n Course page: https://sci2lab.github.io/mehdi/teaching/cs4520/

n is where you get course content (syllabus, lectures, etc.)

n Folio:

n Folio is the website where you will submit all assignments (if

• nline), see your grades

6

Course Description

n This introductory course is designed:

n to give upper level undergraduate a broad overview of many

concepts and algorithms in ML.

n covers the theory and practical algorithms for machine learning

from a variety of perspectives.

n Topics:

n Statistical and probabilistic methods, generative and

discriminative models, linear and logistic regression, decision tree learning, unsupervised learning and clustering and dimensionality reduction.

n In addition, the course covers fundamental concepts such as

training, validation, overfitting, and error rates

7

Prerequisites

n Basics knowledge of probability, linear algebra and

calculus.

n For example, standard probability distributions and also how to

calculate derivatives.

n Significance programming experience

n Python is the language we’ll use

8

Grading

n Attendance and participation: 5% n Homeworks: 20%

n Four homeworks: submitted hardcopy and on Folio n Some exercises, significant programming

n Midterm 03/05/2020 20% n Final exam 05/05/2020 12:30 – 2:30 PM. 25% n Final project: 30%

n Teams of 2 n Define a task, create/acquire data for the task, train ML algorithm(s),

evaluate & report

n Check the course website for details

9

A ≥ 85 85 > B ≥ 80 80 > C ≥ 70 70 > D ≥ 65 F < 65

Homeworks Deadlines

n Assignments are usually due 10-14 days (see

course schedule for deadlines)

n Late submissions for labs/projects are penalized

n 0-24 hours late

• 10 points

n 24-48 hours late

• 20 points

n >48 hours late

n Forgetting to submit is not a valid excuse

n Double check your submission n Save your confirmation email

10

Collaboration Policy & Academic Honesty

n Homeworks to be done individually

n Your own answer and code

n can collaborate in figuring out answers and helping each

• ther solve the problems at high level

n No sharing of pseudocode or code or simulation results n Academic Honesty:

n abide by the University’s academic honesty policy and the

Student Honor Code

n Lack of knowledge of the academic honesty policy is not a

reasonable explanation for a violation

11

Syllabus Questions?

n Familiarize yourself with the syllabus n This PowerPoint highlights key information from the

syllabus but does not cover every subject in detail

n You are responsible for the entirety of the syllabus

n Your HW is to go home & read the syllabus thoroughly

12

A Few Quotes

n “A breakthrough in machine learning would be worth

ten Microsofts” (Bill Gates, Founder, Microsoft)

n “Machine learning is the next Internet”

(Tony Tether, Director, DARPA)

n Machine learning is the hot new thing”

(John Hennessy, President, Stanford)

n “Machine learning is Google’s top priority”

(Eric Schmidt, Chairman, Alphabet)

n “Machine learning is Microsoft Research’s largest investment area” (Peter Lee, Head,

Microsoft Research)

n “‘Data scientist’ is the hottest job title in Silicon Valley”

(Tim O’Reilly, Founder, O’Reilly Media)

13

What is Machine Learning?

n How can we solve a specific problem? n As computer scientists we write a program that encodes a set of

rules that are useful to solve the problem

n In many cases is very difficult to specify those rules, e.g., given a

picture determine whether there is a cat in the image

14

What is Machine Learning?

n Learning systems are not directly programmed to solve a

problem, instead develop own program based on:

n Examples of how they should behave n From trial-and-error experience trying to solve the problem

n Different than standard CS n Want to implement unknown function, only have access e.g.,

to sample input-output pairs (training examples)

n Learning simply means incorporating information from the

training examples into the system

15

What is Machine Learning?

n “The study of computer programs thatimprove automatically

with experience”

n T. Mitchell Machine Learning n Getting computers to program themselves n Writing software is the bottleneck, Let the data do the work

instead!

16

Traditional Programming Machine Learning

Computer Input Program Output Computer Program Input Output

17

Magic?

No, more likegardening

} Seeds = Algorithms } Nutrients = Data } Gardener = Y

• u

} Plants = Programs

18

Sample Applications

n Web search n Computational biology n Finance n E-commerce n Space exploration n Robotics n Information extraction n Social networks n Debugging n [Your favorite area]

19

Tasks that requires machine learning: What makes a 2?

20

Why use machine learning?

n It is very hard to write programs that solve problems like

recognizing a handwritten digit

n What distinguishes a 2 from a 7? n How does our brain do it?

n Instead of writing a program by hand, we collect examples that

specify the correct output for a given input

n A machine learning algorithm then takes these examples and

produces a program that does the job

n The program produced by the learning algorithm may look very

different from a typical hand-written program. It may contain millions of numbers.

n If we do it right, the program works for new cases as well as the

• nes we trained it on.

21

Relationship of Machine Learning to…

n Statistics n Analytics / Data Science n Data Mining n Artificial Intelligence

22

ML in Practice

n Understanding domain, prior knowledge, and goals n Data integration, selection, cleaning n pre-processing, etc. n Learning models n Interpreting results n Consolidating and deploying discovered knowledge n Loop

23

ML in Nutshell

n Hundreds of machine learning algorithms n Tens new every year n Every machine learning algorithm has three components:

n Representation n Evaluation n Optimization

24

Representation

n How do we represent the function from input to

• utput?

n Decision trees n Sets of rules / Logic programs n Graphical models (Bayes/Markov nets) n Neural networks n Support vector machines n Model ensembles n Etc.

25

Evaluation

n Given some data, how can we tell if a function is

“good”?

n Accuracy n Precision and recall n Squared error n Likelihood n Posterior probability n Cost / Utility n Margin n Entropy n K-L divergence n Etc.

26

Optimization

n Given some data, how do we find the “best”

function?

n Combinatorial optimization

n E.g.: Greedy search

n Convex optimization

n E.g.: Gradient descent

n Constrained optimization

n E.g.: Linear programming

27

Learning algorithms are useful in many tasks

• 1. Classification: Determine which discrete category

the example is

28

Examples of Classification

What digit is this?

29

Examples of Classification

Is this a dog?

30

Examples of Classification

What about this one?

31

Examples of Classification

Am I going to pass the exam?

32

Examples of Classification

Do I have diabetes?

33

Learning algorithms are useful in many tasks

• 1. Classification: Determine which discrete category

the example is

• 2. Recognizing patterns: Speech Recognition, facial

identity, etc

34

Examples of Recognizing patterns

Figure: Siri: https://www.youtube.com/watch?v=8ciagGASro0

35

Examples of Recognizing patterns

Figure: Photomath: https://photomath.net/

36

Learning algorithms are useful in many tasks

• 1. Classification: Determine which discrete category

the example is

• 2. Recognizing patterns: Speech Recognition, facial

identity, etc

3.

Recommender Systems: Noisy data, commercial pay-off (e.g., Amazon, Netflix).

37

Examples of Recommendation systems

38

Examples of Recommendation systems

39

Examples of Information Retrieval

40

Examples of Information Retrieval

41

Examples of Information Retrieval

42

Learning algorithms are useful in many tasks

• 1. Classification: Determine which discrete category

the example is

• 2. Recognizing patterns: Speech Recognition, facial

identity, etc

3.

Recommender Systems: Noisy data, commercial pay-off (e.g., Amazon, Netflix).

• 4. Information retrieval: Find documents or images

with similar content

• 5. Computer vision: detection, segmentation, depth

estimation, optical flow, etc

43

Examples of Computer Vision

44

https://blog.insightdatascience.com/generating-custom-photo-realistic-faces-using-ai-d170b1b59255

Examples of Computer Vision

https://www.youtube.com/watch?v=CrnLINlbfFA Fig: https://i.ytimg.com/vi/CrnLINlbfFA/hqdefault.jpg

45

Examples of Computer Vision

[Gatys, Ecker, Bethge. A Neural Algorithm of Artistic Style. Arxiv’15.]

46

Learning algorithms are useful in many tasks

• 1. Classification: Determine which discrete category

the example is

• 2. Recognizing patterns: Speech Recognition, facial

identity, etc

3.

Recommender Systems: Noisy data, commercial pay-off (e.g., Amazon, Netflix).

• 4. Information retrieval: Find documents or images

with similar content

• 5. Computer vision: detection, segmentation, depth

estimation, optical flow, etc

• 6. Robotics: perception, planning, etc

47

Autonomous Driving

48

Flying Robots

Figure: Video: https://www.youtube.com/watch?v=YQIMGV5vtd4

49

Learning algorithms are useful in many tasks

1.

Classification: Determine which discrete category the example is

2.

Recognizing patterns: Speech Recognition, facial identity, etc

3.

Recommender Systems: Noisy data, commercial pay-off (e.g., Amazon, Netflix).

4.

Information retrieval: Find documents or images with similar content

5.

Computer vision: detection, segmentation, depth estimation,

• ptical flow, etc

6.

Robotics: perception, planning, etc

7.

Learning to play games

50

Playing Games: Atari

Figure: Video: https://www.youtube.com/watch?v=V1eYniJ0Rnk

51

Learning algorithms are useful in many tasks

1.

Classification: Determine which discrete category the example is

2.

Recognizing patterns: Speech Recognition, facial identity, etc

3.

Recommender Systems: Noisy data, commercial pay-off (e.g., Amazon, Netflix).

4.

Information retrieval: Find documents or images with similar content

5.

Computer vision: detection, segmentation, depth estimation, optical flow, etc

6.

Robotics: perception, planning, etc

7.

Learning to play games

8.

Recognizing anomalies: Unusual sequences of credit card transactions, panic situation at an airport

9.

Spam filtering, fraud detection: The enemy adapts so we must adapt too

10.

Many more!

52

Types of Learning

n Supervised (inductive) learning

n Training data includes desired outputs

n Unsupervised learning

n Training data does not include desired outputs

n Semi-supervised learning

n Training data includes a few desired outputs

n Reinforcement learning

n Rewards from sequence of actions

53

Supervised Learning

n Given examples of a function (X, F(X)) n Predict function F(X) for new examples X

n Discrete F(X): Classification (1-of-N output (speech

recognition, object recognition, medical diagnosis)

n Continuous F(X): Regression (real-valued output (predicting

market prices, customer rating)

n F(X) = Probability(X): Probability estimation

54

Unsupervised Learning

n Create an internal representation of the input,

capturing regularities/structure in data

n Example:

n Clustering: Discover groups of similar inputs

(documents, images, etc)

55

What We’ll Cover

n Supervised learning

n Decision tree induction n Instance-based learning n Linear Regression n Bayesian learning n Logistic Regression n Neural networks

n Unsupervised learning

n Clustering n Dimensionality reduction

56