Department of Computer Science CSCI 5622: Machine Learning Chenhao - - PowerPoint PPT Presentation

▶
department of computer science csci 5622 machine learning
SMART_READER_LITE
LIVE PREVIEW

Department of Computer Science CSCI 5622: Machine Learning Chenhao - - PowerPoint PPT Presentation

Jan 16, 2023 299 likes •850 views

Department of Computer Science CSCI 5622: Machine Learning Chenhao Tan Lecture 14: PAC learnability Slides adapted from Jordan Boyd-Graber, Chris Ketelsen 1 Announcements Proposal due tomorrow night HW2 regrade requests Peer

slide-1
SLIDE 1

Department of Computer Science CSCI 5622: Machine Learning Chenhao Tan Lecture 14: PAC learnability Slides adapted from Jordan Boyd-Graber, Chris Ketelsen

1

slide-2
SLIDE 2

Announcements

  • Proposal due tomorrow night
  • HW2 regrade requests
  • Peer feedback assignment
  • Midterm example
  • HW4 due in a month

2

slide-3
SLIDE 3

Learning objectives

  • Learn about basics of learning theory
  • Prove some simple bounds on errors and sample sizes
  • Gain some intuition about complexity and overfitting

3

slide-4
SLIDE 4

Simple Example

4

slide-5
SLIDE 5

Simple Example

5

slide-6
SLIDE 6

Simple Example

6

slide-7
SLIDE 7

PAC Learnability

7

slide-8
SLIDE 8

PAC Learnability

8

slide-9
SLIDE 9

PAC Learnability

9

slide-10
SLIDE 10

PAC Learnability

10

slide-11
SLIDE 11

PAC Learnability

11

slide-12
SLIDE 12

PAC Learnability

12

slide-13
SLIDE 13

PAC Learnability

13

slide-14
SLIDE 14

Alien Example

14

slide-15
SLIDE 15

Alien Example

15

slide-16
SLIDE 16

Alien Example

16

slide-17
SLIDE 17

Alien Example

17

slide-18
SLIDE 18

Alien Example

18

slide-19
SLIDE 19

Alien Example

19

slide-20
SLIDE 20

Alien Example

20

slide-21
SLIDE 21

Alien Example

21

slide-22
SLIDE 22

Alien Example

22

slide-23
SLIDE 23

Alien Example

23

slide-24
SLIDE 24

Alien Example

24

slide-25
SLIDE 25

Alien Example

25

slide-26
SLIDE 26

Alien Example

26

slide-27
SLIDE 27

Alien Example

27

slide-28
SLIDE 28

Alien Example

28

slide-29
SLIDE 29

General case, Finite Hypothesis class

29

  • OK, so we saw an example proving PAC learnability for a

specific problem with specific hypothesis and specific algorithm

  • Can we be more general than this?
  • Yes!
  • Today, H is finite
  • Next time, H is infinite
  • Distinction
  • H is finite and c is in H
  • H is finite and c is not in H
slide-30
SLIDE 30

Finite Consistent Hypothesis Class

30

slide-31
SLIDE 31

Finite Consistent Hypothesis Class

31

slide-32
SLIDE 32

Finite Consistent Hypothesis Class

32

slide-33
SLIDE 33

Finite Consistent Hypothesis Class

33

slide-34
SLIDE 34

Finite Consistent Hypothesis Class

34

slide-35
SLIDE 35

Finite Consistent Hypothesis Class

35

slide-36
SLIDE 36

Finite Consistent Hypothesis Class

36

slide-37
SLIDE 37

Finite Consistent Hypothesis Class

37

slide-38
SLIDE 38

Finite Consistent Hypothesis Class

38

slide-39
SLIDE 39

Finite Consistent Hypothesis Class

39

slide-40
SLIDE 40

Finite Consistent Hypothesis Class

40

slide-41
SLIDE 41

Finite Consistent Hypothesis Class

41

slide-42
SLIDE 42

Finite Inconsistent Hypothesis Class

42

slide-43
SLIDE 43

Finite Inconsistent Hypothesis Class

43

slide-44
SLIDE 44

Finite Inconsistent Hypothesis Class

44

slide-45
SLIDE 45

Finite Inconsistent Hypothesis Class

45

slide-46
SLIDE 46

Finite Inconsistent Hypothesis Class

46

slide-47
SLIDE 47

Finite Inconsistent Hypothesis Class

47

slide-48
SLIDE 48

Finite Inconsistent Hypothesis Class

48

slide-49
SLIDE 49

Finite Inconsistent Hypothesis Class

49

slide-50
SLIDE 50

Finite Inconsistent Hypothesis Class

50

slide-51
SLIDE 51

Finite Inconsistent Hypothesis Class

51

slide-52
SLIDE 52

Finite Inconsistent Hypothesis Class

52

slide-53
SLIDE 53

PAC Learnability

  • OK, that was a lot of Math!
  • I expect you to
  • Know the bounds we’ve proved
  • Know which bounds apply to which situations
  • Know how to apply a particular bound to a problem
  • I do not expect you to
  • Know the details of the proofs
  • Be able to prove PAC bounds yourself

53