Introduction CptS 570 Machine Learning School of EECS Washington - - PowerPoint PPT Presentation

Introduction

CptS 570 Machine Learning School of EECS Washington State University

What is Learning?

• Webster
• To gain knowledge or understanding of or skill in by study, instruction or

experience

• To memorize
• Synonym: discover
• To obtain knowledge of for the first time
• May imply acquiring knowledge with little effort or conscious intention (as by

simply being told) or it may imply study and practice

• Knowledge
• Knowing something with familiarity gained through experience or association
• Facts or ideas acquired by study, investigation, observation, or experience
• Deduction? (n!)
• Knowledge representation?
• Performance measure?

What is Machine Learning?

Herbert Simon, CMU

Any process by which a system improves its performance

Expert systems

Acquisition of explicit knowledge

Psychologists

Skill acquisition

Scientists

Theory formation, hypothesis formation and inductive

inference

Tom Mitchell, CMU

A computer program that improves its performance at some

task through experience

Motivations

Automated knowledge engineering

Expertise is scarce Codification of expertise is difficult Expertise frequently consists of a set of test cases Data from measurements, but no information or

knowledge

Only one computer has to learn, then copy Discover new knowledge Understand human learning

Applications

Speech recognition Object recognition Language learning Autonomous navigation Data mining Intelligent agents Cognitive modeling

History

Exploration (1950s and 1960s)

Neurophysiological

Rosenblatt's perceptron

Biological

Simulated evolution

Psychological

Symbol processing systems

Statistical

Control and pattern recognition Samuel's checkers program

Theoretical

Gold's identification in the limit Minsky and Papert's criticism of the perceptron

History

Development of practical algorithms (1970s)

Winston's ARCH

Learned concept of a blocks-world arch

Buchanan and Mitchell's Meta-Dendral

Learned mass-spectrometry prediction rules

Michalski's AQ11

Learned soybean disease diagnosis rules

Quinlan's ID3

Learned chess end-game rules

Fikes, Hart and Nilsson's MACROPS

Learned macro-operators in blocks-world planning

Lenat's AM

Discovered interesting mathematical concepts

History

Explosion of research directions (1980s)

Learning theory Symbolic learning algorithms Connectionist (neural network) learning algorithms Clustering and discovery Explanation-based learning Knowledge-guided inductive learning Analogical and case-based reasoning Genetic algorithms

History

Maturity of the field (1990s)

Statistical comparisons of algorithms Theoretical analyses of algorithms Machine learning = Data mining (?) Successful applications Multi-relational learning Ensemble and Kernel Methods

Mitchell’s Book

Practical approach to study of machine

learning

Methodology snapshot (good one for

1997)