CS 561: Artificial Intelligence Instructor: Prof Hadi Moradi - - PDF document

1

CS 561: Artificial Intelligence

Instructor: Prof Hadi Moradi Instructor: Prof. Hadi Moradi,

moradi@usc.edu

Lectures: M-Th 09:00-10:40, OHE136 Office hours: MW 2:30 – 4:00 pm,

SAL310,

O b i

Or by appointment

TAs: Jeong-Yoon Lee

SAL 112 Office hours: TTH 1:00-2:30 Email: jeongyol@usc.edu

CS 561: Artificial Intelligence

Course web page:

http:/ / www-scf.usc.edu/ ~ csci561a Up to date information, lecture notes Relevant dates, links, etc. Also you may check http://den.usc.edu

Class format: two sections of 45 minutes Course material: Course material:

[AIMA] Artificial I ntelligence: A Modern

Approach, by Stuart Russell and Peter Norvig. 2nd edition

2

CS 561: Artificial Intelligence

Course overview: foundations of symbolic

Course overview: foundations of symbolic

intelligent systems. Agents, search, problem solving, logic, representation, reasoning, symbolic programming, probabilistic reasoning, and robotics.

Prerequisites: CS 455x, i.e.,

programming principles, discrete mathematics

for computing, software design and software engineering concepts. Some knowledge of C/C+ + for some programming assignments.

CS 561: Artificial Intelligence

Grading:

Grading: 25% for midterm 25% for final 40% for homeworks and projects

10% f Q i

10% for Quizzes

3

Practical issues

Class list: use learn usc edu

Class list: use learn.usc.edu

Login with your USC username and

password

I f CSCI 561A is not listed as your courses,

notify the TA.

• t y t e

Submissions: See class web page under

Assignments

submit -user csci561 -tag HW3 HW3.tar.gz

Administrative Issues

Midterm 1: 7/26/10 9:00

10:40pm

Midterm 1: 7/26/10 9:00 - 10:40pm Midterm 2: 8/10/10 9:00 - 10:40pm

See also the class web page: http://den usc edu/ http://den.usc.edu/

4

Why study AI?

Search engines Science Labor Medicine/ Diagnosis Appliances What else?

Humanoid Robots: From Honda to Sony

Walk Turn Stairs

http://world.honda.com/robot/

5

Sony AIBO

movie1

http://www.aibo.com

Natural Language Question Answering

http://www.ai.mit.edu/projects/infolab/ http://aimovie.warnerbros.com

6

Robot Teams

USC robotics Lab

Modular robots self re-assembly.

What is AI?

The exciting new effort to make “The study of mental faculties The exciting new effort to make computers thinks … machine with minds, in the full and literal sense” (Haugeland 1985) “The art of creating machines that perform functions that require intelligence when The study of mental faculties through the use of computational models” (Charniak et al. 1985) A field of study that seeks to explain and emulate intelligent behavior in terms of require intelligence when performed by people” (Kurzweil, 1990) behavior in terms of computational processes” (Schalkol, 1990)

7

AI – The Bigger Picture

Computer Science Philosophy

?

Artificial Intelligence Cognitive Science (Psychology) p y Robotics (Engineering) Neuroscience (Biology)

?

Acting Humanly: The Turing Test

Alan Turing's 1950 article Computing Machinery Alan Turing s 1950 article Computing Machinery

and Intelligence discussed conditions for considering a machine to be intelligent

8

Acting Humanly: The Turing Test What tasks require AI?

“AI is the science and engineering of

• AI is the science and engineering of

making intelligent machines which can perform tasks that require intelligence when performed by humans …”

What tasks require AI?

9

What tasks require AI?

Tasks that require AI:

q

Solving a differential equation Brain surgery Inventing stuff Playing Jeopardy Playing Wheel of Fortune What about walking? What about grabbing stuff? What about pulling your hand away from fire? What about watching TV? What about day dreaming?

Acting Humanly: The Full Turing Test

• Problem:
• Problem:

10

What would a computer need to pass the Turing test?

Communication:

Communication: Memory: Reasoning: Learning:

What would a computer need to pass the Turing test?

Sensing:

Sensing:

M t t l (t t l t t)

Motor control (total test):

11

Thinking Humanly: Cognitive Science

1960 “Cognitive Revolution”:

1960 Cognitive Revolution :

information-processing psychology replaced behaviorism

Thinking Humanly: Cognitive Science

Cognitive science and modeling the activities

Cognitive science and modeling the activities

• f the brain

What level of abstraction? “Knowledge” or

“Circuits”?

How to validate models?

12

Thinking Rationally: Laws of Thought

Aristotle (~ 450 B.C.) attempted to codify

Aristotle ( 450 B.C.) attempted to codify “right thinking”

What are correct arguments/thought

processes?

Thinking Rationally: Laws of Thought

Problems:

13

Acting Rationally: The Rational Agent

Rational behavior: Doing the right thing! Provides the most general view of AI

because it includes:

Acting Rationally: The Rational Agent

Advantages:

Advantages:

14

How to achieve AI?

How is AI research done?

Theoretical Experimental

How to achieve AI?

There are two main lines of research:

There are two main lines of research:

Biological, study humans and imitate their

psychology or physiology.

phenomenal, study and formalize common sense

facts about the world and the problems that the world presents to the achievement of goals. world presents to the achievement of goals.

The two approaches interact to some extent,

and both should eventually succeed. It is a race, but both racers seem to be walking. [John McCarthy]

15

Branches of AI

Logical AI Search Natural language processing pattern recognition Knowledge representation I nference From some facts, others can be inferred.

,

Automated reasoning Learning from experience Planning To generate a strategy for achieving some

goal

AI Prehistory

16

Brief History of AI

Thought

Ancient Times 384 B.C.

• Aristotle

Logic: The science of knowing. Middle Age 1200 Ramon Lull Ars Magnus: a rule-based device to model man's behavior and nature Renaissance Empiricism

Next time implement links

tionally:Laws of T

Renaissance

• Empiricism

Explanation of processes 17th Century

• Gottfried Leibniz

1st system of formal logic 18th Century Rene Descartes Dualism 19th Century 1845

• Charles Babbage

Analytical Engine

• George Boole

Formalization of the Laws of Logic

Thinking Rat

• Formalization of the Laws of Logic

1879-1903

• Gottlob Frege

First-order predicate calculus Early 20th Century 1910-1912

• Russel-Whitehead

Principia Mathematica

Bertrand Russel

1931

• Kurt Godel

Incompleteness Theorem of Logic

Roots of AI in Science:

Aristotle(b.384-): syllogism – formal reasoning Ramon Lull (b.1235): Ars Magna – a machine capable of

answering all questions

Rene Descartes (1596): mind / body separation (dualism);

"cogito ergo sum“

Wilhelm Liebniz (1646-1716): a mechanical concept

generator; "materialism" g ;

Charles Babbage(1792-1871), Ada Lovelace (1815-1860):

Analytical Engine – a general-purpose calculator

George Boole(1815-1864): logic algebras - logical encoding

and calculation of thoughts

Gottlob Frege(1848-1925): predicate calculus

17

Birth of Artificial Intelligence

1940-1956

• 1942
• ENIAC :First digital computer

Grea

• 1943
• Mc Culloch and Pitts

Artificial neural network 1945

• J. Von Neumman

Modern computer architecture 1949

• Claude Shannon

Use of heuristics to solve complex problems

at Expectations

1950

• Alan M.Turing

Computing Machinery and Intelligence: Turing Test 1955

• Herbert Simon,Alan Newell

1st AI program:Logic Theorist

Herbert Simon

1956

• Dartmouth Conference

The Beginning of AI

McCulloch & Pitts

developed theory of artificial neurons (precursor to

ANN's) – 1943

Alan Turing – "Can Machines Think?"

the turing test (1950) the turing machine

Marvin Minsky & Dean Edmonds

first ANN constructed, 1951

John McCarthy

convened the Dartmouth conference that coined the

term artificial intelligence (AI) (1956) and set the research agenda

symbolic AI connectionism

LISP (list processing) 1958 1st AI language

18

The Rise of AI

1957- 1960’s

• 1958
• John McCarthy.

LISP

Growing

1960

• Marvin Minsky

Theory of Frames 1961

• Herbert Simon,Alan Newell

GPS:General Problem Solver

Herbert Simon

1962

• Frank Rosenblatt

Perceptron: Learning in Neural Networks 1965 L tfi A Z d h

g Disenchantment

1965

• Lotfi A. Zadeh

Fuzyy Logic Fuzzy Sets 1968 Joseph Weizenbaum ELIZA: simulates diagnosis by a psychiatrist. 1969

• Marvin Minsky,Seymour Papert

Limitations of Perceptrons

• S. Papert

An Optimistic Start

In the 50's, 60's and early 70's, much exciting progress was being made in AI:

Chess

Claude Shannon, 1950

The Logic Theorist

Alan Newell, Cliff Shaw, Herb Simon, 1957

Checkers (Machine Learning)

Arthur Samuels, 1959

Eliza - NLP

Joseph Weizenbaum, 1966

DENDRAL – Knowledge-Based System

Feigenbaum, Buchanan, Lederberg, 1969

SHRDLU – NLP (Blocks World)

Terry Winnograd, 1972

GPS (General Problem Solver)

Alan Newell & Herb Simon, 1972

19

The 70’s

Bi th d Ri f E t S t Birth and Rise of Expert Systems

1970-mid 1980’s

• 1973
• Alain Colmerauer

PROLOG 1974

• Paul Werbos

Neural Networks Back Propagation Law 1975

• E. Feigenbaum, R. Lindsay.

Dendral

E.FeigenBaum

Edward Shortliffe MYCIN 1976- 1980

• R. Duda, P.Hart, P. Barnett

PROSPECTOR: The first commercial Expert System

P.Hart

1982 John McDermott XCON – "Expert Configurer

The Plateau

In the 70's, AI researchers began to discover that the problem wasn't as easy as it looked!

The Frame Problem

L k f C S R i

Lack of Common Sense Reasoning Combinatorial Explosion The Gap – "Toy" vs. "Real" worlds Perceptrons, by Minsky & Papert (1969) – proved

limitations of perceptron networks and acted to limit significant research in the 70's significant research in the 70 s

Lighthill Report – 1973: curtailed research funding in

British Universities AI developed a reputation as "over-hyped" and unrealistic

20

The 80’s

1982

• John Hopfield

Hopfield Networks

1982

Teuvo Kohonen self-organising feature maps for speech recognitizion

1986

T S j ki

tems works

1986

• Terrence Sejnowski

NETTalk

Rumerhalt,McMelland

Neural Networks Rediscovering of Back-Propagation Learning 1987

• Marvin Minsky

The Society of Minds

ation of Expert Syst tificial Neural Netw

Fuzzy Appliances 1989

• Dean Pomerleau

ALVINN

Commercializa Rebirth of Art

Commercial Success

Despite it's reputation as "over hyped" certain Despite it s reputation as over-hyped , certain AI applications became very successful during the 70's – 80's:

• Expert Systems

Industrial Robotics

• Industrial Robotics
• Planning & Scheduling Applications

AI became a $2,000,000,000 industry by 1988

21

Nowadays…

Early 90’s

• Major advances in all areas of AI, with

significant demonstrations

• significant demonstrations

1995 Birth of Intelligent Systems 1997 The Deep Blue chess program beats Garry Kasparov Late 90’s

• Web crawlers
• AI-based information extraction

programs Intelligent Room and Emotional Agents at MIT's AI Lab 2000- Interactive robot pets The Nomad robot

The Gartner Hype Curve

Interest in AI followed this pattern

Interest in AI followed this pattern,

typical of the hype surrounding new technologies

22

AI State of the art

Have the following been achieved by AI?

World-class chess playing Playing table tennis Cross-country driving Solving mathematical problems

Discover and prove mathematical theories

Discover and prove mathematical theories Engage in a meaningful conversation Understand spoken language Observe and understand human emotions …

Types of expertise (with examples)

Deep cognitive

Judgmental skills

High-level social skills cognitive skills

skills

social skills Highly creative

Musician Senior manager Author, poet

Analytical

Mathemat- i i Economist, Social i ti t ician programmer scientist

Strictly procedural

Typist Driver Social worker

23

A driving example: Grand Challenge

Goal:

Artificial Intelligence Applications

Cognitive Science Artificial Intelligence Robotics Natural Interface Science Applications Applications Interface Applications

• Expert Systems
• Fuzzy Logic
• Genetic Algorithms
• Neural Networks
• Visual Perceptions
• Locomotion
• Navigation
• Tactility
• Natural Language
• Speech Recognition
• Multisensory Interface
• Virtual Reality

24

AI Application Areas in Business

Neural Networks Fuzzy Logic Systems Virtual Reality Genetic Algorithms y Expert Systems AI Application Areas in Business Intelligent Agents

Components of Expert Systems

The Expert System

K l d

Expert Advice

User I t f Inference E i Knowledge Base User Workstation Interface Programs Engine Program

Expert System Development

Workstation

Knowledge Engineering

Knowledge Acquisition Program

Expert and/or Knowledge Engineer

25

Expert System Applications

Decision Management Diagnostic/Troubleshooting Diagnostic/Troubleshooting Maintenance/Scheduling Design/Configuration

Major

Selection/Classification

Major Application Categories

• f Expert Systems

Process Monitoring/Control

General I ntroduction

Course Overview

General I ntroduction

I ntroduction. [AIMA Ch 1] Course Schedule.

Homeworks, exams and grading. Course material, TAs and office hours. Why study AI? What is AI? The Turing test. Rationality. Branches of AI. Research disciplines connected to and at the foundation of AI. Brief history of AI. Challenges for the future. Overview of class syllabus.

26

General I ntroduction

Course Overview

sensors effectors Agent

General I ntroduction

I ntelligent Agents. [AIMA Ch 2] What is

an intelligent agent? Examples. Doing the right thing (rational action). Performance measure.

• Autonomy. Environment and agent design.

Structure of agents Agent types Reflex agents Structure of agents. Agent types. Reflex agents. Reactive agents. Reflex agents with state. Goal-based agents. Utility-based agents. Mobile

• agents. Information agents.

Course Overview (cont.)

9 l

Problem solving and search.

[AIMA Ch 3]

measuring problem. Types of problems. More examples. Basic idea behind search algorithms.

3 l 5 l 9 l

Using these 3 buckets, measure 7 liters of water. Complexity. Combinatorial explosion and NP

completeness.

Polynomial hierarchy.

Traveling salesperson problem

27

Course Overview (cont.)

How can we solve complex problems?

Uninformed search. [AIMA Ch

3]

Depth-first. Breadth-first. Uniform-cost.

3 l 5 l 9 l

Using these 3 buckets, measure 7 liters of water. Depth-limited. Iterative deepening.

• Examples. Properties.

Traveling salesperson problem

Course Overview (cont.)

How can we solve complex problems?

I nformed search. [AIMA Ch 4]

Best-first. A* search. Heuristics.

p p

Hill climbing. Problem of local extrema. Simulated annealing. Traveling salesperson problem

28

Course Overview (cont.)

Practical applications

• f search

Constraint Satisfaction

[AIMA Ch 5]

Backtracking Local search

Course Overview (cont.)

Practical applications

• f search

Game playing

[AIMA Ch 6]

The minimax algorithm.

g

Resource limitations. Aplha-beta pruning. Elements of chance and

non-deterministic games.

tic-tac-toe

29

Course Overview (cont.)

Towards intelligent agents

Agents that reason

logically 1

[AIMA Ch 7]

Knowledge-based

Towards intelligent agents

agents.

Logic and

representation.

Propositional (boolean)

logic.

wumpus world

Course Overview (cont.)

Towards intelligent agents

Agents that reason

logically 2.

[AIMA Ch 7]

Inference in

i i l l i

Towards intelligent agents

propositional logic.

Syntax. Semantics. Examples.

wumpus world

30

Course Overview (cont.)

Building knowledge-based u d g

• edge

ased agents: 1st Order Logic

First-order logic 1. [AIMA Ch 8]

Syntax. Semantics.

Atomic sentences

Atomic sentences. Complex sentences. Quantifiers. FOL knowledge base. Situation calculus.

Course Overview (cont.)

Building knowledge Building knowledge- based agents: 1st Order Logic

First-order logic 2.

[AIMA Ch 9]

Describing actions. Planning. Action sequences.

31

Course Overview (cont.)

Reasoning Logically Reasoning Logically

I nference in first-order logic.

[AIMA Ch 9]

Proofs.

Unification

Unification. Generalized modus ponens. Forward and backward chaining.

Example of backward chaining

Course Overview (cont.)

Representing and Organizing Representing and Organizing Knowledge

Building a knowledge base.

[AIMA Ch 10]

Knowledge bases. Knowledge bases. Vocabulary and rules. Ontologies Organizing knowledge.

An ontology for the sports domain

32

Course Overview (cont.)

Systems that can Plan y Future Behavior

Planning.[AIMA Ch 11]

Definition and goals. Basic representations for

l i planning.

Situation space and plan space. Examples.

Course Overview (cont.)

Learning from Observation g

Decision Trees

[AIMA 18]

Introduction to decision trees. Information theory. Constructing DT. Examples.

33

Course Overview (cont.)

Expert Systems p y

Probabilities + Bayesian Networks

[AIMA 13 + 14]

Basics of probability theory Bayesian rule.

d l

Conditional reasoning. Bayesian Networks. Reasoning under uncertainty

Course Overview (cont.)

Statistical Learning Methods g

Neural Networks.

[AIMA 20]

Human brain structure Neuron and activation function. Forward and backward propagations. Examples.

34

Course Overview (cont.)

Logical Reasoning in the g g Presence of Uncertainty

Fuzzy logic

[Handout]

Introduction to fuzzy logic.

Center of gravity

y g

Linguistic Hedges. Fuzzy inference. Examples.

Center of largest area

Course Overview (cont.)

Machine Learning

Genetic Algorithms

[Handout + AIMA 4]

Genetic algorithm approach. Mutation, Crossover, Fitness function. Examples.

35

Course Overview (cont.)

What challenges remain? g

Towards intelligent machines.

[AIMA Ch 25]

The challenge of robots:

with what we have learned,

what hard problems remain to be solved?

what hard problems remain to be solved? Different types of robots. Tasks that robots are for. Parts of robots. Architectures. Configuration spaces.

robotics@USC

Course Overview (cont.)

What challenges remain? What challenges remain?

Overview and summary. [all of the

above]

What have we learned

What have we learned. Where do we go from here?

robotics@USC

36

Outlook

AI is a very exciting area right now

AI is a very exciting area right now. This course will teach you the

foundations.