for tuesday
play

For Tuesday Reach chapter 18, sections 1-4 Homework: Chapter 12, - PowerPoint PPT Presentation

For Tuesday Reach chapter 18, sections 1-4 Homework: Chapter 12, exercise 7 Program 3 Any questions? And Besides Logic? Semantic networks Frames Semantic Networks Use graphs to represent concepts and the relations


  1. For Tuesday • Reach chapter 18, sections 1-4 • Homework: – Chapter 12, exercise 7

  2. Program 3 • Any questions?

  3. And Besides Logic? • Semantic networks • Frames

  4. Semantic Networks • Use graphs to represent concepts and the relations between them. • Simplest networks are ISA hierarchies • Must be careful to make a type/token distinction: Garfield isa Cat Cat(Garfield) " x (Cat (x)  Feline(x)) Cat isa Feline • Restricted shorthand for a logical representation.

  5. Semantic Nets/Frames • Labeled links can represent arbitrary relations between objects and/or concepts. • Nodes with links can also be viewed as frames with slots that point to other objects and/or concepts.

  6. First Order Representation Rel(Alive,Animals,T) Rel(Flies,Penguins,F) Rel(Flies,Animals,F) Rel(Legs,Bats,2) Birds  Animals Rel(Flies,Bats,T) Mammals  Animals Opus  Penguins Bill  Cats Rel(Flies,Birds,T) Pat  Bats Rel(Legs,Birds,2) Rel(Legs,Mammals,4) Name(Opus,"Opus") Penguins  Birds Name(Bill,"Bill") Cats  Mammals Friend(Opus,Bill) Bats  Mammals Friend(Bill,Opus) Name(Pat,"Pat")

  7. Inheritance • Inheritance is a specific type of inference that allows properties of objects to be inferred from properties of categories to which the object belongs. – Is Bill alive? – Yes, since Bill is a cat, cats are mammals, mammals are animals, and animals are alive. • Such inference can be performed by a simple graph traversal algorithm and implemented very efficiently. • However, it is basically a form of logical inference " x (Cat(x)  Mammal(x)) " x (Mammal(x)  Animal(x)) " x (Animal(x)  Alive(x)) Cat(Bill) |- Alive(Bill)

  8. Backward or Forward • Can work either way • Either can be inefficient • Usually depends on branching factors

  9. Semantic of Links • Must be careful to distinguish different types of links. • Links between tokens and tokens are different than links between types and types and links between tokens and types.

  10. Link Types Link Type Semantics Example A  B Cats  Mammals A subset B A  B Bill  Cats A member B A R B R(A,B) Bill Age 12 " x, x  A  A R B Birds Legs 2 R(x,B) " x  y, x  A  y A R B Birds Parent Birds  B  R(x,y)

  11. Inheritance with Exceptions • Information specified for a type gives the default value for a relation, but this may be over-ridden by a more specific type. – Tweety is a bird. Does Tweety fly? Birds fly. Yes. – Opus is a penguin. Does Opus fly? Penguin's don't fly. No.

  12. Multiple Inheritance • If hierarchy is not a tree but a directed acyclic graph (DAG) then different inheritance paths may result in different defaults being inherited. • Nixon Diamond

  13. Nonmonotonicity • In normal monotonic logic, adding more sentences to a KB only entails more conclusions. if KB |- P then KB  {S} |- P • Inheritance with exceptions is not monotonic (it is nonmonotonic) – Bird(Opus) – Fly(Opus)? yes – Penguin(Opus) – Fly(Opus)? no

  14. • Nonmonotonic logics attempt to formalize default reasoning by allow default rules of the form: – If P and concluding Q is consistent, then conclude Q. – If Bird(X) then if consistent Fly(x)

  15. Defaults with Negation as Failure • Prolog negation as failure can be used to implement default inference. fly(X) :- bird(X), not(ab(X)). ab(X) :- penguin(X). ab(X) :- ostrich(X). bird(opus). ? fly(opus). Yes penguin(opus). ? fly(opus). No

  16. Machine Learning • What do you think it is?

  17. Machine Learning • Defintion by Herb Simon: “Any process by which a system improves performance.”

  18. Tasks • Classification: – medical diagnosis, credit-card applications or transactions, investments, DNA sequences, spoken words, handwritten letters, astronomical images • Problem solving, planning, and acting – solving calculus problems, playing checkers, chess, or backgamon, balancing a pole, driving a car

  19. Performance • How can we measure performance? • That is, what kinds of things do we want to get out of the learning process, and how do we tell whether we’re getting them?

  20. Performance Measures • Classification accuracy • Solution correctness and quality • Speed of performance

  21. Why Study Learning? • (Other than your professor’s interest in it)

  22. Study Learning Because ... • We want computer systems with new capabilities – Develop systems that are too difficult or impossible to construct manually because they require specific detailed knowledge or skills tuned to a particular complex task (knowledge acquisition bottleneck). – Develop systems that can automatically adapt and customize themselves to the needs of individual users through experience, e.g. a personalized news or mail filter, personalized tutoring. – Discover knowledge and patterns in databases, data mining, e.g. discovering purchasing patterns for marketing purposes.

  23. Study Learning Because ... • Understand human and biological learning and teaching better. – Power law of practice. – Relative difficulty of learning disjunctive concepts. • Time is right: – Initial algorithms and theory in place. – Growing amounts of on-line data. – Computational power available.

  24. Designing a Learning System • Choose the training experience. • Choose what exactly is to be learned, i.e. the target function. • Choose how to represent the target function. • Choose a learning algorithm to learn the target function from the experience. • Must distinguish between the learner and the performance element.

  25. Architecture of a Learner Performance System trace of new behavior problem Experiment Critic Generator training learned instances function Generalizer

  26. Training Experience Issues • Direct or Indirect Experience – Direct: Chess boards labeled with correct move extracted from record of expert play. – Indirect: Potentially arbitrary sequences of moves and final games results. • Credit/Blame assignment: – How do we assign blame to individual choices or moves when given only indirect feedback?

  27. More on Training Experience • Source of training data: – “ Random ” examples outside of learner’s control (negative examples available?) – Selected examples chosen by a benevolent teacher (near misses available?) – Ability to query oracle about correct classifications. – Ability to design and run experiments to collect one's own data. • Distribution of training data: – Generally assume training data is representative of the examples to be judged on when tested for final performance.

  28. Supervision of Learning • Supervised • Unsupervised • Reinforcement

  29. Concept Learning • The most studied task in machine learning is inferring a function that classifies examples represented in some language as members or non-members of a concept from pre-classified training examples. • This is called concept learning, or classification.

  30. Simple Example Example Size Color Shape Class 1 small red circle positive 2 big red circle positive 3 small red triangle negative 4 big blue circle negative

  31. Concept Learning Definitions • An instance is a description of a specific item. X is the space of all instances (instance space). • The target concept, c(x), is a binary function over instances. • A training example is an instance labeled with its correct value for c(x) (positive or negative). D is the set of all training examples. • The hypothesis space, H, is the set of functions, h(x), that the learner can consider as possible definitions of c(x). • The goal of concept learning is to find an h in H such that for all <x, c(x)> in D, h(x)= c(x).

  32. Sample Hypothesis Space • Consider a hypothesis language defined by a conjunction of constraints. • For instances described by n features consider a vector of n constraints, <c 1 ,c 2 ,...c> where each c i is either: – ?, indicating that any value is possible for the ith feature – A specific value from the domain of the ith feature –  , indicating no value is acceptable • Sample hypotheses in this language: – <big, red, ?> – <?,?,?> (most general hypothesis) – <  ,  ,  > (most specific hypothesis)

  33. Inductive Learning Hypothesis • Any hypothesis that is found to approximate the target function well over a a sufficiently large set of training examples will also approximate the target function well over other unobserved examples. – Assumes that the training and test examples are drawn from the same general distribution. – This is fundamentally an unprovable hypothesis unless additional assumptions are made about the target concept.

  34. Concept Learning As Search • Concept learning can be viewed as searching the space of hypotheses for one (or more) consistent with the training instances. • Consider an instance space consisting of n binary features, which therefore has 2 n instances. • For conjunctive hypotheses, there are 4 choices for each feature: T, F,  , ?, so there are 4 n syntactically distinct hypotheses, but any hypothesis with a  is the empty hypothesis, so there are 3 n + 1 semantically distinct hypotheses.

Recommend


More recommend