Representing Knowledge Dustin Smith MIT Media Lab July 2008 - - PowerPoint PPT Presentation

Commonsense Computing MIT MediaLab

Representing Knowledge

Dustin Smith MIT Media Lab July 2008

Commonsense Computing MIT MediaLab 2

Big questions

How to get it? How to use it?

Commonsense Computing MIT MediaLab 3

Big questions

How to get it? How to use it?

Commonsense Computing MIT MediaLab 4

Big questions

How to get it? How to use it?

environment

Commonsense Computing MIT MediaLab 4

Big questions

How to get it? How to use it?

sensory environment

Commonsense Computing MIT MediaLab 4

Big questions

How to get it? How to use it?

sensory environment knowledge

Commonsense Computing MIT MediaLab 4

Big questions

How to get it? How to use it?

sensory motor environment knowledge

Commonsense Computing MIT MediaLab 5

Math Computer Science / AI Language

Commonsense Computing MIT MediaLab 6

Representing Learning Combining Items Relations Processes

Commonsense Computing MIT MediaLab 6

Representing Learning Combining Items Relations Processes

Commonsense Computing MIT MediaLab 7

Why math? Compare these two definitions: a) b) “Any number that is multiplied by the sum of two numbers is equal to the sum of the products of it and each

• f the numbers in the first sum.”

Representing Items Math

x(y + z) = xy + xz

Commonsense Computing MIT MediaLab 8

• a set is a collection of arbitrary items, defined by:

(1) listing its members (2) stating a qualifying property (3) stating a set of rules to generate it Examples of sets:

Representing Items Math

A = {1,2,3,4,5} A = {x|x is a positive integer less than 6}

a) 1 ∈ A

b) if x ∈ A and x < 5, then x + 1 ∈ A

A =

Commonsense Computing MIT MediaLab 9

All of the items referred to by the set are known as its extension. The specification of the set’s criteria is its intension:

Representing Items Math

{x|mammal(x) ∧ brown(x)} {dog, bear, kangaroo, ...}

Commonsense Computing MIT MediaLab 10

Representing Items Math

in a ∈ {a, b, c} not in d / ∈ {a, b, c} empty set ∅ subset {a, b, c} ⊂ {a, b, c} superset {a, b, c} ⊃ {a} not subset {a, b, c} ⊂ {a, b} proper subset {a, b, c} ⊆ {a, b, c, d} not proper subset {a, b, c} ⊆ {a, b, c} subset ∅ ⊂ {a, b, c}

Common set operations:

Commonsense Computing MIT MediaLab 11

Representing Items Math

union A ∪ B intersection A ∩ B complement A′ Universe Ω empty set ∅

Cross product:{representing, learning, combining} × {items, relations, processes} = {{representing, items}, {representing, relations}, {representing, processes}, {learning, items}, {learning, relations}, ...}

Commonsense Computing MIT MediaLab 12

Representing Items Math

It’s hard just to talk about items, because items are defined by smaller items (features, properties). And sets of items are related via containment.

Animals (set)

?

Commonsense Computing MIT MediaLab 12

Representing Items Math

It’s hard just to talk about items, because items are defined by smaller items (features, properties). And sets of items are related via containment.

Animals (set) Pets (set)

?

Commonsense Computing MIT MediaLab 12

Representing Items Math

It’s hard just to talk about items, because items are defined by smaller items (features, properties). And sets of items are related via containment.

Animals (set) Pets (set) Dogs (set)

?

Commonsense Computing MIT MediaLab 12

Representing Items Math

It’s hard just to talk about items, because items are defined by smaller items (features, properties). And sets of items are related via containment.

Animals (set) Pets (set) Dogs (set) Fido (member)

?

Commonsense Computing MIT MediaLab 13

Representing Items Math

Animals A(set) Pets P(set)Dogs D (set) Fido f (member)

?

D ⊆ A A ⊃ P f ∈ A f ∈ D f / ∈ P ′ D = {x| has four legs(x) ∧ howls(x) }

Commonsense Computing MIT MediaLab 13

Representing Items Math

Animals A(set) Pets P(set)Dogs D (set) Fido f (member)

?

relationships between sets/categories

D ⊆ A A ⊃ P f ∈ A f ∈ D f / ∈ P ′ D = {x| has four legs(x) ∧ howls(x) }

Commonsense Computing MIT MediaLab 13

Representing Items Math

Animals A(set) Pets P(set)Dogs D (set) Fido f (member)

?

relationships between sets/categories relationships between sets & members

D ⊆ A A ⊃ P f ∈ A f ∈ D f / ∈ P ′ D = {x| has four legs(x) ∧ howls(x) }

Commonsense Computing MIT MediaLab 13

Representing Items Math

Animals A(set) Pets P(set)Dogs D (set) Fido f (member)

?

relationships between sets/categories relationships between sets & members

D ⊆ A A ⊃ P f ∈ A f ∈ D f / ∈ P ′ D = {x| has four legs(x) ∧ howls(x) }

intensional set definition / membership criterion

A relation is a property that holds (or not) between objects. Relation between two sets: A and B:

Representing Relations Math

R ⊆ A × B

A B a b c d e

R = {a, d, b, e, c, d}

aRd bRe cRd

A relation is a property that holds (or not) between objects. Relation between two sets: A and B:

Representing Relations Math

R ⊆ A × B

A B a b c d e

R = {a, d, b, e, c, d}

aRd bRe cRd

A relation is a property that holds (or not) between objects. Relation between two sets: A and B:

Representing Relations Math

R ⊆ A × B

A B a b c d e

R = {a, d, b, e, c, d}

aRd bRe cRd

{1, 2} = {2, 1} 1, 2 = 2, 1

Properties of relations: Reflexive

Representing Relations Math

A B a b c a b

for every x ∈ A, x, x ∈ R

c

Properties of relations: Irreflexive

Representing Relations Math

A B a b c c a b

for every x ∈ A, x, x ∈ R′

Properties of relations: Symmetric

Representing Relations Math

A B a b b a

for every x, y ∈ A, y, x ∈ R

Properties of relations: Asymmetric

Representing Relations Math

A B a b c d e

for every x, y ∈ A, y, x / ∈ R

Properties of relations: Antisymmetric

Representing Relations Math

A B a b c a e

if whenever x, y and y, x ∈ R, then x = y

reflexive → anti − symmetric

Properties of relations: Transitive

Representing Relations Math

A B a b b c

if whenever x, y and y, z ∈ R, then x, z ∈ R

Partial orderings: a binary relationship that is a) reflexive b) antisymmetric c) transitive

Representing Relations Math

⊆

for sets Linear ordering: a partial ordering where (e,g. for real numbers, is a linear ordering)

x ≤ y or y ≤ x for every x, y

≤ ≤

Lattice:

• 1. set L
• 2. partial ordering

3.

Representing Relations Math

≤

∪, ∩ a ∩ b a ∪ b

= greatest lower bound = least upper bound

Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

Representing Relations Math

Leibniz Universal Characteristic (1679):

Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

Representing Relations Math

Leibniz Universal Characteristic (1679):

≤

a b if b % a == 0

a ∩ b a ∪ b

= g.c.d(a,b) = smallest integer divisible. lattice is bounded

Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

Representing Relations Math

Leibniz Universal Characteristic (1679):

Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

Representing Relations Math

Equivalently, we could use a bit string: Primitive Types:

20 = INSECT 21 = ALIV E 22 = MOSTLY WATER 23 = ARTIST 24 = E15320G 25 = STUDIESAI

110110 = Dustin

Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

Representing Relations Math

Equivalently, we could use a bit string: Primitive Types:

20 = INSECT 21 = ALIV E 22 = MOSTLY WATER 23 = ARTIST 24 = E15320G 25 = STUDIESAI

110110 = Dustin

a <= b if b[i] = 1 then a[i] =1

a ∩ b a ∪ b

=logical AND = logical OR

Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

Representing Relations Math

Equivalently, we could use a bit string: Primitive Types:

20 = INSECT 21 = ALIV E 22 = MOSTLY WATER 23 = ARTIST 24 = E15320G 25 = STUDIESAI

110110 = Dustin

Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

Representing Relations Math

Equivalently, we could use a bit string: Primitive Types:

20 = INSECT 21 = ALIV E 22 = MOSTLY WATER 23 = ARTIST 24 = E15320G 25 = STUDIESAI

110110 = Dustin

a <= b if b[i] = 1 then a[i] =1

a ∩ b a ∪ b

=logical AND = logical OR

Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

Representing Relations Math

Equivalently, we could use a bit string: Primitive Types:

20 = INSECT 21 = ALIV E 22 = MOSTLY WATER 23 = ARTIST 24 = E15320G 25 = STUDIESAI

110110 = Dustin

a <= b if b[i] = 1 then a[i] =1

a ∩ b a ∪ b

=logical AND = logical OR

• These markers only

represent conjunctions! (ands)

Representing Relations

People tried this!

• Masterman (1961) - defined 15,000 words in terms of 1000

primitives

• Schank (1975) - reduced the number of primitive acts to 11

Representing Relations Math

People tried this!

• Masterman (1961) - defined 15,000 words in terms of 1000

primitives

• Schank (1975) - reduced the number of primitive acts to 11

But,,,

• No linguistic/psychological evidence for universal set of

primitives

• Languages have families of synonyms (glad, happy, cheerful)

with slightly different meanings -- not disjoint either-or groups.

Propositional Logic Sentences are statements that have a truth value

Representing Items Math

S → {true, false}

P = it will rain today Q = dustin will wear an umbrella P → Q?

Logic: Sentential Connectives

Connective Symbol English Not ¬P not P And P ∨ Q P and Q Or P ∧ Q P or Q Implies P → Q If P then Q Q if P Q is a necessary condition of P P is a sufficient condition of Q Biconditional P ↔ Q P is a sufficient and necessary condition of Q Q is a sufficient and necessary condition of P

Representing Items Math

Logic: Inference Inference is driving new knowledge from old knowledge. Deduction, is a set of rules for truth-preserving transformations over logical statements.

Representing Items Math

Propositional Logic, rules of inference

Representing Items Math

If elephants have wings then, 2+2 = 5 Valid proposition:

Commonsense Computing MIT MediaLab 35

Some representational ideas are in the semantics of the programming language (OOP, Von Neumann, functional), and the programmer can also extend them. Abstraction allows us to name increasingly complicated procedures and data types. Hiding the implementation complexity behind a simple name -- separating the representation from the function.

Representing Items CS/AI

Dog object

feed(*food) pet(*instrument) bark() bark() name weight bark() pee() { hidden from you }

Commonsense Computing MIT MediaLab 36

Representing

Inheritance:

Items CS/AI

Commonsense Computing MIT MediaLab 37

Representing

Inheritance:

Items CS/AI

Dog object

feed(*food) pet(*instrument) name weight bark() pee() { hidden from you }

Animal

class Animal gender = FMALE; end class Dog < Animal name, weight = “,0 bark(); pee(); end gender

Commonsense Computing MIT MediaLab 38

Representing

Three main views of category representations:

• 1. Sufficient and necessary conditions /
• logic. (categories like “game” have no

common properties)

• 2. Exemplars - all instances stored
• 3. Prototypes - one best representative

Items Language

Commonsense Computing MIT MediaLab 39

Representing

Exemplars

Items Language

Prototypes

Commonsense Computing MIT MediaLab 40

Representing

In language items are words, objects are like

• nouns. Their meaning is context-

dependent. Words have various semantic traits when interacting with other words.

Items Language

Commonsense Computing MIT MediaLab 41

Representing

Semantic trait: degree and mode of participation:

• 1. criterial
• 2. expected
• 3. possible
• 4. unexpected
• 5. excluded

Items Language

textual entailment

It’s a dog -| It’s an animal It’s a dog -| It’s a fish

Commonsense Computing MIT MediaLab 42

Representing

Semantic trait: degree and mode of participation:

• 1. criterial
• 2. expected
• 3. possible
• 4. unexpected
• 5. excluded

Items Language

expected/unexpected traits

It’s a dog, but it can bark. It’s a dog, but it can’t bark. It’s a dog, but it can sing. It’s a dog, but it can’t sing.

Commonsense Computing MIT MediaLab 43

Representing

Semantic trait: degree and mode of participation:

• 1. criterial
• 2. expected
• 3. possible
• 4. unexpected
• 5. excluded

Items Language

expected/unexpected traits

It’s a dog, but it can bark. It’s a dog, but it can’t bark. It’s a dog, but it can sing. It’s a dog, but it can’t sing. Expressive paradoxes

Commonsense Computing MIT MediaLab 44

Representing

Semantic trait: degree and mode of participation:

• 1. criterial
• 2. expected
• 3. possible
• 4. unexpected
• 5. excluded

Items Language

if both 2+3 are expressive paradox, then it’s “possible”

It’s a dog and it’s brown (normal) It’s a dog, but it’s brown (why shouldn’t it be?)

Commonsense Computing MIT MediaLab 45

Representing Learning Combining Items Relations Processes

Commonsense Computing MIT MediaLab 45

Representing Learning Combining Items Relations Processes

• Deduction: Derive new knowledge by

exploiting the structure of old knowledge

All Splash Students are Smart John is a Splash Student

Learning Items Philo.

• Deduction: Derive new knowledge by

exploiting the structure of old knowledge

All Splash Students are Smart John is a Splash Student John is Smart

Learning Items Philo.

• Deduction: Derive new knowledge by

exploiting the structure of old knowledge

All Splash Students are Smart John is a Splash Student John is Smart

• by applying

inference meta-rules (modus ponens)

Learning Items Philo.

• Deduction: Derive new knowledge by

exploiting the structure of old knowledge

• Induction: Learning generalizations to fit

data

John is Smart, in Splash and 10 years old. Lisa is Smart, in Splash and 10 years old. Joe is Smart, in Splash and 10 years old.

Learning Items Philo.

• Deduction: Derive new knowledge by

exploiting the structure of old knowledge

• Induction: Learning generalizations to fit

data

John is Smart, in Splash and 10 years old. Lisa is Smart, in Splash and 10 years old. Joe is Smart, in Splash and 10 years old. People in Spash are Smart

Learning Items Philo.

• Deduction: Derive new knowledge by

exploiting the structure of old knowledge

• Induction: Learning generalizations to fit

data

John is Smart, in Splash and 10 years old. Lisa is Smart, in Splash and 10 years old. Joe is Smart, in Splash and 10 years old. People named John, Lisa or Joe are smart?

Learning Items Philo.

• Deduction: Derive new knowledge by

exploiting the structure of old knowledge

• Induction: Learning generalizations to fit

data

John is Smart, in Splash and 10 years old. Lisa is Smart, in Splash and 10 years old. Joe is Smart, in Splash and 10 years old. People 10 years old are smart

Learning Items Philo.

Problems of induction

• Induction is never justified. Hume’s

problem: will the sun rise tomorrow? Assume the past is like the future?

• Inductions are always biased. A priori, all

hypotheses are equally likely?

• Accidental versus law-like hypotheses.

Which properties can be generalized to larger classes?

Learning Items Philo.

Concept Learning

• Functionally, a concept is a mental representation that

divides the world into positive and negative classes.

chair

f(x) → {true, false}

Learning Items Philo.

Concept Learning

• Functionally, a concept is a mental representation that

divides the world into positive and negative classes.

chair

f(x) → {true, false}

Learning Items Philo.

Bruner, Goodnow, Austin (1956) slides from Josh Tenenbaum’s 9.66

Bruner, Goodnow, Austin (1956)

Describing the Microworld

• Shapes = { , , }
• Number = {1, 2, 3}
• Texture = {Shaded, Light, Dark}
• Frame = {Single, Double, Triple}

Number of Concepts:

Bruner, Goodnow, Austin (1956)

Describing the Microworld

• Shapes = { , , }
• Number = {1, 2, 3}
• Texture = {Shaded, Light, Dark}
• Frame = {Single, Double, Triple}

Number of Concepts:

= |Shapes| × |Number| × |Texture| × |frame|

Bruner, Goodnow, Austin (1956)

Describing the Microworld

• Shapes = { , , }
• Number = {1, 2, 3}
• Texture = {Shaded, Light, Dark}
• Frame = {Single, Double, Triple}

Number of Concepts:

= |Shapes| × |Number| × |Texture| × |frame|

= 3 × 3 × 3 × 3 = 81

Bruner, Goodnow, Austin (1956)

Describing the Microworld

• Shapes = { , , }
• Number = {1, 2, 3}
• Texture = {Shaded, Light, Dark}
• Frame = {Single, Double, Triple}

Number of Concepts:

= |Shapes| × |Number| × |Texture| × |frame|

= 3 × 3 × 3 × 3 = 81

9 9

slides from Josh Tenenbaum’s 9.66

+

slides from Josh Tenenbaum’s 9.66

+

slides from Josh Tenenbaum’s 9.66

+

slides from Josh Tenenbaum’s 9.66

+

slides from Josh Tenenbaum’s 9.66

+ +

slides from Josh Tenenbaum’s 9.66

+ +

slides from Josh Tenenbaum’s 9.66

+ +

slides from Josh Tenenbaum’s 9.66

+ + +

slides from Josh Tenenbaum’s 9.66

+ + +

slides from Josh Tenenbaum’s 9.66 “striped and three borders”

Bruner, Goodnow, Austin (1956)

Describing the Microworld

• Shapes = { , , }
• Number = {1, 2, 3}
• Texture = {Shaded, Light, Dark}
• Frame = {Single, Double, Triple}

Number of percepts:

Bruner, Goodnow, Austin (1956)

Describing the Microworld

• Shapes = { , , }
• Number = {1, 2, 3}
• Texture = {Shaded, Light, Dark}
• Frame = {Single, Double, Triple}

Number of percepts:

= |Shapes| × |Number| × |Texture| × |frame|

Bruner, Goodnow, Austin (1956)

Describing the Microworld

• Shapes = { , , }
• Number = {1, 2, 3}
• Texture = {Shaded, Light, Dark}
• Frame = {Single, Double, Triple}

Number of percepts:

= |Shapes| × |Number| × |Texture| × |frame|

= 3 × 3 × 3 × 3 = 81

Bruner, Goodnow, Austin (1956)

Describing the Microworld

• Shapes = { , , }
• Number = {1, 2, 3}
• Texture = {Shaded, Light, Dark}
• Frame = {Single, Double, Triple}

Number of percepts:

= |Shapes| × |Number| × |Texture| × |frame|

= 3 × 3 × 3 × 3 = 81

9 9

generalization lattice

Occam’s Razor

• Favor the simple hypotheses when multiple ones fit

the data

f(x) → θ1x + θ2 f(x) → θ1x2 + θ2x + θ3 f(x) → θ1x7 + θ2x6 + θ3x5 + θ4x4 + θ5x3 + θ6x2 + θ7x + θ8

= = =

Occam’s Razor

• Favor the simple hypotheses when multiple ones fit

the data

f(x) → θ1x + θ2 f(x) → θ1x2 + θ2x + θ3 f(x) → θ1x7 + θ2x6 + θ3x5 + θ4x4 + θ5x3 + θ6x2 + θ7x + θ8

= = =

Occam’s Razor

• Favor the simple hypotheses when multiple ones fit

the data

f(x) → θ1x + θ2 f(x) → θ1x2 + θ2x + θ3 f(x) → θ1x7 + θ2x6 + θ3x5 + θ4x4 + θ5x3 + θ6x2 + θ7x + θ8

= = =

Occam’s Razor

• Favor the simple hypotheses when multiple ones fit

the data

f(x) → θ1x + θ2 f(x) → θ1x2 + θ2x + θ3 f(x) → θ1x7 + θ2x6 + θ3x5 + θ4x4 + θ5x3 + θ6x2 + θ7x + θ8

= = =

Occam’s Razor

• Favor the simple hypotheses when multiple ones fit

the data

f(x) → θ1x + θ2 f(x) → θ1x2 + θ2x + θ3 f(x) → θ1x7 + θ2x6 + θ3x5 + θ4x4 + θ5x3 + θ6x2 + θ7x + θ8

= = =

Occam’s Razor

• Favor the simple hypotheses when multiple ones fit

the data

f(x) → θ1x + θ2 f(x) → θ1x2 + θ2x + θ3 f(x) → θ1x7 + θ2x6 + θ3x5 + θ4x4 + θ5x3 + θ6x2 + θ7x + θ8

= = =

What is the function between x and y?

slides from Josh Tenenbaum’s 9.66

• Given examples of x, y pairs, learn function f(x) → y

What is the function between x and y?

slides from Josh Tenenbaum’s 9.66

• Given examples of x, y pairs, learn function f(x) → y

What is the function between x and y?

slides from Josh Tenenbaum’s 9.66

• Given examples of x, y pairs, learn function f(x) → y

f(x) = sin(x)