Feature Extraction with Description Logics Functional Subsumption - - PowerPoint PPT Presentation

Feature Extraction with Description Logics Functional Subsumption

Rodrigo de Salvo Braz Dan Roth University of Illinois at Urbana-Champaign

A conflict

? Most machine learning algorithms use

feature vectors as inputs.

? Most data is best represented as structured data. ? Feature extraction is the conversion from one to

the other (and may be most of the work).

before person name(“Mohammed Atta”) gender(male) city person date month(April) year(2001) country

Mohammed Atta met with an Iraqi intelligence agent in Prague in April 2001.

meeting participant participant location time name(Iraq) affiliation nationality after word(an) tag(DT) word(intelligence) tag(NN) word(Iraqi) tag(JJ) before before before

... ...

after after after country name(“Czech Republic”) name(Prague)

• rganization

location end begin

Structured data – I

Attributes (node labels) Roles (edge labels)

Structured data – II

Feature Extraction

male name(john) name(mary) female tall

A

name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child

child

child spouse

Structured example

FE

male female male child

child

female child male spouse friend name(jenny) female

Human-written feature types

1 1

feature vector

Feature Extraction

? Typically done in ad hoc fashion: ? Prevents general analysis; ? Prevents Feature Extraction/Learning unified

analysis (e.g. kernels).

? Using a language is tricky ? Type of inference. ? May be intractable if not careful.

A language for declaring which features to generate

participant time nationality meeting country name(Iraq) year(2001) city person date month(April) year(2001) country meeting participant location time name(Iraq) affiliation nationality name(Prague)

• rganization

Feature type specifications by directed trees Example segment

Generating feature vectors

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child spouse spouse child child male tall female male child child female tall child child

? ? ? ?

Example Feature types

Generating feature vectors

1

2 3

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child

1 3 2

spouse spouse

1

Example Feature types

child

Generating feature vectors

1 3 2

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child spouse

1 1 3 2

spouse

Example Feature types

child

Generating feature vectors

2 3 1

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child

1 3 2

spouse spouse

1

Example Feature types

child

Generating feature vectors

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child spouse child male tall

Example Feature types

Nothing like this in the example!

Generating feature vectors

1 3 2

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child spouse

1 3 2

female male child child

1

Example Feature types

Generating feature vectors

1 2, 3

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child spouse

1 3 2

female tall child child

1

Example Feature types

Generating feature vectors

2, 3 1

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child spouse

1 3 2

female tall child child

1

Example Feature types

Generating feature vectors

male name(john) name(mary) female tall A name(jenny) female name(jill) female name(peter) male age(40) name(margot) spouse friend student child child child child spouse spouse child male tall female male child child female tall child child

1 1 1

Example Feature types

child

Feature Description Logics

(AND (SOME spouse ANY) (SOME child (AND male tall))) (SOME spouse (SOME friend female))

spouse child friend spouse male tall female

Subsumption

?A description C subsumes (⊇) a description D if

every individual in D must be in C, no matter the interpretation.

?Subsumption is tractable.

C D

C = (AND (SOME spouse ANY) (SOME child male)) D = (AND (SOME spouse (SOME student ANY)) (SOME child (AND tall male)) (SOME child female))

Feature extraction as subsumption

name(kelly) female name(carol) friend child child female

(SOME child female) Feature type Example

name(john) child

(AND SOME friend (AND name(carol) SOME child (AND name(kelly) female)) SOME child name(john)) Description of node

⊇

Feature extraction as subsumption

name(kelly) female name(carol) friend child child female

(SOME child female) Feature type Example

name(john) child

name(john) Description of node

⊇

name(kelly) female name(carol) friend child child female

(SOME child female) Feature type Example

name(john) child

(AND name(kelly) female) Description of node

Feature extraction as subsumption

⊇

name(kelly) female name(carol) friend child child female

(SOME child female) (AND name(carol) SOME child (AND name(kelly) female)) Feature type Example

active feature!

name(john) child

Feature extraction as subsumption

⊇

Description of node

subject

• bject

A problem in practice

subject

• bject

buy

⊇

Subsumption would be natural in this case but does not occur

dentist car dentist name(patricia) car model(accord) purchase

A problem in practice

subject

• bject

subject

• bject

kill

⊇

name(JFK) name(castro) name(kennedy) kill

A problem in practice

name(schwarzenegger) job governor name(schwarzneger) job actor job

⊇

Make comparison more flexible

?At core of subsumption algorithm is the

comparison of attributes: ... if (attr1 == attr2) ...

?We simply make that a function call:

... if (f (attr1, attr2) == 1) ...

Is this just a hack?

What about the nice DL semantics?

Is this just a hack?

What about the nice DL semantics? In fact, equivalent to “shallow OR” (tractable).

Is this just a hack?

What about the nice DL semantics? In fact, equivalent to “shallow OR” (tractable). Replace any attr by (OR a1 a2 ... an) where f(attr, ai) = 1.

(AND kill (SOME object JFK)) (AND (OR kill murder assassinate) (SOME object (OR JFK kennedy “John F. Kennedy” ...)))

Why not just use shallow OR then?

?Function is an implicit representation. ?We may incorporate procedural knowledge: ? Typos; ? Similar sounding words; ? Context-sensitive knowledge.

Take home message

? Feature Description Logics provides an

expressive way to deal with structured examples.

? Syntax choices render it tractable. ? Allows for FE-learning integrated approaches

like kernels (Cumby & Roth 2003).

? Can be made even more expressive with little

extra cost by functional subsumption.

The End