Query Processing over Incomplete Autonomous Databases - - PowerPoint PPT Presentation

• Query Processing over Incomplete

Autonomous Databases

• !

"# $! %%%&

• Introduction

''%(%)% )&&'%% ''%

– *+(+!+,-+,(,*+

• !

Incompleteness in Web Databases

• "#$
• %&'('#$
• )*+),-

,.+./- *- 0,)). 1.2 3 00+,- 00+- ),+4- .10/4 *4 5+ ,+*- .+/- ..+/- 10*1 *. 6'+

• '#-
• 66

7( 8

• Problem
• !'%9,

)./+

Want a ‘Honda Accord’ with a ‘sedan’ body style for under ‘$12,000’ High Precision Low Recall

"9 :'($;

'

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 '

Many entities corresponding to tuples with missing values might be relevant to the user query

• Low Precision,

Infeasible

Possible Naïve Approaches

>?'#@5A >?'#@5A Low Recall Costly, Infeasible

3+5!6BCB=> -) ''%, ('6! 4+!6%!B> ,)( '6! ()) (%' 7+!BD> -)( '6!+'', )(%' % &'(('

• Outline

56$E 8&9/ :&;* !;#

• The QPIAD Solution

Select Top K Rewritten Queries <3=>'6? <4=>'6@? <7=>'6.

3E>?'@5 A

' <: <' = ' 3 ' ? 4223 ! 4

• @?

4224 ! 7 8 . 4225 ! ?

• @?

4227 B% 5 ' ! 422? B% A 9 ! 4224 ' B ' ? 422A B% ' <: <' = ' 3 ' ? 4223 ! 4

• @?

4224 ! 7 8 . 4225 !

Base Result Set

AFD: Model~> Body style

' <: <' = ' 5(' ?

• @?

4227 B% 2+B B ' ? 422A B% 2+7 Ranked Relevant Uncertain Answers

• LEARN

LEARN REWRITE REWRITE RANK RANK EXPLAIN EXPLAIN

• LEARN

LEARN REWRITE REWRITE RANK RANK EXPLAIN EXPLAIN

• 8$$';

– ()( – *(&%%'% – !''(

Learning Statistics to Support Ranking & Rewriting

• C D!!
• 5 C &&.#E&'#E

;&&.

*8<F-66F' G,H, IJK 2LL63

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

Make, Body ~> Model P(Model=Accord | Make=Honda, Body=Coupe)

• <'

4223 = <: ' ' !&

• )/''>

– <3=>'6? – <4=>'6@? – <7=>'6.

Rewriting to Retrieve Relevant Uncertain Results

! 4225 . 8 ! 4227 @?

• <:

<' = ' ' ? 4223 !

• @?

4224 !

Base Set for Q:(Body=Convt)

AFD: Model~> Body

• #E',

& )

3 '?,@?. 4 '

%+

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

• 8$$';

–

• (& )(

–

• )(/('/&

• Selecting/Ordering Top-K Rewritten Queries

8M *'8

• M *'-

( ) ( ) ( )

R P R P F + ⋅ ⋅ ⋅ + = α α

α

1

• &C /%'F&<
• '/%''

P R R P F = ⋅ =

– B'&:($G

• & '(:

All tuples returned for a single query are ranked equally

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

• 8$$';

–

• (&,C&

– ('C&'

• Explaining Results to the Users

'F+ !) '$>

) *.&

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

• 8$$';

– (= – ((.&

Make, Body ~> Model yields This car is 83% likely to have Model=Accord given that its Make=Honda and Body=Sedan

• Outline

!9/ $6$E :&;* !;#

• #?'H<:H<'H=HH<A

Leveraging Correlation between Data Sources

>?'@5A

<'>3#?'H<:H<'H=HH<A

#?<:H<'H=HH<A

>?'@5A >?<'@5A >?<'@5A I

UNION

#2 F?<'

• 'A

69> '= && % &N =%

AFDs learned from Cars.com

• Handling Aggregate and Join Queries
• (((<
• "<

<: <' = < ' ' B% A2,?22 9 ! 422? 43,352 B% ! 4224 57,4B5 ' ? 4227 ?3,A52 4225 4224 4223 = <:

• '

B% ') 9 ( ') 57,4B5 4224 !

B% B%

A2,?22 A2,?22 < ')

• <: <' =

' '

B% 1* "' B%

'

B% 11

Make=Honda

5 ? 7 4 3 ' !& B% ! B% ! ' <: <' ' ?

• @?

?5A@+)H ?5A@+*

8 .

• 745

?5A@+4H ?5A@+/

' ! t1 + t3 + t2 = 3 t 1 + t 3 + . 9 ( t 2 ) + . 4 ( t 4 ) = 3 . 3

Include a portion of each tuple relative to the probability its missing value matches the query constraint

5?JA@.+.

>?5?JA 8$'@5A

Only include tuples whose most likely missing value matches the query constraint

5?JA@.

• Outline

!9/ 8&9/ K !;#

• QPIAD Web Interface

http://rakaposhi.eas.asu.edu/qpiad

• Empirical Evaluation
• E>

– 5

O !+ O B%,55,222&

– 5

O 9PE:( O 33%,422,222&

– 5

O !',!:& O 34%,?5,222&

• &>

– 7C35Q'%

• :&>

– 32Q&( – '''&) ('

• *>

–

• ('-)('+(+/,,+

– <+(+',(((,R,+ – ('+(+,&,+

• Experimental Results – Ranking & Rewriting

<8:E+-*9-*EC

!6%!B M )S))' %

• Experimental Results – Ranking & Rewriting

<8:E+-*EC ((

!BD M )S))&''( &%%%'

• Experimental Results – General Queries

((( "

• Experimental Results – Learning Methods

!

0.2 0.4 0.6 0.8 1 Year Make Model Price Mileage Body Certified

Accuracy

NBC AFD-Enhanced NBC BayesNet 3 BayesNet 2 Decision Tree

• Experimental Summary
• )(N- (

– M <8:E(&

• *9-*E %(&

– (( M <8:E/)& % '%

• *E
• ('

– #E &'

• <

– (((/( ) (&'' – <8:E( R/ )'(%&

• ''*.&

–

• %(')+++&

– *&#C – *(%)

• Outline

!9/ 8&9/ :&;* 5KF8:

• Related Work
• <(:&E%

– 8%'&&M &&&!'' 9%,TC9%,!'9% – 8%%&&M /'%&'( %) '&%)

• 8%%E%

– 9&')%'%(&%%. – ),) ,''&%' '('%

• <-N-.

– &&.)( %.) – P(&)('%

• ((T

– !&&&&(%%( ,,',( ,,+ – P) /'&'%)%) '%( Our work fits here

• ('

• Contributions
• *)

('/&

– <-)(

• ))( '%

)'( '('%

– #EC*'!

• )(; ('%)

&'

– #C%' (

• #E &R>

– <-)( – # – *.&

• Current Directions – QUIC (CIDR `07 Demo)
• ='''>
• (

> L'!!:M 5$> ';C

• 'E
• E%&&'%>
• '
• '

http://rakaposhi.eas.asu.edu/quic

!?!A>

(9$(

• )(''%

– 5'# – 5&:'# – 5&'# E#

• #

• Problem
• !'9,

)./+

Want a ‘Honda Accord’ with a ‘sedan’ body style for under ‘$12,000’ High Precision Low Recall

"9E

• ''

;

'

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 '

Many entities corresponding to tuples with missing values might be relevant to the user query

• Handling Aggregate and Join Queries
• (((<
• "<

– -&&'

5 ? 7 4 3 ' !& B% ! B% ! ' <: <' ++ ' ?

• @?

?5A@+)H ?5A@+*

8 .

• 745

?5A@+4H ?5A@+/

' ! t1 + .9(t2) + t3 + .4(t4) =3.3 t 1 + t 2 + t 3 = 3

Only include tuples whose most likely missing value matches the query constraint

5?JA@.

Include a portion of each tuple relative to the probability its missing value matches the query constraint

5?JA@.+.

>?5?JA8$'@5A

• 8$$';

– ()( – *(&%%'% – !''(

Learning Statistics to Support Ranking & Rewriting

• C D!!
• 5 C &&.#E&'#E

;&&.

• # M &,-,8:&

*<F-6&<U&--U8:- *8<F-66F' G,H, IJ 2LL63

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

• !
• :&(()%&(

('(')

Incompleteness in Web Databases

"#$

• %&''%'

%&

• %&'('#$
• ''%+(+ +

'& +(+&) &'

• %%

(%''

)*+),- ,.+./- *- 0,)). 1.2 3 00+,- 00+- ),+4- .10/4 *4 5+ ,+*- .+/- ..+/- 10*1 *. 6'+

• '#-
• 66

7( 8

• Introduction

''%(%)% )&&'%% ''%

– *+(+!+,-+,(,*+

,'%('&' &'(&&'%

• Problem
• !'9,

)./+

Want a ‘Honda Accord’ with a ‘sedan’ body style for under ‘$12,000’ High Precision Low Recall

'

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 '

• Problem
• !'9,

)./+

Want a ‘Honda Accord’ with a ‘sedan’ body style for under ‘$12,000’ High Precision Low Recall

"9E

• ''

;

'

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 '

Many entities corresponding to tuples with missing values might be relevant to the user query

• Selecting Top-K Rewritten Queries using F-Measure
• &

V/)

8M *'8

• M *'-

%'8;*++

( ) ( ) ( )

R P R P F + ⋅ ⋅ ⋅ + = α α

α

1

• P9:P>#C%'

)(%&&

– α@*

86-

– αN*

8K-

– αO*

8L-

• P9*>#C'(

&C/%'' ')

We still want the most precise tuples first!

• 9,)'

&C/)(&& %%)&'

RANK RANK

• Ordering Top-K Queries using Estimated Precision
• P)='&C)/,)'

''

P R R P F = ⋅ =

– B'&:( $H$9$$G

• :(/''&)

&'

NOTE: All tuples returned for a single query are ranked equally

– &)'

RANK RANK

• Problem
• !'%9,

)./+

Want a ‘Honda Accord’ with a ‘sedan’ body style for under ‘$12,000’ High Precision Low Recall

'

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 '

• Problem
• !'%9,

)./+

Want a ‘Honda Accord’ with a ‘sedan’ body style for under ‘$12,000’ High Precision Low Recall

"9E

• ''
• :'($;

'

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• 01,222

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ' ' ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 ' '

• ;

3111 ' ' 033,422 032,522

• 5

<: <' = ' ' ' 4223 ; ; ' 4224 '

Many entities corresponding to tuples with missing values might be relevant to the user query

• Query Rewriting in QPIAD

Select Top K Rewritten Queries <3=>'6? <4=>'6@? <7=>'6.

3E>?'#@5A

' <: <' = ' 3 ' ? 4223 ! 4

• @?

4224 ! 7 8 . 4225 ! ?

• @?

4227

• 5

' ! 422?

• A

9 ! 4224 ' B ' ? 422A

• '

<: <' = ' 3 ' ? 4223 ! 4

• @?

4224 ! 7 8 . 4225 !

Base Result Set

AFD: Model~> Body style

' <: <' = ' 5(' ?

• @?

4227

• 2+B

B ' ? 422A

• 2+7

Ranked Relevant Uncertain Answers

• WecanselecttopKrewrittenqueriesusingF

measure FMeasure=(1+α)*P*R/(α*P+R) P– EstimatedPrecision R– EstimatedRecallbasedonPandEstimated Selectivity

• LEARN

LEARN REWRITE REWRITE RANK RANK EXPLAIN EXPLAIN

• Handling Aggregate Queries

&&, ((('!P9%(

• – #C%6CB>%E9'

'$($

3+ :(/'%'%+ 4+ !&(((%&+ 7+ %()/'(<8:E (+ ?+ :)/'.''+ 5+ #&.'' % &=(+ A+ : /&'( /,'&(((( + B+

• (((()(((+

• Learning Statistics to Support Ranking & Rewriting
• ('%(D!!
• (%&&.

#E&'#E'&&.

• (**-)<%'>

– )/'& &< –

• ('%&

&-- – 8&&)(& 8:-

*<F-6&<U&--U8:- *8<F-66F' G,H, IJ 2LL63

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

• Rewriting to Retrieve Relevant Uncertain Results
• #E

&,= '%'' '

! 4225 . 8 7 ! 4227 @?

• ?

' <: <' = ' 3 ' ? 4223 ! 4

• @?

4224 !

Base Set for Q:(Body=Convt)

AFD: Model~> Body

• & )'6! )>

3 *& ('% 4 '& (',

&= '!

• /%,'#EJK,)(

)/%>

– E'%' – #',()/( &'(%)'

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

• Selecting/Ordering Top-K Rewritten Queries
• &

V/)

8M *'8

• M *'-

( ) ( ) ( )

R P R P F + ⋅ ⋅ ⋅ + = α α

α

1

• P9:P>#C%'

)(%&&

• P9*>#C'(&C/

%''')

• P)='&C)/,)'

''

P R R P F = ⋅ =

– B'&:( $H$9$$G

• :(/''

&)&'

NOTE: All tuples returned for a single query are ranked equally

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

• Explaining Results to the Users

>?<:@"''<'@''NP*1HA !) '$>

) *.& > )())(&&W

LEARN LEARN – – REWRITE REWRITE – – RANK RANK – – EXPLAIN EXPLAIN

• Experimental Results – Learning Methods

!

0.2 0.4 0.6 0.8 1 Year Make Model Price Mileage Body Certified

Accuracy

NBC AFD-Enhanced NBC BayesNet 3 BayesNet 2 Decision Tree

(#E'( & #EC*'! &%)

• %)+++&

<8:E%)' )'& )%' !'%

• Handling Aggregate and Join Queries
• (((<

– &&,((( '!P9%(

O #C%6CB>%E9'' $($

• "<

– R/%(''/, )R'' – *'&''%'' ) ''()/ – (&N,'%' /&''/

O 8$$($($''E $Q

• Experimental Results – Ranking & Rewriting

*α #C

> 32/V )/))'

• Combined Effect = α + k

Sets tradeoff

• f precision

& recall Resource limitation on # of rewritten queries

&,))/ ))&%' '%((&

• Experimental Results – Ranking & Rewriting

<8:E+-*9-*EC

!6%!B 9 %&)( '%/ $$ !6%!B &)('%) %/

• Experimental Results – Ranking & Rewriting

<8:E+-*EC ((

!BD &&( ' )' !BD &&&)N(' ' % &(( '%, )( % / % !BD ) /((9 % '

• Experimental Results – Learning Methods

!

0.2 0.4 0.6 0.8 1 Year Make Model Price Mileage Body Certified

Accuracy

NBC AFD-Enhanced NBC BayesNet 3 BayesNet 2 Decision Tree

(#E'( & #EC*'! &%)

• %)+++&

<8:E%)' )'& )%' !'%

• Experimental Results - Extensions

((( "

8'( / ( &',)%( )(& &&.42Q/ 322Q)&' '