Outline • The Niagara Internet Query System – 1:45 - 2:30 • Punctuating Continuous Data Streams East Coast Systems – 2:30 – 3:00 • Break Larissa Smelkov – 3:00 – 3:10 Wayne Wight • Monitoring Streams (Aurora) Joshua Sunshine – 3:10 – 4:00 Andrew See Swetha Shankar • Discussion – 4:00 – 4:30 The Niagara Internet Query System Demo: Querying XML Documents Sample XML: <?xml version="1.0" encoding="ISO-8859-1"?> • Demo: Querying XML Documents <!DOCTYPE W4F_DOC SYSTEM "movies.dtd"> <W4F_DOC> <Movie> • Motivation <Title>Body Shots</Title> <Year>1999</Year> • Partial Results and Differential Operators <Directed_By> <Director>Michael Cristofer</Director> • Synchronization and Streams </Directed_By> <Genres> • NiagaraCQ (Continuous Queries) <Genre>Drama</Genre> </Genres> </Movie> </W4F_DOC> 1
XML - Q L and SEQL XML - Q L and SEQL WHERE Sample XML-QL: Sample SEQL: Movie CONTAINS <Movie> (Title and Director IS <Title>$v68</> “Titles of movies "Terry Gilliam") <Directed_by> Used to locate relevant directed by Terry <Directory>$v71</> URLs for the Query Gilliam” </> Processing Engine. </> IN “*” conform_to Used by the Query http://www.cs.wisc.edu/niagara/data Processing Engine. /xml-movies/dtd $v71 = “Terry Gilliam” Converted to SEQL for CONSTRUCT the Search Engine. <result> <Title>$v68</> </> Next: Partial Results On To Demo… Issues with Querying the Internet • Distributed over a lot of space • Network connections not stable • Sometimes sites not available • Some data might be an infinite stream These factors are a bottleneck in query evaluation. Blocking operators (nest, average) add to the problem The solution??? Next: Motivation and Partial Results 2
Blocking Operators Partial Results Operators that produce output only after Useful for infinite stream of data and also for receiving complete input. slow connections. Traditional QE architecture unsuitable because: • Examples: – Nest Unsuitable assumptions about properties of – Average operators Lack of synchronization support for producing – Set Difference consistent partial results Properties Operators Should Have Flexible Input Property to Produce Partial Results. • If some input streams are available, operators should not block. 1. Flexible input property 2. Anytime property • Example: 3. Non-monotonic input-output property – One website is down, others are not. Partial Results should be available. 3
Anytime Property Non - m o notonic I/O Property • Anytime you ask for a result, the result • Operators should support input that is not should be available. steadily increasing. • Example: • Example: – An average is being computed, and the user – Set Difference: wants the average so far. • Computing {A} – {B}, a new value arrives in {B} that needs to be removed from the output. Differential Operators Synchronization • Operators work on differences between old and • In Order to get new input, rather than re-evaluating the query. partial results, • Each tuple contains old value and new value. operators need to be synchronized • Example: starting with the sources. – Join gets updates from Average and Replicate (More on this with the next – Does not need to re-compute the paper.) join for the entire result set. Next: Synchronization 4
Querying Streams Querying Streams • Sample Data: • Sample Query <NEWSDOC> WHERE <Article> <NEWSDOC> <Company> <Article> <Name>Intel </Name> <Company> “Tell me when a <Symbol>DELL</Symbol> <Symbol>$v1</> </> </> </> </Company> went bankrupt today. IN “http://127.0.0.1/test.php”, company is </Article> <Quotes> mentioned in a … <Quote> <Symbol>$v2</> news stream and <Quotes> <Change>$c</> </> </> has fast falling <Quote> IN “http://127.0.0.1/test2.php”, <Symbol>DELL</Symbol> $v1=$v2, $c < “-10” stock prices.” <Price>120.35</Price> CONSTRUCT <Change>-13</Change> <result><Symbol>$v1</></> </Quote> … Next: NiagaraCQ NiagaraCQ NiagaraCQ • Sub-system of Niagara for Continuous Queries. • Made to support thousands of queries. • Uses timers and changes in source files to trigger queries. • Groups queries based on expression signature to improve performance. 5
NiagaraCQ (cont.) Grouping Queries Plan for Queries vs. Group Plan for Queries. Example XML-QL Queries and their expression signature. Summary Grouping Queries (cont.) (What you should walk away with) • Selection Operators are added incrementally, • Niagara supports XML queries over the most selective first Internet. – For example, “Symbol=INTC” before “Price < 100” • Gives partial results and supports • Join Operators accompanied by Selection streaming input. Operators are added incrementally based on a • Allows continuous queries (NiagaraCQ). cost model – Either split with selection first (few very selective splits), or – Join first, then split (many splits sharing same join) Next: Summary 6
✖ ✭ ☎ ❃ ✟ ✁ ✔ ✂ ❘ ✽ ✭ ✴ ✪ ✟❄ ◆◗ ✩ ✧ ✭ P ✴ ✹ ◆❖ ❑ ✭ ✧ ✁ ✘❁ ✭ ✎ ✙ ✙ ✙ ✁ ✿ ✂ ✎ ✒ ✎ ✿ ✖ ✌ ✁ ✂ ✖ ✎ ✌ ✒ ✙ ✟ ✘ ✂ ✌ ✟ ✮ ✭✲ ✌ ☎ ✏❆ ✁ ✔ ✙ ✎ ✖ ✎ ❀ ✙ ✎ ✂ ❁ ✿ ✖ ✎ ✁ ✔ ✂ ✖ ✁ ✔ ❅ ❄ ✿ ✾ ✘ ✫ ✴ ✫ ✬ ❍ ✭ ★ ✴ ✱ ✮ ✬ ● ✳ ✮ ✝ ✭ ★ ❋ ❉❊ ✰ ❈ ❇ ✛ ✖ ✎ ✞ ✂ ✂ ✂ ✬ ✺ ✲ ✩ ✴ ✹ ✬ ★ ✳ ✹ ✮ ✳ ✫ ✴ ✮ ✬ ✳ ◆ ✲ ✩ ✴ ✳ ✭✲ ◗ ✩ ✳ ✭ ◆ ✧ ✬ ✪ ◗ ❖ ❍ ✭ ✴ ✳ ✮ ✬ ✬ ✪ ✪ ✭ ✩ ✴ ✴ ❖ ◗ ✴ ❖ ✩ ✪ ✩ ★ ✁ ❁ ❈ ✙ ✂ ✿ ☎ ✙ ✁ ✟ ✁ ✂ ☎ ✒ ❉❊ ✘ ✝ ❄ ✁ ✔ ✂ ✁ ✿ ✎ ✔ ❅ ✰ ❋ ✴ ✮ ▲ ❑ ❍ ✮ ✳ ✼ ✫ ✩ ✮ ✩ ✬ ✶ ★ ✭ ★ ✴ ✱ ✮ ✬ ● ✳ ✴ ✮ ✭ ✴ ✘ ☎ ✬ ✶ ✷ ✦✶ ✵ ✣ ✴ ✳ ✱ ✮ ✭✲ ✭✲ ✮✭ ✬ ✮✱ ✯✰ ✭ ✧ ✮ ✧ ✦ ✫ ✮ ✹ ✪ ❁ ✘ ✟ � ✽ ✮ ✩ ✴ ✮✱ ✬ ✲ ✬ ✼ ✴ ✳ ✹ ✳ ✭ ✫✻ ✩ ✺ ✴ ✩ ✩ ✁ ✎ ✒ ✎ ✄ ✟ ✁ ✎ ✌ ✑ ✏ ✍ ✔ ✌ ☞ ✟ ✁ ✞ ✝ ✁ ✂ ✁ � ✓ ✁ ✣ ✌ ✢ ✜ ✙ ✌ ✟ ✌ ✛ ✁ ✚ ✙ ✏ ✌ ✎ ✖ ✘ ✁ ✗ ✏ ✖ ✌ ✏ ✟ ✾✿ ✪ ✒ ✟ ✿ ✝ ✖ ✘ ❂ ✽ ❀ ✘ ✘ � ✙ ✽ ✙ ✁ ✌ ✂ ✟ ✁ ✘❁ ✟ � ☎ ✎ ✾✿ ❁ ✂ ☎ ✘ ✙ ✟ ✘ ✂ ✌ ✟ ✁ ✘ ✘ ✟❄ ✁ ☎ ❃ ✁ ✒ ✘ ✓ ✽ ✖ ✁ ✎ ✌ ✘ ✚ ✽ ✙ ✒ ✌ ✟ ✟ ✂ ✓ ✟ ❀ ✟ ✖ ✘ ✎ ✂ ☎ ✿ ✘ ✓ ✏ ✖ ✘ ✁ ✒ ✌ ✎ ✙ ✁ ☞ ✽ ✙ ✝ ✎ ✂ ✳ ✒ ✑ ✁ ✌ ✿ ✔ ✂ ✌ References • “NiagaraCQ: A Scalable Continuous Query System for Internet Databases”, Jianjun Chen, David J. DeWitt, Feng Tian, Yuan Wang. SIGMOD 2000 , p379-390. ✠☛✡ ✠☛✡ ✄✆☎ • “The Niagara Internet Query System”, Jeffrey Naughton, David DeWitt, David Maier, and many others. ✠✕✡ • “Differential Evaluation of Continual Queries”, L. Liu, C. Pu, R. ✬✆✭ ✤✥✦✆✧ ★✆✩ Barga, T. Zhou, Proceedings of the 16 th International Conference on Distributed Computing Systems. May, 1996. ✬✆✸ • “XML-QL: A Query Language for XML”, A. Deutsch, M. Fernandez, D. Florescu, A. Levy, D. Suciu. Proceedings of the Eighth International World Wide Web Conference, Toronto, May, 1999. ✼❏■ ❑▼▲ ✭✹❏❙ 7
Recommend
More recommend
