Upquery responses in total order with updates Goal: upquery restores state as if present all along. � 17
Upquery responses in total order with updates Goal: upquery restores state as if present all along. 3 2 resulting state 3 respects total order 2 1 � 17
Upquery responses in total order with updates Goal: upquery restores state as if present all along. 3 2 2 resulting state 3 3 resulting state 2 respects total order 2 violates total order 2 1 1 � 17
Upquery responses in total order with updates Goal: upquery restores state as if present all along. 3 2 2 resulting state 3 3 resulting state 2 respects total order 2 violates total order 2 1 1 More complex cases: merged upquery responses, evictions ( Paper ). � 17
Challenges implementing partially-stateful data-flow 1. Concurrent upqueries and forward processing — races! Must maintain correctness under concurrency! 2. Update processing may require absent state � 18
Challenges implementing partially-stateful data-flow 1. Concurrent upqueries and forward processing — races! Must maintain correctness under concurrency! 2. Update processing may require absent state 2 COUNT absent 3 … � 18
Challenges implementing partially-stateful data-flow 1. Concurrent upqueries and forward processing — races! Must maintain correctness under concurrency! 2. Update processing may require absent state Drop updates that touch absent 2 state, future upquery repeats them. COUNT absent 3 … � 18
Challenges implementing partially-stateful data-flow 1. Concurrent upqueries and forward processing — races! Must maintain correctness under concurrency! (see Paper) 2. Update processing may require absent state Drop updates that touch absent 2 state, future upquery repeats them. COUNT absent 3 … � 18
Noria implementation � 19
Noria implementation � 19
Noria implementation MySQL adapter � 19
Noria implementation Data-flow graph Transform MySQL adapter � 19
Noria implementation Data-flow graph Transform MySQL adapter � 19
Noria implementation Data-flow graph Transform • 45k lines of Rust + 15k libraries • RocksDB for base table storage MySQL adapter • ZooKeeper for leader election � 19
Evaluation 1. Can Noria improve a real web application’s performance? 2. How does Noria compare to alternative approaches? 3. Can Noria change queries without downtime? � 20
Evaluation 1. Can Noria improve a real web application’s performance? 2. How does Noria compare to alternative approaches? 3. Can Noria change queries without downtime? Setup Amazon EC2 c5.4xlarge instance (16 vCPUs) Open-loop clients, measuring latency & throughput � 20
Evaluation 1. Can Noria improve a real web application’s performance? 2. How does Noria compare to alternative approaches? 3. Can Noria change queries without downtime? Setup Amazon EC2 c5.4xlarge instance (16 vCPUs) Open-loop clients, measuring latency & throughput comparison with di ff erential dataflow } see Paper multi-machine experiments � 20
Case study: Lobsters (http://lobste.rs) � 21
Case study: Lobsters (http://lobste.rs) ‣ Ruby-on-Rails application with MySQL backend � 21
Case study: Lobsters (http://lobste.rs) ‣ Ruby-on-Rails application with MySQL backend ‣ Hand-optimized by developers to pre-compute aggregations � 21
Case study: Lobsters (http://lobste.rs) ‣ Ruby-on-Rails application with MySQL backend ‣ Hand-optimized by developers to pre-compute aggregations ‣ Noria data-flow with 235 operators, 35 views � 21
Case study: Lobsters (http://lobste.rs) ‣ Ruby-on-Rails application with MySQL backend ‣ Hand-optimized by developers to pre-compute aggregations ‣ Noria data-flow with 235 operators, 35 views ‣ Emulate production load � 21
Can Noria improve Lobsters’ performance? � 22
Can Noria improve Lobsters’ performance? Better Better � 22
Can Noria improve Lobsters’ performance? Better Better � 22
Can Noria improve Lobsters’ performance? Better Better � 22
Can Noria improve Lobsters’ performance? Better Better Noria with natural queries supports 5x MySQL’s throughput. � 22
How does Noria compare to alternatives? Better Better � 23
How does Noria compare to alternatives? Better Better ‣ Zipf-distributed story ID, 95% reads, 5% writes ‣ No TX, all in-memory � 23
How does Noria compare to alternatives? Better Better ‣ Zipf-distributed story ID, 95% reads, 5% writes ‣ No TX, all in-memory � 23
How does Noria compare to alternatives? Better Better ‣ Zipf-distributed story ID, 95% reads, 5% writes ‣ No TX, all in-memory � 24
How does Noria compare to alternatives? Better Better ‣ Zipf-distributed story ID, 95% reads, 5% writes ‣ No TX, all in-memory � 24
How does Noria compare to alternatives? Better Better ‣ Zipf-distributed story ID, Noria outperforms an in-memory key- 95% reads, 5% writes value store and simplifies its interface. ‣ No TX, all in-memory � 24
Can Noria change queries without downtime? Stories Votes COUNT JOIN FILTER 3 2 � 25
Can Noria change queries without downtime? Ratings Stories Votes ⭐⭐⭐ ⭐ COUNT ⭐⭐ JOIN AVG COUNT FILTER 3 JOIN 3 ⭐ 2 1.5 ⭐ � 25
Can Noria change queries without downtime? Better � 26
Can Noria change queries without downtime? Better new table & query added � 26
Recommend
More recommend
![Traditional Netlist SignOff Model ASIC Vendors ASIC Customers [Front End] [Back End] Functional](https://c.sambuz.com/857766/traditional-netlist-signoff-model-s.jpg)
