SLIDE 1
Apache Storm Christopher Little Apache Storm Alternatives Storm - - PowerPoint PPT Presentation
Apache Storm Christopher Little Apache Storm Alternatives Storm - - PowerPoint PPT Presentation
Real-time Web Marketing with Apache Storm Christopher Little Apache Storm Alternatives Storm Hadoop Spark Streaming Processing Model DAG MapReduce DAG Processing Unit Record-at-a-time Batch Mini Batch Latency Sub-second High Few
SLIDE 2
SLIDE 3
Alternatives
Storm Hadoop Spark Streaming Processing Model DAG MapReduce DAG Processing Unit Record-at-a-time Batch Mini Batch Latency Sub-second High Few seconds Fault Tolerance At least once Exactly once Exactly once
SLIDE 4
Architecture (cluster)
Nimbus Zookeeper Zookeeper Supervisor Supervisor Supervisor Worker Process Worker Process Worker Process Worker Process
SLIDE 5
Architecture (cluster)
Nimbus Zookeeper Zookeeper Supervisor Supervisor Supervisor Worker Process Worker Process Worker Process Worker Process
SLIDE 6
Architecture (work) Spout Spout
Bolt Bolt Bolt Bolt
SLIDE 7
Architecture (work) Spout Spout
Bolt Bolt Bolt Bolt
(x, y, z)
SLIDE 8
Architecture (work) Spout Spout
Bolt Bolt Bolt Bolt
(x, y, z)
x3 x5 x1 x8 x3 x10
SLIDE 9
Example Topology - Wordcount Spout
Split Count
(“Mary had a little Mary”) x5 x8 x12 (“Mary”) (“had”) (“a”) (“little”) (“Mary”) (“Mary”, 1) (“had”, 1) (“a”, 1) (“little”, 1) (“Mary”, 2)
SLIDE 10
Example Topology - Wordcount Spout
Split Count
(“Mary had a little Mary”) x5 x8 x12 (“Mary”) (“had”) (“a”) (“little”) (“Mary”) (“Mary”, 1) (“had”, 1) (“a”, 1) (“little”, 1) (“Mary”, 2)
SLIDE 11
Example Topology - Wordcount Spout
Split Count
(“Mary had a little Mary”) x5 x8 x12 (“Mary”) (“had”) (“a”) (“little”) (“Mary”) (“Mary”, 1) (“had”, 1) (“a”, 1) (“little”, 1) (“Mary”, 2) Shuffle() Group(0)
SLIDE 12
Application
SLIDE 13
SLIDE 14
“there is a strong relationship between gaze position and cursor position”
- Chen et al. (2001)
SLIDE 15
Web Analytics
JavaScript Client
Spout
Heat Map
Event Stream
- Clicks
- Mouse movements
- Scroll event
- Form interaction
- Links followed
Realtime Processing
- Filtering
- Windowing
- Aggregation
SLIDE 16
Session Windowing Example
(k1, (v1, 13:02)) (k2, (v2, 13:14)) (k1, (v3, 13:57)) (k1, (v4, 13:20)) (k1, (v1, [13:02, 13:32])) (k2, (v2, [13:14, 13:44])) (k1, (v3, [13:57, 14:27])) (k1, (v4, [13:20, 13:50])) (k1, ([(v1, [13:02, 13:32]) ,(v3, [13:57, 14:27]) ,(v4, [13:20, 13:50])])) (k2, ([(v2, [13:14, 13:44])])) (k1, ([v1, v4], [13:02, 13:50])) (k1, ([v3], [13:57, 14:27])) (k2, ([v2], [13:14, 13:44]))
ParDo ParDo GroupByKey
AssignWindows M e r g e W i n d
- w
s MergeWindows
SLIDE 17