Real-time raise alerts Real-Time Real-time with historical - - PowerPoint PPT Presentation

• Real-time

raise alerts

• Real-time with historical
• Correlate
• Offline
• Develop initial monitoring query
• Back-test
• Progressive

Non-temporal analysis

Engine

+ Fabric

Interactive Query Authoring Real-Time Dashboard

• Performance
• Fabric & language integration
• Query model

Scenarios

• monitor

telemetry & raise alerts

• correlate real-

time with logs

• develop initial

monitoring query

• back-test over

historical logs

• offline analysis

(BI) with early results

• Performance
• Fabric & language integration
• Query model

• Key enabler: performance +

fabric & language integration + query model

struct ClickEvent { long ClickTime; long User; long AdId; } var str = Network.ToStream(e => e.ClickTime, Latency(10secs)); var query = str.Where(e => e.User % 100 < 5) .Select(e => { e.AdId }) .GroupApply( e => e.AdId, s => s.Window(5min).Aggregate(w => w.Count()));

query.Subscribe(e => Console.Write(e)); // write results to console

stream of batches

• More load  larger batches  better throughput

…

𝑝𝑞2

… …

𝑝𝑞1

class DataBatch { long[] SyncTime; ... Bitvector BV; } class UserData_Gen : DataBatch { long[] c_ClickTime; long[] c_User; long[] c_AdId; }

…

𝑝𝑞2

… …

𝑝𝑞1

timestamp payload columns bitvector

str.Where(e => e.User % 100<5); Send(events) ...

Application

Receive(results) On(Batch b) { for i = 0 to b.Size { if !(b.c_User[i]%100 < 5) set b.bitvector[i] } next-operator.On(b) }

Trill

session windows

• Powerful high-perf expression-based user-defined aggregate framework

http://aka.ms/trill