Real-time raise alerts Real-Time Real-time with historical - PowerPoint PPT Presentation

• Real-time raise alerts Real-Time • Real-time with historical Dashboard • Correlate Engine + Fabric • Offline • Develop initial monitoring query • Back-test • Progressive Non-temporal analysis Interactive Query Authoring

• Performance Scenarios • monitor telemetry & raise alerts • Fabric & language integration • correlate real- time with logs • develop initial monitoring query • back-test over historical logs • Query model • offline analysis (BI) with early results

• Performance • Fabric & language integration • Query model

• Key enabler: performance + fabric & language integration + query model

struct ClickEvent { long ClickTime; long User; long AdId; } var str = Network.ToStream(e => e.ClickTime, Latency(10secs)); var query = str.Where(e => e.User % 100 < 5) .Select(e => { e.AdId }) .GroupApply( e => e.AdId, s => s.Window(5min).Aggregate(w => w.Count())); query.Subscribe(e => Console.Write(e)); // write results to console

stream of batches … 𝑝𝑞 2 … 𝑝𝑞 1 … • More load  larger batches  better throughput

… 𝑝𝑞 2 class DataBatch { long[] SyncTime; ... … 𝑝𝑞 1 Bitvector BV; } … class UserData_Gen : DataBatch { long[] c_ClickTime; long[] c_User; payload columns timestamp long[] c_AdId; bitvector }

Application Receive(results) Trill On (Batch b) { for i = 0 to b.Size { if !( b.c_User[i]%100 < 5 ) set b.bitvector[i] } next-operator.On(b) } Send(events) ... str.Where(e => e.User % 100<5);

session windows • Powerful high-perf expression-based user-defined aggregate framework

http://aka.ms/trill