real-time alerting, analytics and reporting at scale with Apache - PowerPoint PPT Presentation

1 real-time alerting, analytics and reporting at scale with Apache Kafka and Apache Ignite @denismagda | @jwfbean | @IMCSUMMIT @denismagda | @jwfbean

2 Hello 👌 @jwfbean @denismagda @denismagda | @jwfbean | @IMCSUMMIT

Digital transformation challenges @denismagda | @jwfbean | @IMCSUMMIT


 4 Digital Transformations Challenges 
 Application Layer ● 10-100x more queries and transactions Web-Scale Apps IoT Mobile Apps Social Media ● 50x more data today as a decade ago 10-100x 
 10-1000x 50x 
 Queries and Faster Analytics Data Storage Transactions (Hours to Sec) (Big Data) ● Overnight analytics become real-time (per sec) Data Layer NoSQL RDBMS Hadoop @denismagda | @jwfbean | @IMCSUMMIT

5 @denismagda | @jwfbean | @IMCSUMMIT

In-Memory Computing and Stream processing • Performance and velocity increases Application Layer Web-Scale Apps IoT Mobile Apps Social Media • Scalability up to petabytes of data GridGain In-Memory Computing Platform Confluent Platform • Act faster by analyzing streams of data Transactional Persistence Event Streaming using SQL language @denismagda | @jwfbean | @IMCSUMMIT

8 Streaming-First Workd @denismagda | @jwfbean | @IMCSUMMIT

9 Kappa Architecture: GridGain and Kafka Connect 💶 @denismagda | @jwfbean | @IMCSUMMIT

Demo @denismagda | @jwfbean | @IMCSUMMIT

Enter Kafka Connect @denismagda | @jwfbean | @IMCSUMMIT

13 CONSUMER PRODUCER Producer 
 Consumer 
 Application Application @denismagda | @jwfbean | @IMCSUMMIT

14 KAFKA CONNECT KAFKA CONNECT CONSUMER PRODUCER Source SMTs SMTs Converter Converter Sink Connector Connector @denismagda | @jwfbean | @IMCSUMMIT

15 Discover connectors, SMTs, and converters @denismagda | @jwfbean | @IMCSUMMIT

16 Discover connectors, SMTs, and converters Descriptions, licensing, support, and more @denismagda | @jwfbean | @IMCSUMMIT

17 Lower the Bar to Enter the World Core developers who use Java/Scala Coding Sophistication streams Core developers who don’t use Java/Scala Data engineers, architects, DevOps/SRE BI analysts User Population @denismagda | @jwfbean | @IMCSUMMIT

Store and process with GridGain @denismagda | @jwfbean | @IMCSUMMIT

19 GridGain: Real-time Streaming and Analytics @denismagda | @jwfbean | @IMCSUMMIT

20 Essential GridGain APIs Co-located Computations Distributed memory-centric Distributed Key-Value storage Read, write and transact with Combines the performance and scale of in- Brings the computations to the servers fast key-value APIs memory computing together with the disk where the data actually resides, eliminating durability and strong consistency in one need to move data over the network system Distributed SQL ACID Transactions Machine and Deep Learning Supports distributed ACID transactions for Set of simple, scalable and efficient tools Horizontally, fault-tolerant distributed SQL key-value as well as SQL operations that allow building predictive machine database that treats memory and disk as learning models without costly data active storage tiers transfers (ETL) @denismagda | @jwfbean | @IMCSUMMIT

21 GridGain SQL For Real-Time Analytics Ignite Node Toronto 2 Montreal Canada Ottawa Calgary 1 Ignite Node 3 2 Mumbai India New Delhi 1. Initial Query 2. Query execution over local data 3. Reduce multiple results in one @denismagda | @jwfbean | @IMCSUMMIT

one last thing… @denismagda | @jwfbean | @IMCSUMMIT

Q&A @denismagda | @jwfbean | @IMCSUMMIT

Thanks! @denismagda dmagda@gridgain.com @jwfbean jwfbean@confluent.io @denismagda | @jwfbean | @IMCSUMMIT @

25