SLIDE 1
@denismagda | @jwfbean | @IMCSUMMIT
1
real-time alerting, analytics and reporting at scale with Apache Kafka and Apache Ignite
@denismagda | @jwfbean
@denismagda | @jwfbean | @IMCSUMMIT
2
Hello 👌
@denismagda @jwfbean
real-time alerting, analytics and reporting at scale with Apache - - PowerPoint PPT Presentation
real-time alerting, analytics and reporting at scale with Apache - - PowerPoint PPT Presentation
1 real-time alerting, analytics and reporting at scale with Apache Kafka and Apache Ignite @denismagda | @jwfbean | @IMCSUMMIT @denismagda | @jwfbean 2 Hello @jwfbean @denismagda @denismagda | @jwfbean | @IMCSUMMIT
SLIDE 2
SLIDE 3
@denismagda | @jwfbean | @IMCSUMMIT
Digital transformation challenges
SLIDE 4
@denismagda | @jwfbean | @IMCSUMMIT
Digital Transformations Challenges
- 10-100x more queries and transactions
- 50x more data today as a decade ago
- Overnight analytics become real-time
4
10-100x Queries and Transactions (per sec) 50x Data Storage (Big Data) 10-1000x Faster Analytics (Hours to Sec)
Application Layer
Web-Scale Apps Mobile Apps IoT Social Media
Data Layer
NoSQL RDBMS Hadoop
SLIDE 5
@denismagda | @jwfbean | @IMCSUMMIT
5
SLIDE 6
SLIDE 7
@denismagda | @jwfbean | @IMCSUMMIT
In-Memory Computing and Stream processing
- Performance and velocity increases
- Scalability up to petabytes of data
- Act faster by analyzing streams of data
using SQL language
Application Layer
Web-Scale Apps Mobile Apps IoT Social Media
GridGain In-Memory Computing Platform
Transactional Persistence
Confluent Platform
Event Streaming
SLIDE 8
@denismagda | @jwfbean | @IMCSUMMIT
8
Streaming-First Workd
SLIDE 9
@denismagda | @jwfbean | @IMCSUMMIT
9
Kappa Architecture:
GridGain and Kafka Connect
💶
SLIDE 10
@denismagda | @jwfbean | @IMCSUMMIT
Demo
SLIDE 11
SLIDE 12
@denismagda | @jwfbean | @IMCSUMMIT
Enter Kafka Connect
SLIDE 13
@denismagda | @jwfbean | @IMCSUMMIT
13
PRODUCER CONSUMER
Producer Application Consumer Application
SLIDE 14
@denismagda | @jwfbean | @IMCSUMMIT
14
PRODUCER CONSUMER
Sink Connector
SMTs
Source Connector
Converter SMTs Converter KAFKA CONNECT KAFKA CONNECT
SLIDE 15
@denismagda | @jwfbean | @IMCSUMMIT
15
Discover connectors, SMTs, and converters
SLIDE 16
@denismagda | @jwfbean | @IMCSUMMIT
16
Discover connectors, SMTs, and converters Descriptions, licensing, support, and more
SLIDE 17
@denismagda | @jwfbean | @IMCSUMMIT
17
User Population Coding Sophistication
Core developers who use Java/Scala Core developers who don’t use Java/Scala Data engineers, architects, DevOps/SRE BI analysts
streams
Lower the Bar to Enter the World
SLIDE 18
@denismagda | @jwfbean | @IMCSUMMIT
Store and process with GridGain
SLIDE 19
@denismagda | @jwfbean | @IMCSUMMIT
19
GridGain: Real-time Streaming and Analytics
SLIDE 20
@denismagda | @jwfbean | @IMCSUMMIT
20
Essential GridGain APIs
Distributed memory-centric storage
Combines the performance and scale of in- memory computing together with the disk durability and strong consistency in one system
Co-located Computations
Brings the computations to the servers where the data actually resides, eliminating need to move data over the network
Distributed Key-Value
Read, write and transact with fast key-value APIs
Distributed SQL ACID Transactions Machine and Deep Learning
Horizontally, fault-tolerant distributed SQL database that treats memory and disk as active storage tiers Supports distributed ACID transactions for key-value as well as SQL operations Set of simple, scalable and efficient tools that allow building predictive machine learning models without costly data transfers (ETL)
SLIDE 21
@denismagda | @jwfbean | @IMCSUMMIT
21
GridGain SQL For Real-Time Analytics
- 1. Initial Query
- 2. Query execution over local data
- 3. Reduce multiple results in one
Ignite Node
Canada
Toronto Ottawa Montreal Calgary
Ignite Node
India
Mumbai New Delhi 1 2 2 3
SLIDE 22
@denismagda | @jwfbean | @IMCSUMMIT
- ne last thing…
SLIDE 23
@denismagda | @jwfbean | @IMCSUMMIT
Q&A
SLIDE 24
@
@denismagda | @jwfbean | @IMCSUMMIT
Thanks!
@denismagda dmagda@gridgain.com @jwfbean jwfbean@confluent.io
SLIDE 25
25