speed up mission critical analytics in the cloud
play

Speed up Mission-Critical Analytics in the Cloud Billy Liu, VP of - PowerPoint PPT Presentation

Speed up Mission-Critical Analytics in the Cloud Billy Liu, VP of Kyligence, Apache Kylin PMC Yiming.liu@kyligence.io + + Formed


  1. Speed up Mission-Critical Analytics in the Cloud Billy Liu, VP of Kyligence, Apache Kylin PMC Yiming.liu@kyligence.io

  2. ����+��� ���������� �����������+��������� • Formed by the creators of Apache Kylin • Kylin: Leading open source OLAP for Big Data • Vision • Unleash big data productivity • Offering • Enterprise Kylin and Managed Analytics Service on Cloud • Funding • Redpoint Ventures, Cisco, CBC Capital and Shunwei Capital • Team • Shanghai & Silicon Valley ������������������ ��

  3. ��������������������������������������������� ���������������������������������������� �������� • Innovation on data structure • Well-designed cube supports sub- seconds query speed on PB/TB dataset • Encoding/Compression/Columnar • Lightweight and Scalable Architecture • Distributed Computing by MR/Spark on YARN • Storage and Parallel Query on HBase • Native on Hadoop • ANSI-SQL • JDBC/ODBC/REST API • Batch & Streaming • Support Batch and Streaming OLAP in one platform ������������������ ��

  4. ���������� ��������� KAP Benchmark SQL on Hadoop ������������������ ��

  5. ���������������,���� ���������������������������������������� • High Performance • Sub-seconds query speed on massive BI dataset Visualization • High Concurrency • Web-scale OLAP query Kyligence Analytics Platform OLAP • Rich Ecosystem Data Mart powered by Apache Kylin • Tableau, PowerBI, MSTR, Qlik • Cloudera, Hortonworks, MapR Big Data • Data Sources HDFS YARN MapReduce Spark Spark SQL Kafka … Platform • Hive/SparkSQL/Kafka • Cognos/Teradata/Oracle/Vertica/GP • Automate Everything Source Data • Auto Cube design based on query pattern intelligently ������������������ ��

  6. ��������� ����������� ��� �� ������ ��� ���� SQL Query Router Mission critical analysis Exploratory analysis Sub-second delay Minute delay SQL on RDBMS Hadoop Cube Intelligent speed up § Spark SQL § Hive § Impala § … ... ������������������ ��

  7. ������������������ 1000+ open source adoptions ������������������� �� SAIC Motor Lenovo Lufax Wealth Mgnt #41 of Fortune 500 #226 of Fortune 500 #1 Fintech in China China Mobile China Pacific Huatai Securities #3 Securities in China Insurance #47 of Fortune 500 #252 of Fortune 500 Huawei OPPO GUOTAI JUNAN Securities #2 Securities in China #83 of Fortune 500 #4 Smart Phone Global ������������������ ��

  8. �������������������� BI OLAP/Data Mart Hadoop Cloud ������������������ ��

  9. ������AB�D��������,�������(��B���������B� ��������� �����������C���B�A�����.����A�����������:D��� • From legacy DW to Hadoop 1000+ Cognos cubes to 2 KAP cubes • Cognos 10.2.2 95% query latency <1s • • 30+ dimensions )D:������ �������:���� Cost reduction by adopting open • source technology KAP )����� ��� �� �B�:� �B��D Hadoop 2.7.2 ������ �:B�A�����������.����C���������� Hive 1.3.0 HBase 1.0.2 ������������������ ��

  10. How Big Data Meets Cloud? ������������������ ��

  11. �������������������������������������������������� Large Companies Security & Privacy Scalability & Elasticity Mission-critical DW migration Continuous Available Resiliency & Redundancy Low performance at web scale Challenges Benefits System Optimization Global Deployment High cost for data-intensive application Cost Efficiency Gap between BI and Big Data Access anytime & anywhere Skills shortage Agility Data movement between RDBMS and Big Data SME ������������������ ��

  12. ����������������������� ���������������������������������������������� Kyligence Analytics Platform • One Click Provision • Dynamic Resizing • Native on Cloud Managed Hadoop • Brings users fully deployed KAP • Enables users to extend or shrink • Integrates deeply with cloud native data and Hadoop stack in minutes computing resource dynamically for on- source, storage, and services on cloud demand workload managed Hadoop ������������������ ��

  13. � � ��� ��������������������� �� VPC User Space Edge Node Analyst Kyligence Analytics Platform Kyligence Console Task Node/Work Node Encrypted Key Cluster Mgnt … AWS EMR Azure HDI Cloud Adaptor Cloud Native Storage Azure Blob AWS S3 … Storage ������������������ ��

  14. ����������� ���� ����� OMS Scheduler AAD Admin Monitor Ingestion Processing Data Service 1 Analysis Sub-second KAP Service Response ERP Database Power BI Devices Blob Storage Excel 2 Data Lake Online Machine Learning Cube Event Hub Logs Cognitiv 3 e Service Blob Storage/Data Lake ������������������ ��

  15. ������������B����������D�����-������� �������������A����������������-������C����� • Cost effective by separating YARN workload ��� • High-throughput, temporary resource ��������� for cubing job ��� �D������� Stable, long-live • resource for analytics ���) services • Allocate resource as �A��� �A��D demand • Enable multiple regions ����D��������( �A�A� ����������� �A���� ���(��A�A� ���� ������ ���� deployment for near EE EE access* Cubes S3 Buckets ������������������ ��

  16. �����������������--�� ��-����������-���������-���������-������������-�������������������� • Dashboard • Quick insight of Kyligence instance • Diagnostics • Figures out system’s bottleneck, issue and exceptions • Optimization • Bring suggestion to turn system to be best • Knowledge Base • Rich knowledge base from daily support tasks ������������������ ��

  17. ��������������������������-�� ��������������-����������� ������������������-�� • Accelerate big data project go to market from months to days Migrate the offline OLAP to be scalable and flexible cloud solution • Empower Big Data as services globally • High Performance, High Concurrency, High Productivity • Seamless Integration with your existing Cloud and BI • • Lower TCO ������������������ ��

  18. Demos ������������������ ��

  19. ������� ��� ��������#� info@kyligence.io | http://kyligence.io | @Kyligence ������������������ ��

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend