mining large datasets case of mining graph data in the
play

Mining Large Datasets: Case of Mining Graph Data in the Cloud - PowerPoint PPT Presentation

Oct 12, 2022 •662 likes •1.36k views

Background Contributions Conclusion Mining Large Datasets: Case of Mining Graph Data in the Cloud Sabeur Aridhi PhD in Computer Science with Laurent dOrazio, Mondher Maddouri and Engelbert Mephu Nguifo 16/05/2014 Sabeur Aridhi Mining

  1. Background Contributions Conclusion Mining Large Datasets: Case of Mining Graph Data in the Cloud Sabeur Aridhi PhD in Computer Science with Laurent d’Orazio, Mondher Maddouri and Engelbert Mephu Nguifo 16/05/2014 Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 1 / 50

  2. Background Contributions Conclusion Context and motivations Application domains Computer networks, Social networks, Bioinformatics, Protein structure Chemoinformatics. Graph representation Chemical compound Data modeling. Identifying relationship patterns and rules. Social network Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 2 / 50

  3. Background Contributions Conclusion Context and motivations Mining graph data Graph mining aims to find patterns, hidden relations and behaviors in data. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 3 / 50

  4. Background Contributions Conclusion Context and motivations Mining graph data Graph mining aims to find patterns, hidden relations and behaviors in data. Mining graph goals Computing graph properties: Density, diameter, radius, ... Mining substructures from graph databases. Substructures: paths, trees, subgraphs. Frequent Subgraph Mining (FSM) task. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 3 / 50

  5. Background Contributions Conclusion Context and motivations Availability of graph data Exponential growth in both size and number of graphs in databases. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 4 / 50

  6. Background Contributions Conclusion Context and motivations Availability of graph data Exponential growth in both size and number of graphs in databases. Availability of graph data sources: The protein data bank (PDB) contains 95280 of protein 3D structures. Facebook loads 60 terabytes of new data every day [Thusoo 2010] . Google processes 20 petabytes of data per day [Dean 2008] . Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 4 / 50

  7. Background Contributions Conclusion Context and motivations Availability of graph data Exponential growth in both size and number of graphs in databases. Availability of graph data sources: The protein data bank (PDB) contains 95280 of protein 3D structures. Facebook loads 60 terabytes of new data every day [Thusoo 2010] . Google processes 20 petabytes of data per day [Dean 2008] . 3Vs of Big Data (Volume, Velocity and Variety). Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 4 / 50

  8. Background Contributions Conclusion Context and motivations Availability of graph data Exponential growth in both size and number of graphs in databases. Availability of graph data sources: The protein data bank (PDB) contains 95280 of protein 3D structures. Facebook loads 60 terabytes of new data every day [Thusoo 2010] . Google processes 20 petabytes of data per day [Dean 2008] . 3Vs of Big Data (Volume, Velocity and Variety). Availability of cloud computing environments. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 4 / 50

  9. Background Contributions Conclusion Context and motivations In this work We are interested to FSM from graph databases. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 5 / 50

  10. Background Contributions Conclusion Context and motivations In this work We are interested to FSM from graph databases. Frequent subgraph mining algorithms Various approaches of FSM. Existing approaches are mainly: Tested on centralized computing systems. Evaluated on relatively small databases. Few works for FSM in the cloud. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 5 / 50

  11. Background Contributions Conclusion Goals Questions Distributed FSM from large graph database. Data/computation distribution. Tuning cloud parameters. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 6 / 50

  12. Background Contributions Conclusion Outline 1 Background 2 Contributions 3 Conclusion Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 7 / 50

  13. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions 3 Conclusion Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 8 / 50

  14. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions Distributed subgraph mining in the cloud 3 Conclusion Contributions Prospects Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 9 / 50

  15. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background Graph A graph is denoted as G = ( V , E ) where V is a set of nodes and E is a set of edges. Subgraph A graph G ′ = ( V ′ , E ′ ) is a subgraph of another graph G = ( V , E ) iff: V ′ ⊆ V , and E ′ ⊆ E ∩ ( V ′ × V ′ ). Density The density of a graph G = ( V , E ) is 2 ·| E | calculated by density ( G ) = ( | V |· ( | V |− 1)) . Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 10 / 50

  16. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions Distributed subgraph mining in the cloud 3 Conclusion Contributions Prospects Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 11 / 50

  17. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background Cloud computing Large number of computers that are connected via Internet. Applications delivered as services. Hardware and system software delivered as services. Pay as you go. Cloud services can be rapidly and elastically provisioned. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 12 / 50

  18. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background Service models Software as a Service (SaaS). Platform as a Service (PaaS), Infrastructure as a Service (IaaS), Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 13 / 50

  19. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions Distributed subgraph mining in the cloud 3 Conclusion Contributions Prospects Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 14 / 50

  20. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background MapReduce framework A framework for processing huge datasets. Large number of computers and task/node failures. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 15 / 50

  21. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background MapReduce framework A framework for processing huge datasets. Large number of computers and task/node failures. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 15 / 50

  22. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 16 / 50

  23. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background SPARK framework A general engine for large-scale data processing. Combine SQL, streaming, and complex analytics. It offers several high-level operators that make it easy to build parallel applications. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 17 / 50

  24. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Background SHARK framework A distributed SQL query engine for Hadoop. Based on SPARK and uses the existing Hive client and metastore. Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 18 / 50

  25. Graph mining Background Cloud computing Contributions Frameworks for large data processing in the cloud Conclusion Related works Outline 1 Background Graph mining Cloud computing Frameworks for large data processing in the cloud Related works 2 Contributions Distributed subgraph mining in the cloud 3 Conclusion Contributions Prospects Sabeur Aridhi Mining Large Datasets - Big Data Forum - Lyon 19 / 50

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data mining is the use of efficient techniques for the analysis of very large collections of data and the extraction of useful and possibly unexpected

2.4k views • 94 slides
Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques

Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques

What is Web Mining? Wh t i W b Mi i What is Web Mining? Wh t i W b Mi i ? ? Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques to automat cally d scover and extract nformat on automatically

774 views • 20 slides
Covered Topics! v Big Graph Data Mining Sampling Ranking v Big Data Management Indexing v

Covered Topics! v Big Graph Data Mining Sampling Ranking v Big Data Management Indexing v

Covered Topics! v Big Graph Data Mining Sampling Ranking v Big Data Management Indexing v Big Data Preprocessing/Cleaning v Big Data Acquisition/Measurement J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,tp:// 1

484 views • 47 slides
Chapter X: Graph Mining Information Retrieval & Data Mining Universitt des Saarlandes,

Chapter X: Graph Mining Information Retrieval & Data Mining Universitt des Saarlandes,

Chapter X: Graph Mining Information Retrieval & Data Mining Universitt des Saarlandes, Saarbrcken Winter Semester 2013/14 X.13- 1 Chapter X: Graph Mining 1. Introduction to Graph Mining 2. Centrality and Other Graph Properties 3.

576 views • 43 slides
DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is

DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is

DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is still no unique answer to this question. A tentative definition: Data mining is the use of efficient techniques for the analysis of very large

1.28k views • 62 slides
Web Mining Web Mining Web mining is the use of data mining techniques to automatically

Web Mining Web Mining Web mining is the use of data mining techniques to automatically

What is Web Mining? What is Web Mining? Web Mining Web Mining Web mining is the use of data mining techniques to automatically discover and extract information from Web documents/services (Etzioni, 1996, CACM 39(11)) Web mining aims to

566 views • 22 slides
Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets?

Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets?

Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets? Data Mining Gain competitive advantages by analyzing data that describes the life of our computerized society. Artificial Intelligence

878 views • 60 slides
GRAPH MINING AND GRAPH KERNELS Part I: Graph Mining Karsten Borgwardt^ and Xifeng Yan*

GRAPH MINING AND GRAPH KERNELS Part I: Graph Mining Karsten Borgwardt^ and Xifeng Yan*

Graph Mining and Graph Kernels GRAPH MINING AND GRAPH KERNELS Part I: Graph Mining Karsten Borgwardt^ and Xifeng Yan* ^University of Cambridge *IBM T. J. Watson Research Center August 24, 2008 | ACM SIG KDD, Las Vegas Graph Mining and Graph

1.28k views • 60 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu March 16, 2016 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification

724 views • 54 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu November 16, 2015 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification

936 views • 38 slides
Data Mining: Concepts and Techniques Chapter 9 Graph mining and Social Network Analysis

Data Mining: Concepts and Techniques Chapter 9 Graph mining and Social Network Analysis

Data Mining: Concepts and Techniques Chapter 9 Graph mining and Social Network Analysis Li Xiong Slides credits: Jiawei Han and Micheline Kamber 1 April 2, 2008 Graph Mining and Social Network Analysis Graph mining Frequent

879 views • 62 slides
Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS

Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS

Large-scale Graph Mining @ Google NY Vahab Mirrokni Google Research New York, NY DIMACS Workshop Large-scale graph mining Many applications Friend suggestions Recommendation systems Security Advertising Benefits Big data available Rich

822 views • 71 slides
Data Mining in Bioinformatics Day 8: Graph Mining for Chemoinformatics and Drug Discovery

Data Mining in Bioinformatics Day 8: Graph Mining for Chemoinformatics and Drug Discovery

Data Mining in Bioinformatics Day 8: Graph Mining for Chemoinformatics and Drug Discovery Chlo-Agathe Azencott Machine Learning & Computational Biology Research Group MPIs Tbingen C.-A. Azencott Graph Mining for Chemoinformatics

654 views • 38 slides
Topic II: Graph Mining Discrete Topics in Data Mining Universitt des Saarlandes, Saarbrcken

Topic II: Graph Mining Discrete Topics in Data Mining Universitt des Saarlandes, Saarbrcken

Topic II: Graph Mining Discrete Topics in Data Mining Universitt des Saarlandes, Saarbrcken Winter Semester 2012/13 T II.Intro- 1 Topic II Intro: Graph Mining 1. Why Graphs? 2. What is Graph Mining 3. Graphs: Definitions 4. Centrality

712 views • 40 slides
CS425: Algorithms for Web Scale Data Most of the slides are from the Mining of Massive Datasets

CS425: Algorithms for Web Scale Data Most of the slides are from the Mining of Massive Datasets

CS425: Algorithms for Web Scale Data Most of the slides are from the Mining of Massive Datasets book. These slides have been modified for CS425. The original slides can be accessed at: www.mmds.org Graph data overview Problems with early

828 views • 80 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course Beijing University of Posts and Telecommunications, Beijing, China July 2019 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies

904 views • 64 slides
Graph Neural Networks E. Daller, S. Bougleux, and L. Brun April 19, 2018 () Graph Neural

Graph Neural Networks E. Daller, S. Bougleux, and L. Brun April 19, 2018 () Graph Neural

Graph Neural Networks E. Daller, S. Bougleux, and L. Brun April 19, 2018 () Graph Neural Networks April 19, 2018 1 / 10 Different Levels/Steps of Recognition Level 0: Hand made classification (Expert Systems) If x > 0 . 3 and y < 1 . 5

889 views • 10 slides
an introduction to git Ewen Cheslack-Postava 1 Outline Review of Version Control

an introduction to git Ewen Cheslack-Postava 1 Outline Review of Version Control

an introduction to git Ewen Cheslack-Postava 1 Outline Review of Version Control Distributed Version Control Repository Model Working Locally Workflows Additional Resources 2 Version Control Basics Developer chooses

827 views • 61 slides
Version Control Systems (Part 2) Devin J. Pohly <djpohly@cse.psu.edu> CMPSC 311:

Version Control Systems (Part 2) Devin J. Pohly <djpohly@cse.psu.edu> CMPSC 311:

Systems and Internet i Infrastructure Security i Institute for Networking and Security Research Department of Computer Science and Engineering Pennsylvania State University, University Park, PA Version Control Systems (Part 2) Devin J.

406 views • 40 slides
GIT in PandaRoot Ralf Kliemt Panda Meeting I 2017 1 A Small GIT Tutorial Repositories

GIT in PandaRoot Ralf Kliemt Panda Meeting I 2017 1 A Small GIT Tutorial Repositories

GIT in PandaRoot Ralf Kliemt Panda Meeting I 2017 1 A Small GIT Tutorial Repositories pandaatfair.githost.io UserSpace PandaRootGroup Local Computer 2 A Small GIT Tutorial Repositories pandaatfair.githost.io UserSpace

257 views • 22 slides
OpenNebula: Experiences at KTH With a deeper dive into emerging data analytics stacks ke

OpenNebula: Experiences at KTH With a deeper dive into emerging data analytics stacks ke

OpenNebula: Experiences at KTH With a deeper dive into emerging data analytics stacks ke Edlund KTH PDC-HPC Center for High Performance Computing KTH HPCViz Data-Intensive

590 views • 37 slides
LIACS Fundamentals Jetty Kleijn | Informatica Bachelorklas 2015-12-01 Discover theworld at

LIACS Fundamentals Jetty Kleijn | Informatica Bachelorklas 2015-12-01 Discover theworld at

LIACS Fundamentals Jetty Kleijn | Informatica Bachelorklas 2015-12-01 Discover theworld at Leiden University Discover theworld at Leiden University Research at LIACS Two clusters Algorithms and Software Technology - - (Foundations

249 views • 20 slides
EvolutionofanEfficient SearchAlgorithmforthe

EvolutionofanEfficient SearchAlgorithmforthe

EvolutionofanEfficient SearchAlgorithmforthe MateInNProbleminChess AmiHauptmanandMosheSipper BenGurionUniversity,Israel 2007HUMIES AWARDSFORHUMANCOMPETITIVERESULTS

162 views • 12 slides
Die Entwicklung der Spielprogrammierung: Von John von Neumann bis zu den hochparallelen

Die Entwicklung der Spielprogrammierung: Von John von Neumann bis zu den hochparallelen

Die Entwicklung der Spielprogrammierung: Von John von Neumann bis zu den hochparallelen Schachmaschinen Alexander Reinefeld Zuse Institut Berlin Humboldt-Universitt zu Berlin Themen der Informatik im historischen Kontext Ringvorlesung

959 views • 70 slides

More recommend