inference in osns via lightweight partial crawls
play

Inference in OSNs via Lightweight Partial Crawls Jithin K. - PowerPoint PPT Presentation

Jul 14, 2023 β€’36 likes β€’1.26k views

Inference in OSNs via Lightweight Partial Crawls Jithin K. Sreedharan Inria, France Konstantin Avrachenkov Bruno Ribeiro Inria, France Purdue University, USA Sigmetrics 2016, June 16 Motivation Estimation and inference in Online Social

  1. Estimator Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 10

  2. Estimator Key property of tours: Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 10

  3. Estimator Length of 𝑙 th tour True value of the contracted graph Key property of tours: 𝑔 𝑣, 𝑀 ∢= 𝑕(𝑣, 𝑀) Samples in 𝑙 th tour Degree of super-node except when 𝑣 or 𝑀 is 𝑇 π‘œ Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 10

  4. Estimator Length of 𝑙 th tour True value of the contracted graph Key property of tours: 𝑔 𝑣, 𝑀 ∢= 𝑕(𝑣, 𝑀) Samples in 𝑙 th tour Degree of super-node except when 𝑣 or 𝑀 is 𝑇 π‘œ Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 10

  5. Estimator Length of 𝑙 th tour True value of the contracted graph Key property of tours: 𝑔 𝑣, 𝑀 ∢= 𝑕(𝑣, 𝑀) Samples in 𝑙 th tour Degree of super-node except when 𝑣 or 𝑀 is 𝑇 π‘œ Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 10

  6. Estimator Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 11

  7. Estimator  Unbiased (unlike asymptotic in [Ribeiro and Towsley β€˜10]) Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 11

  8. Estimator  Unbiased (unlike asymptotic in [Ribeiro and Towsley β€˜10])  Strongly consistent Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 11

  9. Estimator  Unbiased (unlike asymptotic in [Ribeiro and Towsley β€˜10])  Strongly consistent Confidence interval Sampled variance Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 11

  10. Estimator  Unbiased (unlike asymptotic in [Ribeiro and Towsley β€˜10])  Strongly consistent Confidence interval Sampled variance Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 11

  11. Bayesian formulation Find a posterior probability distribution with suitable prior distribution Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 12

  12. Bayesian formulation (contd.) Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 13

  13. Bayesian formulation (contd.) Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 13

  14. Bayesian formulation (contd.) Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 13

  15. Bayesian formulation (contd.) Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 13

  16. Simulations on real-world networks Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 14

  17. Simulations on real-world networks Dogster network: Online social network for dogs ? Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 14

  18. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges 15

  19. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  20. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  21. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  22. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  23. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  24. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  25. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  26. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  27. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  28. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  29. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  30. Simulations on real-world networks: Dogster network Percentage of graph covered: 2.72% (edges), 14.86% (nodes) 415K nodes, 8.27M edges Estimated value 15

  31. Simulations on real-world networks: Friendster network 64K nodes, 1.25M edges Percentage of graph covered: 7.43% (edges), 18.52% (nodes) Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 16

  32. Simulations on real-world networks: Friendster network 64K nodes, 1.25M edges Percentage of graph covered: 7.43% (edges), 18.52% (nodes) Estimated value Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 16

  33. Simulations on real-world networks: Friendster network 64K nodes, 1.25M edges Percentage of graph covered: 7.43% (edges), 18.52% (nodes) Estimated value Estimated value Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 16

  34. Simulations on real-world networks: ADD Health data A friendship network among high school students in USA 1545 nodes, 4003 edges Percentage of graph covered: 10.87% (edges), 19.76% (nodes) Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 17

  35. Simulations on real-world networks: ADD Health data A friendship network among high school students in USA 1545 nodes, 4003 edges Percentage of graph covered: 10.87% (edges), 19.76% (nodes) Estimated value Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 17

  36. Simulations on real-world networks: ADD Health data A friendship network among high school students in USA 1545 nodes, 4003 edges Percentage of graph covered: 10.87% (edges), 19.76% (nodes) Estimated value Estimated value Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 17

  37. What if the super- node is not that β€œsuper”? Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  38. What if the super- node is not that β€œsuper”? Adaptive crawler: super-node gets bigger as crawling progresses Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  39. What if the super- node is not that β€œsuper”? Adaptive crawler: super-node gets bigger as crawling progresses How to add nodes to super-node: Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  40. What if the super- node is not that β€œsuper”? Adaptive crawler: super-node gets bigger as crawling progresses How to add nodes to super-node:  via any method as long as independent of already observed tours Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  41. What if the super- node is not that β€œsuper”? Adaptive crawler: super-node gets bigger as crawling progresses How to add nodes to super-node:  via any method as long as independent of already observed tours  Emulates retrospectively adding new node 𝑗 into super-node from the start Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  42. What if the super- node is not that β€œsuper”? Adaptive crawler: super-node gets bigger as crawling progresses How to add nodes to super-node:  via any method as long as independent of already observed tours  Emulates retrospectively adding new node 𝑗 into super-node from the start  Checks previous tours. Breaks them when 𝑗 is found. Tour 4 Tour 2 Tour 3 Tour 1 Original tour: : node 𝑗 ……. sample 2 sample 1 sample 𝑙 = 𝑇 π‘œ Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  43. What if the super- node is not that β€œsuper”? Adaptive crawler: super-node gets bigger as crawling progresses How to add nodes to super-node:  via any method as long as independent of already observed tours  Emulates retrospectively adding new node 𝑗 into super-node from the start  Checks previous tours. Breaks them when 𝑗 is found.  Start 𝑙 new tours from newly added node 𝑗 ; k ~ negative Binomial distribution (function of degrees of 𝑗, and no of tours) b a d β€œCorrection” tours from 𝒋 : e h f Start at 𝑗 , end in 𝑗 or 𝑇 4 i l r n m p Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  44. What if the super- node is not that β€œsuper”? Adaptive crawler: super-node gets bigger as crawling progresses How to add nodes to super-node:  via any method as long as independent of already observed tours  Emulates retrospectively adding new node 𝑗 into super-node from the start  Checks previous tours. Breaks them when 𝑗 is found.  Start 𝑙 new tours from newly added node 𝑗 ; k ~ negative Binomial distribution (function of degrees of 𝑗, and no of tours) b a d β€œCorrection” tours from 𝒋 : e h f Start at 𝑗 , end in 𝑗 or 𝑇 4 i l r n m p Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  45. What if the super- node is not that β€œsuper”? Adaptive crawler: super-node gets bigger as crawling progresses How to add nodes to super-node:  via any method as long as independent of already observed tours  Emulates retrospectively adding new node 𝑗 into super-node from the start  Checks previous tours. Breaks them when 𝑗 is found.  Start 𝑙 new tours from newly added node 𝑗 ; k ~ negative Binomial distribution (function of degrees of 𝑗, and no of tours) Theorem Dynamic and static super-node sample paths are equivalent in distribution Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 18

  46. From metric 𝜈(𝐻) does network look random ? Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 19

  47. Estimation and hypothesis testing in Chung-Lu or configuration model Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 20

  48. Estimation and hypothesis testing in Chung-Lu or configuration model Assumption: edges labels can be written as a function of node labels Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 20

  49. Estimation and hypothesis testing in Chung-Lu or configuration model Assumption: edges labels can be written as a function of node labels  Does the true value of the given graph belongs to the class of values when the edges are formed purely at random? Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 20

  50. Estimation and hypothesis testing in Chung-Lu or configuration model Assumption: edges labels can be written as a function of node labels  Does the true value of the given graph belongs to the class of values when the edges are formed purely at random? Jithin K. Sreedharan (jithin.sreedharan@inria.fr) 20

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Inference using Partial Information Jeff Miller Harvard University Department of Biostatistics

Inference using Partial Information Jeff Miller Harvard University Department of Biostatistics

Inference using Partial Information Jeff Miller Harvard University Department of Biostatistics ICERM Probabilistic Scientific Computing workshop June 8, 2017 Outline Partial information: What? Why? 1 Need for modular inference framework 2

532 views β€’ 41 slides
inDecentralizedOnlineSocialNetworks OnlineSocialNetworks(OSNs)

inDecentralizedOnlineSocialNetworks OnlineSocialNetworks(OSNs)

Privacy,Cost,andAvailabilityTradeoffs inDecentralizedOnlineSocialNetworks OnlineSocialNetworks(OSNs) areenormouslypopular

178 views β€’ 16 slides
Advanced inference in probabilistic programs Brooks Paige Inference thus far Likelihood

Advanced inference in probabilistic programs Brooks Paige Inference thus far Likelihood

Advanced inference in probabilistic programs Brooks Paige Inference thus far Likelihood weighting / importance sampling MCMC (single-dimension, coded by hand) Lightweight Metropolis-Hastings (update one random choice at a time,

995 views β€’ 86 slides
Large Crawls of the Web for Linguistic Purposes Marco Baroni SSLMIT, University of Bologna

Large Crawls of the Web for Linguistic Purposes Marco Baroni SSLMIT, University of Bologna

Introduction Selecting seed urls Crawling Post-processing Conclusion Large Crawls of the Web for Linguistic Purposes Marco Baroni SSLMIT, University of Bologna Birmingham, July 2005 Marco Baroni Linguistic Crawls Introduction Selecting

1.02k views β€’ 53 slides
Its time to Think Lightweight! www.thinklightweight.com TO D A Y S TO P IC S 1.

Its time to Think Lightweight! www.thinklightweight.com TO D A Y S TO P IC S 1.

Its time to Think Lightweight! www.thinklightweight.com TO D A Y S TO P IC S 1. About Think Lightweight 2. What are Lightweight Structures? 3. Industry Applications 4. Product Line Review 5. Think Lightweight Component

832 views β€’ 43 slides
Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David

Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David

To appear, Proc. 15th International Joint Conference on AI (IJCAI-97), Nagoya, Japan, August, 1997 Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David Poole Department of Computer Science University

220 views β€’ 8 slides
Partial Optimality by Pruning for MAP-inference with General Graphical Models Paul Swoboda,

Partial Optimality by Pruning for MAP-inference with General Graphical Models Paul Swoboda,

Partial Optimality by Pruning for MAP-inference with General Graphical Models Paul Swoboda, Bogdan Savchynskyy, J org Kappes, Christoph Schn orr Heidelberg University, Germany 1/1 Segment the image... 2/1 Optimal labeling 3/1

499 views β€’ 39 slides
Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David

Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David

Probabilistic Partial Evaluation: Exploiting rule structure in probabilistic inference David Poole University of British Columbia 1 Overview Belief Networks Variable Elimination Algorithm Parent Contexts & Structured

259 views β€’ 21 slides
A Partial Solution To the Fundamental Problem of Causal Inference Some of our most important

A Partial Solution To the Fundamental Problem of Causal Inference Some of our most important

A Partial Solution To the Fundamental Problem of Causal Inference Some of our most important questions are causal questions . 100,000 50,000 Gross National Income Per Capita

850 views β€’ 61 slides
Grape growing crawls to the North Situation in Finland Ari Markkula ari.markkula@omenakumpu.com

Grape growing crawls to the North Situation in Finland Ari Markkula ari.markkula@omenakumpu.com

Cold climat seminar Heppenheim 2010 Grape growing crawls to the North Situation in Finland Ari Markkula ari.markkula@omenakumpu.com Length of the day: Sun up time Sun uptime 21:36 19:12 16:48 14:24 Paris 12:00 Houres Helsinki 9:36

157 views β€’ 11 slides
The Effects of Restrictions on Number of Connections in OSNs A Case-Study on Twitter Saptarshi

The Effects of Restrictions on Number of Connections in OSNs A Case-Study on Twitter Saptarshi

The Effects of Restrictions on Number of Connections in OSNs A Case-Study on Twitter Saptarshi Ghosh Gautam Korlam Niloy Ganguly Indian Institute of Technology, Kharagpur, India 3 rd Workshop on Online Social Networks Boston, USA June 22,

521 views β€’ 34 slides
Symbolic Bayesian inference by lazy partial evaluation Chung-chieh Shan (Indiana University)

Symbolic Bayesian inference by lazy partial evaluation Chung-chieh Shan (Indiana University)

Symbolic Bayesian inference by lazy partial evaluation Chung-chieh Shan (Indiana University) Norman Ramsey (Tufts University) November 2015 This research was supported by DARPA grants FA8750-14-2-0007 and FA8750-14-C-0002, NSF grant

721 views β€’ 34 slides
The lightweight beam for Heavyweight applications The impact of this lightweight steel beam will

The lightweight beam for Heavyweight applications The impact of this lightweight steel beam will

The lightweight beam for Heavyweight applications The impact of this lightweight steel beam will revolutionise the international high-rise construction industry Tony Zaccarini and Dr Patrick OBrien The lightweight beam for Heavyweight

417 views β€’ 12 slides
Some Issues related to the Mining of OSNs represented as Graphs (presentation) Conference Paper

Some Issues related to the Mining of OSNs represented as Graphs (presentation) Conference Paper

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/259466622 Some Issues related to the Mining of OSNs represented as Graphs (presentation) Conference Paper February 2013 CITATIONS

536 views β€’ 15 slides
The lightweight beam for Heavyweight applications The impact of this lightweight beam concept

The lightweight beam for Heavyweight applications The impact of this lightweight beam concept

The lightweight beam for Heavyweight applications The impact of this lightweight beam concept will revolutionise the aviation industry Tony Zaccarini and Dr Patrick OBrien The lightweight beam for Heavyweight applications Patent Applied For

479 views β€’ 12 slides
Google Hacking 19 September 2013 Updated August 2015 #s Google's cache is over 95 Petabytes

Google Hacking 19 September 2013 Updated August 2015 #s Google's cache is over 95 Petabytes

Google Hacking 19 September 2013 Updated August 2015 #s Google's cache is over 95 Petabytes Google crawls 300 cached entries per host by default (If the site's SEO ranking is higher, then Google crawls deeper) Getting Google To Scan For You

1.39k views β€’ 114 slides
Web Content Mining Dr. Ahmed Rafea Outline Introduction The Web: Opportunities &

Web Content Mining Dr. Ahmed Rafea Outline Introduction The Web: Opportunities &

Web Content Mining Dr. Ahmed Rafea Outline Introduction The Web: Opportunities & Challenges Techniques Applications Introduction The Web is perhaps the single largest data source in the world. Web mining aims to

417 views β€’ 14 slides
Mining Second Life: Characterizing User Mobility in a Popular Virtual World Chi-Anh La - Pietro

Mining Second Life: Characterizing User Mobility in a Popular Virtual World Chi-Anh La - Pietro

Introduction Mining Second Life Measurement Methodology Results Conclusion Mining Second Life: Characterizing User Mobility in a Popular Virtual World Chi-Anh La - Pietro Michiardi ACM WOSN 2008 Seattle, WA, U.S. Introduction Mining

212 views β€’ 19 slides
jk: Using Dynamic Analysis to Crawl and Test Modern Web Applications Giancarlo Pellegrino (1) ,

jk: Using Dynamic Analysis to Crawl and Test Modern Web Applications Giancarlo Pellegrino (1) ,

jk: Using Dynamic Analysis to Crawl and Test Modern Web Applications Giancarlo Pellegrino (1) , Constantin Tschrtz (2) , Eric Bodden (2) , and Christian Rossow (1) 18th International Symposium on Research in Attacks, Intrusions and Defenses

732 views β€’ 45 slides
Focussed Web Crawling Using RL Searching web for pages relevant to a specific subject No

Focussed Web Crawling Using RL Searching web for pages relevant to a specific subject No

1 Focussed Web Crawling Using RL Searching web for pages relevant to a specific subject No organised directory of web pages Reinforcement Learning Web Crawling : start at one root page, follow links to other pages, follow their Lecture

407 views β€’ 3 slides
CRAWLING WIT ITH Deeksha Kushal Motwani APACHE NUTCH Shailender Joseph Web-Crawling Apache

CRAWLING WIT ITH Deeksha Kushal Motwani APACHE NUTCH Shailender Joseph Web-Crawling Apache

Ashish Kumar Sinha CRAWLING WIT ITH Deeksha Kushal Motwani APACHE NUTCH Shailender Joseph Web-Crawling Apache Nutch Crawling Algorithm in Apache Nutch CONTENTS RestAPI Demo Conclusion Web-Crawling Web-Crawling is a process by

1.31k views β€’ 67 slides
* A new open source language * A concurrent garbage collected language * Builds large programs

* A new open source language * A concurrent garbage collected language * Builds large programs

Go * A new open source language * A concurrent garbage collected language * Builds large programs fast. Not owned by Google. Go ~ Fast Go compilers produce fast code fast. Typical builds take a fraction of a second yet the resulting

538 views β€’ 18 slides
Machine Learning: A Promising Direction for Web Tracking Countermeasures Jason Bau, Jonathan

Machine Learning: A Promising Direction for Web Tracking Countermeasures Jason Bau, Jonathan

Stanford Computer Security Lab Machine Learning: A Promising Direction for Web Tracking Countermeasures Jason Bau, Jonathan Mayer, Hristo Paskov and John C. Mitchell Stanford University Motivation Consumers want control over third-party

208 views β€’ 19 slides
Twi$erEcho : a Distributed Focused Crawler to Support Open

Twi$erEcho : a Distributed Focused Crawler to Support Open

Intl. Workshop on Social Media Applica6ons in News and Entertainment SMANE 2012 @ WWW 2012 v Lyon, April 16th 2012 Twi$erEcho :

290 views β€’ 15 slides

More recommend