Network Text Analysis of R Mailing Lists UseR! Rennes 2009 Angela - - PowerPoint PPT Presentation
Network Text Analysis of R Mailing Lists UseR! Rennes 2009 Angela - - PowerPoint PPT Presentation
Network Text Analysis of R Mailing Lists UseR! Rennes 2009 Angela Bohn, Ingo Feinerer, Kurt Hornik, Patrick Mair, Stefan Theul 7/10/2009 A mailing list social network R -help mailing list: Jan 2008 to May 2009 Number of authors: 5326
A mailing list social network
R-help mailing list: Jan 2008 to May 2009 Number of authors: 5326 Number of mails: 41457 Avg. degree: 4.4 Diameter: 7 Legend:
Author A Author B
answered
Combine SNA and TM
◮ Goal: Combine social network analysis (SNA) and text mining
(TM) to find out more
◮ Data: Mailing lists R-help and R-devel ◮ Packages: sna and tm ◮ Results:
◮ “Interest maps” of R users ◮ Detection of bottlenecks in communication
Data preparation for social network analysis
◮ Create a social network from e-mail headers (tm):
From: dwinsemius at comcast.net (David Winsemius) Date: Thu, 30 Apr 2009 18:49:55 -0400 Subject: [R] Extracting Element from S4 objects In-Reply-To: <23302265.post@talk.nabble.com> Message-ID: <A6039F4E-ABF4-41C5-B03E-FFF32E07C37A@comcast.net>
Author A: ''Hallo, I have a question.'' Author B: ''This is the answer.'' Author C: ''This, too.'' Author D: ''And this.'' Author D: ''And this.'' Author A: ''Thank you.'' Author B Author C Author D Author A
◮ Find aliases:
knoblauch at lyon.inserm.fr (knoblauch) knoblauch at lyon.inserm.fr (Ken Knoblauch) ken.knoblauch at inserm.fr (Kenneth Knoblauch) ken.knoblauch at inserm.fr (Ken Knoblauch)
Levensthein Distance: agrep(base)
Data preparation for text mining
E-mail subjects:
[R] passing args from the command line [R] navigating ggplot viewports [R] how to go to a line in R ...
Term Frequencies (termFreq(tm)):
question data
help function
plot
using
sub package matrix
error
values frame vector list file regression
test model multiple
issue font time variable loop table lattice fedora
value rendering kde plasma plotting plots axis read column
- utput
code
- bject
text variables analysis columns rows string
packages windows glm functions line anova simple row bug dataframe files creating distribution names random reading linux ggplot2 linear series set date factor graph lines
size name logistic results sweave character histogram labels subset xyplot create running installing legend format message graphics numeric install lmer color correlation missing newbie models- bjects
- ptimization
- perator
- dp
- ptim
- dfweave
- bservations
- utside
- ptions
- riginal
- ls
- utputs
- perations
- verdispersion
- peration
- sx
- verlapping
- ptimize
- verlay
- dd
- ption
- btaining
R-help
package
bug
windows
error
function
packages
cmd using check
data install
names code file
building plot files help
suggestion matrix patch behaviour class
argument date text true fails method build functions
factor frame library methods source list version surprising incorrect typo message warning example rfc x11 default documentation numeric
crashes devel false mac memory linux axis multiple
- bject
parallelr sequence graphics foo missing question add dll errors
- utput
size closed extending free segfault time wishlist device name behavior proposed sprintf ubuntu vector embedded lines 2009 calling change compiling integer value vignettes crash intel match type wrong
description level loading summer table google request call format issue path test blas configure cran installing key license pdf write cut development environments extensions glm html infinite shlib view arguments binary checking compile fortran installation null update attributes posixct read rmpi running slot warnings bad conflict minor random solaris vista alias changelog loop print returns 13570 base causes external result tests anrpackage api decreasing empty extension fix generic line loaded mandatory results seed segfaults set sweave unique usage variable cat computer dir environment foreign gui length limit- bjects
- perator
- ptimization
- verflow
- sx
- desolve
- ption
- verplot
- dd
- ffset
- ptim
- ptional
- penmpi
- ptimisation
- ptions
- utdated
- verlap
- verriding
- verwriting
- ps
R-devel
Word Networks
“lattice”
Thierry ONKELINX Jim LemonDeepayan Sarkar
Weidong GuFelix Andrews
Sundar Dorai Raj
Bert Gunter John Kane Stefan Grosse
Dieter Menne Saptarshi GuhaBrian Ripley hadley wickham
Alex Brown Paul Hiemstra Thomas ZumbrunnHenrique Dallazuanna
Ron Bonner Tom Cohen Levi Waldron Patrick Connolly Michael KubovyK Elo
Qian R Aaron Arvey Paulo CardosoCharilaos Skiadas
RINNER Heinrich Ola Caster AndrewjohncloseGabor Grothendieck
Douglas Bates Mark Leeds Steven McKinney Karl Ove Hufthammer Malte Brockmann John Smith Ben Bolker Troels Ring Gavin Simpson Michael Hopkins Mark DiffordChuck Cleland
baptiste auguie
Judith Flores Daniel Ott Suraistephen sefick
Tom Bonen John Fox Richard Cotton Alex KarnerFerry
Paul Boutros Dhruv Sharma Gary Nelson Paul Murrell Marianne Promberger Rolf Turner Scott R Waichler steve Robbie Heremans John Poulsen Greg Snow Ryan Hafen Jon Loehrke Dimitri Liakhovitski remko duursma Christos Argyropoulos Andrew Beckerman Leandro Marino Iago Mosqueira Juliet Hannah Mike Lawrence Mark WilkinsonDan Kortschak Dimitris Rizopoulos Naomi Robbins
jimdareSteve Friedman
Coltrey Mather Thomas Roth Andrew McFadden Ranjan Maitra Gustaf Rydevik Rebecca Fisher Daniel Kornhauser Pieter van der Spek Tyler Smith Sebastien Bihorel rhelp 20 trevva Einar Arnason William Deese Don McKenzie Samuel B Civ USAF AFMC AFRL RVBXI Cable David Winsemius Alex Reynolds Benjamin Tyner francesc montane Bram Kuijper Wolfram Fischer Jim Price audrey Mark Coletti Katell HAMON Richard and Barbara Males Henning Wildhagen Kyle Roberts G Draisma GOUACHE David David Chosid Andreas Krause Patrick Hausmann Chris Barker Brian Desany Joachim Heidemeier Dr PALMIER Patrick CETE NP INFRA TRF Seth W Bigelow Javier PB lost river Mark Heckmann raviWord Networks
“ggplot”
hadley wickham
Paul Murrell Domenico Vistocco P E David Thompson
Thierry ONKELINX
ba8 John Kane Rainer M Krug
Felipe Carrillo
Uwe Ligges Martin Rittner Chris Friedl Tribo LaboyPedro de Barros
Sebastian Weber Ptit Bleuguillaume chaumet Brian Ripley Xavier Chardon
mihalicza peterRonggui Huang Michael Friendly Carsten Jaeger Michael Frumin
Ben Bolker Aric Gregson Megh Dal david f Sorn Norngbaptiste auguie
stephen sefick
Eric Gabor Grothendieck Tylere Couture
galneweinhawDavid Hajage Dave Murray Rust Juliet Hannah steve
Josep Maria Campanera AlsinaWayne F
Harsh Jason Rupert Tom Cohen Avram Aelony Ian Fiske Christopher David Desjardins Ista Zahn Dieter Menne Bernd Weiss haettulegur Marianne Promberger Andreas Christoffersen MUHC Research Sunil SuchindranPaul Emberson
Mike Lawrence Zeljko Vrba Ivan Alves Tena Sakai Ian Fellows Felix Andrews jiho Matias Gallego Liberman simeon duckworth Bernd Engelmann Bernd Ebersberger Jason Law Ingo Michaelis Mikhail Spivakov Thorsten Vogel Williams Scott Edna Bell Henning Wildhagen Tom Bonen Albin Blaschka btcruiser Adam Marsh Elena Schulz Spinu Vitalie Jacob Etches Erich Studerus levyofi Etches Jacob stephen bondWord Networks
“legend”
Henrique Dallazuanna Tom Snowden Marc Schwartz tom soyer John Kane Peter Dalgaard
Lauri NikkinenStanley Ng
Earl GlynnDuncan Murdoch
Deepayan Sarkar
hadley wickham
Gabor Grothendieck Jim Lemon
Thomas Steiner Thierry ONKELINX Pedro de BarrosGavin Simpson
Greg Snow
Georg Otto Hans Joerg Bibiko Uwe Ligges
Yasir Kaheil Paul Johnson Carsten Jaeger Markus Gesmann Tomas Lanczos Matthew Pietrzykowski Alexandre Aguiarbaptiste auguie
stephen sefick
Julia Liu Bernardo Rangel Tura Joachim Heidemeier Dr Felix Andrews Christophe Genolini Mathieu RibatetSimone Giannerini
Martin Maechler Brian Ripley
Dimitri Liakhovitski David Winsemius Julien Beguin Dieter Menne Avram Aelony Aparna Vemuri Rodrigo AluizioJorge Velez
Samuel B Civ USAF AFMC AFRL RVBXI Cable Sherri Heck jimdare Rebecca Fisher Michael Head Christophe Dutang Olivier ETERRADOSSI Mathew Fox Simon Pickett anisha sinnarkarSteve Murray Sarah Goslee
Peter Flom Mike Lawrence Felipe Carrillo Zeljko Vrba Tom Boonen steve rhelp 20 trevva Tom Cohen Bernd Engelmann ming kung Neuer Arkadasch kate yvo Georg Ehret Levi Waldron Agus Susanto Mark Farnell Anne Marie Ternes ba8 Tom Bonen Patrick Hausmann Nelson B Villoria yk Lavan Sarah Vandome mbr Michael Kubovy PALMIER Patrick CETE NP INFRA TRF Paul Emberson valeria pedrina Erich Studerus Giovanni Petris John Luis Ridao Cruz Etches Jacob Wade WallWord Networks
“boxplot”
Brian RipleyHenrique Dallazuanna Petr Pikal Jim Lemon Chuck Cleland
David HewittGabor Grothendieck
Marc Schwartz John Kane
Sharon Kuhlmann BHBaize
Thierry ONKELINX Chris Friedl
hadley wickham
Alex Reynolds Felipe Carrillo Matthias Kohl Peter Alspach Sherri Heck Thomas Adams Stephan Kolassa Marlin Keith Cox Sebastian Luque Michael Kubovy Bill Venables Phil taylor mihalicza peter Birgit Lemcke S Ellison jim holtmanstephen sefick
Megan J Bellamy Erik Iverson Leandro Marino Bernardo Rangel TuraUwe Ligges
Frank E Harrell Adaikalavan Ramasamy Jorge Velezcathelf Greg Snow
Yihui Xie Keith Ponting Coey Minear Antje Ben Bolker Peter Dalgaard goran brostromDeepayan Sarkar
Kenneth Roy Cabrera Torres steve Phillip Porter Ken Knoblauch Petter Hedberg Dimitri Liakhovitski David Winsemius Annette Heisswolf Pooja Jain Coen van Hasselt Andreas Christoffersen Gabriel Rodriguez Mike Lawrence Dieter Menne Zeljko Vrba Richard Yanicky Corinna Schmitt rich Karin Lagesen Marc Bernard Cornelis de Gier Marcin Kozak Tom Cohen Giulio Di Giovanni Marek Bartkuhn Georg Ehret Tubin Paul Adams Sebastian Merz mm745 Rajasekaramya Chad Junkermeier Daniela Garavaglia Murlidharan T Nair Aldi Kraja James Lenihan Amit Patel Samor Gandhi Kenneth TakagiCentrality Measures
Notion lattice ggplot legend boxplot
Thierry ONKELINX Jim Lemon Deepayan Sarkar Weidong Gu Felix Andrews Sundar Dorai Raj Bert Gunter John Kane Stefan Grosse Dieter Menne Saptarshi Guha Brian Ripley hadley wickham Alex Brown Paul Hiemstra Thomas Zumbrunn Henrique Dallazuanna Ron Bonner Tom Cohen Levi Waldron Patrick Connolly Michael Kubovy K Elo Qian R Aaron Arvey Paulo Cardoso Charilaos Skiadas RINNER Heinrich Ola Caster Andrewjohnclose Gabor Grothendieck Douglas Bates Mark Leeds Steven McKinney Karl Ove Hufthammer Malte Brockmann John Smith Ben Bolker Troels Ring Gavin Simpson Michael Hopkins Mark Difford Chuck Cleland baptiste auguie Judith Flores Daniel Ott Surai stephen sefick Tom Bonen John Fox Richard Cotton Alex Karner Ferry Paul Boutros Dhruv Sharma Gary Nelson Paul Murrell Marianne Promberger Rolf Turner Scott R Waichler steve Robbie Heremans John Poulsen Greg Snow Ryan Hafen Jon Loehrke Dimitri Liakhovitski remko duursma Christos Argyropoulos Andrew Beckerman Leandro Marino Iago Mosqueira Juliet Hannah Mike Lawrence Mark Wilkinson Dan Kortschak Dimitris Rizopoulos Naomi Robbins jimdare Steve Friedman Coltrey Mather Thomas Roth Andrew McFadden Ranjan Maitra Gustaf Rydevik Rebecca Fisher Daniel Kornhauser Pieter van der Spek Tyler Smith Sebastien Bihorel rhelp 20 trevva Einar Arnason William Deese Don McKenzie Samuel B Civ USAF AFMC AFRL RVBXI Cable David Winsemius Alex Reynolds Benjamin Tyner francesc montane Bram Kuijper Wolfram Fischer Jim Price audrey Mark Coletti Katell HAMON Richard and Barbara Males Henning Wildhagen Kyle Roberts G Draisma GOUACHE David David Chosid Andreas Krause Patrick Hausmann Chris Barker Brian Desany Joachim Heidemeier Dr PALMIER Patrick CETE NP INFRA TRF Seth W Bigelow Javier PB lost river Mark Heckmann ravi hadley wickham Paul Murrell Domenico Vistocco P E David Thompson Thierry ONKELINX ba8 John Kane Rainer M Krug Felipe Carrillo Uwe Ligges Martin Rittner Chris Friedl Tribo Laboy Pedro de Barros Sebastian Weber Ptit Bleu guillaume chaumet Brian Ripley Xavier Chardon mihalicza peter Ronggui Huang Michael Friendly Carsten Jaeger Michael Frumin Ben Bolker Aric Gregson Megh Dal david f Sorn Norng baptiste auguie stephen sefick Eric Gabor Grothendieck Tylere Couture galneweinhaw David Hajage Dave Murray Rust Juliet Hannah steve Josep Maria Campanera Alsina Wayne F Harsh Jason Rupert Tom Cohen Avram Aelony Ian Fiske Christopher David Desjardins Ista Zahn Dieter Menne Bernd Weiss haettulegur Marianne Promberger Andreas Christoffersen MUHC Research Sunil Suchindran Paul Emberson Mike Lawrence Zeljko Vrba Ivan Alves Tena Sakai Ian Fellows Felix Andrews jiho Matias Gallego Liberman simeon duckworth Bernd Engelmann Bernd Ebersberger Jason Law Ingo Michaelis Mikhail Spivakov Thorsten Vogel Williams Scott Edna Bell Henning Wildhagen Tom Bonen Albin Blaschka btcruiser Adam Marsh Elena Schulz Spinu Vitalie Jacob Etches Erich Studerus levyofi Etches Jacob stephen bond Henrique Dallazuanna Tom Snowden Marc Schwartz tom soyer John Kane Peter Dalgaard Lauri Nikkinen Stanley Ng Earl Glynn Duncan Murdoch Deepayan Sarkar hadley wickham Gabor Grothendieck Jim Lemon Thomas Steiner Thierry ONKELINX Pedro de Barros Gavin Simpson Greg Snow Georg Otto Hans Joerg Bibiko Uwe Ligges Yasir Kaheil Paul Johnson Carsten Jaeger Markus Gesmann Tomas Lanczos Matthew Pietrzykowski Alexandre Aguiar baptiste auguie stephen sefick Julia Liu Bernardo Rangel Tura Joachim Heidemeier Dr Felix Andrews Christophe Genolini Mathieu Ribatet Simone Giannerini Martin Maechler Brian Ripley Dimitri Liakhovitski David Winsemius Julien Beguin Dieter Menne Avram Aelony Aparna Vemuri Rodrigo Aluizio Jorge Velez Samuel B Civ USAF AFMC AFRL RVBXI Cable Sherri Heck jimdare Rebecca Fisher Michael Head Christophe Dutang Olivier ETERRADOSSI Mathew Fox Simon Pickett anisha sinnarkar Steve Murray Sarah Goslee Peter Flom Mike Lawrence Felipe Carrillo Zeljko Vrba Tom Boonen steve rhelp 20 trevva Tom Cohen Bernd Engelmann ming kung Neuer Arkadasch kate yvo Georg Ehret Levi Waldron Agus Susanto Mark Farnell Anne Marie Ternes ba8 Tom Bonen Patrick Hausmann Nelson B Villoria yk Lavan Sarah Vandome mbr Michael Kubovy PALMIER Patrick CETE NP INFRA TRF Paul Emberson valeria pedrina Erich Studerus Giovanni Petris John Luis Ridao Cruz Etches Jacob Wade Wall Brian Ripley Henrique Dallazuanna Petr Pikal Jim Lemon Chuck Cleland David Hewitt Gabor Grothendieck Marc Schwartz John Kane Sharon Kuhlmann B HBaize Thierry ONKELINX Chris Friedl hadley wickham Alex Reynolds Felipe Carrillo Matthias Kohl Peter Alspach Sherri Heck Thomas Adams Stephan Kolassa Marlin Keith Cox Sebastian Luque Michael Kubovy Bill Venables Phil taylor mihalicza peter Birgit Lemcke S Ellison jim holtman stephen sefick Megan J Bellamy Erik Iverson Leandro Marino Bernardo Rangel Tura Uwe Ligges Frank E Harrell Adaikalavan Ramasamy Jorge Velez cathelf Greg Snow Yihui Xie Keith Ponting Coey Minear Antje Ben Bolker Peter Dalgaard goran brostrom Deepayan Sarkar Kenneth Roy Cabrera Torres steve Phillip Porter Ken Knoblauch Petter Hedberg Dimitri Liakhovitski David Winsemius Annette Heisswolf Pooja Jain Coen van Hasselt Andreas Christoffersen Gabriel Rodriguez Mike Lawrence Dieter Menne Zeljko Vrba Richard Yanicky Corinna Schmitt rich Karin Lagesen Marc Bernard Cornelis de Gier Marcin Kozak Tom Cohen Giulio Di Giovanni Marek Bartkuhn Georg Ehret Tubin Paul Adams Sebastian Merz mm745 Rajasekaramya Chad Junkermeier Daniela Garavaglia Murlidharan T Nair Aldi Kraja James Lenihan Amit Patel Samor Gandhi Kenneth TakagiMost central persons Deepayan Sarkar, Sundar Dorai Raj, baptiste auguie hadley wickham, Thierry ONKELINX Duncan Murdoch, hadley wickham, Greg Snow Gabor Grothendieck, hadley wickham
lattice ggplot legend boxplot Deepayan Sarkar Sundar Dorai Raj baptiste auguie hadley wickham Thierry ONKELINX Duncan Murdoch Greg Snow Gabor Grothendieck
Results: Interest maps
question data help function plot using sub package matrix error frame values vector list file regression test model multiple issue time variable loop table lattice value plotting plots axis read column
- utput
- bject
- bjects
- ptim
Gabor Grothendieck
Henrique Dallazuanna
jim holtman
Duncan Murdoch
Peter Dalgaard Wacek Kusnierczyk
Brian Ripley
Greg Snow
hadley wickham
Deepayan Sarkar
Ted HardingJorge Velez
Rolf Turner John Kane John FoxFrank E Harrell Jim Lemon David Winsemius
Erik Iverson Uwe Ligges Daniel Nordlund Bert GunterDouglas Bates
Ben Bolker Henrik Bengtsson tom soyer Charilaos Skiadas Andrew Robinson Philippe Grosjean Wensui Liu esmail bonakdarian Kingsford Jones Sherri Heck Eik Vettorazzi Ravi Varadhan Muhammad AzamResults: Communication bottlenecks
Gabor Grothendieck Henrique Dallazuanna jim holtman Duncan Murdoch Peter Dalgaard Wacek Kusnierczyk Brian Ripley Greg Snow hadley wickham Deepayan Sarkar Ted Harding Jorge Velez Rolf Turner John Kane John Fox Frank E Harrell Jim Lemon David Winsemius Erik Iverson Uwe Ligges Daniel Nordlund Bert Gunter Douglas Bates Ben Bolker Henrik Bengtsson tom soyer Charilaos Skiadas Andrew Robinson Philippe Grosjean Wensui Liu esmail bonakdarian Kingsford Jones Sherri Heck Eik Vettorazzi Ravi Varadhan Muhammad Azam
Results: Communication bottlenecks
Good.
B A
Can be improved.
C D
Thank you!
Packages: sna: Carter T. Butts (2008). Social Network Analysis with
- sna. Journal of Statistical Software 24/6.
tm: I. Feinerer, K. Hornik, and D. Meyer (2008). Text Mining Infrastructure in R. Journal of Statistical Software 25/5. References:
- C. Bird, A. Gourley, P. Devanbu, M. Gertz, and A.
- Swaminathan. Mining email social networks. In Proceedings
- f the 2006 international workshop on Mining software
- repositories. ACM, New York, 2006.