social media studies Understanding content, user behaviors and - - PowerPoint PPT Presentation

▶
social media studies
SMART_READER_LITE
LIVE PREVIEW

social media studies Understanding content, user behaviors and - - PowerPoint PPT Presentation

Jun 28, 2023 253 likes •519 views

Research frontiers in online social media studies Understanding content, user behaviors and information diffusion Emilio Ferrara Center for Complex Networks and Systems Research School of Informatics and Computing Indiana University

slide-1
SLIDE 1

Research frontiers in online social media studies

Understanding content, user behaviors and information diffusion

Emilio Ferrara Center for Complex Networks and Systems Research School of Informatics and Computing Indiana University Bloomington

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-2
SLIDE 2

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-3
SLIDE 3

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-4
SLIDE 4

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-5
SLIDE 5

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-6
SLIDE 6

Data Collection

  • Twitter Streaming API (10% sample of total traffic)
  • August, 2010 – present
  • ~5TB Compressed
  • Real-time access to data from last 9 months related

to 3 themes: US Politics, Social Movements, News

Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI August 8, 2013

slide-7
SLIDE 7

Detecting early signatures of persuasion in information cascades

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-8
SLIDE 8

Scope of the project

  • Data acquisition in streaming scenario from

Social Media (Twitter, FB)

  • Extraction of information tokens, so-called

memes

  • Clustering of memes
  • Meme clusters classification

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-9
SLIDE 9

Architecture

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-10
SLIDE 10

Problem statement

  • Goal: clustering a large volume of tweets, in a

streaming scenario, in topics based on their similarity.

  • Challenges: tweets text is too sparse for

classification, we need to exploit further features:

  • Network structure
  • Temporal signature
  • Meta-data

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-11
SLIDE 11

Meme definition

  • @Mention: the user addresses another user mentioning its username

(Twitter syntax: @)

  • #Hashtag: the user tags its message with a “concept” (syntax: #)
  • URL: a message can include one/multiple URL(s) in extended or

shortened format

  • Phrase: whatever remains after removing mentions, hashtags and

URLs, stemming verbs/nouns, removing stop-words and punctuation.

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-12
SLIDE 12

Advantages of using memes

  • More granularity: each tweet is assigned to at least one (or more)

memes

  • Efficiency in real-time scenario: each incoming tweet is directly

assigned to its meme/s without additional overhead

  • Memes can be aggregated each other forming clusters of topics

related by content/structure similarity

  • We define a set of similarity measures:
  • Common user similarity
  • Common tweets similarity
  • Common document similarity

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-13
SLIDE 13

Network, memes & content relations

  • Social network
  • Memes
  • Content

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-14
SLIDE 14

Meme similarity measures

  • Common users similarity
  • Common tweet similarity
  • Content similarity

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-15
SLIDE 15

CUS: Common User Similary

Cosine similarity:

= 0.77 Example:

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-16
SLIDE 16

Example: Pmeme1 = {tweet1, tweet2, tweet5} Pmeme2 = {tweet1, tweet2, tweet5, tweet6}

CTS: Common Tweet Similarity

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-17
SLIDE 17

CDS: Common Document Similarity

We use once again the cosine similarity but using the TF-IDF matrix

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-18
SLIDE 18

Linear combination similarity score:

Example: CTS(i,j) = 0.8 CUS(i,j) = 0.7 CDS(i,j) = 0.9 Different weighting schema:

L(i,j) = 0.5 * 0.8 + 0.5 * 0.9 = 0.85 L(i,j) = 0.33 * 0.8 + 0.33 * 0.7 + 0.33 * 0.9 = 0.79 L(i,j) = 1 * 0.9 = 0.9 (MAX)

Linear Combination

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-19
SLIDE 19

Evaluation method

Building a ground-truth dataset:

  • Thematic, hand-picked keywords (Twitter trends)
  • ~ 2K tweets with keywords classified as trending
  • All tweets collected from a given day
  • The dataset contains 9 different classes (imbalanced size)

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-20
SLIDE 20

Goals

1.Determining how much correct is our clustering solution compared to the ground truth. 2.Determining the best trade-off in the clustering algorithm configuration considering:

  • Quality of obtained partitioning
  • Number of obtained clusters
  • Size of obtained clusters

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-21
SLIDE 21

Evaluation metrics

Mutual information: Normalized mutual information:

Individual entropies: H(X), H(Y) Joint entropy: H(X,Y) Conditional entropies: H(X|Y), H(Y|X)

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-22
SLIDE 22

Clustering algorithms

  • We investigated different clustering algorithms:
  • Static hierarchical clustering
  • Hierarchical stream-clustering
  • Online K-means clustering
  • Algorithms have been evaluated against each
  • ther to determine:
  • Tweet clustering vs. meme clustering
  • Best clustering algorithm
  • Content vs. network features

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-23
SLIDE 23

Stream-clustering quality

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-24
SLIDE 24

Cluster Number/Size

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-25
SLIDE 25

Summary

  • Efficient clustering must exploit meta-data, content

and network information

  • Introduction of memes (hashtags, mentions, URLs,

phrases) as a ‘pre-clustering’ step improves performance

  • Efficient, scalable, robust clustering algorithm

adaptable for working in streaming scenario

  • Room for performance improvement adding further

features and exploiting parameter tuning for similarity measures

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI

slide-26
SLIDE 26

Current/Future Work

  • Design of a distributed architecture supporting:
  • Distributed MapReduce-based storage with replication;

currently evaluating:

  • Hadoop/Hbase
  • Riak
  • Cassandra/Solandra
  • Large data-storage required:
  • ~50M tweets/day (increasing) ~300G/day

uncompressed data

  • Data compression (gz, lzw), support for JSON
  • Low latency, data-storage & access & analysis in close-to-

real-time scenario

August 8, 2013 Summer Workshop on Algorithms and Cyberinfrastructure for large scale optimization/AI