Sockpuppets in Online Discussions: Use and Abuse Srijan Kumar Jure Leskovec Justin Cheng V.S. Subrahmanian An Army of Me: Sockpuppets in Online Discussion Communities. S. Kumar, J. Cheng, J. Leskovec and V.S. Subrahmanian. Proceedings of World Wide Web Conference, 2017 (WWW 2017). Best Paper Award Honorable Mention.
2
Example bdiaz209 posts only on this discussion to bdiaz209 April 28 2013, 11PM Possibly the best blog I’ve ever read major props to you support and defend Eric_17 Eric_17 April 28 2013, 12AM Thanks. I knew Marvel fans would try to flame me, but they have nothing other than “oh that’s your opinion” instead of coming up with their own argument Fellstrike April 29 2013, 6PM Quit talking to yourself, *******. Get back on your meds if you’re going to do that 3
4
Sockpuppets in Wikipedia USE ABUSE 5
Sockpuppets in online discussions 6
Data: Sockpuppets 2.9M 2.1M 62M Users Articles Posts 7
Defining sockpuppets No ground truth sockpuppet labels! (Surprise?!) We adopt currently used definition from Wikipedia, after statistical validation for our task, as follows: Sockpuppets are accounts that post from the same IP address in the same discussion very close in time (15 min), in at least 3 different instances. 3,656 1,623 Sockpuppets Puppetmasters Note: we use the IP addresses for definition, but not detection 8
Characteristics of sockpuppets 9
How to compare sockpuppets and ordinary users? We have to match! For each sockpuppet, match an ordinary user that makes similar number of posts on similar discussions 10
Where do sockpuppets post? 11
Relation between pair of sockpuppets jakey008 Feb 5 2013, 2PM should have read the reviews first :( Upvote each other more ricobeans27 Feb 5 2013, 3PM p < 10 -3 Couldn’t agree more. Falcon-X32 Feb 5 2013, 3PM I agree. You are absolutely right! Smoothzilla Feb 5 2013, 3PM Thanks for your support!!!! Interact more with each other p < 10 -3 12
Do puppetmasters lead double lives? Double life hypothesis: Puppetmaster maintains distinct personality for the two sockpuppets Sockpuppet 2 Ordinary Sockpuppet 1 More simiar Less similar Similarity is measured as cosine similarity between user posts’ features: LIWC, sentiment, number of words, etc. 13
Do puppetmasters lead double lives? Alternate hypothesis: Puppetmaster operates both sockpuppets similarly Sockpuppet 1 Sockpuppet 2 Ordinary Less similar More similar Similarity is measured as cosine similarity between user posts’ features: LIWC, sentiment, number of words, etc. 14
Do puppetmasters lead double lives? Non-sockpuppet Sockpuppet 2 Sockpuppet 1 Both sockpuppets are more similar to each other p < 10 -3 “Good sock/Bad sock” not common 15
Why are sockpuppets created? Only for deception? 16
Deceptiveness Hypothesis: Deceptive sockpuppets of the same master have very different usernames. Non-Pretenders Pretenders Sock pairs Random pairs 0 100 200 300 Number of pairs 2/3 1/3 5 15 20 0 10 Levenshtein distance between usernames srijan srijan2 srijan theRealBatman 17
Pretender vs Non-pretender Sockpuppets srijan Feb 5 2013, 2PM best article i have read!!! More opinionated ricobeans27 Feb 5 2013, 3PM p < 10 -3 But this article doesn’t make any sense theRealBatman Feb 5 2013, 3PM YOU ARE STUPID AND A ***** srijan Feb 5 2013, 3PM i agree.. these morons dont know a thing Downvoted and Swear more reported more p < 10 -3 p < 10 -3 18
How are sockpuppets used? Do sockpuppets always support one another? 19
Neutral sockpuppets We quantify the amount of support by counting assenting, negation and dissenting words from LIWC srijan Feb 5 2013, 3PM best article ever! theRealBatman Feb 5 2013, 3PM why so? 60% Neutral 20
Supporter sockpuppets We quantify the amount of support by counting assenting, negation and dissenting words from LIWC srijan Feb 5 2013, 3PM best article ever! theRealBatman Feb 5 2013, 3PM Totally agree!! 60% 30% Neutral Supporter 21
Dissenter sockpuppets We quantify the amount of support by counting assenting, negation and dissenting words from LIWC srijan Feb 5 2013, 3PM best article ever! theRealBatman Feb 5 2013, 3PM I don’t think so 60% 30% 10% Neutral Supporter Dissenter 22
Supportiveness and Deceptiveness Pretender Probability of being a pretender 1.0 Non-pretender 0.74 0.70 0.58 0.5 0.42 0.30 0.26 0.0 Neutral Supporter Dissenter Deception is important to create an illusion of public consensus 23
Detecting sockpuppets 24
Features Activity Community Post Number of posts, Number of words, Number of upvotes and number of replies, characters, etc., downvotes, reciprocity of posts, LIWC counts, Fraction of reported posts, age of account, Readability, Is account reported, … Sentiment, … … Note: we are not using the IP based features 25
Is an account a sockpuppet? 26
Is an account a sockpuppet? Baseline Post 0.57 Community 0.54 Activity 0.59 All 0.68 0.5 0.6 0.7 0.8 0.9 1.0 AUC 27
Do two accounts belong to the same person? 28
Do two accounts belong to the same person? Baseline Post 0.80 Community 0.56 Activity 0.86 All 0.91 0.5 0.6 0.7 0.8 0.9 1.0 AUC 29
What’s next? • Being implemented at Reddit and Wikipedia • Creating algorithmic models for detection (random walks, deep learning, etc.) 30
You may also be interested in Tutorials on misbehavior and misinformation: • – Data-Driven Approaches towards Malicious Behavior Modeling. Jiang et al., SIGKDD 2017 – Antisocial Behavior on the Web: Characterization and Detection. Kumar et al., WWW 2017 Hoaxes in Wikipedia • – Disinformation on the Web: Impact, Characterisitics and Detection of Wikipedia Hoaxes. Kumar et al., WWW 2016 Vandals in Wikipedia • – VEWS: A Wikipedia Vandal Early Warning System. Kumar et al., SIGKDD 2015 Language and deception • – Linguisitic Harbingers of Betrayal: A Case Study on an Online Strategic Game. Niculae et al., ACL 2015 Social network algorithm for troll detection • – Accurately Detecting Trolls in Slashdot Zoo via Decluttering. Kumar et al., ASONAM 2014 More details at: http://cs.stanford.edu/~srijan 31
Upcoming workshop at WSDM 2018 MIS2: Misinformation and Misbehavior Mining on the Web MIS 2 Feb 9, 2018 at Los Angeles, CA Held in conjunction with WSDM 2018 Submissions due: Nov 15, 2017 Notifications due: Dec 7, 2017 Best paper awards of USD 1,000 sponsored by Please submit your papers! Completed research papers, short papers, works in progress, extended abstracts are welcome!
Recommend
More recommend
