Social Media in Survey Research Casey Langer Tesfaye American - - PowerPoint PPT Presentation

▶
social media in survey research
SMART_READER_LITE
LIVE PREVIEW

Social Media in Survey Research Casey Langer Tesfaye American - - PowerPoint PPT Presentation

Feb 06, 2023 164 likes •629 views

Reporting Data in the Social Media World Panel Title: Practical Applications for Social Media in Survey Research Casey Langer Tesfaye American Institute of Physics AAPOR 69 th Annual Conference 5/16/2014 2 Problem It has always been

slide-1
SLIDE 1

AAPOR 69th Annual Conference 5/16/2014

Casey Langer Tesfaye American Institute of Physics

Reporting Data in the Social Media World Panel Title: Practical Applications for Social Media in Survey Research

slide-2
SLIDE 2

AAPOR 69th Annual Conference 5/16/2014

Problem

  • It has always been difficult to control the

trajectory of published data

  • Can we leverage social media to increase
  • Awareness of trajectory?
  • Accessibility of findings?
  • Accuracy of interpretations?

2

slide-3
SLIDE 3

AAPOR 69th Annual Conference 5/16/2014

Proposed solution- topical case study

  • Twitter search stream
  • Women + physics
  • September + October 2013

3

At the Solvay Conference on Physics in 1927, the only woman in attendance was Marie Curie (bottom row, third from left). Source: http://ontd-political.livejournal.com/10806258.html#ixzz30UyWOvOz

slide-4
SLIDE 4

AAPOR 69th Annual Conference 5/16/2014

4

slide-5
SLIDE 5

AAPOR 69th Annual Conference 5/16/2014

Proposed solution- topical case study

  • Twitter search stream
  • Women + physics
  • September + October 2013
  • Find references to survey data
  • Look for patterns

5

slide-6
SLIDE 6

AAPOR 69th Annual Conference 5/16/2014

Proposed solution- topical case study

  • Twitter search stream
  • Women + physics
  • September + October 2013
  • Find references to survey data
  • Look for patterns
  • Goal: practical, data-driven recommendations

6

slide-7
SLIDE 7

AAPOR 69th Annual Conference 5/16/2014

Corey S. Powell ?@coreyspowell Oct 10 Men and women really do communicate differently...according to an analysis of old Enron emails. https://medium.com/the-physics-arxiv-blog/e7dbd9d6a518 … View summary Reply Retweet Favorite More Olivier DA COSTA ?@olivierdacosta Oct 10 Why Are There Still So Few Women in Science? ¦ NYT - Meg Urry, professor of physics and astronomy at Yale http://nyti.ms/18ROWK9 View summary Reply Retweet Favorite More Physics ?@Physics360 Oct 10 Women: Life Sciences v. the Lifeless Sciences: My current Taki's article reflects upon an impassioned New York... http://bit.ly/19jPruC Expand Reply Retweet Favorite More James Holloway ?@grasshapa Oct 9 “Data Mining Reveals the Emotional Differences in Emails Written by Men and Women” by @arxivblog https://medium.com/the-physics-arxiv-blog/e7dbd9d6a518 … View summary Reply Retweet Favorite More Matthew Feickert ?@HEPfeickert Oct 9 For those applying to grad school in physics, you should seriously consider a department's Female Friendliness. | http://www.aps.org/programs/women/female- friendly/index.cfm … Expand Reply Retweet Favorite More

7

slide-8
SLIDE 8

AAPOR 69th Annual Conference 5/16/2014

Working with Twitter data

  • Twitter data needs a framework or structure
  • Generalizing conclusions is tricky territory
  • Completeness
  • Representativeness
  • BUT there is much to be learned

8

slide-9
SLIDE 9

AAPOR 69th Annual Conference 5/16/2014

Data Structure

  • First goal: transform the data into a usable form

9

Text 1 url 1 Text 2 url 2 Text 3

slide-10
SLIDE 10

AAPOR 69th Annual Conference 5/16/2014

Some basic numbers

  • Dataset had 1894 lines, which collapsed down

to 251 tweets

10

slide-11
SLIDE 11

AAPOR 69th Annual Conference 5/16/2014

Groom the data

  • Isolate relevant tweets
  • Some don’t reference survey data or relevant

articles

11

slide-12
SLIDE 12

AAPOR 69th Annual Conference 5/16/2014

Grooming text data is challenging

  • I only want references to pieces that report

survey data

  • but how?

12

slide-13
SLIDE 13

AAPOR 69th Annual Conference 5/16/2014

Some Tweets are easy to remove

13

slide-14
SLIDE 14

AAPOR 69th Annual Conference 5/16/2014

Some are harder to judge

14

slide-15
SLIDE 15

AAPOR 69th Annual Conference 5/16/2014

Automating is the way to go

  • In a larger scale data set, manual investigations

are impossible

  • Start with a sample, develop a strategy, hone

and measure its effectiveness

15

slide-16
SLIDE 16

AAPOR 69th Annual Conference 5/16/2014

Structure the data

survey data Original source link auto text Auto link handle Tweet text date 1 http://www.nyt imes.com/2013 /10/06/magazi ne/why-are- there-still-so- few-women-in- science.html Why Are There Still So Few Women in Science? http: //nyti .ms/1 8RO WK9 @Sandra AusmaPE ng Only 20% of physics PhDs are awarded to

  • women. And only

14% of US physics professors are

  • women. #STEM

#persevere 10/14

16

slide-17
SLIDE 17

AAPOR 69th Annual Conference 5/16/2014

Some metrics

  • 40% of the tweets containing links led to survey

data (n=74)

  • Only 6 of those tweets linked to original

sources

17

slide-18
SLIDE 18

AAPOR 69th Annual Conference 5/16/2014

Looking for a starting point

  • How to group Tweets or sources that belong

together?

18

slide-19
SLIDE 19

AAPOR 69th Annual Conference 5/16/2014

Link duplication

  • Many different ways to look at links
  • Link in Tweet (lowest # of dupes)
  • Full link of referred url
  • Autolink from url

19

slide-20
SLIDE 20

AAPOR 69th Annual Conference 5/16/2014

Link duplication

  • By Autotext: 52 of 74 Tweets were in duplicate

sets

20

slide-21
SLIDE 21

AAPOR 69th Annual Conference 5/16/2014

Largest set: 25 Tweets

21

slide-22
SLIDE 22

AAPOR 69th Annual Conference 5/16/2014

NY Times article

  • Date range of Tweets October 3-20

22

Oct 3 Oct 5 Oct 7 Oct 9 Oct 13 Oct 15 Oct 20 1 2 3 4 5 6 7

slide-23
SLIDE 23

AAPOR 69th Annual Conference 5/16/2014

The Viral Myth and Justin Bieber Effect

23

Source: http://www.technologyreview.com/view/512271/researchers-peek-at-the-structure-of-the-viral-internet/ Original paper: http://research.microsoft.com/pubs/176494/diffusion.pdf

slide-24
SLIDE 24

AAPOR 69th Annual Conference 5/16/2014

Second largest set: 13 Tweets

24

slide-25
SLIDE 25

AAPOR 69th Annual Conference 5/16/2014

Let’s Talk about Autotext

25

slide-26
SLIDE 26

AAPOR 69th Annual Conference 5/16/2014

Why Are There Still So Few Women in Science? Long but interesting. Why there are fewer women in Physics. Why are their so few women in science (and especially physics)? An amazing longform piece by Eileen Pollack. Physics grad ponders, "Why so few #womeninscience?" Women and science still don't mix, and this former physics student wants to know why At the Solvay Conference on Physics in 1927, the only woman in attendance was Marie Curie. Before @EileenPollack had an MFA & led workshops in Tent: Creative Writing, she studied physics. #womeninscience Tho useful, this piece tells only part of the story. RT @AliciaMorga My high school didn't even have a physics class Former top physics pupil at Yale seeks out former classmates and professors to figure out why there are still so... Fascinating read on why, for ex, only 1/5 of Physics PhDs go to women "@nytimes: Why are so few women in science? As one of *very* few women in the physics program @UTAustin in the mid-80s, this hits 2 close 2 home. Excellent review by the NYT of the physics world for women... Challenges we have all faced with stereotypes My HS biology, chemistry & physics teachers were women. Unusual and fortunate for me. #stem #edu 11 girls out of 85 in my college physics class. Trying hard to recruit more We need content rich teachers, but no mention of as a career for Yale Physics majors? #WhatitTakes #EducationNation Women in science: one-fifth of physics PhD's in the US are awarded to women and only half of those are Americans. Interesting read. Why are there still so few women in science? More women in medical research than physics, luckily. Only 14% of US physics professors are #women. Is a change in culture the key to parity? My @Penn physics PhD class was 50% female. #goupenn Why Are There Still So Few Women in Science? Why Are There Still So Few Women in Science? If only one person had said, "You can do this." Only 20% of physics PhDs are awarded to women. And only 14% of US physics professors are women. #STEM #persevere Mostly, though, I didn’t go on in physics because not a single professor encouraged me to go to graduate school. Female Physics Professor @Yale discusses the barriers for women in science Well that's alarming: "Of all the physics professors in the United States, only 14 percent are women."

26

slide-27
SLIDE 27

AAPOR 69th Annual Conference 5/16/2014

Tweets come from a variety of sources

  • Text comes from:
  • Autotext
  • Title
  • photo caption
  • article content
  • from the user themselves

27

slide-28
SLIDE 28

AAPOR 69th Annual Conference 5/16/2014

Tweets use a variety of strategies

  • Strategies include
  • Summarizing the story
  • Personalizing the story
  • Pulling out key points
  • Commentary about the article

28

slide-29
SLIDE 29

AAPOR 69th Annual Conference 5/16/2014

Women and science still don't mix, and this former physics student wants to know why Women and science still don't mix, and this former physics student wants to know why: It's been more tha... Women and science still don't mix, and this former physics student wants to know why Women and science still don't mix, and this former physics student wants to know why - It’s been more than... Women and science still don't mix, and this former physics student wants to know why Women and science still don't mix much & this former physics student wants to know why - Women and science still don't mix, and this former physics student wants to know why Women and science still don't mix, and this former physics student wants to know why Women and science still don't mix, and this former physics student wants to know why | The Verge Yahoo News Tech : Women and science still don't mix, and this former physics student wants to know why Women and science still don't mix, and this former physics student wants to ... - The Verge Women and science still don't mix, and this former physics student wants to ... - The Verge Women and science still don't mix, and this former physics student wants to ... - The Verge

29

slide-30
SLIDE 30

AAPOR 69th Annual Conference 5/16/2014

And yet only one Tweet used the autolink:

http://cur.lv/3vzwl http://bit.ly/16l2E82 http://flip.it/p3hy9 http://tmblr.co/Ze3LpvwhZlC4 http://vrge.co/GBeUYb http://www.theverge.com/2013/10/3/4798754/why-more-women-dont-work-in-sciences http://dlvr.it/44ZZqz http://bit.ly/1758iI2 http://www.theverge.com/2013/10/3/4798754/why-more-women-dont-work-in-sciences http://dlvr.it/44ZFq4 http://dlvr.it/44ydyH http://dlvr.it/44ydwn http://bit.ly/19nUqyh

30

slide-31
SLIDE 31

AAPOR 69th Annual Conference 5/16/2014

None of the autotext included stats

31

slide-32
SLIDE 32

AAPOR 69th Annual Conference 5/16/2014 #USA: #Women earn 60% of B.A.'s - but only 20% in computer #science, 20% in physics & 18% in engineering @Physicsupdate20: All-Male Physics Departments Are Not Proof of Bias Against Hiring Women 11 girls out of 85 in my college physics class. Trying hard to recruit more All-Male Physics Departments Are Not Proof of Bias Against Hiring Women ... - Lab Manager Laboratory News All-male physics departments are not proof of bias against hiring women, analysis finds As one of *very* few women in the physics program @UTAustin in the mid-80s, this hits 2 close 2 home. At the Solvay Conference on Physics in 1927, the only woman in attendance was Marie Curie. Fascinating read on why, for ex, only 1/5 of Physics PhDs go to women "@nytimes: Why are so few women in science? Girls in communities with women in #STEM are more likely to take Physics —a key indicator of interest in engineering Girls who grow up in communities with many women working in #STEM fields are more likely to take Physics classes: In communities with large numbers of women employed in #STEM, more females take high school #physics. Interesting read. Why are there still so few women in science? More women in medical research than physics, luckily. MT @ASCETweets: Girls in cities w/ women in #STEM more likely to take Physics—an indicator of engineering interest MT @SkanskaUSA @ASCETweets Girls in communities w/ women in #STEM are more likely to take Physics. MT @skanskausa: MT @ASCETweets Girls in communities w /#STEM women more likely 2take Physics- interest in engineering My @Penn physics PhD class was 50% female. #goupenn Why Are There Still So Few Women in Science? My HS biology, chemistry & physics teachers were women. Unusual and fortunate for me. #stem #edu My retweets by gender: 40% women, 60% men. #tweeq Only 14% of US physics professors are #women. Is a change in culture the key to parity? Only 20% of physics PhDs are awarded to women. And only 14% of US physics professors are women. #STEM #persevere Per the chart with this article, only physics has fewer female PhDs than philosophy : RT @ASCETweets Girls in communities w/ women in #STEM are more likely to take Physics — interest in engineering RT @ThePublicSquare: Why women earn only 39% of chemistry, 34% of economics and 20% of physics PhDs. The American Physics Society released ranking of 145 most female-friendly depts. Tho useful, this piece tells only part of the story. RT @AliciaMorga My high school didn't even have a physics class Well that's alarming: "Of all the physics professors in the United States, only 14 percent are women." Why Don't Girls Grok Physics?: I am from India and went to a high school where 60 percent of the girls ... Women earn 60% of bach deg but 20% of comp sci deg, 20% of physics deg, 18% percent of eng deg Women in science: one-fifth of physics PhD's in the US are awarded to women and only half of those are Americans. Women who study science less likely than men to end up working in physics, maths & eng but much more likely to teach

32

slide-33
SLIDE 33

AAPOR 69th Annual Conference 5/16/2014

Accuracy

  • Easy to spot large scale misinterpretations
  • But how to catch small?
  • What do misinterpretations look like?
  • Case study? Corpus?
  • Comment streams or a variety of searches can

provide more data

33

slide-34
SLIDE 34

AAPOR 69th Annual Conference 5/16/2014

People of Twitter

  • Tweeters are independent content curators
  • We can’t control what they say or how they say

it or even necessarily be aware of all of it

  • But maybe we can afford the kind of talk we

want

34

slide-35
SLIDE 35

AAPOR 69th Annual Conference 5/16/2014

In summary

  • Twitter behavior patterned, but far less

streamlined than we might have assumed

35

slide-36
SLIDE 36

AAPOR 69th Annual Conference 5/16/2014

In summary

  • What are the key metrics?*
  • Timeline of Tweets
  • Twitter handles (counts, networks)
  • Source of text

36

slide-37
SLIDE 37

AAPOR 69th Annual Conference 5/16/2014

Some guidelines

  • Consider your goals:
  • To be credited with analysis
  • To spread correct information
  • Make sharing easy
  • Link to specific article, not a dynamic page

stream

  • Prime headlines, captions, autotext, and links

for sharing

  • Be aware of key players

37

slide-38
SLIDE 38

AAPOR 69th Annual Conference 5/16/2014

Potential areas for future work

Monitor Tweets over time

  • Set up API & web crawler
  • Generate and check custom metrics (e.g.

dashboard)

  • Store whole articles and compare text to Tweet
  • Isolate & compare citations of stats?

38

slide-39
SLIDE 39

AAPOR 69th Annual Conference 5/16/2014

Potential areas for future work

  • Network structure of dissemination for stories
  • Find key nodes in your network

39

slide-40
SLIDE 40

AAPOR 69th Annual Conference 5/16/2014

Questions? Comments? Suggestions?

  • Email me:

Casey Langer Tesfaye: clanger@aip.org

40

slide-41
SLIDE 41

AAPOR 69th Annual Conference 5/16/2014

Extra slides start here

41

Really? There’s more? I’m not amused.

slide-42
SLIDE 42

AAPOR 69th Annual Conference 5/16/2014

42

slide-43
SLIDE 43

AAPOR 69th Annual Conference 5/16/2014

Questions

  • Can we know our networks?
  • What kinds of searches can or should we

monitor and how?

  • How can we set up our research results in the

most shareable way?

43

slide-44
SLIDE 44

AAPOR 69th Annual Conference 5/16/2014

Research Questions

  • Research questions can evolve from the data
  • Timeline of dissemination
  • Network of dissemination
  • Use of autotext
  • Original sources?
  • Correct interpretations?

44

slide-45
SLIDE 45

AAPOR 69th Annual Conference 5/16/2014

Sharelines

45

slide-46
SLIDE 46

AAPOR 69th Annual Conference 5/16/2014

Network analysis?

  • Only one account tweeted links to survey data

more than once

46