The rise and decline of an open collaboration system: How - - PowerPoint PPT Presentation

▶
the rise and decline of an open collaboration system
SMART_READER_LITE
LIVE PREVIEW

The rise and decline of an open collaboration system: How - - PowerPoint PPT Presentation

Feb 17, 2023 105 likes •683 views

The rise and decline of an open collaboration system: How Wikipedia's reaction to popularity is causing its decline ...and other SCIENCE with Aaron. whoami? whoami? Staff Researcher Working with Dario, Oliver & Fabrice on AFTv5

slide-1
SLIDE 1

The rise and decline of an

  • pen collaboration system:

How Wikipedia's reaction to popularity is causing its decline

...and other SCIENCE with Aaron.

slide-2
SLIDE 2

whoami?

slide-3
SLIDE 3

whoami?

  • Staff Researcher

○ Working with Dario, Oliver & Fabrice on AFTv5

slide-4
SLIDE 4

whoami?

  • Staff Researcher

○ Working with Dario, Oliver & Fabrice on AFTv5

  • PhD Candidate @ UMN

○ Live in the tundra* & telecommute

*Minnesota's mean temperature is similar to SF -- much larger standard deviation.

slide-5
SLIDE 5

whoami?

  • Staff Researcher

○ Working with Dario, Oliver & Fabrice on AFTv5

  • PhD Candidate @ UMN

○ Live in the tundra* & telecommute

  • WSOR intern lead last summer

*Minnesota's mean temperature is similar to SF -- much larger standard deviation.

slide-6
SLIDE 6

whoami?

  • Staff Researcher

○ Working with Dario, Oliver & Fabrice on AFTv5

  • PhD Candidate @ UMN

○ Live in the tundra* & telecommute

  • WSOR intern lead last summer
  • Wikipedian

○ Volunteer on Research:Committee ○ User scripts ○ Gnome

*Minnesota's mean temperature is similar to SF -- much larger standard deviation.

slide-7
SLIDE 7

Agenda

slide-8
SLIDE 8

Agenda

  • 1. Introduction
slide-9
SLIDE 9

Agenda

  • 1. Introduction
  • 2. Recent research (i.e. Rise & Decline)
slide-10
SLIDE 10

Agenda

  • 1. Introduction
  • 2. Recent research (i.e. Rise & Decline)
  • 3. Ongoing/future work
slide-11
SLIDE 11

Agenda

  • 1. Introduction
  • 2. Recent research (i.e. Rise & Decline)
  • 3. Ongoing/future work

Stop me if you have a quick question.

slide-12
SLIDE 12

Agenda

  • 1. Introduction
  • 2. Recent research (i.e. Rise & Decline)
  • 3. Ongoing/future work

Stop me if you have a quick question. Please wait on discussion.

slide-13
SLIDE 13
slide-14
SLIDE 14

SCIENCE SCIENCE

SCIENCE

SCIENCE

SCIENCE

SCIENCE SCIENCE

W S O R

slide-15
SLIDE 15

WSOR11: Templates --> newcomers

  • R. S. Geiger, A. Halfaker, M. Pinchuk & S. Walling, Defense Mechanism or Socialization Tactic, Accepted to ICWSM'12
slide-16
SLIDE 16

WSOR2011: Reverts predict survival

slide-17
SLIDE 17

WSOR11: Confound

  • What if newcomers are getting worse?
slide-18
SLIDE 18

WSOR11: Confound

  • What if newcomers are getting worse?

○ Increased rejection ○ Decreased retention == good! ○ Warning templates == good!

slide-19
SLIDE 19

New results: The decline

  • What: Rejection of good newcomers
  • How: Tools (huggle) that facilitate rejection
  • Why: Policy/Guidelines calcified against

newcomer input

slide-20
SLIDE 20

New results: Rejection & retention

2100 newcomers from 2001-2011 Edit session = First editing experience Thanks:

  • Oliver Keyes
  • Maryana Pinchuk
  • Steven Walling
  • Stuart Geiger

Quality: 1. Vandals 2. Bad-faith 3. Good-faith 4. Golden

slide-21
SLIDE 21

New results: Rejection & retention

Rising rate of rejection Decreasing retention Consistent quality since 2006

slide-22
SLIDE 22

New results: Rejection & retention

Quality newcomers X Rejection = Decreased retention

slide-23
SLIDE 23

New results: Rejection & retention

Logistic regression model: What predicts survival?

  • Determines significance (i.e. non-random)
  • Compare the amount of effect
  • Controls for confounding factors
slide-24
SLIDE 24

New results: Rejection & retention

Logistic regression model: What predicts survival?

  • Determines significance (i.e. non-random)
  • Compare the amount of effect
  • Controls for confounding factors

Confounds:

  • editor quality
  • temporal effects
  • investment
  • rejection type (reverted, deleted)
  • sent a message (e.g. welcomed or

warned)

slide-25
SLIDE 25

New results: Rejection & retention

Logistic regression model: What predicts survival?

  • Determines significance (i.e. non-random)
  • Compare the amount of effect
  • Controls for confounding factors

Confounds:

  • editor quality
  • temporal effects
  • investment
  • rejection type (reverted, deleted)
  • sent a message (e.g. welcomed)

Rejection still a significant negative effect!

slide-26
SLIDE 26

New results: Wiki-Tools

  • Hypothesis: Quality control tools like huggle

are exacerbating the negative effect of rejection.

slide-27
SLIDE 27

New results: Wiki-Tools

Model: Significant, negative effect.

  • reverted (coef: -0.48, p=0.01)
  • reverted-with-tool (coef: -2.51, p=0.02)

○

5X MORE BAD*!

* Measured as a decrease in the log-odds of survival

slide-28
SLIDE 28

New results: Wiki-Tools

Model: Significant, negative effect.

  • reverted (coef: -0.48, p=0.01)
  • reverted-with-tool (coef: -2.51, p=0.02)

○

5.5X MORE BAD*!

* Measured as a decrease in the log-odds of survival

ON THE RISE!

slide-29
SLIDE 29

New results: Wiki-Tools

  • Are reverting tools like huggle part of the

problem?

Yes. Why?

slide-30
SLIDE 30

Bold -> Revert -> Discuss Cycle

Bold edit reverted? consensus? Discuss No Yes No Yes

slide-31
SLIDE 31

Bold -> Revert -> Discuss Cycle

Bold edit reverted? consensus? Discuss No Yes No Yes 1. Boldly make the edit you think is right. 2. If you get reverted, talk about it on the discussion page. 3. Form consensus and move on.

slide-32
SLIDE 32

Bold -> Revert -> Discuss Cycle

Bold edit reverted? consensus? Discuss No Yes No Yes 1. Boldly make the edit you think is right. 2. If you get reverted, talk about it on the discussion page. 3. Form consensus and move on.

  • Initiation: Reverted editor posts on talk

page

  • Reciprocation: Reverting editor

responds

slide-33
SLIDE 33

Bold -> Revert -> Discuss Cycle

Initiation Reciprocation

slide-34
SLIDE 34

New results: Wiki-Tools

slide-35
SLIDE 35

New results: Wiki-Tools

humans

slide-36
SLIDE 36

New results: Wiki-Tools

humans hugglers

slide-37
SLIDE 37

New results: Wiki-Tools

humans hugglers robots

slide-38
SLIDE 38
slide-39
SLIDE 39

Notice: Someone you reverted wants to ask you why! Click here to tell them they're wrong.

slide-40
SLIDE 40

Why are conditions getting so bad for newbies?

slide-41
SLIDE 41

Why are conditions getting so bad for newbies?

  • Political economist Elinor Ostrom:

Rules must be...

  • a. well-matched to local circumstances
  • b. malleable by the governed individuals
slide-42
SLIDE 42

Why are conditions getting so bad for newbies?

  • Political economist Elinor Ostrom:

Rules must be...

  • a. well-matched to local circumstances
  • b. malleable by the governed individuals

Policies & Guidelines ~ Rules of governance Is it getting harder for newbies to affect them? And what about essays? (less formal norms)

slide-43
SLIDE 43
slide-44
SLIDE 44

Model: Logistic regression What predicts rejection?

  • Contributions to essays reverted less overall
  • Contributing getting harder over time

○ Except for essays

  • Younger editors reverted more

○ Except for essays

All reported effects are statistically significant.

slide-45
SLIDE 45

Model: Logistic regression What predicts rejection?

  • Contributions to essays reverted less overall
  • Contributing getting harder over time

○ Except for essays

  • Younger editors reverted more

○ Except for essays

All reported effects are statistically significant.

slide-46
SLIDE 46

Summary!

slide-47
SLIDE 47

Summary!

  • Reverts scare away newbies

○ Good newbies are getting reverted more

slide-48
SLIDE 48

Summary!

  • Reverts scare away newbies

○ Good newbies are getting reverted more

  • Huggle reverts are worse

○ Tool users don't discuss their reverts

slide-49
SLIDE 49

Summary!

  • Reverts scare away newbies

○ Good newbies are getting reverted more

  • Huggle reverts are worse

○ Tool users don't discuss their reverts

  • Newbies can't affect the rules that govern

them

○ They seem to have turned to essays, but essays don't carry the same weight.

slide-50
SLIDE 50

So what?

  • Can't stop reverting bad edits - Quality
slide-51
SLIDE 51

So what?

  • Can't stop reverting bad edits - Quality
  • Can't stop using tools - Efficiency
slide-52
SLIDE 52

So what?

  • Can't stop reverting bad edits - Quality
  • Can't stop using tools - Efficiency
  • Can't open up policy - Consistency
slide-53
SLIDE 53

So what?

  • Can't stop reverting bad edits - Quality
  • Can't stop using tools - Efficiency
  • Can't open up policy - Consistency
  • HuggleSnuggle

○ If we know how to find bad new users, we know how to find the good ones too.

slide-54
SLIDE 54

Newbies

Huggle & Snuggle

slide-55
SLIDE 55

What's next?

  • Experimental interfaces

○ Wikignome [http://en.wikipedia.org/wiki/User:EpochFail/Wikignome/Sandbox] ○ Mr. Clean [http://en.wikipedia.org/wiki/User:EpochFail/MrClean/Sandbox]

  • Blurring the divide between reader and

editor

○ Readers have value (AFTv5) ■ Informs value of visual editor ○ Building tools for readers: ■ History - Reading list - Watchlist

  • Community health

○ Predict future declines ○ Survival models - Needs a visualization

slide-56
SLIDE 56

Thanks!

SCIENCE SCIENCE

SCIENCE

SCIENCE

SCIENCE

SCIENCE SCIENCE

Aaron Halfaker

aaron.halfaker@gmail.com User:EpochFail