SLIDE 1
lessons from mechanical turk and turkopticon, 20082015 six - - PowerPoint PPT Presentation
lessons from mechanical turk and turkopticon, 20082015 six - - PowerPoint PPT Presentation
lessons from mechanical turk and turkopticon, 20082015 six silberman scope you might care about this if... the most important things some workers are casual; others are professionals mostly, workers are not the narrowly selfish
SLIDE 2
SLIDE 3
you might care about this if...
SLIDE 4
the most important things
SLIDE 5
some workers are casual;
- thers are professionals
SLIDE 6
mostly, workers are not the narrowly selfish “rational actors”
- f classical economic theory
SLIDE 7
markets are not isolated, homogeneous, “frictionless” spaces but parts of a larger complex system with incomplete information and imperfect competition
SLIDE 8
market designers should address workers’ concerns
SLIDE 9
professional workers are
- verlooked allies in the process
- f improving outcomes
SLIDE 10
we may need new
- rganizational models
SLIDE 11
this is research!
SLIDE 12
mechanical turk turkopticon theory so what?
SLIDE 13
mechanical turk
SLIDE 14
the basic process
SLIDE 15
requesters post tasks workers do tasks requesters approve or reject
SLIDE 16
tasks
SLIDE 17
search result relevance evaluation transcription and translation writing content moderation data cleaning and metadata creation usability testing behavioral and market research
SLIDE 18
requesters
SLIDE 19
big companies government agencies startups researchers
SLIDE 20
workers
SLIDE 21
75-80% US-based; rest India half women, half men half born in 1980s median US HH income: $50K/yr median IN HH income: $10K/yr
SLIDE 22
most work is done by a small part of the worker population
SLIDE 23
serious Turkers contribute a lot
- f unpaid labor to create an
effective and supportive professional community
SLIDE 24
$2/hr – $400/day
SLIDE 25
wages experience (years and # of tasks) community participation specialized software use reliance on Turking income
SLIDE 26
complications
SLIDE 27
rejections scale, communication complexity, expectations distrust
SLIDE 28
turkopticon
SLIDE 29
- rigin story
SLIDE 30
uncertainty about payment unaccountable and arbitrary rejections fraudulent tasks prohibitive time limits long pay delays uncommunicative requesters and admins cost of errors borne by workers low pay
SLIDE 31
turking with turkopticon
SLIDE 32
- utcomes
SLIDE 33
complications
SLIDE 34
evolution
SLIDE 35
situatedly rational actors in complex polycentric systems
SLIDE 36
rational actors in perfect markets
SLIDE 37
preferences given and fixed at birth
SLIDE 38
economic actors maximize
SLIDE 39
actors act freely
SLIDE 40
complete information
SLIDE 41
efficient markets
SLIDE 42
no (low) barriers to entry
SLIDE 43
perfect competition
SLIDE 44
pareto optimality
SLIDE 45
preferences socially constructed
SLIDE 46
economic actors “satisfice” and have “other-regarding preferences”
SLIDE 47
actors face constrained choices, exercise power over each other
SLIDE 48
limited information
SLIDE 49
herd behavior and other “irrational” phenomena shape market dynamics
SLIDE 50
market power
SLIDE 51
- ther criteria for evaluating
market outcomes, e.g., fairness
SLIDE 52
no invisible hand
SLIDE 53
institutions shape outcomes
SLIDE 54
institutions are “the prescriptions that humans use to organize all forms of repetitive and structured human interactions”
SLIDE 55
situated rationality
SLIDE 56
institutional situations are interlinked, creating polycentric systems
SLIDE 57
crowd work is a polycentric system populated by situatedly rational actors
SLIDE 58
so what?
SLIDE 59