incentives in crowdsourcing a game theoretic approach
play

Incentives in Crowdsourcing: A Game-theoretic Approach ARPITA GHOSH - PowerPoint PPT Presentation

Aug 20, 2023 •575 likes •1.48k views

Incentives in Crowdsourcing: A Game-theoretic Approach ARPITA GHOSH Cornell University NIPS 2013 Workshop on Crowdsourcing: Theory, Algorithms, and Applications Incentives in Crowdsourcing: A Game-theoretic Approach 1 / 26 Users on the Web:

  1. Incentives in Crowdsourcing: A Game-theoretic Approach ARPITA GHOSH Cornell University NIPS 2013 Workshop on Crowdsourcing: Theory, Algorithms, and Applications Incentives in Crowdsourcing: A Game-theoretic Approach 1 / 26

  2. Users on the Web: Online collective effort Contribution online from the crowds: Incentives in Crowdsourcing: A Game-theoretic Approach 2 / 26

  3. Users on the Web: Online collective effort Contribution online from the crowds: Reviews (Amazon, Yelp), online Q&A sites (Y! Answers, Quora, StackOverflow), discussion forums Wikipedia Social media: Blogs, YouTube, . . . Incentives in Crowdsourcing: A Game-theoretic Approach 2 / 26

  4. Users on the Web: Online collective effort Contribution online from the crowds: Reviews (Amazon, Yelp), online Q&A sites (Y! Answers, Quora, StackOverflow), discussion forums Wikipedia Social media: Blogs, YouTube, . . . Crowdsourcing: Paid and unpaid; microtasks and challenges Amazon Mechanical Turk, Citizen Science (GalaxyZoo, FoldIt), Games with a Purpose, contests (Innocentive, Topcoder) Incentives in Crowdsourcing: A Game-theoretic Approach 2 / 26

  5. Users on the Web: Online collective effort Contribution online from the crowds: Reviews (Amazon, Yelp), online Q&A sites (Y! Answers, Quora, StackOverflow), discussion forums Wikipedia Social media: Blogs, YouTube, . . . Crowdsourcing: Paid and unpaid; microtasks and challenges Amazon Mechanical Turk, Citizen Science (GalaxyZoo, FoldIt), Games with a Purpose, contests (Innocentive, Topcoder) Online education: Peer-learning, peer-grading Incentives in Crowdsourcing: A Game-theoretic Approach 2 / 26

  6. Incentives and collective effort Quality, participation varies widely across systems Incentives in Crowdsourcing: A Game-theoretic Approach 3 / 26

  7. Incentives and collective effort Quality, participation varies widely across systems How to incentivize high participation and effort? Incentives in Crowdsourcing: A Game-theoretic Approach 3 / 26

  8. Incentives and collective effort Quality, participation varies widely across systems How to incentivize high participation and effort? Two components to designing incentives: Social psychology: What constitutes a reward? Incentives in Crowdsourcing: A Game-theoretic Approach 3 / 26

  9. Incentives and collective effort Quality, participation varies widely across systems How to incentivize high participation and effort? Two components to designing incentives: Social psychology: What constitutes a reward? Rewards are limited : How to allocate among self-interested users? A game-theoretic framework for incentive design Incentives in Crowdsourcing: A Game-theoretic Approach 3 / 26

  10. The game-theoretic approach to incentive design System design induces rules specifying allocation of rewards Self-interested users choose actions to maximize own payoff Incentives in Crowdsourcing: A Game-theoretic Approach 4 / 26

  11. The game-theoretic approach to incentive design System design induces rules specifying allocation of rewards Self-interested users choose actions to maximize own payoff Participation (‘Endogenous entry’) Incentives in Crowdsourcing: A Game-theoretic Approach 4 / 26

  12. The game-theoretic approach to incentive design System design induces rules specifying allocation of rewards Self-interested users choose actions to maximize own payoff Participation (‘Endogenous entry’) Revealing information truthfully (ratings, opinions, . . . ) Incentives in Crowdsourcing: A Game-theoretic Approach 4 / 26

  13. The game-theoretic approach to incentive design System design induces rules specifying allocation of rewards Self-interested users choose actions to maximize own payoff Participation (‘Endogenous entry’) Revealing information truthfully (ratings, opinions, . . . ) Effort: Quality of content (UGC sites) Output accuracy (crowdsourcing) Quantity: Number of contributions, attemped tasks Speed of response (Q&A forums), . . . Incentive design : Allocate reward to align agent’s incentives with system Incentives in Crowdsourcing: A Game-theoretic Approach 4 / 26

  14. Incentive design for crowdsourcing Reward allocation problem varies across systems: Incentives in Crowdsourcing: A Game-theoretic Approach 5 / 26

  15. Incentive design for crowdsourcing Reward allocation problem varies across systems: Why? Constraints, reward regimes, vary with nature of reward: Monetary; social-psychological (attention, status, . . . ) Attention rewards: Diverging [GM11, GH11]; subset constraints [GM12] Money-like rewards: Bounded; sum constraints [GM12] Incentives in Crowdsourcing: A Game-theoretic Approach 5 / 26

  16. Incentive design for crowdsourcing Reward allocation problem varies across systems: Why? Constraints, reward regimes, vary with nature of reward: Monetary; social-psychological (attention, status, . . . ) Attention rewards: Diverging [GM11, GH11]; subset constraints [GM12] Money-like rewards: Bounded; sum constraints [GM12] Observability of (value of) agents’ output Can only reward what you can see Incentives in Crowdsourcing: A Game-theoretic Approach 5 / 26

  17. Incentive design for crowdsourcing Reward allocation problem varies across systems: Why? Constraints, reward regimes, vary with nature of reward: Monetary; social-psychological (attention, status, . . . ) Attention rewards: Diverging [GM11, GH11]; subset constraints [GM12] Money-like rewards: Bounded; sum constraints [GM12] Observability of (value of) agents’ output Can only reward what you can see Perfect rank-ordering: Contests [. . . ] Imperfect: Noisy votes in UGC [EG13, GH13] Unobservable: Judgement elicitation [DG13] Incentives in Crowdsourcing: A Game-theoretic Approach 5 / 26

  18. Learning & incentives in user-generated content Joint work with Patrick Hummel, ITCS’13 The setting: User-generated content (Reviews, Q&A forums, comments, videos, articles, . . . ) Quality of contributions varies widely: Incentives in Crowdsourcing: A Game-theoretic Approach 6 / 26

  19. Learning & incentives in user-generated content Joint work with Patrick Hummel, ITCS’13 The setting: User-generated content (Reviews, Q&A forums, comments, videos, articles, . . . ) Quality of contributions varies widely: Sites want to display best contributions Incentives in Crowdsourcing: A Game-theoretic Approach 6 / 26

  20. Learning & incentives in user-generated content Joint work with Patrick Hummel, ITCS’13 The setting: User-generated content (Reviews, Q&A forums, comments, videos, articles, . . . ) Quality of contributions varies widely: Sites want to display best contributions But quality is not directly observable: Incentives in Crowdsourcing: A Game-theoretic Approach 6 / 26

  21. Learning & incentives in user-generated content Joint work with Patrick Hummel, ITCS’13 The setting: User-generated content (Reviews, Q&A forums, comments, videos, articles, . . . ) Quality of contributions varies widely: Sites want to display best contributions But quality is not directly observable: Infer quality from viewer votes Incentives in Crowdsourcing: A Game-theoretic Approach 6 / 26

  22. Learning & incentives in user-generated content Joint work with Patrick Hummel, ITCS’13 The setting: User-generated content (Reviews, Q&A forums, comments, videos, articles, . . . ) Quality of contributions varies widely: Sites want to display best contributions But quality is not directly observable: Infer quality from viewer votes How to display contributions to optimize overall viewer experience? Incentives in Crowdsourcing: A Game-theoretic Approach 6 / 26

  23. A multi-armed bandit problem Incentives in Crowdsourcing: A Game-theoretic Approach 7 / 26

  24. A multi-armed bandit problem Learning contribution qualities: Multi-armed bandit problem Arms: Contributions Success probability: Contribution’s ‘quality’ Incentives in Crowdsourcing: A Game-theoretic Approach 7 / 26

  25. A multi-armed bandit problem Learning contribution qualities: Multi-armed bandit problem Arms: Contributions Success probability: Contribution’s ‘quality’ Contributors: Agents with cost to quality, benefit from views Incentives in Crowdsourcing: A Game-theoretic Approach 7 / 26

  26. A multi-armed bandit problem Learning contribution qualities: Multi-armed bandit problem Arms: Contributions Success probability: Contribution’s ‘quality’ Contributors: Agents with cost to quality, benefit from views Arms are endogenous ! Contributors choose whether to participate, content quality Incentives in Crowdsourcing: A Game-theoretic Approach 7 / 26

  27. A multi-armed bandit problem Learning contribution qualities: Multi-armed bandit problem Arms: Contributions Success probability: Contribution’s ‘quality’ Contributors: Agents with cost to quality, benefit from views Arms are endogenous ! Contributors choose whether to participate, content quality What is a good learning algorithm in this setting? Incentives in Crowdsourcing: A Game-theoretic Approach 7 / 26

  28. Overview Strategic contributors: Decide participation, quality Viewers vote on displayed contributions Mechanism: Decides which contribution to display Metric: Equilibrium regret Incentives in Crowdsourcing: A Game-theoretic Approach 8 / 26

  29. Model: Content and feedback Contribution quality q : Probability of viewer upvote Incentives in Crowdsourcing: A Game-theoretic Approach 9 / 26

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Incentives and Game-Theoretic Considerations in Bitcoin Jonathan Katz 1 Background 2 Basic

Incentives and Game-Theoretic Considerations in Bitcoin Jonathan Katz 1 Background 2 Basic

Incentives and Game-Theoretic Considerations in Bitcoin Jonathan Katz 1 Background 2 Basic game theory Normal-form game N players; each player P i has a set of available actions A i A = A i x x A N is the set of outcomes

950 views • 91 slides
Incentives and Game-Theoretic Considerations in Bitcoin Jonathan Katz 1 Bitcoin provides a

Incentives and Game-Theoretic Considerations in Bitcoin Jonathan Katz 1 Bitcoin provides a

Incentives and Game-Theoretic Considerations in Bitcoin Jonathan Katz 1 Bitcoin provides a rich playground in which to explore the effects of rational behavior Jonathan Katz 2 Background 3 Basic game theory Normal - form

640 views • 50 slides
A Game-Theoretic Approach to Network Security Mohammad Pirani and Henrik Sandberg Department of

A Game-Theoretic Approach to Network Security Mohammad Pirani and Henrik Sandberg Department of

A Game-Theoretic Approach to Network Security Mohammad Pirani and Henrik Sandberg Department of Automatic Control KTH Royal Institute of Technology Outline Defense mechanisms in cyber physical systems security Game-theoretic

489 views • 24 slides
Towards a Realistic Model of Incentives in Interdomain Routing: Decoupling Forwarding from

Towards a Realistic Model of Incentives in Interdomain Routing: Decoupling Forwarding from

Static Analysis of Decoupling The Game-Theoretic Approach Towards a Realistic Model of Incentives in Interdomain Routing: Decoupling Forwarding from Signaling Aaron D. Jaggard DIMACS Rutgers University Joint work with Vijay Ramachandran

729 views • 38 slides
on Power System Flexibility: A Game Theoretic Approach Donghoon Ryu, Jinsoo Kim Resource

on Power System Flexibility: A Game Theoretic Approach Donghoon Ryu, Jinsoo Kim Resource

16 th IAEE European Conference Ljubljana, 25-28 August 2019 The Influence of Energy Prosumer's Arbitrage Strategy on Power System Flexibility: A Game Theoretic Approach Donghoon Ryu, Jinsoo Kim Resource Economics Lab. Hanyang University

445 views • 18 slides
Allocation of Hydroelectric Economic Rent Using a Cooperative Game Theoretic Approach by Egill

Allocation of Hydroelectric Economic Rent Using a Cooperative Game Theoretic Approach by Egill

The 44th International Universities Power Engineering Conference, September, 1-4, 2009, Glasgow, Scotland Allocation of Hydroelectric Economic Rent Using a Cooperative Game Theoretic Approach by Egill Benedikt Hreinsson Department of

196 views • 18 slides
A Game-Theoretic Approach to Decentralized Optimal Power Allocation for Cellular Networks Shruti

A Game-Theoretic Approach to Decentralized Optimal Power Allocation for Cellular Networks Shruti

Introduction Model Decentralized mechanism Conclusion A Game-Theoretic Approach to Decentralized Optimal Power Allocation for Cellular Networks Shruti Sharma Ph.D. candidate, Electrical Engineering and Computer Science and Demos Teneketzis

986 views • 40 slides
Mechanism Design 4/27/18 Game-Theoretic Modeling Determine what actions agents could take.

Mechanism Design 4/27/18 Game-Theoretic Modeling Determine what actions agents could take.

Mechanism Design 4/27/18 Game-Theoretic Modeling Determine what actions agents could take. Determine the agents incentives: Assign a utility for each agent to every outcome. Compute Nash equilibria to predict how agents will

244 views • 13 slides
Managing the Weakest Link: A Game-Theoretic Approach for the Mitigation of Insider Threats Aron

Managing the Weakest Link: A Game-Theoretic Approach for the Mitigation of Insider Threats Aron

Managing the Weakest Link: A Game-Theoretic Approach for the Mitigation of Insider Threats Aron Laszka 1 , 2 Benjamin Johnson 3 ottle 4 Pascal Sch Jens Grossklags 1 ohme 4 Rainer B 1 Pennsylvania State University 2 Budapest University of

456 views • 44 slides
A Game-Theoretic Approach for Alert Prioritization Aron Laszka, Yevgeniy Vorobeychik, Daniel

A Game-Theoretic Approach for Alert Prioritization Aron Laszka, Yevgeniy Vorobeychik, Daniel

A Game-Theoretic Approach for Alert Prioritization Aron Laszka, Yevgeniy Vorobeychik, Daniel Fabbri, Chao Yan, Bradley Malin Intrusion Detection Detection and mitigation of cyber-attacks is of crucial importance; however, attackers try to

846 views • 16 slides
A Game-Theoretic Approach for Runtime Capacity Allocation in MapReduce Eugenio Gianniti * , Danilo

A Game-Theoretic Approach for Runtime Capacity Allocation in MapReduce Eugenio Gianniti * , Danilo

A Game-Theoretic Approach for Runtime Capacity Allocation in MapReduce Eugenio Gianniti * , Danilo Ardagna * , Michele Ciavotta * , Mauro Passacantando * * * Politecnico di Milano ** Universit di Pisa Slide 1 POLITECNICO DI MILANO Goals

421 views • 23 slides
Game Theoretic Pragmatics Michael Franke Preliminaries Game Theory Fundamentals Interpretation

Game Theoretic Pragmatics Michael Franke Preliminaries Game Theory Fundamentals Interpretation

Game Theoretic Pragmatics Michael Franke Preliminaries Game Theory Fundamentals Interpretation Games IBR Interpreting BiOT Game Theoretic Pragmatics . . . aims for mathematically precise models of language use and interpretation . . .

730 views • 61 slides
Game-Theoretic Approach to Security Problems Antonn Ku cera ech Joint work with Tom

Game-Theoretic Approach to Security Problems Antonn Ku cera ech Joint work with Tom

Game-Theoretic Approach to Security Problems Antonn Ku cera ech Joint work with Tom Brzdil, David Klaka, Tom Lamser, and Vojt Rehk (IJCAI 2018, AAMAS 2018) Brno, September 2018 Antonn Ku cera (FI MU Brno)

383 views • 16 slides
CIMAT, Guanajuato, Mexico 5th Workshop on Game-Theoretic Probability and Related Topics, 13-15

CIMAT, Guanajuato, Mexico 5th Workshop on Game-Theoretic Probability and Related Topics, 13-15

CIMAT, Guanajuato, Mexico 5th Workshop on Game-Theoretic Probability and Related Topics, 13-15 November 2014 A Prequential Approach to Financial Risk Management Mark Davis Department of Mathematics Imperial College London

690 views • 37 slides
Derandomization in Game- Theoretic Probability Kenshi Miyabe, Meiji University, Japan (joint

Derandomization in Game- Theoretic Probability Kenshi Miyabe, Meiji University, Japan (joint

Derandomization in Game- Theoretic Probability Kenshi Miyabe, Meiji University, Japan (joint work with Akimichi Takemura) Fifth Workshop on Game-Theoretic Probability and Related Topics 13 Nov 2014 This talk is mainly based on

393 views • 28 slides
crowdsourcing workflow control Nate Tucker and Perry Green barriers to effective crowdsourcing

crowdsourcing workflow control Nate Tucker and Perry Green barriers to effective crowdsourcing

crowdsourcing workflow control Nate Tucker and Perry Green barriers to effective crowdsourcing Last time: Ensure proper incentives: Positive and negative Social and economic This time: How can we help people

482 views • 12 slides
Game Theoretic Security Framework for Quantum Key Distribution Walter O. Krawec Fei Miao

Game Theoretic Security Framework for Quantum Key Distribution Walter O. Krawec Fei Miao

Game Theoretic Security Framework for Quantum Key Distribution Walter O. Krawec Fei Miao Department of Computer Science Department of Computer Science University of Connecticut University of Connecticut Storrs, CT USA Storrs, CT USA

1.26k views • 60 slides
Using Game Theory to analyze Risk to Privacy Lisa Rajbhandari Einar A. Snekkenes Agenda

Using Game Theory to analyze Risk to Privacy Lisa Rajbhandari Einar A. Snekkenes Agenda

Using Game Theory to analyze Risk to Privacy Lisa Rajbhandari Einar A. Snekkenes Agenda Introduction Background Issues focused on this paper Why Game Theory? A privacy scenario Limitations Conclusion 2 Introduction

821 views • 19 slides
Algorithmic Game Theory and Scheduling Eric Angel, Evripidis Bampis, Fanny Pascual IBISC,

Algorithmic Game Theory and Scheduling Eric Angel, Evripidis Bampis, Fanny Pascual IBISC,

CIRM, May 06 Algorithmic Game Theory and Scheduling Eric Angel, Evripidis Bampis, Fanny Pascual IBISC, University of Evry, France Outline Scheduling vs. Game Theory Stability, Nash Equilibrium Price of Anarchy Coordination

697 views • 37 slides
Recent results in game theoretic mathematical finance Nicolas Perkowski HumboldtUniversit

Recent results in game theoretic mathematical finance Nicolas Perkowski HumboldtUniversit

Recent results in game theoretic mathematical finance Nicolas Perkowski HumboldtUniversit at zu Berlin May 31st, 2017 Thera Stochastics In Honor of Ioannis Karatzass 65th Birthday Based on joint work with R. Lochowski (Warsaw),

441 views • 31 slides
CSC321 Lecture 9: Generalization Roger Grosse Roger Grosse CSC321 Lecture 9: Generalization 1 /

CSC321 Lecture 9: Generalization Roger Grosse Roger Grosse CSC321 Lecture 9: Generalization 1 /

CSC321 Lecture 9: Generalization Roger Grosse Roger Grosse CSC321 Lecture 9: Generalization 1 / 27 Overview Weve focused so far on how to optimize neural nets how to get them to make good predictions on the training set. How do we make

905 views • 27 slides
Generalization of Deep Learning 1 Yuan YAO HKUST Some Theories are limited but help:

Generalization of Deep Learning 1 Yuan YAO HKUST Some Theories are limited but help:

Generalization of Deep Learning 1 Yuan YAO HKUST Some Theories are limited but help: Approximation Theory and Harmonic Analysis : What functions are represented well by deep neural networks, without suffering the curse of dimensionality and

568 views • 15 slides
Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Checkpoint: The bigger picture Supervised learning: instances, concepts, and

895 views • 42 slides
On the Generalization Benefjt of Noise in Stochastic Gradient Descent Samuel L. Smith, Erich

On the Generalization Benefjt of Noise in Stochastic Gradient Descent Samuel L. Smith, Erich

On the Generalization Benefjt of Noise in Stochastic Gradient Descent Samuel L. Smith, Erich Elsen and Soham De ICML 2020 Joint work with Soham De Erich Elsen With thanks to: Esme Sutherland, James Martens, Yee Whye Teh Sander Dieleman,

778 views • 25 slides

More recommend