Components and Functions of Crowdsourcing Systems A Systematic - - PowerPoint PPT Presentation

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement

Lars Hetmank Dresden, 27.02.2013

Components and Functions of Crowdsourcing Systems

A Systematic Literature Review

Components and Functions of Crowdsourcing Systems Slide 2 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Short Vita

• Dipl.-Wirt.-Inf. Lars Hetmank, TU Dresden
• Research interests:
• Information and Knowledge Management
• Semantic Web and Linked Data
• Crowdsourcing
• Affiliations:
• Manifesto for a Standard on Knowledge Exchange in

Social Knowledge Management Environments

• Knowledge Research Center (KRC) e.V. Dresden

10.03.13

Components and Functions of Crowdsourcing Systems Slide 3 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Motivation

Practical relevance:

• development of a crowdsourcing system (CSS) and

its integration into existing IT infrastructure is risky, cost-intensive, and time-consuming,

• structured development process required

Theoretical relevance:

• various theoretical contribution in the area of

crowdsourcing

• however, little has been investigated to define a

crowdsourcing system and its technical design aspects precisely

10.03.13

Components and Functions of Crowdsourcing Systems Slide 4 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Method: Systematic Literature Review

10.03.13

Research Questions (RQs): RQ1: How and in which detail are CSSs defined in current research literature? What design aspects do they cover? RQ2: What type of components and functions of a CSS can be conceptualized?

Review Protocol Plan Systematic Literature Review Selected Studies Relevant Studies Conduct Search Screen Papers Concept Map Extract Data Result Process

Components and Functions of Crowdsourcing Systems Slide 5 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Search Strategy #1

• Population: peer-reviewed conference proceedings

and journal papers in English since 2006; dissertations, newspaper articles, unpublished works

• r non-scientific articles were not considered
• Search Resources: ACM Digital Library, Ebscohost

(Academic Search Complete and Business Source Complete), Emerald, IEEE Xplore Digital Library, Sage Journals, ScienceDirect, SpringerLink and Wiley

• Search Terms: crowdsourcing system, crowdsourcing

application, and crowdsourcing platform (in singular and plural)

10.03.13

Components and Functions of Crowdsourcing Systems Slide 6 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Search Strategy #2

• Inclusion criteria:
• define or at least propose a description of what CSSs are

(RQ 1)

• address design issues of CSSs (RQ 2)
• r classify or give an overview of CSSs (RQ 2).
• Exclusion criteria:
• use CSSs for evaluation purposes but do not address design

issues

10.03.13

Components and Functions of Crowdsourcing Systems Slide 7 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Conduct Search

Step 1: a) review the abstract, introduction, and conclusion b) check inclusion and exclusion criteria c) label publication and research type à Relevant studies = 72

10.03.13

Step 2: a) read article, find and record the definitions of CSS b) set keyword for each article to identify components and functions of CSSs

Publica(on ¡type ¡ ¡ Research ¡type ¡ ¡

Components and Functions of Crowdsourcing Systems Slide 8 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Results: Crowdsourcing System Definitions (RQ1)

• 17 definitions or descriptions of the term crowdsourcing

system, application, or platform

• vary in detail and none of them cover all of the four derived

perspectives

10.03.13

“Crowdsourcing Platform is a trusted broker ensuring that providers successfully complete the task requests and that requestors pay for the

• charges. Crowdsourcing Platform issues authentication credentials for

requestors and providers when they join the platform, stores details about skill-set, history of completed requests, handles charging and payments, and manages platform misuse. Crowdsourcing platform can execute crowdsourcing requests in a number of different modes, by advertising them on the marketplace, allowing providers to bid for them, or in the form of a competition, where requestor identifies criteria to be used for selection of the winning submission. Crowdsourcing platform may further allow requestors and providers to team-up.” (Vukovic, 2009)

Components and Functions of Crowdsourcing Systems Slide 9 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Perspectives

10.03.13

Crowd- sourcing System

• rganizational

perspective process perspective human-centric perspective technical perspective

broker

requester crowd APIs user interface standardization history tracking authentication workflow support payment mechanism crowdsourcing process social interaction

mediator, marketplace, interface, or trusted broker software components, technical functions,

• r data objects

human brainpower and collective intelligence

algorithm

detailed actions performed to data

• bjects or users

Perspec(ves ¡of ¡crowdsourcing ¡systems ¡(source: ¡own ¡illustra(on) ¡

Components and Functions of Crowdsourcing Systems Slide 10 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Results: Components and Functions (RQ 2)

10.03.13 Register user Evaluate user Form user group Enable coordination User management Design task Assign task Task management Evaluate contribution Select contribution Contribution management Define workflow Manage workflow Workflow management Components ¡and ¡func(ons ¡of ¡crowdsourcing ¡systems ¡(source: ¡own ¡illustra(on) ¡ ¡

Components and Functions of Crowdsourcing Systems Slide 11 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Conclusion and Next Steps

• development of an integrated CSS definition
• overview of reusable software components

and functions may be used as a checklist

• dependencies between the elements are not

well represented

• designing a semantic model to support

knowledge-intensive crowdsourcing activities

10.03.13

Components and Functions of Crowdsourcing Systems Slide 12 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

References

(1) Kitchenham, B.: Guidelines for performing Systematic Literature Reviews in Software

• Engineering. (2007).

(2) Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic literature reviews in software engineering – A systematic literature review. Information and Software Technology. 51, 7–15 (2009). (3) Brereton, P., Kitchenham, B. a., Budgen, D., Turner, M., Khalil, M.: Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software. 80, 571–583 (2007). (4) Wieringa, R., Maiden, N., Mead, N., Rolland, C.: Requirements engineering paper classification and evaluation criteria: a proposal and a

• discussion. Requirements Engineering. 11, 102–

107 (2006). (5) Bortz, J., Döring, N.: Forschungsmethoden und Evaluation für Human- und

• Sozialwissenschaftler. Springer, Heidelberg

(2009).

10.03.13

h>p://larshetmank.com/documents/wi2013_css_concept_map.pdf ¡ ¡

Components and Functions of Crowdsourcing Systems Slide 13 / 13

Fakultät Wirtschaftswissenschaften | Lehrstuhl für Wirtschaftsinformatik, insbes. Informationsmanagement | Prof. Schoop

Contact

Email: lars@hetmank.de Twitter: https://twitter.com/larshetmank

10.03.13