Algorithms for Online Labour Marketplaces Stefano Leonardi - - PowerPoint PPT Presentation

▶
algorithms for online labour marketplaces
SMART_READER_LITE
LIVE PREVIEW

Algorithms for Online Labour Marketplaces Stefano Leonardi - - PowerPoint PPT Presentation

Dec 22, 2023 219 likes •1.06k views

Algorithms for Online Labour Marketplaces Stefano Leonardi Sapienza University of Rome Based on work with Aris Anagnostopoulos (Sapienza Univ.) Carlos Castillo (UPF, Barcelona) Adriano Fazzone (Sapienza Univ.) Aris Gionis (Aalto Univ.)

slide-1
SLIDE 1

Algorithms for Online Labour Marketplaces

Stefano Leonardi Sapienza University of Rome Based on work with

Aris Anagnostopoulos Carlos Castillo Adriano Fazzone Aris Gionis Evimaria Terzi (Sapienza Univ.) (UPF, Barcelona) (Sapienza Univ.) (Aalto Univ.) (Boston University)

slide-2
SLIDE 2

Online labor marketplaces

  • We will see an increase in the sophistication of

systems that use and guide user actions

  • Require models and algorithms to capture the

human elements

  • What skills people have
  • Efficiency
  • Time availability
  • Human-human relationship
  • Incentive and behavioral issues
  • Human errors / disagreements
  • Work organization
slide-3
SLIDE 3
  • Polymath project:
  • Open source community:

Online collaborative systems

Several success stories indicate that much more is possible:

  • Tagging/geotagging systems:
  • Content creation systems:
  • Online labor markets:
  • Crowdsourcing:
slide-4
SLIDE 4

This lecture

We will look at two specific problems:

  • How can we form teams of experts online

when compatibility between experts is modelled by a social network

  • How can we decide online when to use
  • utsourced workers, when to hire workers in a

team and when to fire inactive workers

slide-5
SLIDE 5

This lecture

We like to solve the above problems while achieving:

  • Good performance of formed teams on

allocated tasks

  • Fair distribution of the task load between

experts

  • Low coordination overhead within a team
  • Good trade-offs between outsourcing and

hiring/salary cost

slide-6
SLIDE 6
  • Polymath project:
  • Open source community:

Online collaborative systems

Several success stories indicate that much more is possible:

  • Tagging/geotagging systems:
  • Content creation systems:
  • Online labor markets:
  • Crowdsourcing:
slide-7
SLIDE 7

Boston University Slideshow Title Goes Here

Team formation

slide-8
SLIDE 8

Boston University Slideshow Title Goes Here

Team formation

Industrial and business settings

Cluster hires: Which experts should be hired? Online collaborations: Can teams really work online?

slide-9
SLIDE 9

Boston University Slideshow Title Goes Here

Team formation

Educational settings

Traditional classroom: How to create good study groups? Massive Online Courses (MOOCS): How to bring in social aspects?

slide-10
SLIDE 10

Boston University Slideshow Title Goes Here

Team formation

Research environments

Writing proposals with others Cluster hires with diversity Collaborative problem solving

slide-11
SLIDE 11

2001

slide-12
SLIDE 12

2001

Organizer Insider Co-organizer Security expert Mechanic Mechanic Electronics expert Explosives expert Acrobat Con-man Pick-pocket thief

slide-13
SLIDE 13
slide-14
SLIDE 14
slide-15
SLIDE 15
slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18
  • Desk – Team sizes over time
slide-19
SLIDE 19

The Online Team Formation Problem

slide-20
SLIDE 20

Boston University Slideshow Title Goes Here

Related work

slide-21
SLIDE 21

Boston University Slideshow Title Goes Here

Set-cover view of team formation

Experts Single task

slide-22
SLIDE 22

Boston University Slideshow Title Goes Here

Set-cover view of team formation

Experts Single task

slide-23
SLIDE 23

Basic formulation: set cover

10011101 10010010 10001101 00010101 Vector of skills Vector of skills Task

Problem: Given a pool of experts, a single task hire the minimum-cost subset of experts that can complete (i.e., cover) the task Facts:

  • The problem is NP-hard
  • Greedy algorithm is a a good approximation algorithm
slide-24
SLIDE 24

Setting

  • Pool of people with different skills
  • Stream of tasks/jobs arriving online
  • Tasks have some skill requirements
  • Create teams on-the-fly for each job

– Select the right team – Satisfy various criteria

slide-25
SLIDE 25

Criteria

  • Fitness

– E.g. success rate, maximize expected number of successful tasks – Depends on:

– People skills – Ability to coordinate

  • Fairness: everybody should be involved in roughly the same

number of tasks

  • Efficiency:

– Cost of outsourced tasks vs cost of hired workers

  • Trade-offs may appear: do you see how?
slide-26
SLIDE 26

Basic formulation: Skills and people

  • n People/Experts
  • m Skills
  • Each person has some

skills

slide-27
SLIDE 27

Basic formulation: jobs & teams

  • Stream of k Jobs/Tasks
  • A job requires some skills
  • k Teams are created
  • nline
  • A team must cover all

job skills 10011101 10010010 10001 101 00010101 Load:

slide-28
SLIDE 28

Coordination cost

  • Coordination cost measures the compatibility of the team

members

  • Example of

:

– Degree of knowledge – Time-zone difference – Past collaboration

:

  • Select teams that minimizes coordination cost

– Steiner-tree cost – Diameter – Sum of distances

slide-29
SLIDE 29

Framework

  • Jobs/Tasks (k)
  • People (n)
  • Skills (m)
  • Teams (k)
  • Distance between people
  • Team coordination cost
  • Score/fitness
  • Load
slide-30
SLIDE 30

Binary Profiles

In this talk (and most the work): Binary skill profiles

  • A person either has a skill or not
  • Team has a skill if a person has it
  • A job either requires it or not
  • Score of a team Q for task J
  • Covering problem
  • Other options are available
slide-31
SLIDE 31

Online Balanced Task Covering

slide-32
SLIDE 32
  • 1. Balanced task covering
  • Cover all the jobs
  • Objective
  • NP-hard problem even with k

= 2

  • Offline setting has a randomized approx. algo.

That succeeds with prob 1 – δ with ratio

  • Does it exist an O(1)-APX?
slide-33
SLIDE 33

Our modeling approach

  • Set a desirable coordination cost upper bound B
  • Online solve

Load of person i

Team j covers job j Bounded coordination cost

  • Must concurrently solve various combinatorial problems:

– Set cover – Steiner tree – Online makespan minimization

slide-34
SLIDE 34

Our modeling approach

Job p1 p2 p3 p4 p5 p6 p7 Qj 1

ü

ü ü Q1 = {p2, p4, p5} 2 ü ü ü Q2 = {p1, p4, p6} 3 ü ü Q3 = {p3, p4} 4 ü ü ü Q4 = {p1, p5, p7} 5 ü ü ü ü Q5 = {p2, p3. p4, p5} 6 ü ü ü Q6 = {p3, p5, p6} 7 ü ü Q7 = {p1, p2} 8 ü ü ü ü ü Q8 = {p1, p2, p3, p4, p7} 9 ü ü ü Q9 = {p3, p4, p5} Load 4 4 5 6 5 2 2

slide-35
SLIDE 35

Balanced task covering – Online

  • Evaluate by competitive ratio

– Compare with optimal offline assignment – Offline has full information

  • Simple heuristics

– Assemble the team of minimum size – Assemble the team that minimize the maximum load of a person: – Assemble the team that minimize the sum of the loads of the team: – Competitive ratios are bad:

  • In practice some are OK
slide-36
SLIDE 36

Algorithm ExpLoad

When a task arrives at time t

  • Weight each person p by
  • Select team Q that covers all task skills and minimizes
  • Weighted set cover problem
  • Theorem.

Competitive ratio =

Load of p at time t

slide-37
SLIDE 37

Experiments

slide-38
SLIDE 38

Mapping of data to problem instances Summary statistics

slide-39
SLIDE 39
slide-40
SLIDE 40
slide-41
SLIDE 41
slide-42
SLIDE 42
slide-43
SLIDE 43

Online Balanced Task Covering with Coordination Cost

slide-44
SLIDE 44
  • 2. Coordination cost
  • Have not taken into account coordination cost
  • Distance between people
  • Team coordination cost
  • Select teams that minimizes

– Steiner-tree cost – Diameter – Sum of distances

slide-45
SLIDE 45

Coordination cost

  • Steiner-tree cost
  • Diameter
  • Sum of distances
slide-46
SLIDE 46

Conflicting goals

  • We want solutions that minimize

– Load – Coordination cost

and satisfy each job.

slide-47
SLIDE 47

Our modeling approach

  • Set a desirable coordination cost upper bound B
  • Online solve
  • 3 different problems for the 3 different coordination costs
  • This talk: focus on Steiner tree coordination cost
slide-48
SLIDE 48

Algorithm

At every step t:

  • Combine ExpLoad with coordination cost constraint Þ
  • Find a team that:

– Covers all required skills – Satisfies – Minimizes

  • How?
slide-49
SLIDE 49

At every step t

  • Incorporate to the graph
  • Solve a variant of Steiner tree. Get a solution that

– Covers all required skills – Satisfies – α-approximates

  • Different graphs in the family tradeoff between α, β

= Þ

slide-50
SLIDE 50

Result

We wanted:

  • Theorem. The algorithm satisfies:
  • Can obtain α, β = O(log(n, m, k))
slide-51
SLIDE 51

Group Steiner Tree

  • Group Steiner Tree: Construct a Steiner tree that

connects at least one node for each group

  • Heuristics for Group Steiner Tree:
  • 1. LLT [Lappas, Liu, Terzi, KDD 2009]

– Connect each skill to all experts that own the skill – Construct a Steiner tree connecting all skills of

slide-52
SLIDE 52

Group Steiner tree

  • 2. Set Cover (SC): Cover all skills with experts.

At each step select the most effective expert cost-effectiveness: # newly covered skills distance to experts selected so far plus * ExpLoad of the expert

slide-53
SLIDE 53

Experiments Bibsonomy

Experts = prolific authors Task = interview scientists Distance = f( #collaborations ) Optimize over

slide-54
SLIDE 54

Experiments Bibsonomy

Experts = prolific authors Task = interview scientists Distance = f( #collaborations )

slide-55
SLIDE 55

Experiments IMDB

Experts = directors Task = find a cast Distance = f( #common actors directed )

slide-56
SLIDE 56

Online Team Formation with Outsourcing

slide-57
SLIDE 57

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skills
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-58
SLIDE 58

w1={s1 } w2={s2 , s4 } w3={s2 , s3 }

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-59
SLIDE 59

J 1={s1 , s2} t

w1={s1 } w2={s2 , s4 } w3={s2 , s3 }

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-60
SLIDE 60

J 1={s1 , s2} t

w1 , w2 Hired: Outsourced: Cost: λ1+ λ2 w1={s1 } w2={s2 , s4 } w3={s2 , s3 } ―

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-61
SLIDE 61

J 1={s1 , s2} J 2={s1 , s3} t

w1 , w2 Hired: Outsourced: Cost: λ1+ λ2 w1={s1 } w2={s2 , s4 } w3={s2 , s3 } ―

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-62
SLIDE 62

J 1={s1 , s2} J 2={s1 , s3} t

w1 , w2 w1 w3 Hired: Outsourced: Cost: λ1+ λ2 w1={s1 } w2={s2 , s4 } w3={s2 , s3 } C1 + σ1 + λ3 ―

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-63
SLIDE 63

J 1={s1 , s2} J 2={s1 , s3} J 3={s2 , s3} t

w1 , w2 w1 w3 Hired: Outsourced: Cost: λ1+ λ2 w1={s1 } w2={s2 , s4 } w3={s2 , s3 } C1 + σ1 + λ3 ―

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-64
SLIDE 64

J 1={s1 , s2} J 2={s1 , s3} J 3={s2 , s3} t

w1 , w2 w1 w3 w1 , w3 ― Hired: Outsourced: Cost: λ1+ λ2 σ1 + C3 + σ3 w1={s1 } w2={s2 , s4 } w3={s2 , s3 } C1 + σ1 + λ3 ―

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-65
SLIDE 65

J 1={s1 , s2} J 2={s1 , s3} J 3={s2 , s3} J 4={s2 , s3 , s4} t

w1 , w2 w1 w3 w1 , w3 ― Hired: Outsourced: Cost: λ1+ λ2 σ1 + C3 + σ3 w1={s1 } w2={s2 , s4 } w3={s2 , s3 } C1 + σ1 + λ3 ―

Team Formation with Outsourcing

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-66
SLIDE 66

Team Formation with Outsourcing

J 1={s1 , s2} J 2={s1 , s3} J 3={s2 , s3} J 4={s2 , s3 , s4} t

w1 , w2 w1 w3 w1 , w3 ― w3 w2 Hired: Outsourced: Cost: λ1+ λ2 σ1 + C3 + σ3 σ3 + λ2 ― w1={s1 } w2={s2 , s4 } w3={s2 , s3 } C1 + σ1 + λ3

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Create teams of workers for solving tasks/jobs that arrive online
  • Tasks and workers are represented as a set of skills
  • At each time step a new task arrives
  • A team must be created to cover all the task skils
  • Each member of the team can be either hired or a freelance worker
  • Each worker wr has a hiring (Cr), salary (σr), and outsourcing (λr) cost
  • Goal: design an online, cost-minimizing algorithm for hiring, firing, and
  • utsourcing.
slide-67
SLIDE 67

Quality of online algorithms

Competitive approximation ratio of an online algorithm: Goal: Design a polynomial-time online algorithm for the TFO problem with a small competitive approximation ratio.

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-68
SLIDE 68

Methodology

1. TFO-LumpSum: no salary and no firing. Design a polynomial time online algorithm with a logarithmic competitive approximation ratio. 2. TFO: full version. Design a polynomial time online algorithm with a logarithmic competitive approximation ratio, by modifying the algorithm for TFO-LumpSum.

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-69
SLIDE 69

Methodology

1. TFO-LumpSum: no salary and no firing. Design a polynomial time online algorithm with a logarithmic competitive approximation ratio. 2. TFO: full version. Design a polynomial time online algorithm with a logarithmic competitive approximation ratio, by modifying the algorithm for TFO-LumpSum.

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-70
SLIDE 70
  • 𝑦6 = 1 if worker 𝑋6 is hired, 0 otherwise.
  • 𝑔

6: = 1 if worker 𝑋6 is outsourced for performing task 𝐾:, 0 otherwise.

TFO-LumpSum: Online primal–dual technique

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-71
SLIDE 71
  • 𝑦6 = 1 if worker 𝑋6 is hired, 0 otherwise.
  • 𝑔

6: = 1 if worker 𝑋6 is outsourced for performing task 𝐾:, 0 otherwise.

TFO-LumpSum: Online primal–dual technique

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-72
SLIDE 72

TFO-LumpSum: Algorithm

Competitive approximation ratio: Running time:

When job 𝐾< arrives: Step 1: Increase potentials: Step 2: Perform randomized rounding to decide which worker to hire and to whom to outsource

n: total number of workers. m: total number of skills. C*: maximum hiring cost.

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-73
SLIDE 73

Methodology

1. TFO-LumpSum: no salary and no firing. Design a polynomial time online algorithm with a logarithmic competitive approximation ratio. 2. TFO: full version. Design a polynomial time online algorithm with a logarithmic competitive approximation ratio, by modifying the algorithm for TFO-LumpSum.

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-74
SLIDE 74

TFO

  • Theorem. There exists a a polynomial time online algorithm for

TFO with competitive approximation ratio

m: total number of skills. C*: maximum hiring cost. T*: number of tasks in the stream. n: total number of workers.

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Proof. Use online primal–dual schema with a more complicated

set of integer and linear programs.

slide-75
SLIDE 75

TFO

  • Theorem. There exists a a polynomial time online algorithm for

TFO with competitive approximation ratio

m: total number of skills. C*: maximum hiring cost. T*: number of tasks in the stream. n: total number of workers.

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Proof. Use online primal–dual schema with a more complicated

set of integer and linear programs.

slide-76
SLIDE 76

Experiments: Datasets

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

Generation of the stream of tasks:

  • Pick a random task as pivot.
  • With probability 1-1/p, pick the next task within those whose Jaccard

similarity with the pivot is at least 0.5.

  • With probability 1/p, pick another random task as a pivot.
slide-77
SLIDE 77

Experiments: TFO vs. Heuristics

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

Generation of the stream of tasks:

  • Pick a random task as pivot.
  • With probability 1-1/100, pick the next task within those whose Jaccard

similarity with the pivot is at least 0.5.

  • With probability 1/100, pick another random task as a pivot.
slide-78
SLIDE 78

Experiments: TFO vs. Always Outsource

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-79
SLIDE 79

Experiments: TFO vs. Always Outsource

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

slide-80
SLIDE 80

Conclusions

Aris Anagnostopoulos Algorithms for Hiring and Outsourcing in the Online Marketplace London, KDD 2018

  • Defined a novel online team formation problem in a hire-or-
  • utsource setting
  • Designed polynomial-time online algorithms with competitive

approximation ratios

  • Shown the applicability of our algorithmic solutions, by

performing experiments using data from online outsourcing marketplaces

  • Showed the practical use of the online primal–dual schema

Future work:

  • Relax/test some of the modeling assumptions
  • k-TFO: # of hired workers can be at most a fixed number k
slide-81
SLIDE 81

Future directions

Modeling

  • Several human elements: capabilities, cooperation, etc.
  • Application dependent

Learning

  • Learning profiles of experts
  • Learn coordination based on performance

Algorithmic

  • Matching problems
  • How to train experts
  • Explore-exploit tradeoff
slide-82
SLIDE 82

Future directions

Game-theoretic

  • Incentives for participation and rewarding mechanisms
  • Issues on cooperation / altruism / trust
slide-83
SLIDE 83

Thanks!

Questions, comments, etc.: Stefano: http://www.dis.uniroma1.it/~leon