The Hiring Problem: Going Beyond Secretaries Sergei Vassilvitskii - - PowerPoint PPT Presentation

▶
the hiring problem
SMART_READER_LITE
LIVE PREVIEW

The Hiring Problem: Going Beyond Secretaries Sergei Vassilvitskii - - PowerPoint PPT Presentation

Aug 17, 2023 196 likes •1.15k views

The Hiring Problem: Going Beyond Secretaries Sergei Vassilvitskii (Yahoo!) Andrei Broder (Yahoo!) Adam Kirsch (Harvard) Ravi Kumar (Yahoo!) Michael Mitzenmacher (Harvard) Eli Upfal (Brown) The Secretary Problem Interview candidates for a

slide-1
SLIDE 1

The Hiring Problem:

Going Beyond Secretaries

Sergei Vassilvitskii (Yahoo!) Andrei Broder (Yahoo!) Adam Kirsch (Harvard) Ravi Kumar (Yahoo!) Michael Mitzenmacher (Harvard) Eli Upfal (Brown)

slide-2
SLIDE 2

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-3
SLIDE 3

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-4
SLIDE 4

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-5
SLIDE 5

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-6
SLIDE 6

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-7
SLIDE 7

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-8
SLIDE 8

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-9
SLIDE 9

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-10
SLIDE 10

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. n

slide-11
SLIDE 11

The Secretary Problem

Interview candidates for a position one at a time. After each interview decide if the candidate is the best. Goal: maximize the probability of choosing the best candidate. This is not about hiring secretaries, but about decision making under uncertainty. n

slide-12
SLIDE 12

The Hiring Problem

slide-13
SLIDE 13

The Hiring Problem

A startup is growing and is hiring many employees: Want to hire good employees Can’t wait for the perfect candidate

slide-14
SLIDE 14

The Hiring Problem

A startup is growing and is hiring many employees: Want to hire good employees Can’t wait for the perfect candidate Many potential objectives. Explore the tradeoff between number of interviews & the average quality.

slide-15
SLIDE 15

The Hiring model

Candidates arrive one at a time. Assume all have iid uniform(0,1) quality scores - For applicant denote it by . (Can deal with other distributions, not this talk) i iq

slide-16
SLIDE 16

The Hiring model

Candidates arrive one at a time. Assume all have iid uniform(0,1) quality scores - For applicant denote it by . (Can deal with other distributions, not this talk) During the interview: Observe Decide whether to hire or reject i iq iq

slide-17
SLIDE 17

Strategies

slide-18
SLIDE 18

Strategies

Hire above a threshold.

slide-19
SLIDE 19

Strategies

Hire above a threshold. Hire above the minimum or maximum.

slide-20
SLIDE 20

Strategies

Hire above a threshold. Hire above the minimum or maximum. Lake Wobegon Strategies: “Lake Wobegon: where all the women are strong, all the men are good looking, and all the children are above average”

slide-21
SLIDE 21

Strategies

Hire above a threshold. Hire above the minimum or maximum. Lake Wobegon Strategies: Hire above the average (mean or median)

slide-22
SLIDE 22

Strategies

Hire above a threshold. Hire above the minimum or maximum. Lake Wobegon Strategies: Hire above the average Side note: [Google Research Blog - March ‘06]: “... only hire candidates who are above the mean of the current employees...”

slide-23
SLIDE 23

Threshold Hiring

Set a threshold , hire if . t iq ≥ t

slide-24
SLIDE 24

Threshold Hiring

Set a threshold , hire if . t iq ≥ t

slide-25
SLIDE 25

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t

slide-26
SLIDE 26

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t

slide-27
SLIDE 27

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t

slide-28
SLIDE 28

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t t

slide-29
SLIDE 29

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t t

slide-30
SLIDE 30

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t t

slide-31
SLIDE 31

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t t t

slide-32
SLIDE 32

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t t t

slide-33
SLIDE 33

Threshold Hiring

Set a threshold , hire if . t iq ≥ t t t t

slide-34
SLIDE 34

Threshold Analysis

Set a threshold , hire if . Easy to see that average quality approaches Hiring rate . t iq ≥ t 1 1 − t 1 + t 2

slide-35
SLIDE 35

Threshold Analysis

Set a threshold , hire if . Easy to see that average quality approaches Hiring rate . t iq ≥ t 1 1 − t Quality stagnates and does not increase with time. 1 + t 2

slide-36
SLIDE 36

Hire only if better than everyone already hired.

Maximum Hiring

slide-37
SLIDE 37

Hire only if better than everyone already hired.

Maximum Hiring

slide-38
SLIDE 38

Hire only if better than everyone already hired.

Maximum Hiring

slide-39
SLIDE 39

Hire only if better than everyone already hired.

Maximum Hiring

slide-40
SLIDE 40

Hire only if better than everyone already hired.

Maximum Hiring

slide-41
SLIDE 41

Hire only if better than everyone already hired.

Maximum Hiring

slide-42
SLIDE 42

Hire only if better than everyone already hired.

Maximum Hiring

slide-43
SLIDE 43

Hire only if better than everyone already hired.

Maximum Hiring

slide-44
SLIDE 44

Hire only if better than everyone already hired.

Maximum Hiring

slide-45
SLIDE 45

Hire only if better than everyone already hired.

Maximum Hiring

slide-46
SLIDE 46

Hire only if better than everyone already hired.

Maximum Hiring

slide-47
SLIDE 47

Hire only if better than everyone already hired.

Maximum Hiring

slide-48
SLIDE 48

Hire only if better than everyone already hired.

Maximum Hiring

slide-49
SLIDE 49

Hire only if better than everyone already hired.

Maximum Hiring

slide-50
SLIDE 50

Maximum Analysis

Start with employee of quality Let be the i-th candidate hired q hi

slide-51
SLIDE 51

Maximum Analysis

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi

slide-52
SLIDE 52

Maximum Analysis

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn−1

slide-53
SLIDE 53

Maximum Analysis

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn−1 gn ∼ Unif(0, gn−1)

slide-54
SLIDE 54

Maximum Analysis

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn−1 gn ∼ Unif(0, gn−1) E[gn|gn−1] = gn−1 2

slide-55
SLIDE 55

Maximum Analysis

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn−1 gn ∼ Unif(0, gn−1) E[gn|gn−1] = gn−1 2 E[gn] = 1 − q 2n

slide-56
SLIDE 56

Maximum Analysis

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn−1 gn ∼ Unif(0, gn−1) E[gn|gn−1] = gn−1 2 E[gn] = 1 − q 2n Very high quality!

slide-57
SLIDE 57

Maximum Analysis

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn−1 gn ∼ Unif(0, gn−1) E[gn|gn−1] = gn−1 2 E[gn] = 1 − q 2n Extremely slow hiring!

slide-58
SLIDE 58

Lake Wobegon Strategies

slide-59
SLIDE 59

Lake Wobegon Strategies

Above the mean: Average quality after n hires: 1 − 1 √n

slide-60
SLIDE 60

Lake Wobegon Strategies

Above the mean: Average quality after n hires: Above the median: Median quality after n hires: 1 − 1 √n 1 − 1 n

slide-61
SLIDE 61

Lake Wobegon Strategies

Above the mean: Average quality after n hires: Above the median: Median quality after n hires: Surprising: Tight concentration is not possible Hiring above mean converges to a log-normal distribution 1 − 1 √n 1 − 1 n

slide-62
SLIDE 62

Hiring Above Mean

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi

slide-63
SLIDE 63

Hiring Above Mean

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn (in+1)q ∼ Unif(1 − gn, 1)

slide-64
SLIDE 64

Hiring Above Mean

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn (in+1)q ∼ Unif(1 − gn, 1) Therefore: gn+1 ∼ n + 1 n + 2gn + 1 n + 2Unif(0, gn)

slide-65
SLIDE 65

Hiring Above Mean

Start with employee of quality Let be the i-th candidate hired Focus on the gap: q hi gi = 1 − (hi)q gi Conditioned on : gn (in+1)q ∼ Unif(1 − gn, 1) = gn

  • 1 − Unif(0, 1)

n + 2

  • Therefore:

gn+1 ∼ n + 1 n + 2gn + 1 n + 2Unif(0, gn)

slide-66
SLIDE 66

Hiring Above Mean

gn+1 = gn

  • 1 − Unif(0, 1)

n + 2

slide-67
SLIDE 67

Hiring Above Mean

Expand: gn+1 = gn

  • 1 − Unif(0, 1)

n + 2

  • gn+t = gn

t

  • i=1
  • 1 − Unif(0, 1)

n + i + 1

slide-68
SLIDE 68

Hiring Above Mean

Expand: gn+1 = gn

  • 1 − Unif(0, 1)

n + 2

  • gn+t = gn

t

  • i=1
  • 1 − Unif(0, 1)

n + i + 1

  • Therefore:

gn ∼ (1 − q)

n

  • i=1
  • 1 − Unif(0, 1)

i + 1

slide-69
SLIDE 69

Hiring Above Mean

Expand: gn+1 = gn

  • 1 − Unif(0, 1)

n + 2

  • gn+t = gn

t

  • i=1
  • 1 − Unif(0, 1)

n + i + 1

  • Therefore:

gn ∼ (1 − q)

n

  • i=1
  • 1 − Unif(0, 1)

i + 1

  • E[gn] = (1 − q)

n

  • i=1
  • 1 −

1 2(i + 1)

  • = Θ( 1

√n)

slide-70
SLIDE 70

Hiring Above Mean

Conclusion: Average quality after hires: Time to hire employees: Very weak concentration results. n 1 − Θ( 1 √n) Θ(n3/2) n

slide-71
SLIDE 71

Hiring Above Median

Start with employee of quality When we have employees. Compute median of the scores Hire next applicants with scores above q 2k + 1 2 mk mk

slide-72
SLIDE 72

Hiring Above Median

mk

slide-73
SLIDE 73

Hiring Above Median

mk New hires: with quality above mk

slide-74
SLIDE 74

Hiring Above Median

mk

slide-75
SLIDE 75

Hiring Above Median

mk+1

slide-76
SLIDE 76

Hiring Above Median

mk

slide-77
SLIDE 77

Hiring Above Median

mk Inductive hypothesis: Unif(mk, 1)

slide-78
SLIDE 78

Hiring Above Median

mk Inductive hypothesis: Unif(mk, 1) New hires: are Unif(mk, 1)

slide-79
SLIDE 79

Hiring Above Median

mk Unif(mk, 1)

slide-80
SLIDE 80

Hiring Above Median

mk+1 Unif(mk, 1)

slide-81
SLIDE 81

Hiring Above Median

mk+1 Unif(mk+1, 1)

slide-82
SLIDE 82

Hiring Above Median

mk+1 Unif(mk+1, 1) Induction holds.

slide-83
SLIDE 83

Hiring Above Median

Unif(mk, 1) mk+1

slide-84
SLIDE 84

The median:

Hiring Above Median

Unif(mk, 1) mk+1

slide-85
SLIDE 85

Hiring Above Median

Unif(mk, 1) mk+1 The median: smallest of uniform r.v., each k + 1 Unif(mk, 1)

slide-86
SLIDE 86

Hiring Above Median

Unif(mk, 1) mk+1 The median: smallest of uniform r.v., each k + 1 Unif(0, g′

k)

slide-87
SLIDE 87

Hiring Above Median

Unif(mk, 1) mk+1 g′

k+1|g′ k ∼ g′ kBeta(k + 2, 1)

The median: smallest of uniform r.v., each k + 1 Unif(0, g′

k)

slide-88
SLIDE 88

Hiring Above Median

Expand (like the means): g′

k ∼ g k

  • i=1

Beta(i + 1, 1)

slide-89
SLIDE 89

Hiring Above Median

Expand (like the means): E[g′

n] = Θ( 1

n) g′

k ∼ g k

  • i=1

Beta(i + 1, 1)

slide-90
SLIDE 90

Hiring Above Median

Expand (like the means): E[g′

n] = Θ( 1

n) Caveat: this is the median gap! The mean gap is: Θ(log n n ) g′

k ∼ g k

  • i=1

Beta(i + 1, 1)

slide-91
SLIDE 91

Hiring Above Median

Conclusion: Average quality after hires: Time to hire employees: Again, very weak concentration results. n n 1 − Θ(log n n ) Θ(n2)

slide-92
SLIDE 92

Extensions

Interview preprocessing: Self selection: quality may increase over time Errors: Noisy estimates on scores. Firing: Periodically fire bottom 10% (Jack Welch Strategy) Similar analysis to the median. Many more... iq

slide-93
SLIDE 93

Thank You