Surveys Classes on Tuesday (Nov 26)? One (long) lecture for project - - PowerPoint PPT Presentation
Surveys Classes on Tuesday (Nov 26)? One (long) lecture for project presentation or two separate lectures? 1 CS6501: T opics in Learning and Game Theory (Fall 2019) Learning From Strategically Revealed Samples Instructor: Haifeng Xu
1
Ø Classes on Tuesday (Nov 26)? Ø One (long) lecture for project presentation or two
separate lectures?
CS6501: T
(Fall 2019) Learning From Strategically Revealed Samples
Instructor: Haifeng Xu
Part of slides by Hanrui Zhang
3
Ø Introduction and An Example Ø Formal Model and Results Ø Learning from Strategic Samples: Other Works
4
A new postdoc
applicant Alice
She has 50
papers and I only want to read 3.
The Trouble of Bob, a Professor of Rocket Science
5
Current postdoc Charlie is happy . . .
Give me 3 papers by Alice that I need to read.
Charlie is excited
about hiring Alice
6
They know what each other is thinking…
I got to pick best 3 papers to persuade Bob, so that he will hire Alice.
Charlie shall pick best 3 papers by Alice – I need to calibrate for that
7
ØSetup: (binary-)classify distributions with
label 𝑚 ∈ {, 𝑐}
samples drawn from distributions
ØGoal: accept good ones (𝑚 = ) and reject
bad ones (𝑚 = 𝑐)
ØPrevious example: a postdoc candidate = a
distribution (over papers)
Alice is waiting to hear from bob
8
Principal Reacts by Committing to a Policy
ØPrincipal (Bob) commits to and announces a policy to agent Charlie
I will hire Alice if you give me 3 good papers, or 2 excellent papers
and I want Alice to be first author on at least 2 of them.
9
Charlie is reading through Alice’s 50 papers…
ØHas access to 𝑜(= 50) samples (papers) from distribution 𝑚 (Alice)
ØCan choose 𝑛(= 3) samples as his report
10
ØHas access to 𝑜(= 50) samples (papers) from distribution 𝑚 (Alice)
ØCan choose 𝑛(= 3) samples as his report ØAgent (Charlie) sends his report to Bob
principal (Bob), aiming to persuade Bob to accept distribution 𝑚 (Alice)
Charlie found 3 papers by Alice meeting bob’s criteria He is sure bob will hire Alice upon seeing these 3 papers
11
ØBob observes Charlie’s report, and makes a decision according to
the policy he announced
it looks like
Alice is doing
good work, so let’s hire her.
I read the 3 papers you sent
but the other two are incredible.
12
ØUniversity admissions
13
ØUniversity admissions
ØClassify loan lending decisions
APPROVED
14
ØUniversity admissions
ØClassify loan lending decisions
ØDecide which restaurants to go based on Yelp rating
ØHiring job candidates in various scenarios
15
ØUniversity admissions
ØClassify loan lending decisions
ØDecide which restaurants to go based on Yelp rating
ØHiring job candidates in various scenarios ØNote: this problem deserves study even you do classification
manually instead of using an automated classifier
16
Ø Introduction and An Example Ø Formal Model and Results Ø Learning from Strategic Samples: Other Works
17
ØA distribution 𝑚 ∈ {, 𝑐} arrives, which can be good (𝑚 = ) or bad
(𝑚 = 𝑐)
ØAn agent has access to 𝑜 i.i.d. samples from 𝑚, from which he
chooses a subset of exactly 𝑛 samples as his report
ØPrincipal observes agent’s report, and decides whether to accept
18
a distribution 𝑚 ∈ {, 𝑐} arrives 𝑚 generates 𝑜 iid samples 𝐸 = {𝑒2, ⋯ , 𝑒4} Principal commits to a policy Π(𝑆) ∈ [0, 1] that maps report 𝑆 to probability of accepting 𝑆 Agent receives 𝐸 and report 𝑆 = {𝑠
2, … , 𝑠 <} ⊆ 𝐸
probability 𝑞 = Π(𝑆)
Objective: maximize prob of accepting 𝑚 Objective: accept g and reject b
19
ØThis is the same as distinguishing two distributions from samples
be 100% certain)
Fact: Let 𝜗 = max
C [ 𝑇 − 𝑐 𝑇 ] be total variation (TV) distance
between , 𝑐. Then Ω(1/𝜗H) samples to distinguish , 𝑐 with constant success probability.
Note: (𝑇) = Pr
K∼M(𝑦 ∈ 𝑇) is accumulated probability for 𝑦 ∈ 𝑇
20
ØThis is the same as distinguishing two distributions from samples
be 100% certain) 𝑐
− 𝑐
OP
Illustration of TV distance Formally, − 𝑐
OP = Q K:M K ST(K)
[ 𝑦 − 𝑐(𝑦)]𝑒𝑦
Fact: Let 𝜗 = max
C [ 𝑇 − 𝑐 𝑇 ] be total variation (TV) distance
between , 𝑐. Then Ω(1/𝜗H) samples to distinguish , 𝑐 with constant success probability
21
ØThis is the same as distinguishing two distributions from samples
be 100% certain)
Fact: Let 𝜗 = max
C [ 𝑇 − 𝑐 𝑇 ] be total variation (TV) distance
between , 𝑐. Then Ω(1/𝜗H) samples to distinguish , 𝑐 with constant success probability Proof
ØFirst, compute S∗ = arg max
C [ 𝑇 − 𝑐 𝑇 ]
ØIdea: try to estimate value of 𝑚(𝑇∗) where 𝑚 ∈ {, 𝑐}
ØHow to estimate 𝑚(𝑇∗) from samples?
ØΩ(1/𝜗H) samples suffices to distinguish random variable
(𝑇∗) from 𝑐(𝑇∗)
22
ØThis is the same as distinguishing two distributions from samples
be 100% certain)
Fact: Let 𝜗 = max
C [ 𝑇 − 𝑐 𝑇 ] be total variation (TV) distance
between , 𝑐. Then Ω(1/𝜗H) samples to distinguish , 𝑐 with constant success probability Remarks
ØWhen agent is not strategic, performance depends on TV
distance in the form of Ω
2 XY
23
“Tough” World
ØA good candidate writes a good paper w.p. 0.05 ØA bad candidate writes a good paper w.p. 0.005 ØAll candidates have 𝑜 = 50 papers, and the professor wants to
read only 𝑛 = 1 good candidate Q: What is a reasonable principal policy?
24
“Tough” World
ØA good candidate writes a good paper w.p. 0.05 ØA bad candidate writes a good paper w.p. 0.005 ØAll candidates have 𝑜 = 50 papers, and the professor wants to
read only 𝑛 = 1 good candidate Q: What is a reasonable principal policy?
ØAccept iff the reported paper is good
1 − 0.05 [\ ≈ 0.92
1 − 0.005 [\ ≈ 0.22
ØWhat happens if agent not strategic? ØStrategic selection actually helps principal!
à almost cannot distinguish
25
“Easy” World
ØA good candidate writes a good paper w.p. 0.05 0.95 ØA bad candidate writes a good paper w.p. 0.005 0.05 ØAll candidates have 𝑜 = 50 papers, and the professor wants to
read only 𝑛 = 1 good candidate
26
“Easy” World
ØA good candidate writes a good paper w.p. 0.05 0.95 ØA bad candidate writes a good paper w.p. 0.005 0.05 ØAll candidates have 𝑜 = 50 papers, and the professor wants to
read only 𝑛 = 1 good candidate Policy: Accept iff the reported paper is good
ØGood candidate is accepted with prob 𝑞M = 1 −
1 − 0.95 [\ ≈ 1
ØA bad candidate is accepted with prob 𝑞T = 1 −
1 − 0.05 [\ ≈ 0.92
ØWhat happens if agent not strategic? ØHere, strategic selection hurts principal!
à can distinguish easily
27
ØThat is, principle tries to use the “most distinguishable” sample
Theorem: Any pareto optimal deterministic policy satisfies: 1. It orders sample space based on likelihood ratio (𝑦)/𝑐(𝑦) 2. Limiting acceptance probability satisfy: 𝑞M + 1 − 𝑞T a = 1 where 𝑠 = max
K
(𝑦)/𝑐(𝑦) is maximum likelihood ratio
Pareto frontier when 𝑠 = 3 Note: can define error rate = min de
df
28
ØThat is, principle tries to use the “most distinguishable” sample ØIn strategic environment, likelihood ratio (𝑦)/𝑐(𝑦) matters
Theorem: Any pareto optimal deterministic policy satisfies: 1. It orders sample space based on likelihood ratio (𝑦)/𝑐(𝑦) 2. Limiting acceptance probability satisfy: 𝑞M + 1 − 𝑞T a = 1 where 𝑠 = max
K
(𝑦)/𝑐(𝑦) is maximum likelihood ratio
29
ØThis is not exactly optimal ØBut, error rate decrease exponentially in 𝑛 ØWhat is optimal like?
Theorem: There is a deterministic policy: 1. Which orders the sample space 2. Whose limiting error rate is at most exp(−𝑛 1 − 𝑠i\.[ H/2)
30
Ø Introduction and An Example Ø Formal Model and Results Ø Learning from Strategic Samples: Other Works
31
ØLearning from samples that are strategically transformed
32
ØStrategic behaviors are costly
APPROVED
33
ØHow to induce the correct strategic behaviors
Paper: How Do Classifiers Induce Agents To Invest Effort Strategically by Kleinberg and Raghavan
Haifeng Xu
University of Virginia hx4ad@virginia.edu