Preference Modeling Lirong Xia Todays Schedule Modeling random - PowerPoint PPT Presentation

Preference Modeling Lirong Xia

Today’s Schedule Ø Modeling random preferences • Random Utility Model Ø Modeling preferences over lotteries • Prospect theory 2

Parametric Preference Learning Decisions Ground truth + Statistical model MLE, Bayesian, etc … 3

Parametric ranking models Ø A statistical model has three parts • A parameter space: Θ • A sample space: S = Rankings( A ) n • A = the set of alternatives, n=#voters • assuming votes are i.i.d. • A set of probability distributions over S: {Pr θ ( s ) for each s ∈ Rankings( A ) and θ ∈ Θ} 4

Example Ø Condorcet’s model for two alternatives Ø Parameter space Θ={ , } Ø Sample space S = { , } n Ø Probability distributions, i.i.d. Pr( | ) = Pr( | ) = p> 0.5 5

Mallows’ model [Mallows-1957] Ø Fixed dispersion 𝜒 <1 Ø Parameter space • all full rankings over candidates Ø Sample space • i.i.d. generated full rankings Ø Probabilities: Pr W ( V ) ∝ 𝜒 Kendall ( V , W ) 6

Example: Mallows for Eric Kyle Stan Ø Probabilities: 𝑎 = 1 + 2𝜒 + 2𝜒 ( + 𝜒 ) > > > > ⁄ 1 𝑎 ⁄ 𝜒 𝑎 > > > > > > 𝜒 ( 𝑎 ⁄ ⁄ 𝜒 𝑎 Truth > > > > 𝜒 ( 𝑎 ⁄ 𝜒 ) 𝑎 7 ⁄

Random utility model (RUM) [Thurstone 27] Ø Continuous parameters: Θ =( θ 1 ,…, θ m ) • m : number of alternatives • Each alternative is modeled by a utility distribution μ i • θ i : a vector that parameterizes μ i Ø An agent’s latent utility U i for alternative c i is generated independently according to μ i ( U i ) Ø Agents rank alternatives according to their perceived utilities • Pr ( c 2 ≻ c 1 ≻ c 3 | θ 1 , θ 2 , θ 3 ) = Pr U i ∼ μ i ( U 2 >U 1 >U 3 ) θ 2 θ 3 θ 1 U 3 U 1 U 2 8

Generating a preference-profile Ø Pr ( Data | θ 1 , θ 2 , θ 3 ) = ∏ V ∈ Data Pr( V | θ 1 , θ 2 , θ 3 ) Parameters θ 3 θ 2 θ 1 Agent n Agent 1 … P n = c 1 ≻ c 2 ≻ c 3 P 1 = c 2 ≻ c 1 ≻ c 3 9

Plackett-Luce model Ø μ i ’ s are Gumbel distributions • A.k.a. the Plackett-Luce (P-L) model [BM 60, Yellott 77] Ø Alternative parameterization λ 1 ,…, λ m λ 1 λ 2 λ m − 1 Pr( c 1  c 2  c m | λ 1  λ m ) = ×  × × λ 1 +  + λ m λ 2 +  + λ m λ m − 1 + λ m c 1 is the top choice in { c 1 ,…, c m } c 2 is the top choice in { c 2 ,…, c m } c m -1 is preferred to c m Ø Pros: • Computationally tractable • Analytical solution to the likelihood function – The only RUM that was known to be tractable McFadden • Widely applied in Economics [McFadden 74] , learning to rank [Liu 11], and analyzing elections [GM 06,07,08,09] Ø Cons: may not be the best model 10

Example > > > > 10× 4 1 10× 1 4 9 6 > > > > 10× 5 1 10× 1 5 5 1 4 9 5 Truth > > > > 10× 5 4 10× 4 5 6 5 11

RUM with normal distributions Ø μ i ’ s are normal distributions • Thurstone’sCase V [Thurstone 27] Ø Pros: • Intuitive • Flexible Ø Cons: believed to be computationally intractable • No analytical solution for the likelihood function Pr( P | Θ ) is known ∞ ∞ ∞ Pr( c 1   c m | Θ ) =  µ m ( U m ) µ m − 1 ( U m − 1 )  µ 1 ( U 1 ) dU 1  ∫ ∫ ∫ dU m − 1 dU m U m U 2 −∞ … 12 U m : from - ∞ to ∞ U m- 1 : from U m to ∞ U 1 : from U 2 to ∞

Decision making 13

Maximum likelihood estimators (MLE) Model: M r “Ground truth” θ … V 1 V 2 V n Ø For any profile P= ( V 1 ,…, V n ) , • The likelihood of θ is L ( θ , P )=Pr θ ( P )= ∏ V ∈ P Pr θ ( V ) • The MLE mechanism MLE ( P ) = argmax θ L ( θ , P ) • Decision space = Parameter space 14

Bayesian approach Ø Given a profile P= ( V 1 ,…, V n ) , and a prior distribution 𝜌 over Θ Ø Step 1: calculate the posterior probability over Θ using Bayes’ rule • Pr( θ | P ) ∝ 𝜌 ( θ ) Pr θ ( P ) Ø Step 2: make a decision based on the posterior distribution • Maximum a posteriori (MAP) estimation • MAP ( P ) = argmax θ Pr( θ | P ) • Technically equivalent to MLE when 𝜌 is uniform 15

Example • Θ={ , } Pr( | ) • S = { , } n = Pr( | ) • Probability distributions: = 0.6 • Data P = {10@ + 8@ } • MLE – L(O)=Pr O (O) 6 Pr O (M) 4 = 0.6 10 0.4 8 – L(M)=Pr M (O) 6 Pr M (M) 4 = 0.4 10 0.6 8 – L(O)>L(M), O wins • MAP: prior O:0.2, M:0.8 – Pr(O|P) ∝ 0.2 L(O) = 0.2 × 0.6 10 0.4 8 – Pr(M|P) ∝ 0.8 L(M) = 0.8 × 0.4 10 0.6 8 16 – Pr(M|P)> Pr(O|P), M wins

Decision making under uncertainty • You have a biased coin: head w/p p Credit: Panos Ipeirotis & Roy Radner – You observe 10 heads, 4 tails – Do you think the next two tosses will be two heads in a row? • Bayesian Ø MLE-based approach – the ground truth is • there is an unknown captured by a belief distribution but fixed ground truth – Compute Pr( p |Data) • p = 10/14=0.714 assuming uniform prior • Pr(2heads| p= 0.714 ) – Compute Pr(2heads|Data)=0.485<0 =(0.714) 2 =0.51>0.5 .5 • Yes! – No! 17

Prospect Theory: Motivating Example Ø Treat lung cancer with Radiation or Surgery Radiation Surgery 100% immediately survive 90% immediately survive Q1 22% 5-year survive 34% 5-year survive 0% die immediately 10% die immediately Q2 78% die in 5 years 66% die in 5 years Ø Q1: 18% choose Radiation Ø Q2: 49% choose Radiation 18

More Thoughts Ø Framing Effect • The baseline/starting point matters • Q1: starting at “the patient dies” • Q2: starting at “the patient survives” Ø Evaluation • subjective value (utility) • perceptual likelihood (e.g. people tend to overweight low probability events) 19

Prospect Theory Kahneman Ø Framing Phase (modeling the options) • Choose a reference point to model the options • O = {o 1 ,…,o k } Ø Evaluation Phase (modeling the preferences) • a value function v: O à R • a probability weighting function π: [0,1] à R Ø For any lottery L= (p 1 ,…,p k ) ∈ Lot(O) V(L) = ∑ π(pi)v(o i ) 20

Example: Insurance Ø potential loss of $1000 @1% Ø Insurance fee $15 Ø Q1 (reference point: current wealth) • Buy: Pay $15 for sure. V = v(-15) • No: $0@99% + $-1000@1%. V = π(.99)v(0) + π(.01)v(- 1000) = π(.01)v(-1000) Ø Q2 (reference point: current wealth-1000) • Buy: $985 for sure. V = v(985) • No: $1000@99% + $0@1%. V = π(.99)v(1000) + π(.01)v(0) = π(.99)v(1000) 21