Preference Modelling and Learning Paolo Viappiani LIP6 - CNRS, - - PowerPoint PPT Presentation

▶

Aug 11, 2023 411 likes •1.13k views

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Preference Modelling and Learning Paolo Viappiani LIP6 - CNRS, Universit Pierre et Marie Curie www-desir.lip6.fr/ viappianip/ DMRS workshop,

SLIDE 1

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Preference Modelling and Learning

Paolo Viappiani

LIP6 - CNRS, Université Pierre et Marie Curie www-desir.lip6.fr/∼viappianip/

DMRS workshop, Bolzano 18 September 2014

1/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 2

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Outline

1

Languages for Preferences

2

Utility-based Representations

3

Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

2/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 3

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Preference Handling Systems are Everywhere

Not only recommender systems Computational advertisement Intelligent user interfaces Cognitive assistants Personalized medecine Personal Robots What the theory has to say about preferences?

3/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 4

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

What are Preferences?

Preferences are “rational” desires. Preferences are at the basis of any decision aiding activity. There are no decisions without preferences. Preferences, Values, Objectives, Desires, Utilities, Beliefs,...

4/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 5

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Are We Rational Decision Makers?

NO. Human decision makers are (often) irrational and

inconsistent. (work of Nobel prize winner Daniel Kahneman) Moreover preferences are often constructed during the “decision process” itself (considering specific examples)

5/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 6

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Binary relations

Preference Relation ⊆ A × A is a reflexive binary relation x y stands for x is at least as good as y can be decomposed into an asymmetric and a symmetric part Asymmetric part Strict preference ≻: x y ∧ ¬(y x) Symmetric part Indifference ∼1: x y ∧ y x Incomparability ∼2: ¬(x y) ∧ ¬(y x)

6/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 7

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Preference Statements

People will not directly provide their preference relation . Rather, they will provide statements about preferred states of affair. Consider sentences of the type: I like red shoes. I do not like brown sugar. I prefer Obama to McCain. I do not want tea with milk. Cost is more important than safety. I prefer flying to Athens than having a suite at Istanbul.

7/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 8

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Representation Problem

Often impossible to state explicitly the preference relation (), especially when A is large → need for a compact representation! Logical Languages Weighted Logics Conditional Logics ... Graphical Languages Conditional Preference networks (CP nets) Conditional Preference networks with trade-offs Generalized Additive Independence networks

8/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 9

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Representation Languages

A Compact representation is useful so that preferences can be formulated with statements that encompass several alternatives A preference statement: “I prefer red cars over to blue cars” But....What does this exactly mean? All red cards are preferred to blue cars? Some red cards are preferred to blue cars? There is at lest one red car preferred to a blue car? I prefer the average red car to the average blue car? The need of a principled “semantic” for preference statements

9/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 10

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Logical Languages

Preference Logics Logical languages for preferences aim at giving a “semantic” to preference statements Φ is a preference formula (for example: color red) In logic you write x Φ to say that x has the “feature” expressed by formula Φ Mod(Φ) are the alternative where Φ holds Von Wright semantics The statement “I prefer Φ to Ψ” actually means preferring the state of affairs Φ ∧ ¬Ψ to Ψ ∧ ¬Φ Still not enough... Several preference semantics: strong, optimistic, pessimistic, opportunistic, ceteris paribus

10/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 11

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

From Statements to Relations: Semantics

Preference Statements I prefer Φ to Ψ (Φ and Ψ are propositional formula) Preference for Φ ∧ ¬Ψ over Ψ ∧ ¬Φ (von Wright’s interpretation) Common case boolean preferences, where Φ is a variable and Ψ its negation (I prefer furnished apartment rather the unfurnished) Different Semantics Let be a preference relation. satisfies the preference statements if it holds x y for:

for every x, y ∈ A : x Φ ∧ ¬Ψ, y Ψ ∧ ¬Φ (Strong semantics) iff x Φ ∧ ¬Ψ, y Ψ ∧ ¬Φ and additionally they have the same evaluation for the other variables (Ceteris Paribus) x and y are maximal elements of satisfying Φ ∧ ¬Ψ and Ψ ∧ ¬Φ respectively (Optimistic) x and y are minimal elements of satisfying Φ ∧ ¬Ψ and Ψ ∧ ¬Φ respectively (Pessimistic) x is maximal, y minimal elements of satisfying Φ ∧ ¬Ψ and Ψ ∧ ¬Φ respectively (Opportunistic) 11/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 12

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Ceteris Paribus: From Statements to Networks

Preferential Independence Key notion in preference reasoning. It is analogous to probabilistic independence. CP preference a preferred to a′ ceteris paribus iff ab a′b ∀b Conditional Preference a preferred to a′ given c iff ab′ a′b′c for a given c CP networks The notion of conditional preferential indipendence constitutes the main building block to develop graphical models for compactly representing complex preferences → CP networks

12/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 13

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

CP nets

Formalization Each variable X is associated with a set of parents Pa(X) and a conditional preference table (CPT). The CPT assigns, for each combination of values of the parents, a total order

n the values that X can take.

CP-nets Research question: is assignment x preferred to y ? how to find undominated assignment of variables to values? Start by nodes with no parents, assign best value, look at the children,.. Technique of “worsening flip” sequence Notice: strong analogy with Bayesian networks

13/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 14

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Utility Representation

Utility function u : X → [0, 1] Ideal item x⊤ such that u(x⊤) = 1 and u(x⊥) = 0 (scaling) Representing, eliciting u difficult in explicit form Flat utility representation often unrealistic Color Shape Position Weight ... Utility item1 red round top 1kg ... 0.82 item2 blue square top 2kg ... 1 item3 green square bottom 5kg ... 0.96 ... ... ... ... ... ... ...

14/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 15

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Additive Utility Functions

Additive representation (common in MAUT) Sum of local utility functions ui over attributes (or local value functions vi multiplied by scaling weights) Exponential reduction in the number of needed parameters u(x) =

ui(xi) =

αivi(xi) (1) Color v1 red 1.0 blue 0.7 green 0.0 Shape v2 round square 0.2 star 1 Importance for attribute “color”: α1 = 0.2, for “shape”: α2 = 0.3. Notice: many algorithms in the recommender system community (for example matrix factorization techniques) implicitly assume an additive model!

15/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 16

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Generalized Additive Utility

Sum of local utility functions uI over sets of attributes (or local value functions vi multiplied by scaling weights) Higher descriptive power than strictly additive utilities, while still having a manageable number of parameters u(x) =

uJi (xJi ) =

αJi vJi (xJi ) (2) where Ji is a set of indices, xJi the projection of x on Ji and m the number of factors.

Color Shape vcolor,shape red round 0.9 red square 1.0 red star 0.5 blue round 0.4 ... ... ... Position vposition top 1 bottom

Importance for factor “color+shape”: αJ1 = 0.2, for “position”: αJ2 = 0.3.

16/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 17

Languages for Preferences Utility-based Representations Preference Elicitation / Learning

Utility of Money

Money is a “special” attribute with monotonic preference Expected Monetary Value (EMV) = Expected Utility (note: explanation to the St. Petersburg paradox) For most people: U(100$) > 0.5U(200$) U(money) is concave (most common case): risk averse decision maker U(money) is linear: risk neutral U(money) is convex: risk seeker Concept of Certainty Equivalent (CE) Quasi-linear utility scale: utility expressed in a monetary scale Moreover, influence of “status quo” → Prospect Theory [Kahneman]

17/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 18

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Classic Approaches for Utility Elicitation

Assessment of multi attribute utility functions

Typically long list of questions Focus on high risk decision Goal: learn utility parameters (weights) up to a small error

Which queries?

Local: focus on attributes in isolation Global: compare complete outcomes

Standard Gamble Queries (SGQ)

Choose between option x0 for sure or a gamble < x⊤, l, x⊥, 1−l > (best option x⊤ with probability l, worst option x⊥ with probability 1−l)

18/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 19

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

(Standard) Elicitation of Additive Models

Consider an attribute (for example, color)

Ask for the best value (say, red) Ask for worst value (gray) Ask local standard gable for each remaining color to assess it local utility value (value function)

Bound queries can be asked

Refine intervals on local utility values

Scaling factors

Define reference outcome Ask global queries in order to assess the difference in utility

ccurring when, starting from the reference outcome,

“moving” a particular attribute to the best / worst

19/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 20

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Automated Elicitation vs Classic Elicitation

Problems with the classic view Standard gamble queries (and similar queries) are difficult to respond Large number of parameters to assess Unreasonable precision required Cognitive or computational cost may outweigh benefit Automated Elicitation and Recommendation Important points: Cognitively plausible forms of interaction Incremental elicitation until a decision is possible We can often make optimal decisions without full utility information Generalization across users

20/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 21

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Adaptive Utility Elicitation

Utility-based Interactive Recommender System: Bel: belief about the user’s utility function u Opt(Bel): optimal decision given incomplete beliefs about u Algorithm: Adaptive Utility Elicitation

Repeat until Bel meets some termination condition

Ask user some query

Observe user response r

Update Bel given r

Recommend Opt(Bel) Types of Beliefs

Probabilistic Uncertainty: distribution of parameters, updated using Bayes
Strict Uncertainty: feasible region (if linear constraints: convex polytope)

21/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 22

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Which Recommendations? Which Queries?

Several Framewors The questions:

How to represent the uncertainty over an utility function? How to aggregate this uncertainty? Make recommendations ? How to choose the next query? How to recommend a set, a ranking, ... Minimax Regret Maximin Bayesian Knoweldge constraints constraints

prob. distribtion

representation Which option minimax maximin expected to recommend? regret utility utility Which query worst-case worst-case expected value to ask next? regret reduction maximin improvement

f information

Possible other choices: Hurwitz criterion, and others. Hybrid models: minimax regret with expected regret reduction,...

22/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 23

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Minimax Regret

Intuition

Adversarial game; the recommender selects the item reducing the “regret” wrt the “best” item when the uniknonw parameters are chosen by the adversary

Robust criterion for decision making under uncertainty [Savage; Kouvelis] Showed to be effective when used for decision making under utility uncertainty [Boutilier et al., 2006] and as driver for elicitation

Advantages No heavy Bayesian updates No prior assumption required MMR computation suggests queries to ask to the user Limitations

No account for noisy responses Formulation of the optimization depends on the assumption about the utility

23/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 24

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Minimax Regret

Assumption: a set of feasible utility functions W is given The pairwise max regret PMR(x, y; W) = max

w∈W u(y; w) − u(x; w)

The max regret MR(x; W) = max

y∈X PMR(x, y; W)

The minimax regret MMR(W) of W and the minimax optimal item x∗

MMR(W) = min

x∈X MR(x, W)

x∗

= arg min

x∈X MR(x, W) 24/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 25

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Example

item feature1 feature2 a 10 14 b 8 12 c 7 16 d 14 9 e 15 6 f 16

Linear utility model with normalized utility weights (w1 + w2 = 1); u(x; w)=(1−w2)x1 +w2x2 =(x2 −x1)w2 + x1 Notice: it is a 1 dimensional problem

Initially, we only know that w2 ∈ [0, 1] PMR(a, f; w2)= maxw2 u(f; w2)−u(a; w2) = maxw2 6(1−w2) − 14w2 = maxw2 6 − 20w2 = 6 (for w2 = 0) PMR(a, b; w2)=maxw2 u(b; w2)−u(a; w2)<0 (a dominates b; there can’t be regret in choosing a instead of b!) PMR(a, c; w2)=maxw2 −3(1−w2)−2w2 =2 (for w2 = 1) ....

25/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 26

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Example (continued)

item feature1 feature2 a 10 14 b 8 12 c 7 16 d 14 9 e 15 6 f 16

Linear utility model with normalized utility weights (w1 + w2 = 1); u(x; w)=(1−w2)x1 +w2x2 =(x2 −x1)w2 + x1 Notice: it is a 1 dimensional problem

Computation of the pairwise regret table.

PMR(·, ·) a b c d e f MR a

2 4 5 6 6 b 2 4 6 7 8 8 c 3 1 7 8 9 9 d 5 3 7 1 2 7 e 8 6 10 3 1 10 f 14 12 16 9 6 16

The MMR-optimal solution is a, adversarial choice is f, and minimax regret value is 6. In reality no need to compute the full table (tree search methods) [Braziunas, PhD Thesis, 2011] Now, we want to ask a new query to improve the

decision. A very successful strategy (thought

generally not optimal!) is the current solution strategy: ask user to compare a and f

26/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 27

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

INCREMENTAL ECLITATION WITH MINIMAX REGRET

OBSERVATIONS Si P ⊂ P′ alors : ΘP′ ⊂ ΘP PMR(x, y; ΘP′) ≤ PMR(x, y; ΘP) for all x, y ∈ X MR(x; ΘP′) ≤ MR(x; ΘP) for all x ∈ X MMR(ΘP′) ≤ MMR(ΘP) → Adding preference statements cannot increase MMR (and often decreases); justification for interactive elicitation with questions. WHICH STRATEGY TO ASK QUESTIONS? Worst-case Minimax Regret : arg min

q∈Q max r

MMR(X; ΘP∪r) Current Choice Strategy : x∗ y∗ or x∗ y∗? (do you prefer x∗ or y∗ ?)

27/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 28

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

A graphical illustration for linear utility model (1/3)

28/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 29

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

A graphical illustration for linear utility model (2/3)

29/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 30

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

A graphical illustration for linear utility model (3/3)

30/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 31

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Minimax Regret Computation

Computation of Pairwise Max Regret as Linear Program

Objective function: maxw∈W w · (y − x) Usually W expressed by linear constraints such as w ·a ≥ w ·b for a b ∈ P (a set of comparisons)

Computation of Minimax Regret

Naive approach: test all n2−n combinations of choices Better idea: implement a search problem [Braziunas, 2011] i=choice of recommender, j=choice of adversary UB: upper bound on minimax regret (max regret of best solution found) LBi: lower bound on the max regret of option i After testing i against j: LBi ← max(LBi, PMR(i, j)) Whenever LBi ≥ UB: prune option i Empirically, a small number of PMR checks is needed

31/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 32

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Example of Minimax Regret Computation

Example Complete “pairwise max regret" table PMR(i, j) j=1 j=2 j=3 j=4 MR(i) i=1 1 2 3 3 i=2 2 2 2 2 i=3 4 1 1 4 i=4 3 2 3 3 Evaluation (0 PMR checks) PMR(i, j) j=1 j=2 j=3 j=4 LBi i=1 ? ? ? i=2 ? ? ? i=3 ? ? ? i=4 ? ? ? UB +Inf

32/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 33

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Example of Minimax Regret Computation

Example Complete “pairwise max regret" table PMR(i, j) j=1 j=2 j=3 j=4 MR(i) i=1 1 2 3 3 i=2 2 2 2 2 i=3 4 1 1 4 i=4 3 2 3 3 Evaluation (1 PMR checks) PMR(i, j) j=1 j=2 j=3 j=4 LBi i=1 1 ? ? 1 i=2 ? ? ? i=3 ? ? ? i=4 ? ? ? UB +Inf

33/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 34

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Example of Minimax Regret Computation

Example Complete “pairwise max regret" table PMR(i, j) j=1 j=2 j=3 j=4 MR(i) i=1 1 2 3 3 i=2 2 2 2 2 i=3 4 1 1 4 i=4 3 2 3 3 Evaluation (3 PMR checks) PMR(i, j) j=1 j=2 j=3 j=4 LBi i=1 1 2 3 3 i=2 ? ? ? i=3 ? ? ? i=4 ? ? ? UB 3

34/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 35

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Example of Minimax Regret Computation

Example Complete “pairwise max regret" table PMR(i, j) j=1 j=2 j=3 j=4 MR(i) i=1 1 2 3 3 i=2 2 2 2 2 i=3 4 1 1 4 i=4 3 2 3 3 Evaluation (6 PMR checks) PMR(i, j) j=1 j=2 j=3 j=4 LBi i=1 1 2 3 3 i=2 2 2 2 2 i=3 ? ? ? i=4 ? ? ? UB 2

35/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 36

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Example of Minimax Regret Computation

Example Complete “pairwise max regret" table PMR(i, j) j=1 j=2 j=3 j=4 MR(i) i=1 1 2 3 3 i=2 2 2 2 2 i=3 4 1 1 4 i=4 3 2 3 3 Evaluation (7 PMR checks) PMR(i, j) j=1 j=2 j=3 j=4 LBi i=1 1 2 3 3 i=2 2 2 2 2 i=3 4 ? ? 4 i=4 ? ? ? UB 2

36/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 37

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Example of Minimax Regret Computation

Example Complete “pairwise max regret" table PMR(i, j) j=1 j=2 j=3 j=4 MR(i) i=1 1 2 3 3 i=2 2 2 2 2 i=3 4 1 1 4 i=4 3 2 3 3 Evaluation (8 PMR checks) PMR(i, j) j=1 j=2 j=3 j=4 LBi i=1 1 2 3 3 i=2 2 2 2 2 i=3 4 ? ? 4 i=4 3 ? ? 3 UB 2

37/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 38

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Different Aggregation Functions

x = (x1, . . . , xn), y = (y1, . . . , yn) → x y ⇔ u(x; ω) ≥ u(y; ω) Weighting parameters provide a control on: the type of compromise sought in Multicriteria Decision Making the attitude towards equity in Social Choice Standard weighting parameters (weights attached to criteria)

Weighted sum : u(x; ω) =

ωixi

Weighted Tchebycheff : u(x; ω) = max

i∈[ [1;n] ] {ωi x∗

i −xi

x∗

i −x∗i }

Rank-dependent weighting parameters (weights attached to ranks)

OWA : u(x; w) =

wix(i)

Choquet : u(x; v) =

i=1
x(i) − x(i−1)
v(X(i))

38/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 39

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Application to OWA

Positive normalized weights (general case) u(x; w) =

n

wix(i) non-linear in x but linear in w ! u(x, w) ≥ u(y, w) ⇐ ⇒ w.x↑ ≥ w.y↑ where x↑ is the vector x sorted by increasing order → preference of type x y is equivalent to a linear inequality Positive normalized decreasing weights (fair optimization)

OWA with decreasing weights: wi > wj whenever i < j ensures the compatibility with Pigou-Dalton transfers, i.e: ∀i, j : xj > xi, ∀ε ∈ (0, xj − xi), (x1, xi + ε, . . . , xj − ε, xn) ≻ (x1, . . . , xn) → add inequalities of type wi − wi+1 ≥ δ with δ > 0

39/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 40

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Regret Computation for OWA

Dataset Manipulation x↑ = (x(1), ..., x(n)) permutation of features from worst to best PMR for OWA max

w

w.(y↑ − x↑) (3) s.t. 0 ≤ wi ≤ 1 ∀i (4)

wi = 1 (5) wi − wi+1 ≥ δ (6) w.a↑ ≥ w.b↑ ∀a, b s.t. a b ∈ P (7)

40/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 41

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Application to Choquet integrals

Cv(x) = x(1)v(X(1)) +

n

i=2
x(i) − x(i−1)
v(X(i))

x(i) ≤ x(i+1) for all i = 1, . . . , n − 1 and X(i) = {j ∈ N, xj ≥ x(i)}, Example ∅ {1} {2} {3} {1, 2} {1, 3} {2, 3} {1, 2, 3} v 0.1 0.2 0.3 0.5 0.6 0.7 1 x = (10, 6, 14) and y = (10, 12, 8) Cv(x) = 6 + (10 − 6)v({1, 3}) + (14 − 10)v({3}) = 9.6 Cv(y) = 8 + (10 − 8)v({1, 2}) + (12 − 10)v({2}) = 9.4 With Cv we observe that x is preferred to y.

41/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 42

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Fairness and convex capacities in the Choquet integral

v is said to be convex or supermodular when v(A ∪ B) + v(A ∩ B) ≥ v(A) + v(B) for all A, B ⊆ N, Proposition (Chateauneuf and Tallon, 1999)

When preferences are represented by a Choquet integral, then choosing v convex is equivalent to the following property: ∀x1, x2, . . . xp ∈ Rn, ∀k ∈ {1, 2, . . . , p} and ∀i ∈ {1, 2, . . . , p}, λi ≥ 0 such that p

i=1 λi = 1 we have:

Cv(x1) = Cv(x2) = . . . = Cv(xp) ⇒ Cv(

λixi) ≥ Cv(xk)

Example (convex v and fairness)

x = (18, 18, 0), y = (0, 18, 18), z = (18, 0, 18) t = (12, 12, 12) = (x + y + z)/3 v convex ⇒ [Cv(x) = Cv(y) = Cv(z) ⇒ t x, t y, t z]

42/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 43

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

COMPUTATION OF PMR FOR CHOQUET: GENERAL CASE

CAPACITY The function v : 2N → [0, 1] is a normalized capacity if: v(∅) = 0, v(N) = 1 v(A) ≤ v(B) for all A ⊂ B ⊆ N (monotonicity)

43/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 44

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

COMPUTATION OF PMR FOR CHOQUET: GENERAL CASE

CAPACITY The function v : 2N → [0, 1] is a normalized capacity if: v(∅) = 0, v(N) = 1 v(A) ≤ v(B) for all A ⊂ B ⊆ N (monotonicity) PMR(x, y; ΘP) = max

Cv(y) − Cv(x) s.t. v(∅) = 0 v(N) = 1 v(A) ≤ v(A ∪ {i}) ∀A ⊂ N, ∀i ∈ N\A Cv(a) ≥ Cv(b) ∀a, b s.t. a b ∈ P LP with an exponentiel number of variables and constraints At most 2n variables in the objective function

43/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 45

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Choquet integral and Möbius masses

An alternative representation in terms of the Möbius inverse: Möbius inverse and Möbius masses To any set-function v : 2N → R is associated m : 2N → R a mapping called Möbius inverse, defined by: ∀A ∈ N, m(A) =

B⊆A

(−1)|A\B|v(B), v(A) =

B⊆A

m(B) Coefficients m(B) for B ⊆ A are called Möbius masses. Remark: v convex if Möbius masses positive (belief function, Shafer 1976) Choquet integral as a function of Möbius masses Cv(x) =

B⊆A

m(B)

i∈B

xi

2-additive capacity (m(B) = 0 iff |B| > 2) → capacity completely characterized by (n2 + n)/2 coefficients. Good compromise between compacity of the model and expressivity. 44/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 46

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

PMR for 2-additive Choquet

m vector of variables (moebsius mass) encoding a capacity v: m = (m1, . . . , mn, m11, m12, . . . , m1n, m23, . . . , m2n . . . , mn−1n) ¯ x = (x1, . . . , xn, x11, x12, . . . , x1n, x23, . . . , x2n . . . , xn−1n), xij = xi ∧ xj Therefore we can write Cv(x) = m.(¯ x) The case of a 2-additive capacity

max

m.(¯ x − ¯ y) (8) s.t.

j∈J

mij ≥ 0, i =1,. . ., n; ∀J : {i} ⊆ J ⊆ N (9)

mi +

j=i+1

mij = 1 (10) m.¯ a ≥ m.¯ b ∀a, b s.t. a b ∈ P

Constraint 11 ensures that the capacity is normalized. Monotonicity (9) requires n 2n−1 constraints. Only for 2-additive capacities: efficient formulation in term of convex combination of extreme masses [Hullermeier 2012]. 45/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 47

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

PMR for 2-additive Belief Functions

m vector of variables (moebsius mass) encoding a capacity v: m = (m1, . . . , mn, m11, m12, . . . , m1n, m23, . . . , m2n . . . , mn−1n) ¯ x = (x1, . . . , xn, x11, x12, . . . , x1n, x23, . . . , x2n . . . , xn−1n), xij = xi ∧ xj Cv(x) = m.(¯ x) belief functions are capacities such that mi,j ≥ 0

The case of (2-additive) belief functions max

m

m.(¯ x − ¯ y) (11) s.t.

n

mi +

n

j=i+1

mij = 1 (12) mi ≥ 0, i =1, . . . , n, (13) mij ≥ 0, i =1, . . . , n, j =i + 1, . . . , n, (14) m.¯ a ≥ m.¯ b ∀a, b s.t. a b ∈ P

46/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 48

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

PREFERENCE QUERIES OF TYPE “1A0 Λ ?”

COMPARISON BETWEEN: 1A0, the fictions alternative such that 1A0i = 1 if i ∈ A and 1A0i = 0

therwise, and

Λ, the fictious alternative such that Λi = λ for all i ∈ N. OBSERVATIONS Cv(1A0) = v(A) for all A ⊆ N Cv(Λ) = λ for all λ ∈ R CONSEQUENCES 1A0 Λ ⇔ v(A) ≥ λ ⇔ v(B) ≥ λ for all B ⊇ A 1A0 Λ ⇔ v(A) ≤ λ ⇔ v(B) ≤ λ for all B ⊆ A

47/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 49

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

PREFERENCE QUERIES OF TYPE “1A0 Λ ?”

COMPARISON BETWEEN: 1A0, the fictions alternative such that 1A0i = 1 if i ∈ A and 1A0i = 0

therwise, and

Λ, the fictious alternative such that Λi = λ for all i ∈ N. OBSERVATIONS Cv(1A0) = v(A) for all A ⊆ N Cv(Λ) = λ for all λ ∈ R CONSEQUENCES 1A0 Λ ⇔ v(A) ≥ λ ⇔ v(B) ≥ λ for all B ⊇ A 1A0 Λ ⇔ v(A) ≤ λ ⇔ v(B) ≤ λ for all B ⊆ A PROPOSITION 1 : COMPATIBILITY WITH PREFERENCES P Updating [lB, uB] for all B ⊇ A (resp. B ⊆ A) at each insertion of preferences 1A0 Λ (resp. 1A0 Λ) in P, the compatibility with P is assured.

47/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 50

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

COMPUTATION OF PMR FOR CHOQUET

Let P be preferences statements obtained by asking only 1A0 Λ queries. PROPOSITION 2 : EXISTENCE OF A COMPATIBLE CAPACITY Let v|D : 2N → [0, 1] be a partial function defined on D ⊆ 2N such that:

v|D(A) ∈ [lA, uA] for all A ∈ D
v|D(A) ≤ v|D(B) for all A, B ∈ D such that A ⊂ B

In this case, v|D can be completed in a capacity of ΘP.

48/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 51

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

COMPUTATION OF PMR FOR CHOQUET

Let P be preferences statements obtained by asking only 1A0 Λ queries. PROPOSITION 2 : EXISTENCE OF A COMPATIBLE CAPACITY Let v|D : 2N → [0, 1] be a partial function defined on D ⊆ 2N such that:

v|D(A) ∈ [lA, uA] for all A ∈ D
v|D(A) ≤ v|D(B) for all A, B ∈ D such that A ⊂ B

In this case, v|D can be completed in a capacity of ΘP. PMR(x, y; ΘP) = max

Cv(y) − Cv(x) s.t. v(X(i+1)) ≤ v(X(i)) ∀i ∈ {1, . . . , n − 1} v(Y(i+1)) ≤ v(Y(i)) ∀i ∈ {1, . . . , n − 1} v(X(i)) ≤ v(Y(j)) ∀i, j ∈ N s.t. X(i) ⊂ Y(j) and X(i) ⊂ Y(j+1) v(Y(i)) ≤ v(X(j)) ∀i, j ∈ N s.t. Y(i) ⊂ X(j) and Y(i) ⊂ X(j+1) uX(i) ≤ v(X(i)) ≤ lX(i)

∀i∈N

uY(i) ≤ v(Y(i)) ≤ lY(i) ∀i ∈ N

48/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 52

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

COMPUTATION OF PMR FOR CHOQUET

A closer look at the Objective Function

Remember that Cv(x) = x(1)v(X(1)) + n

i=2

x(i) − x(i−1)
v(X(i)).

Cv(y) − Cv(x) =

A∈{X(i) | X(i)=Y(i)}

−(x(i) − x(i−1))vA + (15)

A∈{Y(i) | Y(i)=X(i)}

(y(i) − y(i−1))vA + (16)

A∈{X(i) | X(i)=Y(i)}

(y(i) − y(i−1) − x(i) + x(i−1))vA. (17) Objective function has to be maximized: vA will be as small as possible for all A ∈ {X(i) | X(i) = Y(i)} and as large as possible for all A ∈ {Y(i) | Y(i) = X(i)}.

49/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 53

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

COMPUTATION OF PMR FOR CHOQUET

Let P be preferences statements obtained by asking only 1A0 Λ queries. PMR(x, y; ΘP) = max

Cv(y) − Cv(x) s.t. v(X(i+1)) ≤ v(X(i)) ∀i ∈ {1, . . . , n − 1} v(Y(i+1)) ≤ v(Y(i)) ∀i ∈ {1, . . . , n − 1} v(X(i)) ≤ v(Y(j)) ∀i, j ∈ N s.t. X(i) ⊂Y(j) and X(i) ⊂Y(j+1) v(Y(i)) ≤ v(X(j)) ∀i, j ∈ N s.t. Y(i) ⊂X(j) and Y(i) ⊂X(j+1) Y(i) ⊂X(j), Y(i) ⊂X(j+1), and Y(i−1) ⊂X(j) uX(i) ≤ v(X(i)) ≤ lX(i)

∀i∈N

uY(i) ≤ v(Y(i)) ≤ lY(i) ∀i ∈ N → LP with at most 2(n−1) variables et 3(n−1)+2n = 5n − 3 constraints. [Benabbou et al., 2014]

50/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 54

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

QUERY STRATEGIES FOR CHOQUET

SELECTION CRITERIA: WORST-CASE MINIMAX REGRET min

A⊆N min Λ max{MMR(X; ΘP∪{1A0Λ}), MMR(X; ΘP∪{1A0Λ})}

(myopic measure of “value of information”) OBSERVATIONS For all A ⊆ N: MMR(X; ΘP∪{1A0Λ}) is a function of λ decreasing on [lA, uA] MMR(X; ΘP∪{1A0Λ}) function of λ increasing on [lA, uA] These two functions have the same maximum CONSEQUENCES MMR(X; ΘP∪{1A0Λ}) and MMR(X; ΘP∪{1A0Λ}) have necessarily an intersection. Minimization of Λ can be done with dichotomic search.

51/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 55

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

QUERY STRATEGIES FOR CHOQUET

OBSERVATION 1 : There are exactly 2n − 2 possible subsets of criteria A ⊆ N. HEURISTIC Consider only the attributes implicated in the objective PMR(x∗, y∗; ΘP) for x∗ and y ∗ that are associated with minimax regret. REMARK 2 : It is possible that no single query can reduce MMR TIE-BREAKING Choose A minimizing λ = lA + uA 2 : 0.5 MMR(X; ΘP∪{1A0Λ}) + 0.5 MMR(X; ΘP∪{1A0Λ}) [Benabbou et al., 2014]

52/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 56

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Experiment 1

10-dimensional Knapsack problem with 1000 items. Interactive elicitation with a simulated user; minimax-regret assuming a Choquet integral, when the user is answering according to different underlying models. 53/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 57

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Experiment 2

Datasets of 100 alternatives evaluated on 10 criteria and characterized by a set of performance vectors X a are randomly generated; constructed in such a way that n

i=1 xa i

= 1 for all x ∈ X a, where a ∈ {0.5, 1, 2} so as to

btain different types of Pareto sets.

54/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 58

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Which Recommendations? Which Queries?

Several Framewors The questions:

How to represent the uncertainty over an utility function? How to aggregate this uncertainty? Make recommendations ? How to choose the next query? How to recommend a set, a ranking, ... Minimax Regret Maximin Bayesian Knoweldge constraints constraints

prob. distribtion

representation Which option minimax maximin expected to recommend? regret utility utility Which query worst-case worst-case expected value to ask next? regret reduction maximin improvement

f information

Possible other choices: Hurwitz criterion, and others. Hybrid models: minimax regret with expected regret reduction,...

55/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 59

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Bayesian Framework for Recommendation and Elicitation

Let’s assume utility u(x; w) parametric in w for a given structure, for example u(x; w)=w · x P(w) probability distribution over utility function Expected utility of a given item x EU(x)=

u(x) P(w) dw

Current expected utility of best recommendation x∗ EU∗ =maxx∈A EU(x); x∗ =arg maxx∈A EU(x) When a new preference is known (for instance, user prefers apples over orange), the distribution is updated according to Bayes (Monte Carlo methods, Expectation Propagation)

(possible prior distribution) (distribution updated after user feedback) 56/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 60

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Response Models

Model the user’s cognitive ability of answering correctly to a preference query Noiseless responses (unrealistic but often assumed in research papers!) Constant error (can model distraction, e.g. clicking on the wrong icon) Logistic error (Boltzmann distribution), a commonly used probabilistic response model for comparison/choice queries: “Among options in set S, which one do you prefer?”

probability of response “x is my preferred item in S”

Pr(S → x) = eγu(x)

y∈S eγu(y)

γ is a temperature parameter (how “noisy” is the user). For comparison queries (“Is item1 better than item2?”) P(selecting 1st item) as a function of the difference in utility 57/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 61

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

What Query to Ask Next?

The problem can be modeled as a POMDP [Boutilier, AAAI 2002], however impractical to solve for non trivial cases Idea: ask query with highest “value”, a posteriori improvement in decision quality In a Bayesian approach (Myopic) Expected Value of Information EVOIθ(q) =

r∈R

Pθ(r)EU∗

θ|r − EU∗ θ

where R is the set of possible responses (answers); θ is the current belief distribution and θ|r the posterior and Pθ(r) the prior probability of a given response. Ask query q∗ = arg max EVOIθ(q) with highest EVOI In non-Bayesian setting, one can use non probabilistic measures of decision improvement (for example, worst-case regret reduction, ...)

58/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 62

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Efficient Computation of Queries

Optimal Query Sets and Recommendation Sets How to optimize VoI depends on the query type, in general too many possible queries to iterate over them For comparison queries there are n(n−1)

possible queries For choice queries of size k there are n

candidate queries

For bound queries, continuous space Optimal Query Sets and Recommendation Sets Characterization of the myopically optimal choice query [Viappiani and Boutilier, 2010] Tight connection with problem of generating a set of recommendations The optimization problem is submodular → greedy optimization gives strong guarantees Lazy evaluation techniques are computationally very efficient (<1 second for large datasets; naive methods several hours...)

59/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 63

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Experimental Results (Bayesian Elicitation)

We know that the myopic value of information of the queries posed by the greedy strategies are close to the optimum. Empirical results show that they are also effective when evaluated in an iterative process; the recommended item converges to the “true” best item in few cycles.

60/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 64

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Application: Active Collaborative Filtering

Incremental Elicitation of Rating Profiles Recommender systems such as Netflix asks you to rate more movies, in order to improve their preference models and provide better recommendations in the future. This can be seen as an implicit “query” asking for your preferred movies among the ones presented.

61/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 65

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

Other Approaches

Point-wise methods (Support Vector Machines) Sorting / Ordered classification problems Learning preference relations (qualitative preferences) Learning CP networks Relational learning Rankings: neighbour models, probabilistic approaches (Mallows, Babington-Smith),...

62/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 66

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

The Road Ahead

Preference learning must deal with “biased” decision makers: more work at the intersection of behavioral decision theory and principled mathematical methods Learning/elicitation wrt Prospect Theory Social context of personalization Utility-based evaluation of “contagion” effect in social networks Sequential decision problems (MDPs / reinforcement learning) and sequential optimization of elicitation (optimization of VOI for long horizon)

63/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 67

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

References 1 (Fundamentals: Preference Modeling)

L. J. Savage, The Foundations of Statistics, Wiley, New York, 1954.

P . Kouvelis and G. Yu, Robust Discrete Optimization and Its Applications, Kluwer, Dordrecht, 1997. Roberts F .S, Measurement theory, with applications to Decision Making, Utility and the Social Sciences, Addison-Wesley, Boston, 1979. Roubens M., Vincke Ph., Preference Modeling, Springer Verlag, Berlin, 1985. Fishburn P .C., Interval Orders and Interval Graphs, J. Wiley, New York, 1985. Fodor J., Roubens M., Fuzzy preference modelling and multicriteria decision support, Kluwer Academic, Dordrecht, 1994. Pirlot M., Vincke Ph., Semi Orders, Kluwer Academic, Dordrecht, 1997. Fishburn P .C., Preference structures and their numerical representations, Theoretical Computer Science, vol. 217, 359-383, 1999. Öztürk M., Tsoukiàs A., Vincke Ph., Preference Modelling, in M. Ehrgott, S. Greco, J. Figueira (eds.), State of the Art in Multiple Criteria Decision Analysis, Springer Verlag, Berlin, 27 - 72, 2005.

64/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 68

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

References 2

Prefererence Representation and Languages Craig Boutilier, Ronen I. Brafman, Carmel Domshlak, Holger H. Hoos, David

Poole. CP-nets: A Tool for Representing and Reasoning with Conditional Ceteris

Paribus Preference Statements. JAIR 21: 135-191 (2004) Souhila Kaci. Working with Preferences: Less Is More. Cognitive Technologies, Springer 2011. Behavioral Decision Theory and Biases Kahneman, Daniel; Paul Slovic and Amos Tversky. Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, 1982. Gilovich, Thomas and Dale Griffin, Daniel Kahneman. Heuristics and biases: The psychology of intuitive judgment, Cambridge University Press, 2002. John W. Payne, James R. Bettman and Eric J. Johnson. The Adaptive Decision Maker, Cambridge University Press, 1993. Daniel Kahneman. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011.

65/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 69

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

References 3 (Incremental Utility Elicitation/Learning)

Urszula Chajewska, Daphne Koller, Ronald Parr. Making Rational Decisions Using Adaptive Utility Elicitation. AAAI 2000: 363-369. Craig Boutilier. A POMDP Formulation of Preference Elicitation Problems. AAAI/IAAI 2002: 239-246.

C. Boutilier, R. Patrascu, P

. Poupart, D. Schuurmans. Constraint-based

ptimization and utility elicitation using the minimax decision criterion. Artificial

Intelligence 170(8- 9): 686-713 (2006). Paolo Viappiani, Craig Boutilier. Regret-based optimal recommendation sets in conversational recommender systems. RecSys 2009: 101-108. Paolo Viappiani, Craig Boutilier. Optimal Bayesian Recommendation Sets and Myopically Optimal Choice Query Sets. NIPS 2010: 2352-2360. Darius Braziunas, Craig Boutilier. Assessing regret-based preference elicitation with the UTPREF recommendation system. ACM EC 2010: 219-228.

D. Braziunas. Decision-theoretic elicitation of generalized additive utilities, Ph.D.

dissertation, University of Toronto, 2011.

A. F. Tehrani, W. Cheng, K. Dembczynski, E. Hullermeier. Learning monotone

nonlinear models using the Choquet integral. Machine Learning 89(1-2): 183-211 (2012) Paolo Viappiani, Christian Kroer. Robust Optimization of Recommendation Sets with the Maximin Utility Criterion. ADT 2013: 411-424. Nawal Benabbou, Patrice Perny, Paolo Viappiani. Incremental Elicitation of Choquet Capacities for Multicriteria Decision Making. ECAI 2014: 87-92.

66/67 Paolo Viappiani Preference Modelling and Learning

SLIDE 70

Languages for Preferences Utility-based Representations Preference Elicitation / Learning Standard vs Automated Elicitation Minimax-Regret Bayesian Approaches Discussion and Future Works

References 4 (Applications, Intelligent User Interfaces)

Bart Peintner, Paolo Viappiani, Neil Yorke-Smith. Preferences in Interactive Systems: Technical Challenges and Case Studies. AI Magazine 29(4): 13-24 (2008). Pearl Pu, Boi Faltings, Li Chen, Jiyong Zhang, Paolo Viappiani. Usability Guidelines for Product Recommenders Based on Example Critiquing Research. Recommender Systems Handbook 2011: 511-545. Paolo Viappiani, Boi Faltings, Pearl Pu. Preference-based Search using Example-Critiquing with Suggestions. JAIR 27: 465-503 (2006). Krzysztof Z. Gajos, Daniel S. Weld, Jacob O. Wobbrock. Automatically generating personalized user interfaces with Supple. Artificial Intelligence 174(12-13): 910-950 (2010). Markus Stolze, Michael Strobel. Recommending as Personalized Teaching - Towards Credible Needs-based eCommerce Recommender Systems in Designing personalized user experiences in eCommerce, pages 293-313 (2004). Myers, K.; Berry, P .; Blythe, J.; Conley, K.; Gervasio, M.; McGuinness, D.; Morley, D.; Pfeffer, A.; Pollack, M.; and Tambe, M. 2007. An Intelligent Personal Assistant for Task and Time Management. AI Magazine 28(2): 47-61.

67/67 Paolo Viappiani Preference Modelling and Learning