SLIDE 1
Long-Range Planning and Behavioral Biases: A Computational Approach
Jon Kleinberg
Including joint work with Manish Raghavan and Sigal Oren.
Long-Range Planning and Behavioral Biases: A Computational Approach - - PowerPoint PPT Presentation
Long-Range Planning and Behavioral Biases: A Computational Approach - - PowerPoint PPT Presentation
Long-Range Planning and Behavioral Biases: A Computational Approach Jon Kleinberg Including joint work with Manish Raghavan and Sigal Oren. Cornell University Long-Range Planning Growth in on-line systems where users and groups have long
SLIDE 2
SLIDE 3
Badges on Stack Overflow
−60 −40 −20 20 40 60
Number of days relative to badge win
2 4 6 8 10
Number of actions per day
Civic Duty
Qs As Q-votes A-votes
−60 −40 −20 20 40 60
Number of days relative to badge win
2 4 6 8 10 12 14
Number of actions per day
Electorate
Qs As Q-votes A-votes
Badges, Milestones, and Incentives
The Placement Problem: Given a desired mixture of actions, how should one define milestones to (approximately) induce these actions? How do badges and milestones derive their value? Social / Motivational / Transactional?
Antin-Churchill 2011, Deterding et al 2011, Chawla-Hartline-Sivan 2012, Easley-Ghosh 2013, Anderson-Huttenlocher-Kleinberg-Leskovec 2013
SLIDE 4
Planning and Time-Inconsistency
Tacoma Public School System
Fundamental behavioral process: Making plans for the future.
Plans can be multi-step. Natural model: agents chooses optimal sequence given costs and benefits.
What could go wrong?
Costs and benefits are unknown, and/or genuinely changing over time. Time-inconsistency.
SLIDE 5
Planning and Time-Inconsistency
Fundamental behavioral process: Making plans for the future.
Plans can be multi-step. Natural model: agents chooses optimal sequence given costs and benefits.
What could go wrong?
Costs and benefits are unknown, and/or genuinely changing over time. Time-inconsistency.
SLIDE 6
Planning and Time-Inconsistency
Fundamental behavioral process: Making plans for the future.
Plans can be multi-step. Natural model: agents chooses optimal sequence given costs and benefits.
What could go wrong?
Costs and benefits are unknown, and/or genuinely changing over time. Time-inconsistency.
SLIDE 7
Why did George Akerlof not make it to the post office?
Agent must ship a package sometime in next n days. One-time effort cost c to ship it. Loss-of-use cost x each day hasn’t been shipped.
SLIDE 8
Why did George Akerlof not make it to the post office?
Agent must ship a package sometime in next n days. One-time effort cost c to ship it. Loss-of-use cost x each day hasn’t been shipped. An optimization problem: If shipped on day t, cost is c + tx. Goal: min
1≤t≤n c + tx.
Optimized at t = 1.
SLIDE 9
Why did George Akerlof not make it to the post office?
Agent must ship a package sometime in next n days. One-time effort cost c to ship it. Loss-of-use cost x each day hasn’t been shipped. An optimization problem: If shipped on day t, cost is c + tx. Goal: min
1≤t≤n c + tx.
Optimized at t = 1. In Akerlof’s story, he was the agent, and he procrastinated: Each day he planned that he’d do it tomorrow. Effect: waiting until day n, when it must be shipped, and doing it then, at a significantly higher cumulative cost.
SLIDE 10
Why did George Akerlof not make it to the post office?
Agent must ship a package sometime in next n days. One-time effort cost c to ship it. Loss-of-use cost x each day hasn’t been shipped. A model based on present bias [Akerlof 91; cf. Strotz 55, Pollak 68]
Costs incurred today are more salient: raised by factor b > 1.
On day t:
Remaining cost if sent today is bc. Remaining cost if sent tomorrow is bx + c. Tomorrow is preferable if (b − 1)c > bx.
SLIDE 11
Why did George Akerlof not make it to the post office?
Agent must ship a package sometime in next n days. One-time effort cost c to ship it. Loss-of-use cost x each day hasn’t been shipped. A model based on present bias [Akerlof 91; cf. Strotz 55, Pollak 68]
Costs incurred today are more salient: raised by factor b > 1.
On day t:
Remaining cost if sent today is bc. Remaining cost if sent tomorrow is bx + c. Tomorrow is preferable if (b − 1)c > bx.
General framework: quasi-hyperbolic discounting [Laibson 1997]
Cost/reward c realized t units in future has present value βδtc Special case: δ = 1, b = β−1, and agent is naive about bias. Can model procrastination, task abandonment [O’Donoghue-Rabin08], and benefits of choice reduction [Ariely and Wertenbroch 02, Kaur-Kremer-Mullainathan 10]
SLIDE 12
Cost Ratio
Cost ratio: Cost incurred by present-biased agent Minimum cost achievable Across all stories in which present bias has an effect, what’s the worst cost ratio? max stories S cost ratio(S).
SLIDE 13
Cost Ratio
Cost ratio: Cost incurred by present-biased agent Minimum cost achievable Across all stories in which present bias has an effect, what’s the worst cost ratio? max stories S cost ratio(S). ???
SLIDE 14
A Graph-Theoretic Framework
s c d t e b
8 2 2 16 8 8
a
16 2
Use graphs as basic structure to represent scenarios
[Kleinberg-Oren 2014]
Agent plans to follow cheapest path from s to t. From a given node, immediately outgoing edges have costs multplied by b > 1.
SLIDE 15
A Graph-Theoretic Framework
s c d t e b
8 2 2 16 8 8
a
16 2
36 32 34
Use graphs as basic structure to represent scenarios
[Kleinberg-Oren 2014]
Agent plans to follow cheapest path from s to t. From a given node, immediately outgoing edges have costs multplied by b > 1.
SLIDE 16
A Graph-Theoretic Framework
s c d t e b
8 2 2 16 8 8
a
16 2
24 20
Use graphs as basic structure to represent scenarios
[Kleinberg-Oren 2014]
Agent plans to follow cheapest path from s to t. From a given node, immediately outgoing edges have costs multplied by b > 1.
SLIDE 17
Example: Akerlof’s Story as a Graph
v1 t s v2 c c c v3 v4 v5 c c c x x x x x
Node vi = reaching day i without sending the package.
SLIDE 18
Paths with Rewards
s a t b
3 5 2 6
10 12
reward 11
12
Variation: agent only continues on path if cost ≤ reward at t. Can model abandonment: agent stops partway through a completed path. Can model benefits of choice reduction: deleting nodes can sometimes make graph become traversable.
SLIDE 19
Paths with Rewards
s a t b
3 5 2 6
10 11
reward 11
12
Variation: agent only continues on path if cost ≤ reward at t. Can model abandonment: agent stops partway through a completed path. Can model benefits of choice reduction: deleting nodes can sometimes make graph become traversable.
SLIDE 20
Paths with Rewards
s a t b
3 5 2 6
10 12
reward 11
12
Variation: agent only continues on path if cost ≤ reward at t. Can model abandonment: agent stops partway through a completed path. Can model benefits of choice reduction: deleting nodes can sometimes make graph become traversable.
SLIDE 21
Paths with Rewards
s a t b
3 5 2 6
10 11
reward 11
12
Variation: agent only continues on path if cost ≤ reward at t. Can model abandonment: agent stops partway through a completed path. Can model benefits of choice reduction: deleting nodes can sometimes make graph become traversable.
SLIDE 22
A More Elaborate Example
s v01 v02 v10 v11 v12 v20 v21 v22 v30 v31 t 2 + 4 + 4 =10 13 20
Three-week short course with two projects.
Reward of 16 from finishing the course. Effort cost in a given week: 1 from doing no project, 4 from doing one, 9 from doing both. vij = the state in which i weeks of the course are done and the student has completed j projects.
SLIDE 23
A More Elaborate Example
s v01 v02 v10 v11 v12 v20 v21 v22 v30 v31 t 2 + 4 + 4 =10 13 20
Three-week short course with two projects.
Reward of 16 from finishing the course. Effort cost in a given week: 1 from doing no project, 4 from doing one, 9 from doing both. vij = the state in which i weeks of the course are done and the student has completed j projects.
SLIDE 24
A More Elaborate Example
s v01 v02 v10 v11 v12 v20 v21 v22 v30 v31 t 2 + 9 = 11 12 19
Three-week short course with two projects.
Reward of 16 from finishing the course. Effort cost in a given week: 1 from doing no project, 4 from doing one, 9 from doing both. vij = the state in which i weeks of the course are done and the student has completed j projects.
SLIDE 25
A More Elaborate Example
s v01 v02 v10 v11 v12 v20 v21 v22 v30 v31 t 2 + 4 + 4 =10 13 20 18
Three-week short course with two projects.
Reward of 16 from finishing the course. Effort cost in a given week: 1 from doing no project, 4 from doing one, 9 from doing both. vij = the state in which i weeks of the course are done and the student has completed j projects.
SLIDE 26
A More Elaborate Example
s v01 v02 v10 v11 v12 v20 v21 v22 v30 v31 t 2 + 4 + 4 =10 13 20
Three-week short course with two projects.
Reward of 16 from finishing the course. Effort cost in a given week: 1 from doing no project, 4 from doing one, 9 from doing both. vij = the state in which i weeks of the course are done and the student has completed j projects.
SLIDE 27
A More Elaborate Example
s v01 v02 v10 v11 v12 v20 v21 v22 v30 v31 t 2 + 4 + 4 =10 13 20
Three-week short course with two projects.
Reward of 16 from finishing the course. Effort cost in a given week: 1 from doing no project, 4 from doing one, 9 from doing both. vij = the state in which i weeks of the course are done and the student has completed j projects.
SLIDE 28
A More Elaborate Example
s v01 v02 v10 v11 v12 v20 v21 v22 v30 v31 t 2 + 4 + 4 =10 13 20 12 19
Three-week short course with two projects.
Reward of 16 from finishing the course. Effort cost in a given week: 1 from doing no project, 4 from doing one, 9 from doing both. vij = the state in which i weeks of the course are done and the student has completed j projects.
SLIDE 29
A Bad Example for the Cost Ratio
v1 t s v2 c3 c c2 v3 v4 v5 c4 c5 c6 x x x x x
Cost ratio can be roughly bn, and this is essentially tight. Can we characterize the instances with exponential cost ratio? Goal, informally stated: Must any instance with large cost ratio contain Akerlof’s story as a sub-structure?
SLIDE 30
Characterizing Bad Instances via Graph Minors
Graph H is a minor of graph G if we can contract connected subsets of G into “super-nodes” so as to produce a copy of H. In the example: G has a K4-minor.
SLIDE 31
Characterizing Bad Instances via Graph Minors
Graph H is a minor of graph G if we can contract connected subsets of G into “super-nodes” so as to produce a copy of H. In the example: G has a K4-minor.
SLIDE 32
Characterizing Bad Instances via Graph Minors
Graph H is a minor of graph G if we can contract connected subsets of G into “super-nodes” so as to produce a copy of H. In the example: G has a K4-minor.
SLIDE 33
Characterizing Bad Instances via Graph Minors
v1 t s v2 c3 c c2 v3 v4 v5 c4 c5 c6 x x x x x
SLIDE 34
Characterizing Bad Instances via Graph Minors
SLIDE 35
Characterizing Bad Instances via Graph Minors
The k-fan Fk: the graph consisting of a k-node path, and one more node that all others link to. Theorem For every λ > 1 there exists ε > 0 such that if the cost ratio is > λn, then the underlying undirected graph of the instance contains an Fk-minor for k = εn.
SLIDE 36
Choice Reduction
s a t b
3 5 2 6
10 11
reward 11
12
Choice reduction problem: Given G, not traversable by an agent, is there a subgraph of G that is traversable?
Our initial idea: if there is a traversable subgraph in G, then there is a traversable subgraph that is a path. But this is not the case.
Results:
A characterization of the structure of minimal traversable subgraphs. NP-completeness [Feige 2014, Tang et al 2015]
SLIDE 37
Choice Reduction
s a b t c
2 3 6 6 2 reward 12
Choice reduction problem: Given G, not traversable by an agent, is there a subgraph of G that is traversable?
Our initial idea: if there is a traversable subgraph in G, then there is a traversable subgraph that is a path. But this is not the case.
Results:
A characterization of the structure of minimal traversable subgraphs. NP-completeness [Feige 2014, Tang et al 2015]
SLIDE 38
Sophistication
Sophisticated agents [O’Donoghue-Rabin 1999] Can successfully anticipate their behavior in the future. Plan in the present based on this awareness. Example: It’s Thursday; a progress report must be written and submitted by Saturday at midnight. Cost to do it Thursday = 3. Cost to do it Friday = 5. Cost to do it Saturday = 9. A struggle between three selves: one for each of Thurs, Fri, Sat. On Saturday: must be done for cost of 9. Your Friday self perceives the cost as 2 · 5 = 10 > 9. Makes the Saturday self do it. Your Thursday self perceives the cost as 2 · 3 = 6. But doesn’t want to leave the decision to the Friday self (since 6 < 9).
SLIDE 39
Sophisticated Planning on a Graph
v1 t s v2
9 3 5
10 9 9 6
A graph-theoretic model of sophisticated planning
[Kleinberg-Oren-Raghavan 2016] There is a “self” for each node. Working backward in a topological ordering of the graph, determine what the self at node v will do, given known behaviors at later nodes.
SLIDE 40
Sophisticated Planning on a Graph
v1 t s v2
9 3 5
10 9 9 6
A graph-theoretic model of sophisticated planning
[Kleinberg-Oren-Raghavan 2016] There is a “self” for each node. Working backward in a topological ordering of the graph, determine what the self at node v will do, given known behaviors at later nodes.
SLIDE 41
Sophisticated Planning on a Graph
v1 t s v2
9 3 5
10 9 9 6
A graph-theoretic model of sophisticated planning
[Kleinberg-Oren-Raghavan 2016] There is a “self” for each node. Working backward in a topological ordering of the graph, determine what the self at node v will do, given known behaviors at later nodes.
SLIDE 42
Worst-Case Performance for Sophisticated Agents
v1 t s v2
9 1 c
10 9 c b
Sophisticated agent can be c times worse than optimal, for any c ≤ b.
SLIDE 43
Worst-Case Performance for Sophisticated Agents
v1 t s v2
9 1 c
10 9 c b
Sophisticated agent can be c times worse than optimal, for any c ≤ b. Theorem [Kleinberg-Oren-Raghavan 2016]: In every instance G, a sophisticated agent incurs at most b times the optimal cost.
Worst case is exponentially better than in the case of naive agents.
SLIDE 44
Further Directions
s a b t c
2 3 6 6 2 reward 12