Introduction to Dynamic Programming For use in Population - - PowerPoint PPT Presentation

Introduction to Dynamic Programming

For use in Population Applications J.R. Walker

Department of Economics University of University of Wisconsin–Madison

Economics of Population, Spring 2012

JRW DP

Life Cycle Decision Making

Human Capital Formation: Amount and type of schooling, timing Household formation. Marriage, Cohabitation, Separation, Divorce Childbearing: Number of children, timing of 1st, spacing

JRW DP

Life Cycle Decision Making

Cont.

Health: payoffs from healthy/risky behaviors Migration: Where to move? When to move? Employment, Savings, Retirement

JRW DP

INVESTMENT DECISIONS

May problems are investment decisions. Thus tools learned here, useful elsewhere

JRW DP

Dynamic problems

Typical problem: Life cycle consumption. For a given sequence of income by age a: y(a) Determine optimal consumption plan from today to (known) end of life.

JRW DP

Symbolic representation

V(c0, c1, . . . , cT) = max

c0,c1,...,cT U(c0, c1, . . . , cT)

(1) subject to: y0, y1, . . . , yT given (2)

JRW DP

Feasible Set

Defining the feasible consumption sequences depends on structure of capital market. For example, if can borrow or lend at same interest (all periods) Then budget constraint is: At+1 = (1 + r)At + yt − ct, t = 0, 1, . . . , T, A0 = a0

JRW DP

DP simplification

DP breaks this one T + 1–dimensional problem into T + 1 one dimensional problems We known how to solve one dimensional problems.

JRW DP

Bellman Principle

Two fundamental principles of life: Golden Rule: Do unto to others . . . Bellman’s Principle: Don’t worry how you got here, do the best you can.

JRW DP

Backward Induction

Example: Optimal Consumption Plan

We will study ”finite horizon (lifetime) problems.” Last Period, T < ∞ Period T: enumerate all feasible situations (states, denoted x). In consumption example, the states defined by asset levels AT (and age). Determine the best course of action in period T for each state, x. (maxcT u(cT), s.t. cT ≤ AT = ⇒ ˆ cT(AT), Remember best action ˆ cT(x) and payoff V(x) = u(ˆ cT(x)).

JRW DP

Backward Induction

Continued

Period T − 1: enumerate all feasible states xT−1. Determine best course of action in period T − 1 for each state using Bellman’s Principle. VT−1 = max

cT−1∈CT−1

{u(cT−1) + βVT(AT)} c ∈ C feasible consumption; β discount factor. Perfect capital markets: AT = (1 + r)AT−1 + yT−1 − cT−1

JRW DP

Recursion

Notice recursion. In period T we solved for the optimal consumption so VT(AT) is known. The budget constraint links assets in periods T − 1, T by the consumption level of period T − 1. Thus, we have enough information to determine the best action (optimal consumption) in period T − 1.

JRW DP

Continue back to current period

Title of slide says it all. Continue these steps back to period 0. For a given initial asset level, A0 = a0, and sequence of incomes

• yj

T

j=0, we have solved the dynamic problem

ˆ c0, ˆ c1, . . . , ˆ cT.

JRW DP

Original Meaning of DP

In 1958 when Bellman labeled the solution procedure Dynamic Programming there were so few people writing code, the term programmer didn’t exist. Usage at that time, programming meant planning — Dynamic Planning. Planning indeed: Solved consumption for entire life (cradle to grave).

JRW DP

Example 1: Capital Budgeting

Our firm has $5 million to invest in mfg plants for expansion. Each plant has a few projects for on how it intends to the money. Each project gives the cost (c) and the tot revenue (r expected Each will enact one project.

JRW DP

Example 2: Shortest Path

Picture of Network.

JRW DP

Commonalities

List of commonalities of examples 1 and 2.

JRW DP