Introduction to Dynamic Programming For use in Population Applications J.R. Walker Department of Economics University of University of Wisconsin–Madison Economics of Population, Spring 2012 JRW DP
Life Cycle Decision Making Human Capital Formation: Amount and type of schooling, timing Household formation. Marriage, Cohabitation, Separation, Divorce Childbearing: Number of children, timing of 1st, spacing JRW DP
Life Cycle Decision Making Cont. Health: payoffs from healthy/risky behaviors Migration: Where to move? When to move? Employment, Savings, Retirement JRW DP
INVESTMENT DECISIONS May problems are investment decisions. Thus tools learned here, useful elsewhere JRW DP
Dynamic problems Typical problem: Life cycle consumption. For a given sequence of income by age a : y ( a ) Determine optimal consumption plan from today to (known) end of life. JRW DP
Symbolic representation V ( c 0 , c 1 , . . . , c T ) = c 0 , c 1 ,..., c T U ( c 0 , c 1 , . . . , c T ) max (1) subject to: y 0 , y 1 , . . . , y T given (2) JRW DP
Feasible Set Defining the feasible consumption sequences depends on structure of capital market. For example, if can borrow or lend at same interest (all periods) Then budget constraint is: A t + 1 = ( 1 + r ) A t + y t − c t , t = 0 , 1 , . . . , T , A 0 = a 0 JRW DP
DP simplification DP breaks this one T + 1–dimensional problem into T + 1 one dimensional problems We known how to solve one dimensional problems. JRW DP
Bellman Principle Two fundamental principles of life: Golden Rule: Do unto to others . . . Bellman’s Principle: Don’t worry how you got here, do the best you can. JRW DP
Backward Induction Example: Optimal Consumption Plan We will study ”finite horizon (lifetime) problems.” Last Period, T < ∞ Period T : enumerate all feasible situations (states, denoted x ). In consumption example, the states defined by asset levels A T (and age). Determine the best course of action in period T for each state, x . (max c T u ( c T ) , s.t. c T ≤ A T = ⇒ ˆ c T ( A T ) , Remember best action ˆ c T ( x ) and payoff V ( x ) = u (ˆ c T ( x )) . JRW DP
Backward Induction Continued Period T − 1: enumerate all feasible states x T − 1 . Determine best course of action in period T − 1 for each state using Bellman’s Principle. V T − 1 = max { u ( c T − 1 ) + β V T ( A T ) } c T − 1 ∈ C T − 1 c ∈ C feasible consumption; β discount factor. Perfect capital markets: A T = ( 1 + r ) A T − 1 + y T − 1 − c T − 1 JRW DP
Recursion Notice recursion. In period T we solved for the optimal consumption so V T ( A T ) is known. The budget constraint links assets in periods T − 1 , T by the consumption level of period T − 1. Thus, we have enough information to determine the best action (optimal consumption) in period T − 1. JRW DP
Continue back to current period Title of slide says it all. Continue these steps back to period 0. For a given initial asset level, A 0 = a 0 , and sequence of � T � incomes j = 0 , we have solved the dynamic problem y j ˆ c 0 , ˆ c 1 , . . . , ˆ c T . JRW DP
Original Meaning of DP In 1958 when Bellman labeled the solution procedure Dynamic Programming there were so few people writing code, the term programmer didn’t exist. Usage at that time, programming meant planning — Dynamic Planning. Planning indeed: Solved consumption for entire life (cradle to grave). JRW DP
Example 1: Capital Budgeting Our firm has $5 million to invest in mfg plants for expansion. Each plant has a few projects for on how it intends to the money. Each project gives the cost ( c ) and the tot revenue ( r expected Each will enact one project. JRW DP
Example 2: Shortest Path Picture of Network. JRW DP
Commonalities List of commonalities of examples 1 and 2. JRW DP
Recommend
More recommend
