s Prt - PowerPoint PPT Presentation

❚❤❡ ❝❛❦❡✲❡❛t✐♥❣ ♣r♦❜❧❡♠ ❙✐♠♣❧❡st ♣♦ss✐❜❧❡ ❧✐❢❡✲❝②❝❧❡ ❝♦♥s✉♠♣t✐♦♥✲s❛✈✐♥❣s ♣r♦❜❧❡♠ ◮ ■♥t❡rt❡♠♣♦r❛❧ ♣r♦❜❧❡♠ ♦❢ ❛ ❝♦♥s✉♠❡r ❧✐✈✐♥❣ ❢♦r T ♣❡r✐♦❞s ❛♥❞ ❡♥❞♦✇❡❞ ✇✐t❤ ✐♥✐t✐❛❧ ✇❡❛❧t❤ a ✶ ✐♥ ♣❡r✐♦❞ t = ✶ ◮ ❍❡r ❣♦❛❧✿ t♦ ❛❧❧♦❝❛t❡ t❤❡ ❝♦♥s✉♠♣t✐♦♥ ♦❢ t❤✐s ✇❡❛❧t❤ ♦✈❡r ❤❡r T ♣❡r✐♦❞s ♦❢ ❧✐❢❡ ✐♥ ♦r❞❡r t♦ ♠❛①✐♠✐s❡ ❤❡r ❧✐❢❡t✐♠❡ ✇❡❧❧❜❡✐♥❣ ◮ ❈♦♥s✉♠♣t✐♦♥ ✐s ❞✐✈✐s✐❜❧❡✿ ❛ ❝♦♥t✐♥✉♦✉s ❞❡❝✐s✐♦♥ ✈❛r✐❛❜❧❡ ◮ ❆♥② r❡♠❛✐♥✐♥❣ ✇❡❛❧t❤ ✐♥ ♣❡r✐♦❞ t ✐s ♣r♦❞✉❝t✐✈❡✱ ❣❡♥❡r❛t✐♥❣ k ( a ) ✉♥✐ts ♦❢ ✇❡❛❧t❤ t♦ ❝♦♥s✉♠❡ ✐♥ t❤❡ ❢✉t✉r❡ ◮ ◆♦ ♦✉tst❛♥❞✐♥❣ ❞❡❜ts ❛r❡ ❛❧❧♦✇❡❞ ❛t t❤❡ ❡♥❞ ♦❢ ❧✐❢❡ ◮ ❆♥❞ ❛♥② r❡♠❛✐♥✐♥❣ ✇❡❛❧t❤ ❛t t❤❡ ❡♥❞ ♦❢ ❧✐❢❡ ✐s ♦❢ ♥♦ ✈❛❧✉❡

❋♦r♠❛❧ ♠♦❞❡❧ � T β t − ✶ u ( c t ) max ( c ✶ ,..., c T ) ∈ C T t = ✶ s✳t a t + ✶ = k ( a t − c t ) ❢♦r t = ✶ , . . . , T a T + ✶ ≥ ✵ a ✶ ( ∈ A ) ❣✐✈❡♥ ◮ P❡r✲♣❡r✐♦❞ ✇❡❧❧❜❡✐♥❣ u ✿ ✐♥❝r❡❛s✐♥❣ ✐♥ ❝♦♥s✉♠♣t✐♦♥ ◮ ❈♦♥s✉♠♣t✐♦♥✿ ❝❤♦✐❝❡ ✈❛r✐❛❜❧❡✱ ✇✐t❤ ❞♦♠❛✐♥ C ✭❤❡r❡ R + ✵ ♦r R + ✱ ❞❡♣❡♥❞✐♥❣ ♦♥ u ✮ ◮ ❆ss❡ts ✐s t❤❡ st❛t❡ ✈❛r✐❛❜❧❡✱ ✇✐t❤ ❞♦♠❛✐♥ A ✭❤❡r❡ R + ✵ ♦r R + ✮ ◮ k ✿ ❧❛✇ ♦❢ ♠♦t✐♦♥ ❢♦r ❛ss❡ts✱ ❛ ♣♦s✐t✐✈❡ ❛♥❞ ✐♥❝r❡❛s✐♥❣ ❢✉♥❝t✐♦♥ ✐♥ A k ( a t − c t ) = R ( a t − c t ) ✇❤❡r❡ R = ✶ + r ✐s t❤❡ ✐♥t❡r❡st ❢❛❝t♦r

❈❧❛ss✐❝❛❧ s♦❧✉t✐♦♥ ◮ ❖❜❥❡❝t✐✈❡ ❢✉♥❝t✐♦♥ ✐s C ✶ ✭❝♦♥t✐♥✉♦✉s❧② ❞✐✛❡r❡♥t✐❛❜❧❡✮✿ ✐♥t❡r✐♦r ♦♣t✐♠✉♠ s❛t✐s✜❡s ❢♦❝ ◮ ❈❧❛ss✐❝❛❧ s♦❧✉t✐♦♥✿ ❛tt❛❝❦ ♣r♦❜❧❡♠ ❞✐r❡❝t❧② ❜② s♦❧✈✐♥❣ ❛❧❧ ✐ts ❢♦❝✬s ◮ ❯s❡❢✉❧ t♦ ✇r✐t❡ ♠♦❞❡❧ r❡str✐❝t✐♦♥s ♠♦r❡ ❝♦♠♣❛❝t❧② ❜② ♥♦t✐♥❣ t❤❛t t❤❡ ❧❛✇ ♦❢ ♠♦t✐♦♥ ❢♦r ❛ss❡ts t♦❣❡t❤❡r ✇✐t❤ t❤❡ ✐♥✐t✐❛❧ ❝♦♥❞✐t✐♦♥ ✐♠♣❧② T � R T − t + ✶ c t a T + ✶ = R T a ✶ − t = ✶ ◮ ❚❤❡r❡❢♦r❡✱ t❤❡ ❝♦♥s✉♠❡r✬s ♣r♦❜❧❡♠ ❢♦r ❛ ❣✐✈❡♥ a ✶ ≥ ✵ ✐s T T � � β t − ✶ u ( c t ) R ✶ − t c t ≤ a ✶ max s✳t ( c ✶ ,..., c T ) ∈ ( C ) T t = ✶ t = ✶

❈❧❛ss✐❝❛❧ s♦❧✉t✐♦♥✿ ❊✉❧❡r ❡q✉❛t✐♦♥ ■ ◮ ▲❛❣r❛♥❣✐❛♥ ❢♦r t❤✐s ♣r♦❜❧❡♠ � T � T � � β t − ✶ u ( c t ) − λ R ✶ − t c t − a ✶ L = t = ✶ t = ✶ ◮ ❲✐t❤ ♥❡❝❡ss❛r② ❢♦❝✬s ✇✐t❤ r❡s♣❡❝t t♦ c t ✱ ❢♦r t = ✶ , . . . , T ✿ β t − ✶ u ′ ( c t ) = λ R ✶ − t ◮ P✉tt✐♥❣ t♦❣❡t❤❡r t✇♦ s✉❜s❡q✉❡♥t ❝♦♥❞✐t✐♦♥s ②✐❡❧❞s u ′ ( c t ) = β Ru ′ ( c t + ✶ ) ❢♦r t = ✶ , . . . , T − ✶ ✭✶✮ ◮ ❚❤❡s❡ ❛r❡ t❤❡ ❊✉❧❡r ❡q✉❛t✐♦♥s ❢♦r t❤✐s ♣r♦❜❧❡♠

❈❧❛ss✐❝❛❧ s♦❧✉t✐♦♥✿ ❊✉❧❡r ❡q✉❛t✐♦♥ ■■ u ′ ( c t ) = β Ru ′ ( c t + ✶ ) ❢♦r t = ✶ , . . . , T − ✶ ◮ ❊✉❧❡r ❡q✉❛t✐♦♥✿ ❡st❛❜❧✐s❤❡s r❡❧❛t✐♦♥s❤✐♣ ❜❡t✇❡❡♥ ❝♦♥s✉♠♣t✐♦♥ ✐♥ s✉❜s❡q✉❡♥t ♣❡r✐♦❞s ◮ ❇✉t ♥♦t t❤❡ ❝♦♥s✉♠♣t✐♦♥ ❧❡✈❡❧ ◮ ❋♦r t❤❛t ✇❡ ♥❡❡❞ t❤❡ ❜✉❞❣❡t ❝♦♥str❛✐♥t ◮ ❚❤❡ ❑✉❤♥✲❚✉❝❦❡r ❝♦♥❞✐t✐♦♥s ❞♦ ❥✉st t❤❛t

❈❧❛ss✐❝❛❧ s♦❧✉t✐♦♥✿ ❑✉❤♥✲❚✉❝❦❡r ❝♦♥❞✐t✐♦♥s ◮ ❚❤❡ ❑✉❤♥✲❚✉❝❦❡r ❝♦♥❞✐t✐♦♥s ❢♦r t❤✐s ♣r♦❜❧❡♠✿ � T � � � T R ✶ − t c t − a ✶ R ✶ − t c t ≤ a ✶ λ ≥ ✵ , λ = ✵ , t = ✶ t = ✶ ◮ ■❢ u str✐❝t❧② ✐♥❝r❡❛s✐♥❣ ✭ u ′ > ✵✮✿ ◮ λ > ✵✿ ✰✈❡ ♠❛r❣✐♥❛❧ ✈❛❧✉❡ ♦❢ r❡❧❛①✐♥❣ t❤❡ ❜✉❞❣❡t ❝♦♥str❛✐♥t ◮ � t = ✶ ,..., T R ✶ − t c t = a ✶ ✿ ❝♦♥s✉♠❡r ❜❡tt❡r ♦✛ ❜② ❝♦♥s✉♠✐♥❣ ❛❧❧ a ✶ ◮ ❚❤❡♥ a T + ✶ = ✵ ✭✷✮ ◮ ❚♦❣❡t❤❡r✱ t❤❡ T ❝♦♥❞✐t✐♦♥s ✭ ❄❄ ✮ ❛♥❞ ✭ ❄❄ ✮ ❞❡t❡r♠✐♥❡ t❤❡ T ✐♥t❡r✐♦r ♦♣t✐♠❛❧ ❝♦♥s✉♠♣t✐♦♥ ❝❤♦✐❝❡s

❈♦r♥❡r s♦❧✉t✐♦♥s ◮ ❯♣ t♦ ❤❡r❡ ✇❡ ❛ss✉♠❡❞ t❤❛t t❤❡ s♦❧✉t✐♦♥ ✐s ✐♥t❡r✐♦r ◮ ❚❤❡ ❊✉❧❡r ❝♦♥❞✐t✐♦♥s ❛❧❧♦✇✐♥❣ ❢♦r ❝♦r♥❡r s♦❧✉t✐♦♥s ❛r❡ u ′ ( c t ) ≤ β Ru ′ ( c t + ✶ ) ❢♦r t❤❡ ♣♦ss✐❜✐❧✐t② ♦❢ c t = ✵ u ′ ( c t ) ≥ β Ru ′ ( c t + ✶ ) ♦r ❢♦r t❤❡ ♣♦ss✐❜✐❧✐t② ♦❢ c t = a t ◮ ❚②♣✐❝❛❧ ❝❤♦✐❝❡s ♦❢ ✉t✐❧✐t② ❢✉♥❝t✐♦♥s ❛r❡ ❝♦♥t✐♥✉♦✉s❧② ❞✐✛❡r❡♥t✐❛❜❧❡ ❛♥❞ ♠♦♥♦t♦♥✐❝❛❧❧② ✐♥❝r❡❛s✐♥❣ ✐♥ R + ✱ ✇✐t❤ t❤❡ ❛❞❞✐t✐♦♥❛❧ ❢♦❧❧♦✇✐♥❣ ♣r♦♣❡rt②✿ c t → ✵ + u ′ ( c t ) = + ∞ c t → ✵ + u ( c t ) = −∞ lim ❛♥❞ lim ■♥ t❤✐s ❝❛s❡ ❛ s♦❧✉t✐♦♥✱ ✐❢ ✐t ❡①✐sts✱ ✐s ✐♥t❡r✐♦r

❚❤❡ ❝❛❦❡✲❡❛t✐♥❣ ♣r♦❜❧❡♠ ❙✐♠♣❧❡ ❡①❛♠♣❧❡✿ ❈❘❘❆ ✉t✐❧✐t②

❈❘❘❆ ✉t✐❧✐t② ◮ ❆ ❝♦♥✈❡♥✐❡♥t ❛♥❞ ♣♦♣✉❧❛r s♣❡❝✐✜❝❛t✐♦♥ ♦❢ t❤❡ ✉t✐❧✐t② ❢✉♥❝t✐♦♥ ✭ γ > ✵✮ u ( c ) = c ✶ − γ ✶ − γ γ − ✶ ✐s t❤❡ ❡❧❛st✐❝✐t② ♦❢ ✐♥t❡rt❡♠♣♦r❛❧ s✉❜st✐t✉t✐♦♥ ◮ ■t ✐s ❣❡♥❡r❛❧❧② ❛❝❝❡♣t❡❞ t❤❛t γ ≥ ✶✱ ✐♥ ✇❤✐❝❤ ❝❛s❡✱ ❢♦r c ∈ R + u ( c ) < ✵ , lim c → ✵ u ( c ) = −∞ , lim c → + ∞ u ( c ) = ✵ u ′ ( c ) > ✵ , lim c → ✵ u ′ ( c ) = + ∞ , lim c → + ∞ u ′ ( c ) = ✵

❈❘❘❆ ✉t✐❧✐t②✿ s♦❧✉t✐♦♥ ■ ◮ ❚❤❡ ♣r♦❜❧❡♠ ✐s T T � β t − ✶ c ✶ − γ � t R ✶ − t c t ≤ a ✶ max s✳t ✶ − γ ( c ✶ ,..., c T ) ∈ ( R + ) T t = ✶ t = ✶ ◮ ❊✉❧❡r ❡q✉❛t✐♦♥s✿ c t = ( β R ) − ✶ c − γ = β Rc − γ γ c t + ✶ ⇒ ❢♦r t = ✶ , . . . , T − ✶ t t + ✶ ◮ ❇② s✉❝❝❡ss✐✈❡ s✉❜st✐t✉t✐♦♥✿ t − ✶ γ c ✶ c t = ( β R )

❈❘❘❆ ✉t✐❧✐t②✿ s♦❧✉t✐♦♥ ■■ ◮ ❚❤❡ ❜✉❞❣❡t ❝♦♥str❛✐♥t ❛♥❞ ♦♣t✐♠❛❧✐t② ❝♦♥❞✐t✐♦♥ ✐♠♣❧② � R ✶ − t c t a ✶ = t = ✶ ,..., T � � t − ✶ � ✶ − γ ✶ γ R = c ✶ β γ t = ✶ ,..., T � ✶ ✶ − γ α t − ✶ γ R = c ✶ ✇❤❡r❡ α = β γ t = ✶ ,..., T ◮ ❚❤❡ s♦❧✉t✐♦♥ ❢♦r t = ✶ , . . . , T ✿ ✶ − α ✶ − α t − ✶ γ a ✶ c ✶ = ❛♥❞ c t = ✶ − α T ( β R ) ✶ − α T a ✶

❈❘❘❆ ✉t✐❧✐t②✿ s♦❧✉t✐♦♥ ■■■ ■♥ ❣❡♥❡r❛❧✱ ✐❢ t❤❡ ♦♣t✐♠✐s❛t✐♦♥ ♣r♦❜❧❡♠ st❛rts ❛t t✐♠❡ t ❛s ❢♦❧❧♦✇s T T � � β τ − t c ✶ − γ τ R τ − t c τ ≤ a τ max s✳t ✶ − γ ( c t ,..., c T ) ∈ ( R + ) T − t + ✶ τ = t τ = t t❤❡ s♦❧✉t✐♦♥ ❢♦r c t ✐s ✶ − α c t = ✶ − α T − t + ✶ a t ❚❤✐s ✐s t❤❡ ❝♦♥s✉♠♣t✐♦♥ ❢✉♥❝t✐♦♥ ✱ ❛ ❧✐♥❡❛r ❢✉♥❝t✐♦♥ ♦❢ ❛ss❡ts ✐❢ ✉t✐❧✐t② ✐s ❈❘❘❆

❈❘❘❆ ✉t✐❧✐t②✿ ❝♦♥s✉♠♣t✐♦♥ ♦✈❡r t❤❡ ❧✐❢❡✲❝②❝❧❡ t − ✶ ✶ − α γ a ✶ β R ❞❡t❡r♠✐♥❡s t❤❡ ♣r♦✜❧❡ ♦❢ t❤❡ s♦❧✉t✐♦♥✿ c t = ✶ − α T ( β R ) .06 .05 consumption .04 .03 .02 20 30 40 50 60 age r=4% r=2.5% r=1% β = ✶ . ✵✷✺ − ✶ ❛♥❞ ✐♥✐t✐❛❧ ❛ss❡ts ❛r❡ a ✷✵ = ✶✳

❚❤❡ ❝❛❦❡✲❡❛t✐♥❣ ♣r♦❜❧❡♠ ❊①✐st❡♥❝❡ ❛♥❞ ✉♥✐q✉❡♥❡ss ♦❢ s♦❧✉t✐♦♥

❲❤❡♥ ❝❛♥ t❤❡ ❡①✐st❡♥❝❡ ♦❢ t❤❡ ♦♣t✐♠✉♠ ❜❡ ❣✉❛r❛♥t❡❡❞❄ ◮ ❋❡❛s✐❜✐❧✐t② s❡t✿ s♣❛❝❡ ♦❢ ❝❤♦✐❝❡s s❛t✐s❢②✐♥❣ t❤❡ ♣r♦❜❧❡♠ ❝♦♥str❛✐♥ts     �  ( c ✶ , . . . , c T ) ∈ C T : R ✶ − t c t ≤ a ✶ C ✶ : T ( a ✶ ) =  t = ✶ ,..., T ✇❤❡r❡ t②♣✐❝❛❧❧② C = R + ◮ ❆♣♣❧② ❲❡✐❡rstr❛ss t❤❡♦r❡♠ t♦ ❡♥s✉r❡ ❡①✐st❡♥❝❡ ♦❢ s♦❧✉t✐♦♥✿ ▲❡t u : C → R ❜❡ ❝♦♥t✐♥✉♦✉s ❛♥❞ s✉♣♣♦s❡ C ✶ : T ( a ✶ ) ⊂ C T ✐s ♥♦♥✲❡♠♣t② ❛♥❞ ❝♦♠♣❛❝t✳ ❚❤❡♥ t❤❡ ❝♦♥s✉♠❡r✬s ♣r♦❜❧❡♠ � β t − ✶ u ( c t ) max ( c ✶ ,..., c T ) ∈C ✶ : T ( a ✶ ) t = ✶ ,..., T ❤❛s ❛ s♦❧✉t✐♦♥ ✐♥ C ✶ : T ( a ✶ ) ❢♦r ❛♥② a ✶ ∈ A ✳

❲❤❡♥ ✐s t❤❡ ♦♣t✐♠✉♠ ✐♥t❡r✐♦r ❛♥❞ ✉♥✐q✉❡❄ ◮ ❚②♣✐❝❛❧ ❝♦♥s✉♠❡r✬s ♣r♦❜❧❡♠✿ u ✐s str✐❝t❧② ✐♥❝r❡❛s✐♥❣✱ ❝♦♥❝❛✈❡ ❛♥❞ C ✶ ◮ ❚❤❡♥ t❤❡ s✉♠ ♦❢ ♣❡r✲♣❡r✐♦❞ ✉t✐❧✐t✐❡s ✐s ❛❧s♦ str✐❝t❧② ✐♥❝r❡❛s✐♥❣✱ ❝♦♥❝❛✈❡ ❛♥❞ C ✶ ◮ ❆❧s♦ ❛ss✉♠❡ t❤❛t t❤❡ ❢❡❛s✐❜✐❧✐t② s❡t C ✶ : T ( a ✶ ) ✐s ♥♦♥✲❡♠♣t② ❛♥❞ ❝♦♠♣❛❝t ◮ ❯♥❞❡r t❤❡s❡ ❝♦♥❞✐t✐♦♥s t❤❡ s♦❧✉t✐♦♥ ✐s ✉♥✐q✉❡ ◮ ■t ✐s ❛❧s♦ ✐♥t❡r✐♦r ✭ T > ✶✮ ◮ ❇✉t ✐❢ ✇❡ ❤❛❞ ❛ ❝♦♥✈❡① u ✿ ❝♦r♥❡r s♦❧✉t✐♦♥

❉②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ ❚❤❡ ❇❡❧❧♠❛♥ ❡q✉❛t✐♦♥

❉②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ ◮ ❉②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ s♣❧✐ts t❤❡ ❜✐❣ ♣r♦❜❧❡♠ ✐♥t♦ s♠❛❧❧❡r ♣r♦❜❧❡♠s t❤❛t ❛r❡ ♦❢ s✐♠✐❧❛r str✉❝t✉r❡ ❛♥❞ ❡❛s✐❡r t♦ s♦❧✈❡ ◮ ❚❤❡ tr✐❝❦ ✐s t♦ ✜♥❞ t❤❡ ❧✐♠✐t❡❞ s❡t ♦❢ ✈❛r✐❛❜❧❡s t❤❛t ❝♦♠♣❧❡t❡❧② ❞❡s❝r✐❜❡ t❤❡ ❞❡❝✐s✐♦♥ ♣r♦❜❧❡♠ ✐♥ ❡❛❝❤ ♣❡r✐♦❞ ✕ t❤❡ st❛t❡ ◮ ❚❤❡♥ t❤❡ s♦❧✉t✐♦♥ ♦❢ t❤❡s❡ ♣r♦❜❧❡♠s ♦✈❡r ❛ s♠❛❧❧ st❛t❡✲s♣❛❝❡ ❞❡t❡r♠✐♥❡s ❛ s❡t ♦❢ ♣♦❧✐❝② ❢✉♥❝t✐♦♥s✿ ♦♣t✐♠❛❧ ❝♦♥s✉♠♣t✐♦♥ ✐s h t ( a t ) ❢♦r t = ✶ , . . . T ◮ ❉P r❡t✉r♥s ❛ ❣❡♥❡r❛❧ s♦❧✉t✐♦♥✿ ✐t s♦❧✈❡s t❤❡ ❡♥t✐r❡ ❢❛♠✐❧② ♦❢ ♣r♦❜❧❡♠s ♦❢ t❤❡ s❛♠❡ t②♣❡ ◮ ❚❤❡ s♣❡❝✐✜❝ s♦❧✉t✐♦♥ t♦ ♦✉r ♣r♦❜❧❡♠ ❝❛♥ ❜❡ ❝♦♥str✉❝t❡❞ r❡❝✉rs✐✈❡❧②✱ ❜② ✐t❡r❛t✐♥❣ = h t ( a t ) c t = R ( a t − c t ) a t + ✶ st❛rt✐♥❣ ❢r♦♠ t❤❡ ❣✐✈❡♥ a ✶

Pr♦❜❧❡♠ s♣❡❝✐✜❝❛t✐♦♥ ■ ◮ ■♥ ♦✉r ♣r♦❜❧❡♠✱ t❤❡ ❧❡✈❡❧ ♦❢ ❛ss❡ts ❛t t❤❡ st❛rt ♦❢ ♣❡r✐♦❞ t s✉♠♠❛r✐s❡s ❛❧❧ t❤❡ ✐♥❢♦r♠❛t✐♦♥ ♥❡❡❞❡❞ t♦ s♦❧✈❡ ❢♦r ❝♦♥s✉♠♣t✐♦♥ ◮ ❚❤❡ ❢❡❛s✐❜✐❧✐t② s❡t ❛t t✐♠❡ t ❢♦r t❤❡ s❡q✉❡♥❝❡ ♦❢ ♣r❡s❡♥t ❛♥❞ ❢✉t✉r❡ ❝♦♥s✉♠♣t✐♦♥ ❝❤♦✐❝❡s ❣✐✈❡♥ a t ∈ A ✐s     �  ( c t , . . . , c T ) ∈ C T − t + ✶ : R t − τ c τ ≤ a t C t : T ( a t ) =  τ = t ,..., T ◮ ■❢ ❝♦♥s✉♠♣t✐♦♥ ♠✉st ❜❡ ♣♦s✐t✐✈❡ ✐♥ ❡✈❡r② ♣❡r✐♦❞✱ t❤❡♥ C = A = R + ❛♥❞ t❤❡ ❢❡❛s✐❜✐❧✐t② s❡t ❛t t✐♠❡ t ✐s  { c t > ✵ : a t + ✶ = R ( a t − c t ) > ✵ } ✐❢ t < T  C t ( a t ) =  { c t > ✵ : a t + ✶ = R ( a t − c t ) ≥ ✵ } ✐❢ t = T

Pr♦❜❧❡♠ s♣❡❝✐✜❝❛t✐♦♥ ■■ ◮ ❚❤❡ ♣r♦❜❧❡♠ ♦❢ ❛ ❝♦♥s✉♠❡r ✇✐t❤ ❛ss❡ts a t ❛t t✐♠❡ t ✐s � β τ − t u ( c τ ) V t ( a t ) = max ( c t ,..., c T ) ∈C t : T ( a t ) τ = t ,..., T ◮ V t ✐s t❤❡ ✈❛❧✉❡ ❢✉♥❝t✐♦♥ ◮ ■♥❞✐r❡❝t ❧✐❢❡t✐♠❡ ✉t✐❧✐t②✿ ♠❡❛s✉r❡s ♠❛① ✉t✐❧✐t② t❤❛t ❛ss❡ts a t ❝❛♥ ❞❡❧✐✈❡r ◮ ■t ✐s ❛ ❢✉♥❝t✐♦♥ ♦❢ a t ❛❧♦♥❡ ◮ ❉❡♣❡♥❞❡♥❝❡ ♦♥ a t ❛r✐s❡s t❤r♦✉❣❤ t❤❡ ❢❡❛s✐❜✐❧✐t② s❡t

Pr♦❜❧❡♠ s♣❡❝✐✜❝❛t✐♦♥ ■■■ ❚❤❡ ✈❛❧✉❡ ❢✉♥❝t✐♦♥ ❝❛♥ ❜❡ ❞❡✜♥❡❞ r❡❝✉rs✐✈❡❧② � β τ − t u ( c τ ) V t ( a t ) = max ( c t ,..., c T ) ∈C t : T ( a t ) τ = t ,..., T                 T �   β τ − ( t + ✶ ) u ( c τ ) = max u ( c t ) + β max     c t ∈C t ( a t ) ( c t + ✶ ,..., c T ) ∈C t + ✶ : T ( a t + ✶ )     τ = t + ✶   � �� �     V t + ✶ ( a t + ✶ ) = c t ∈C t ( a t ) { u ( c t ) + β V t + ✶ ( R [ a t − c t ]) } max

❚❤❡ ❇❡❧❧♠❛♥ ❡q✉❛t✐♦♥ ■ c t ∈C t ( a t ) { u ( c t ) + β V t + ✶ ( R [ a t − c t ]) } V t ( a t ) = max ◮ ❚❤✐s ✐s ❛ ❢✉♥❝t✐♦♥❛❧ ❡q✉❛t✐♦♥✿ r❡❝✉rs✐✈❡ ❢♦r♠✉❧❛t✐♦♥ ◮ ❇r❡❛❦s t❤❡ ❧❛r❣❡ ❧✐❢❡❝②❝❧❡ ♣r♦❜❧❡♠ ✐♥ s♠❛❧❧❡r st❛t✐❝ ♣r♦❜❧❡♠s ◮ ❑❡②✿ ♠❡♠♦r②❧❡ss ♣r♦❝❡ss ❞❡♣❡♥❞s ♦♥❧② ♦♥ t❤❡ ✈❛❧✉❡ ♦❢ st❛t❡ ✈❛r✐❛❜❧❡s ❛t t❤❡ t✐♠❡ ♦❢ ❞❡❝✐s✐♦♥ ◮ Pr✐♥❝✐♣❧❡ ♦❢ ❖♣t✐♠❛❧✐t②✿ ✐❢ t❤❡ ❝♦♥s✉♠❡r ❜❡❤❛✈❡s ♦♣t✐♠❛❧❧② ✐♥ t❤❡ ❢✉t✉r❡✱ ❛❧❧ t❤❛t ♠❛tt❡rs ❢♦r t❤❡ s♦❧✉t✐♦♥ ❛t t✐♠❡ t ✐s t❤❡ ❞❡❝✐s✐♦♥ ♦❢ ❤♦✇ ♠✉❝❤ t♦ ❝♦♥s✉♠❡ t♦❞❛② ◮ V t + ✶ ❡①✐sts ✭❜② r❡❝✉rs✐♦♥✮ ❜✉t ✐s ✉♥❦♥♦✇♥✦

❚❤❡ ❇❡❧❧♠❛♥ ❡q✉❛t✐♦♥ ■■ ❖❢t❡♥ ✉s❡❢✉❧ t♦ r❡❢♦r♠✉❧❛t❡ t❤❡ ♣r♦❜❧❡♠ ✐♥ t❡r♠s ♦❢ s❛✈✐♥❣s ❞❡❝✐s✐♦♥s ◮ ❉❡✜♥❡ t❤❡ ♣❛②♦✛ ❢✉♥❝t✐♦♥ ❛s � � a t − a t + ✶ f ( a t , a t + ✶ ) = u = u ( c t ) R ◮ ❚❤❡♥ t❤❡ ❝♦♥s✉♠♣t✐♦♥✴s❛✈✐♥❣s ♣r♦❜❧❡♠ ✐s ❡q✉✐✈❛❧❡♥t❧② s♣❡❝✐✜❡❞ ❛s V t ( a t ) = a t + ✶ ∈D t ( a t ) { f ( a t , a t + ✶ ) + β V t + ✶ ( a t + ✶ ) } max ✇❤❡r❡ t❤❡ ❢❡❛s✐❜✐❧✐t② s❡t ❛t t✐♠❡ t ✭❢♦r C = A = R + ✮  � � a t + ✶ > ✵ : a t − a t + ✶ R − ✶ > ✵ ,  ✐❢ t < T D t ( a t ) = � �  a t + ✶ ≥ ✵ : a t − a t + ✶ R − ✶ > ✵ ✐❢ t = T

❚❤❡ ❇❡❧❧♠❛♥ ❡q✉❛t✐♦♥ ■■■ ◮ ❚❤❡ s♦❧✉t✐♦♥ ✐s g t ( a t ) = arg max { f ( a t , a t + ✶ ) + β V t + ✶ ( a t + ✶ ) } a t + ✶ ∈D t ( a t ) ◮ ❊①✐sts ❛♥❞ ✐s ✉♥✐q✉❡ ✉♥❞❡r t❤❡ ❝♦♥❞✐t✐♦♥s ❞✐s❝✉ss❡❞ ❡❛r❧✐❡r✿ ◮ f r❡❛❧✲✈❛❧✉❡❞✱ str✐❝t❧② ✐♥❝r❡❛s✐♥❣ ✭❞❡❝r❡❛s✐♥❣✮ ✐♥ t❤❡ ✜rst ✭s❡❝♦♥❞✮ ❛r❣✉♠❡♥t✱ ❝♦♥❝❛✈❡ ❛♥❞ C ✶ ✐♥ ❜♦t❤ ❛r❣✉♠❡♥ts ◮ D ✐s ♥♦♥✲❡♠♣t② ❛♥❞ ❝♦♠♣❛❝t ◮ ❯♥❞❡r t❤❡s❡ ❝♦♥❞✐t✐♦♥s g ✐s ❛❧s♦ ❝♦♥t✐♥✉♦✉s ◮ ▼♦r❡♦✈❡r✱ V ✐♥❤❡r✐ts s♦♠❡ ♦❢ t❤❡ ♣r♦♣❡rt✐❡s ♦❢ f ◮ ❝♦♥t✐♥✉✐t②✱ ♠♦♥♦t♦♥✐❝✐t② ❛♥❞ ❝♦♥❝❛✈✐t② ◮ ❞✐✛❡r❡♥t✐❛❜✐❧✐t② ❛t ♣♦✐♥ts a ∈ A ✐♥ ✇❤✐❝❤ t❤❡ s♦❧✉t✐♦♥ ✐s ✐♥t❡r✐♦r

❉②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ ❘❡❝✉rs✐✈❡ s♦❧✉t✐♦♥

❘❡❝✉rs✐✈❡ s♦❧✉t✐♦♥ a t + ✶ ∈D t ( a t ) { f ( a t , a t + ✶ ) + β V t + ✶ ( a t + ✶ ) } V t ( a t ) = max ❑❡② ✐♥s✐❣❤t ♦❢ ❞②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣✿ t❤❡ ✉♥❦♥♦✇♥ V ❝❛♥ ❜❡ ♣✐♥♥❡❞ ❞♦✇♥ ❜② ❜❛❝❦✇❛r❞ ✐♥❞✉❝t✐♦♥ ◮ ❚❤✐s ❤✐❣❤❧✐❣❤ts t❤❡ ✉s❡❢✉❧♥❡ss ♦❢ t❤❡ ❇❡❧❧♠❛♥ ❡q✉❛t✐♦♥ ◮ ❆♥❞ ✐♥s♣✐r❡s t❤❡ ♥✉♠❡r✐❝❛❧ str❛t❡❣② t♦ s♦❧✈❡ ♠♦❞❡❧s ✇✐t❤ ♥♦ ❝❧♦s❡❞✲❢♦r♠ s♦❧✉t✐♦♥

▲❛st ♣❡r✐♦❞ ❙♦❧✉t✐♦♥ str❛t❡❣②✿ st❛rt ❢r♦♠ ♣❡r✐♦❞ T ❛♥❞ ♠♦✈❡ ❜❛❝❦✇❛r❞s ❛s t❤❡ ❢✉t✉r❡ ✈❛❧✉❡ ❢✉♥❝t✐♦♥✱ t❤❡ ❝♦♥t✐♥✉❛t✐♦♥ ✈❛❧✉❡✱ ✐s ❞❡t❡r♠✐♥❡❞ ◮ ❚❤❡ ♣r♦❜❧❡♠ ✐♥ t❤❡ ❧❛st ♣❡r✐♦❞ ✐s V T ( a T ) = a T + ✶ ∈D T ( a T ) { f ( a T , a T + ✶ ) } max ✇❤❡r❡ D T ( a T ) = [ ✵ , Ra T ] ◮ ❚❤❡ s♦❧✉t✐♦♥ ✐s ✭❢♦r ❛♥② a T ∈ A ✮ g T ( a T ) = ✵ ✇✐t❤ ✈❛❧✉❡ V T ( a T ) = f ( a T , ✵ ) = u ( a T )

▲❛st ❜✉t ♦♥❡ ♣❡r✐♦❞ ◮ ❙✐♥❝❡ V T ( a T ) = u ( a T ) ✱ t❤❡ ♣r♦❜❧❡♠ ❛t T − ✶ ✐s ❦♥♦✇♥ V T − ✶ ( a T − ✶ ) = a T ∈D T − ✶ ( a T − ✶ ) { f ( a T − ✶ , a T ) + β V T ( a T ) } max ◮ ❯♥❞❡r ❞✐✛❡r❡♥t✐❛❜✐❧✐t② ♦❢ t❤❡ ♠❛①✐♠✐s✐♥❣ ❢✉♥❝t✐♦♥✱ ❛♥ ✐♥t❡r✐♦r ♦♣t✐♠✉♠ s❛t✐s✜❡s t❤❡ ❢♦❝✬s ✭❢♦r ❛♥② a T − ✶ ∈ A f ✷ ( a T − ✶ , a T ) + β V ′ g T − ✶ ( a T − ✶ ) ✐s t❤❡ s♦❧✉t✐♦♥ t♦ T ( a T ) = ✵ ◮ ❙♦ t❤❡ ✈❛❧✉❡ ❢✉♥❝t✐♦♥ ❛t T − ✶ ✐s ✭❢♦r ❡❛❝❤ a T − ✶ ∈ A ✮ V T − ✶ ( a T − ✶ ) = f ( a T − ✶ , g T − ✶ ( a T − ✶ )) + β V T ( g T − ✶ ( a T − ✶ ))

P❡r✐♦❞ t ▼♦✈❡ ❜❛❝❦✇❛r❞s ✐♥ s✐♠✐❧❛r st❡♣s ◮ ❖♥❝❡ t❤❡ ✈❛❧✉❡ ❢✉♥❝t✐♦♥ ❢♦r ♣❡r✐♦❞ t + ✶ ❤❛s ❜❡❡♥ ❞❡t❡r♠✐♥❡❞✱ s♦❧✈❡ ✭❢♦r ❡❛❝❤ a t ∈ A ✮ g t ( a t ) = arg max { f ( a t , a t + ✶ ) + β V t + ✶ ( a t + ✶ ) } a t + ✶ ∈D t ( a t ) ◮ ❚❤❡ s♦❧✉t✐♦♥ ❝❛♥ t❤❡♥ ❜❡ ✉s❡❞ t♦ ❜✉✐❧❞ V t ✭❢♦r ❡❛❝❤ a t ∈ A ✮✿ V t ( a t ) = f ( a t , g t ( a t )) + β V t + ✶ ( g t ( a t ))

❙♦❧✉t✐♦♥ t♦ ♦✉r s♣❡❝✐✜❝ ♣r♦❜❧❡♠ ◮ ❚❤❡ s♣❡❝✐✜❝ ♣r♦❜❧❡♠ ✇❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ✐s ❢✉❧❧② ❝❤❛r❛❝t❡r✐s❡❞ ❜② t❤❡ ✐♥✐t✐❛❧ ❝♦♥❞✐t✐♦♥✱ a ✶ ◮ ❚♦ ❝♦♥str✉❝t t❤❡ s♦❧✉t✐♦♥✱ ✇❡ ✉s❡ t❤❡ ♣♦❧✐❝② ❢✉♥❝t✐♦♥s g t ❛♥❞ ✐t❡r❛t❡✱ ❢♦r t = ✶ , . . . , T c t = a t − R − ✶ g t ( a t ) ❛♥❞ a t + ✶ = g t ( a t )

❉②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ ❖♣t✐♠❛❧✐t② ❝♦♥❞✐t✐♦♥s

❖♣t✐♠❛❧✐t② ❝♦♥❞✐t✐♦♥s ■ ◮ ❚❤❡ t②♣✐❝❛❧ ♣r♦❜❧❡♠ ✐♥ ❡❝♦♥♦♠✐❝s ❛ss✉♠❡s t❤❛t t❤❡ ✉t✐❧✐t② ❢✉♥❝t✐♦♥ ✐s str✐❝t❧② ✐♥❝r❡❛s✐♥❣✱ ❝♦♥❝❛✈❡ ❛♥❞ ❝♦♥t✐♥✉♦✉s❧② ❞✐✛❡r❡♥t✐❛❜❧❡ ✭✐♥ ❝♦♥s✉♠♣t✐♦♥✮✱ ❛♥❞ t❤❛t t❤❡ ❢❡❛s✐❜✐❧✐t② s♣❛❝❡ ✐s ❝❧♦s❡❞ ❛♥❞ ❜♦✉♥❞❡❞ ◮ ❯♥❞❡r t❤❡s❡ ❝♦♥❞✐t✐♦♥s t❤❡ s♦❧✉t✐♦♥ ✐s ✉♥✐q✉❡ ❛♥❞ V ✐s ❞✐✛❡r❡♥t✐❛❜❧❡ ◮ ❆♥❞ t❤❡ ✜rst ♦r❞❡r ❝♦♥❞✐t✐♦♥s ❛r❡ ♥❡❝❡ss❛r② ❛♥❞ s✉✣❝✐❡♥t ❢♦r ❛♥ ✐♥t❡r✐♦r ♦♣t✐♠✉♠

❖♣t✐♠❛❧✐t② ❝♦♥❞✐t✐♦♥s ■■ ◮ ❚❤❡ ♣r♦❜❧❡♠ ❛t t✐♠❡ t ✐s V t ( a t ) = a t + ✶ ∈D t ( a t ) { f ( a t , a t + ✶ ) + β V t + ✶ ( a t + ✶ ) } max ◮ ❚❤❡ ❢♦❝ ❛t t✐♠❡ t ✐s f ✷ ( a t , a t + ✶ ) + β V ′ t + ✶ ( a t + ✶ ) = ✵ ◮ ❯s❡ t❤❡ ❡♥✈❡❧♦♣❡ ❝♦♥❞✐t✐♦♥ t♦ ✇♦r❦♦✉t V ′ t + ✶ ( a t + ✶ ) f ✶ ( a t , a t + ✶ ) + f ✷ ( a t , a t + ✶ ) ∂ a t + ✶ t + ✶ ( a t + ✶ ) ∂ a t + ✶ V ′ + β V ′ t ( a t ) = ∂ a t ∂ a t � � ∂ a t + ✶ f ✷ ( a t , a t + ✶ ) + β V ′ = f ✶ ( a t , a t + ✶ ) + t + ✶ ( a t + ✶ ) ∂ a t � �� � ❢♦❝ ❛t t f ✶ ( a t , a t + ✶ ) = u ′ ( h t ( a t )) =

❖♣t✐♠❛❧✐t② ❝♦♥❞✐t✐♦♥s ■■■ P✉t t❤❡ ❢♦❝ t♦❣❡t❤❡r ✇✐t❤ t❤❡ ❡♥✈❡❧♦♣❡ ❝♦♥❞✐t✐♦♥ t♦ ❣❡t t❤❡ ❊✉❧❡r ❡q✉❛t✐♦♥ f ✷ ( a t , a t + ✶ ) + β f ✶ ( a t + ✶ , a t + ✷ ) = ✵ u ′ ( c t ) = β Ru ′ ( c t + ✶ ) ⇔ � � a t − a t + ✶ s✐♥❝❡✿ u ( c t ) = f ( a t , a t + ✶ ) = u R ❛♥❞ s♦✿ f ✶ ( a t , a t + ✶ ) = u ′ ( c t ) ❛♥❞ f ✷ ( a t , a t + ✶ ) = − u ′ ( c t ) R

❉②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ ◆✉♠❡r✐❝❛❧ s♦❧✉t✐♦♥

◆✉♠❡r✐❝❛❧ s♦❧✉t✐♦♥ ◮ ❚❤❡ ❝❛❦❡✲❡❛t✐♥❣ ♣r♦❜❧❡♠ ✐s ❡❛s② t♦ s♦❧✈❡ ♦♥ t❤❡ ♣❛♣❡r ◮ ❇✉t ✐t ✐s ❛♥ ✐♥str✉❝t✐✈❡ ❡①❛♠♣❧❡ t♦ ♣❧❛② ✇✐t❤ ♥✉♠❡r✐❝❛❧❧② ◮ ❙♦♣❤✐st✐❝❛t❡❞ ❡♥♦✉❣❤ t♦ r❡q✉✐r❡ ♠♦st ♦❢ t❤❡ ♥✉♠❡r✐❝❛❧ tr✐❝❦s ✉s❡❞ ✐♥ ♠♦r❡ ❝♦♠♣❧✐❝❛t❡❞ ♠♦❞❡❧s ◮ ❇✉t ❡❛s② ❡♥♦✉❣❤ t♦ ❦❡❡♣ t❤❡ ❞✐s❝✉ss✐♦♥ s✐♠♣❧❡ ◮ ❈❛♥ ❜❡ ✉s❡❞ t♦ ❞❡♠♦♥str❛t❡ t❤❡ ❝♦♠♣❛r❛t✐✈❡ ❛❞✈❛♥t❛❣❡s ♦❢ ✈❛r✐♦✉s ♥✉♠❡r✐❝❛❧ ♣r♦❝❡❞✉r❡s s✐♥❝❡ t❤❡ s♦❧✉t✐♦♥ ✐s ❦♥♦✇♥✦

❈♦♠♣✉t❡rs ❞♦ ♥♦t ❦♥♦✇♥ ✐♥✜♥✐t② ✶✳ ▼♦❞❡❧ s♣❡❝✐✜❝❛t✐♦♥ ◮ ❈❘❘❆ ✉t✐❧✐t② ✐s ❣r❡❛t t♦ ❡♥s✉r❡ t❤❛t ❝♦♥s✉♠❡rs ❛✈♦✐❞ ❣❡tt✐♥❣ ❝❧♦s❡ t♦ ③❡r♦ ❝♦♥s✉♠♣t✐♦♥ ◮ ❚❤❡ s❛♠❡ ❞♦❡s ♥♦t ❤♦❧❞ ❢♦r ❝♦♠♣✉t❛t✐♦♥❛❧ s♦❧✉t✐♦♥s✿ ❡①tr❡♠❡ ✈❛❧✉❡s ❝❛✉s❡ t❤❡ r♦✉t✐♥❡ t♦ ❝r❛s❤ ⇒ ❇♦✉♥❞ s♦❧✉t✐♦♥ s♣❛❝❡ t♦ ✐ts r❡❧❡✈❛♥t ♣❛rts t♦ ❛✈♦✐❞ ♣r♦❜❧❡♠s ✷✳ ❉✐s❝r❡t✐s❡ st❛t❡ s♣❛❝❡ ◮ ❙❡❧❡❝t ❣r✐❞ ✐♥ ❛ss❡ts A = { a i } i = ✶ ,..., n a ◮ ❙♦❧✈❡ ♣r♦❜❧❡♠ ♦♥❧② ❢♦r ♣♦✐♥ts ✐♥ t❤❡ ❣r✐❞ ◮ ❆♣♣r♦①✐♠❛t❡ ✉♥❦♥♦✇♥ ❢✉♥❝t✐♦♥s ♥✉♠❡r✐❝❛❧❧② ♦✉ts✐❞❡ t❤❡ ❣r✐❞

❆❧❣♦r✐t❤♠ ❢♦r r❡❝✉rs✐✈❡ s♦❧✉t✐♦♥ � a i � ✶✳ P❛r❛♠❡t❡r✐s❡ ♠♦❞❡❧ ❛♥❞ s❡❧❡❝t ❣r✐❞ ✐♥ ❛ss❡ts✿ i = ✶ ,..., n a ✷✳ ❈❤♦♦s❡ st♦♣♣✐♥❣ ❝r✐t❡r✐♦♥ ǫ > ✵ � a i � ✸✳ ❙t♦r❡ V T + ✶ = ✵ ❢♦r ❛❧❧ i = ✶ , . . . , n a ✹✳ ▲♦♦♣ ♦✈❡r t ❜❛❝❦✇❛r❞s✿ t = T , . . . , ✶ ❋♦r ❡❛❝❤ i = ✶ , . . . , n a � � � � a i − a t + ✶ + β � ✹✳✶ ❈♦♠♣✉t❡ g i t = arg max a t + ✶ ∈D t ( a i ) u V t + ✶ ( a t + ✶ ) R � � � � g i a i − + β � ✹✳✷ ❈♦♠♣✉t❡ V i g i t = u t V t + ✶ t R ✹✳✸ ❆♣♣r♦①✐♠❛t❡ V t ♦✈❡r ✐ts ❡♥t✐r❡ ❞♦♠❛✐♥ t♦ ❣❡t � V t ❛♥❞ st♦r❡ ✐t ❚❤✐s st❡♣ ✐s ♦♣t✐♦♥❛❧ ✿ ❝❛♥ ❜❡ ❞♦♥❡ ❞✐r❡❝t❧② ✐♥ st❡♣ ✹✳✶ ♦r s❦✐♣♣❡❞ ❛❧t♦❣❡t❤❡r✱ ❞❡♣❡♥❞✐♥❣ ♦♥ t❤❡ s♦❧✉t✐♦♥ ♠❡t❤♦❞ ✲ ♠♦r❡ t♦ ❢♦❧❧♦✇

❙♦❧✉t✐♦♥ ❛t ❡❛❝❤ ♣♦✐♥t ◮ ❙t❡♣ ✹✳✶ ✐s t❤❡ ✭❝♦♠♣✉t❛t✐♦♥❛❧❧②✮ ❤❡❛✈② ♣❛rt ♦❢ t❤❡ s♦❧✉t✐♦♥ ❛❧❣♦r✐t❤♠ ◮ ❚❤❡r❡ ❛r❡ t✇♦ ♠❛✐♥ ✇❛②s ♦❢ ✜♥❞✐♥❣ t❤❡ ♦♣t✐♠✉♠ g i t ◮ ❯s❡ ❛ s❡❛r❝❤ ❛❧❣♦r✐t❤♠ t♦ ❧♦♦❦ ❢♦r t❤❡ ✈❛❧✉❡ ♦❢ s❛✈✐♥❣s a t + ✶ t❤❛t ♠❛①✐♠✐s❡ V t ( a t ) ❚❤✐s ✐s t❤❡ ♣r♦❝❡❞✉r❡ ✐♠♣❧✐❝✐t ✐♥ t❤❡ ❛❧❣♦r✐t❤♠ ✇❡ ♣r❡s❡♥t❡❞ ◮ ❖r ❧♦♦❦ ❢♦r t❤❡ r♦♦t ♦❢ t❤❡ ❊✉❧❡r ❡q✉❛t✐♦♥ u ′ ( c t ) = β RV ′ ( a t + ✶ ) ✈s♣❛❝❡✵✳✶❝♠ ❲❡ ✇✐❧❧ ❞✐s❝✉ss t❤✐s s♦❧✉t✐♦♥ ❧❛t❡r

❙♦❧✉t✐♦♥ ❛t ❡❛❝❤ ♣♦✐♥t ✉s✐♥❣ t❤❡ ❢♦❝✿ ❛ tr✐❝❦ ■ ◮ ❯s❡❢✉❧ tr✐❝❦ ✉♥❞❡r ❈❘❘❆✿ s♣❡❡❞ ✉♣ ❛♥❞ ✐♠♣r♦✈❡ ❛❝❝✉r❛❝② ♦❢ s♦❧✉t✐♦♥ ◮ ❚❤❡ ❊✉❧❡r ❡q✉❛t✐♦♥ ✐s c t = ( β R ) − ✶ /γ � � − ✶ /γ c − γ = β RV ′ V ′ t + ✶ ( a t + ✶ ) ⇔ t + ✶ ( a t + ✶ ) t ◮ ❇✉t s✐♥❝❡ ✭❡♥✈❡❧♦♣❡ ❝♦♥❞✐t✐♦♥✮ � � − γ a t + ✶ − g t + ✶ ( a t + ✶ ) t + ✶ ( a t + ✶ ) = u ′ ( h t + ✶ ( a t + ✶ )) = V ′ R ◮ ❚❤❡ s♦❧✉t✐♦♥ ✐s t❤❡ ❧❡✈❡❧ ♦❢ s❛✈✐♥❣s a t + ✶ t❤❛t s❛t✐s✜❡s � � a t + ✶ − g t + ✶ ( a t + ✶ ) a t − a t + ✶ = ( β R ) − ✶ /γ R R � �� � � �� � c t c t + ✶

❙♦❧✉t✐♦♥ ❛t ❡❛❝❤ ♣♦✐♥t ✉s✐♥❣ t❤❡ ❢♦❝✿ ❛ tr✐❝❦ ■■ � � a t + ✶ − g t + ✶ ( a t + ✶ ) a t − a t + ✶ = ( β R ) − ✶ /γ h t + ✶ ( a t + ✶ ) = ( β R ) − ✶ /γ R R � �� � � �� � c t c t + ✶ ◮ ❚❤✐s ✐s ❛ ❧✐♥❡❛r ✭✐♥ a t + ✶ ✮ ❡q✉❛t✐♦♥ ✐♥ ♥♦♥✲st♦❝❤❛st✐❝ ♣r♦❜❧❡♠s ◮ ▼♦r❡ ❣❡♥❡r❛❧❧②✱ t❤❡ ♣♦❧✐❝② ❢✉♥❝t✐♦♥ h ✐s t②♣✐❝❛❧❧② ♥♦t ✈❡r② ♥♦♥✲❧✐♥❡❛r ◮ ❙♦ ❛❧❧ ✇❡ ♥❡❡❞ ✐s t♦✿ ✶✳ ❙t♦r❡ h t ( a i ) ❛❢t❡r s♦❧✈✐♥❣ ❝♦♥s✉♠❡rs ♣r♦❜❧❡♠ ❛t t✐♠❡ t ✷✳ ✏❈♦♥♥❡❝t t❤❡ ♣♦✐♥ts✑ t♦ ❛♣♣r♦①✐♠❛t❡ ❢✉♥❝t✐♦♥ h ❛♥❞ ♦❜t❛✐♥ t❤❡ s♦❧✉t✐♦♥ ♦✈❡r t❤❡ ❡♥t✐r❡ ❞♦♠❛✐♥✿ ▲✐♥❡❛r ■♥t❡r♣♦❧❛t✐♦♥ ◮ ◆♦t✐❝❡ t❤❛t V ✐s ♥♦t ♥❡❡❞❡❞ t♦ s♦❧✈❡ t❤❡ ♣r♦❜❧❡♠ ✉s✐♥❣ t❤❡ ❢♦❝

❆♣♣r♦①✐♠❛t✐♥❣ t❤❡ ✈❛❧✉❡ ❢✉♥❝t✐♦♥ ◮ ❆ ❜❛❞ ✐❞❡❛✿ t♦ r❡❧② ♦♥ s✐♠♣❧❡ ✭❧✐♥❡❛r✮ ❛♣♣r♦①✐♠❛t✐♦♥s ♦❢ V t♦ s♦❧✈❡ ♠♦❞❡❧ ❛s V ❝❛♥ ❜❡ ❤✐❣❤❧② ♥♦♥✲❧✐♥❡❛r ◮ ❇✉t ♦♥❡ ♠❛② st✐❧❧ ♥❡❡❞ t❤❡ ✈❛❧✉❡ ❢✉♥❝t✐♦♥✱ ❡✈❡♥ ✇❤❡♥ r❡❧②✐♥❣ ♦♥ t❤❡ ❢♦❝ ❢♦r t❤❡ s♦❧✉t✐♦♥✿ ◮ t♦ st✉❞② t❤❡ ✈❛❧✉❡ ♦❢ ❞✐✛❡r❡♥t ♣♦❧✐❝② ✐♥t❡r✈❡♥t✐♦♥s ◮ ♦r ❛tt✐t✉❞❡s t♦✇❛r❞s r✐s❦ ♦♥❝❡ ✉♥❝❡rt❛✐♥t② ✐s ❝♦♥s✐❞❡r❡❞ ◮ ❚✇♦ ❛❧t❡r♥❛t✐✈❡s t♦ ❛♣♣r♦①✐♠❛t❡ V ◮ ▼♦r❡ r❡❧✐❛❜❧❡ ❛♣♣r♦①✐♠❛t✐♦♥ ♠❡t❤♦❞✿ s❤❛♣❡✲♣r❡s❡r✈✐♥❣ s♣❧✐♥❡s ◮ ❘❡❞✉❝❡ ♥♦♥✲❧✐♥❡❛r✐t② ❜② ❛♣♣❧②✐♥❣ s❡❧❡❝t❡❞ tr❛♥s❢♦r♠❛t✐♦♥✱ t❤❡♥ ❛♣♣r♦①✐♠❛t❡ ❜② ❧✐♥❡❛r ✐♥t❡r♣♦❧❛t✐♦♥ ❋♦r ❛ ❈❘❘❆ ✉t✐❧✐t②✿ ✶ Ψ t ( a t ) = [( ✶ − γ ) V t ( a t )] ✶ − γ

Pr❛❝t✐❝❛❧ s❡ss✐♦♥ ✶

■♥❝♦♠❡ ♣r♦❝❡ss

❆❞❞ ✐♥❝♦♠❡ ♣r♦❝❡ss ◮ ❏✉st ❛❞❞✐♥❣ ❛♥ ✐♥❝♦♠❡ ♣r♦❝❡ss ❞♦❡s ♥♦t ♠✉❝❤ ❝❤❛♥❣❡ t❤❡ ❧✐❢❡❝②❝❧❡ ♣r♦❜❧❡♠ ◮ ❇✉t r❛✐s❡s ✐♥t❡r❡st✐♥❣ ✐ss✉❡s ♦❢ ❤♦✇ t♦ ❞❡❛❧ ✇✐t❤ t❤❡ ❝r❡❞✐t ♠❛r❦❡ts ◮ ❙✉♣♣♦s❡ t❤❡ ❝♦♥s✉♠❡r ❤❛s ❛ str❡❛♠ ♦❢ ✐♥❝♦♠❡ ♦✈❡r t✐♠❡ y t = w ( a t , t ) ◮ ❋♦r t❤❡ ♠♦♠❡♥t✱ s✉♣♣♦s❡ { y t } t = ✶ ,..., T ✐s ❦♥♦✇♥ ❜② t❤❡ ❝♦♥s✉♠❡r ❢r♦♠ t✐♠❡ t = ✶

■♥❝♦♠❡ ♣r♦❝❡ss ❈r❡❞✐t ▼❛r❦❡ts

❋✉♥❝t✐♦♥✐♥❣ ❝r❡❞✐t ♠❛r❦❡ts ■ ◮ ■❢ ❝r❡❞✐t ♠❛r❦❡ts ❛r❡ ❝♦♠♣❧❡t❡✱ t❤❡ ❝♦♥s✉♠❡r ♠❛② ❜♦rr♦✇ t♦ ❜r✐♥❣ ✐♥❝♦♠❡ ❢♦r✇❛r❞ ◮ ❆ss❡ts ❛t t✐♠❡ t ❝❛♥ ❜❡ ♥❡❣❛t✐✈❡ ◮ ❇♦rr♦✇✐♥❣ ❧✐♠✐t❡❞ ❜② ❛❜✐❧✐t② t♦ r❡♣❛② ◮ ❉♦♠❛✐♥ ♦❢ ♣♦ss✐❜❧❡ ✈❛❧✉❡s ♦❢ ❛ss❡ts ❝❤❛♥❣❡s ♦✈❡r t✐♠❡✱ ❞❡♣❡♥❞✐♥❣ ♦♥ t✐♠❡ ❧❡❢t t♦ r❡♣❛② ❞❡❜ts ❛♥❞ t❡r♠✐♥❛❧ ❝♦♥❞✐t✐♦♥ ◮ ❚❤❡ ♣r♦❜❧❡♠ ♦❢ t❤❡ ❝♦♥s✉♠❡r ❛t t✐♠❡ t ❢♦r ❛ss❡ts a t a t + ✶ { f ( a t , y t , a t + ✶ ) + β V t + ✶ ( a t + ✶ , y t + ✶ ) } V t ( a t , y t ) = max a t + ✶ = R ( a t + y t − c t ) s✳t✳ y t + ✶ = w ( a t + ✶ , t + ✶ ) a T + ✶ ≥ ✵ c t > ✵ ❛♥❞

❋✉♥❝t✐♦♥✐♥❣ ❝r❡❞✐t ♠❛r❦❡ts ■■ ◮ ❚❤❡ ❢❡❛s✐❜✐❧✐t② s♣❛❝❡ ❛t t✐♠❡ t < T ✐s         � T a t + ✶ : a t + y t − a t + ✶ R ( t + ✶ ) − τ y τ > ✵ D t ( a t , y t ) = > ✵ , a t + ✶ +   R   � �� �   τ = t + ✶ c t � � � T R ( t + ✶ ) − τ y τ , R ( a t + y t ) = − τ = t + ✶ ◮ ❆t t✐♠❡ T D T ( a T , y T ) = [ ✵ , R ( a T + y T ))

❋✉♥❝t✐♦♥✐♥❣ ❝r❡❞✐t ♠❛r❦❡ts ■■■ ◮ ❚❤❡ ❝♦♠♣❛❝t s♣❡❝✐✜❝❛t✐♦♥ ♦❢ t❤❡ ♣r♦❜❧❡♠ ✐s V t ( a t , y t ) = a t + ✶ ∈D t ( a t , y t ) { f ( a t , y t , a t + ✶ ) + β V t + ✶ ( a t + ✶ , y t + ✶ ) } max s✳t✳ y t = w ( a t , t ) ❢♦r ❛❧❧ t ◮ ❋♦❝ ✐s ❊✉❧❡r ❡q✉❛t✐♦♥ u ′ ( c t ) = β Ru ′ ( c t + ✶ ) ◮ ❚❤❡ st❛t❡ s♣❛❝❡ ✐s ♥♦✇ ✷✲❞✐♠❡♥s✐♦♥❛❧ ◮ ❆❧t❤♦✉❣❤ ✐t ✐s ❡❛s② t♦ r❡❞✉❝❡ t♦ ✶ ❞✐♠❡♥s✐♦♥ ✐♥ t❤✐s ❝❛s❡ ❜② ♥♦t✐♥❣ t❤❛t a t + ✶ = R ( a t + w ( a t , t ) − c t ) ◮ ❈♦♠♣✉t❛t✐♦♥✲✇✐s❡✱ r❡❞✉❝✐♥❣ t❤❡ ❞✐♠❡♥s✐♦♥❛❧✐t② ♦❢ t❤❡ st❛t❡ s♣❛❝❡ ✐s t❤❡ ♠♦st t✐♠❡✲s❛✈✐♥❣ ♣r♦❝❡❞✉r❡

❙✐♠♣❧❡ ❡①❛♠♣❧❡✿ ❈❘❘❆ ✉t✐❧✐t② t − ✶ γ c ✶ ◮ ❲✐t❤ ❈❘❘❆ ✉t✐❧✐t② t❤❡ ❊✉❧❡r ❡q✉❛t✐♦♥ ✐♠♣❧✐❡s c t = ( β R ) ◮ ❚❤❡ ✈❛❧✉❡ ♦❢ t♦t❛❧ ❧✐❢❡t✐♠❡ ✇❡❛❧t❤ ❛t t = ✶ ✐s � R ✶ − t y t W = a ✶ + t = ✶ ,..., T ◮ ❚♦t❛❧ ❝♦♥s✉♠♣t✐♦♥ ✐s � � � β R ✶ − γ � t − ✶ γ c ✶ R ✶ − t c t = C = t = ✶ ,..., T t = ✶ ,..., T ◮ ❨✐❡❧❞✐♥❣✱ ❢♦r t = ✶ , . . . , T ✶ − α t − ✶ ✶ ✶ − γ γ R c t = ( β R ) ✶ − α T W ✇❤❡r❡ α = β γ γ

❈❘❘❆ ✉t✐❧✐t②✿ ♣r♦✜❧❡s ❢♦r ❛ ♣❛t✐❡♥t ❝♦♥s✉♠❡r increasing income constant income 2 4 3 0 2 −2 1 −4 0 20 30 40 50 60 20 30 40 50 60 age age consumption income assets r = ✹ % ❛♥❞ β = ✶ . ✵✷✺ − ✶ ✳ ■♥✐t✐❛❧ ❛ss❡ts ❛r❡ a ✶ = ✶✳ ■♥❝♦♠❡ ♣r♦✜❧❡s ❛s ♣❧♦tt❡❞✳

❈❘❘❆ ✉t✐❧✐t②✿ ✐♥tr♦❞✉❝✐♥❣ r❡t✐r❡♠❡♥t 15 10 5 0 −5 20 30 40 50 60 70 age consumption income assets r = ✹ % ❛♥❞ β = ✶ . ✵✷✺ − ✶ ✳ ■♥✐t✐❛❧ ❛ss❡ts ❛r❡ a ✶ = ✶✳ ■♥❝♦♠❡ ♣r♦✜❧❡s ❛s ♣❧♦tt❡❞✳

❈r❡❞✐t ❝♦♥str❛✐♥ts ■ ◮ ■❢ ❝r❡❞✐t ✐s r❛t✐♦♥❡❞✱ t❤❡ ❝♦♥s✉♠❡r ♠❛② ❜❡ ✇✐❧❧✐♥❣ t♦ ❝♦♥s✉♠❡ ♠♦r❡ t❤❛♥ s❤❡ ❝❛♥ ❛✛♦r❞ ✐♥ t❤❡ s❤♦rt t❡r♠ ◮ ■♥ t❤❡ ❛❜s❡♥❝❡ ♦❢ ❝r❡❞✐t✱ t❤❡ ❢❡❛s✐❜✐❧✐t② s❡t ✐s r❡str✐❝t❡❞ t♦ � � a t + ✶ : a t + y t − a t + ✶ D t ( a t , y t ) = > ✵ , a t + ✶ ≥ ✵ R ◮ ❚❤✐s ✐♠♣❧✐❡s t❤❛t t❤❡ ❝♦♥s✉♠❡r✬s ❜❡st ❝❤♦✐❝❡ ♠❛② ❜❡ ❛ ❝♦r♥❡r s♦❧✉t✐♦♥

❈r❡❞✐t ❝♦♥str❛✐♥ts ■■ ◮ ❚❤❡ ♣r♦❜❧❡♠ ♦❢ t❤❡ ❝♦♥s✉♠❡r ❛t t✐♠❡ t ❢♦r ❛ss❡ts a t ✐s ♥♦✇ V t ( a t , y t ) = max a t + ✶ { f ( a t , y t , a t + ✶ ) + β V t + ✶ ( a t + ✶ , y t + ✶ ) } s✳t✳ a t + ✶ = R ( a t + y t − c t ) y t + ✶ = w ( a t + ✶ , t + ✶ ) c t > ✵ ❛♥❞ a t + ✶ ≥ ✵ ◮ ❚❤❡r❡ ❛r❡ T ✐♥❡q✉❛❧✐t② r❡str✐❝t✐♦♥s ✐♥ ❛ss❡ts ♥♦✇✱ s♦ ✇❡ ❤❛✈❡ T ✜rst ♦r❞❡r ❛♥❞ ❑✉❤♥ ❚✉❝❦❡r ❝♦♥❞✐t✐♦♥s✿ f ✸ ( a t , y t , a t + ✶ ) + β f ✶ ( a t + ✶ , y t + ✶ , a t + ✷ ) = λ t ❢♦r t = t = ✶ , . . . , T − ✶ λ t a t + ✶ = ✵ , λ t ≥ ✵ , a t + ✶ ≥ ✵ a T + ✶ = ✵ ❢♦r t = T

❈r❡❞✐t ❝♦♥str❛✐♥ts ■■■ ❚❤❡ s♦❧✉t✐♦♥ ✐s min { a t + y t , r♦♦t ♦❢ u ′ ( c t ) = β Ru ′ ( c t + ✶ ) } c t = ♦r max { ✵ , r♦♦t ♦❢ f ✸ ( a t , y t , a t + ✶ ) + β f ✶ ( a t + ✶ , y t + ✶ , a t + ✷ ) = ✵ } a t + ✶ =

■♥❝♦♠❡ ♣r♦❝❡ss ◆✉♠❡r✐❝❛❧ s♦❧✉t✐♦♥

❙♦❧✉t✐♦♥ ❛❧❣♦r✐t❤♠ ❚❤❡ r❡❝✉rs✐✈❡ s♦❧✉t✐♦♥ ✐♥ ♣r❛❝t✐❝❡✿ ❛❧♠♦st ❡①❛❝t❧② ❛s ❜❡❢♦r❡ � � a i ✶✳ P❛r❛♠❡t❡r✐s❡ ♠♦❞❡❧ ❛♥❞ s❡❧❡❝t ❣r✐❞s ✐♥ a t ✿ t i = ✶ ,..., n a ✷✳ ❈❤♦♦s❡ st♦♣♣✐♥❣ ❝r✐t❡r✐♦♥ ǫ > ✵ � � a i ✸✳ ❙t♦r❡ V T + ✶ = ✵ ❢♦r ❛❧❧ i = ✶ , . . . , n a T + ✶ ✹✳ ▲♦♦♣ ♦✈❡r t ❜❛❝❦✇❛r❞s✿ t = T , . . . , ✶ ❋♦r ❡❛❝❤ i = ✶ , . . . , n a � � � � t , t ) − a t + ✶ + β � ✹✳✶ ❈♦♠♣✉t❡ g i a i t + w ( a i t = arg max u V t + ✶ ( a t + ✶ ) R a t + ✶ ∈D t ( a i t ) � � � � g i + β � ✹✳✷ ❈♦♠♣✉t❡ V i a i t + w ( a i g i t = u t , t ) − t V t + ✶ t R

❈♦♠♣✉t❛t✐♦♥❛❧ s♦❧✉t✐♦♥✿ ❛❞❞✐t✐♦♥❛❧ ✐ss✉❡s ✶✳ ❉✐♠❡♥s✐♦♥ ♦❢ st❛t❡ s♣❛❝❡✿ r❡❞✉❝❡ t♦ ✶ ✐♥ s♦❧✉t✐♦♥ a t + ✶ = R ( a t + w ( a t , t ) − c t ) ✷✳ P♦s✐t✐✈❡ ❝♦♥s✉♠♣t✐♦♥✿ ♠❛② ❜❡ tr✐❝❦② t♦ ❡♥s✉r❡ ✇✐t❤ ❛♣♣r♦①✐♠❛t❡❞ ⇒ ❢✉♥❝t✐♦♥s ✐♠♣♦s❡ ♠✐♥✐♠✉♠ ❝♦♥s✉♠♣t✐♦♥ c min > ✵ ✸✳ ❋✉♥❝t✐♦♥✐♥❣ ❝r❡❞✐t ♠❛r❦❡ts✿ ❣r✐❞ ✐♥ ❛ss❡ts ❝❤❛♥❣❡s ♦✈❡r t✐♠❡ ◮ ▲♦✇❡r ❜♦✉♥❞ ❛t t ❡♥s✉r❡s ❞❡❜t ❝❛♥ ❜❡ r❡♣❛✐❞ ❛♥❞ c min ✐s ❛✛♦r❞❛❜❧❡ � � R t − τ y τ ≥ R t − τ c min a t + τ = t ..., T τ = t ..., T ◮ ❯♣♣❡r ❜♦✉♥❞ ❛t t r❡❛❝❤❡❞ ✐❢ ❝♦♥s✉♠❡s c min ✐♥ ❛❧❧ ♣❡r✐♦❞s t♦ t � R t − τ ( y τ − c min ) a t ≤ R t − ✶ a ✶ + τ = ✶ ..., t − ✶

Pr❛❝t✐❝❛❧ s❡ss✐♦♥ ✷

❙t♦❝❤❛st✐❝ ♦♣t✐♠✐s❛t✐♦♥

❙t♦❝❤❛st✐❝ ♣r♦❜❧❡♠s ◮ ▼♦st ✐♥t❡r❡st✐♥❣ ♣r♦❜❧❡♠s ✐♥ ❡❝♦♥♦♠✐❝s ✐♥✈♦❧✈❡ s♦♠❡ s♦rt ♦❢ ✉♥✐♥s✉r❛❜❧❡ r✐s❦ ◮ ❚❤❡ s♦❧✉t✐♦♥ t♦ t❤❡ ❞②♥❛♠✐❝ ♣r♦❜❧❡♠ ✇✐❧❧ ❞❡♣❡♥❞ ❝r✉❝✐❛❧❧② ♦♥ ✶✳ ❤♦✇ ♠✉❝❤ r✐s❦ ❝♦♥s✉♠❡rs ❢❛❝❡ ✷✳ t❤❡✐r ❛tt✐t✉❞❡s t♦✇❛r❞s r✐s❦ ◮ ❲❡ ❝♦♥s✐❞❡r ❛ st♦❝❤❛st✐❝ ✐♥❝♦♠❡ ♣r♦❝❡ss t♦ ❢♦r♠❛❧✐s❡ ✉♥❝❡rt❛✐♥t② ◮ ❆♥❞ ❞♦ s♦ ✐♥ ❛ ♣❛rs✐♠♦♥✐♦✉s ✇❛②✱ ✉s✐♥❣ ▼❛r❦♦✈ ♣r♦❝❡ss❡s

❙t♦❝❤❛st✐❝ ♦♣t✐♠✐s❛t✐♦♥ ▼❛r❦♦✈ ♣r♦❝❡ss❡s

❙✉♣❡r ❜r✐❡❢ ✐♥tr♦❞✉❝t✐♦♥ t♦ st♦❝❤❛st✐❝ ▼❛r❦♦✈ ♣r♦❝❡ss❡s ■ ❙t♦❝❤❛st✐❝ ♣r♦❝❡ss✿ s❡q✉❡♥❝❡ { y t } t = ✶ ,... ♦❢ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s✴✈❡❝t♦rs ❚❤❡ ▼❛r❦♦✈ ♣r♦♣❡rt② ◮ ❙✉♣♣♦s❡ { y t } t = ✶ , ✷ ,... ✐s ❞❡✜♥❡❞ ♦♥ t❤❡ s✉♣♣♦rt Y ◮ ❚❤❡♥ { y t } s❛t✐s✜❡s t❤❡ ▼❛r❦♦✈ ♣r♦♣❡rt② ✐❢✱ ❢♦r ❛❧❧ y ∈ Y Pr♦❜ ( y t + ✶ = y | y t , . . . , y ✶ ) = Pr♦❜ ( y t + ✶ = y | y t ) ❢♦r ❞✐s❝r❡t❡ Y Pr♦❜ ( y t + ✶ < y | y t , . . . , y ✶ ) = Pr♦❜ ( y t + ✶ < y | y t ) ❢♦r ❝♦♥t✐♥✉♦✉s Y

❙✉♣❡r ❜r✐❡❢ ✐♥tr♦❞✉❝t✐♦♥ t♦ st♦❝❤❛st✐❝ ▼❛r❦♦✈ ♣r♦❝❡ss❡s ■■ ◮ ❚❤❡ ❝♦♥❞✐t✐♦♥❛❧ ♣r♦❜❛❜✐❧✐t✐❡s ❛r❡ ❦♥♦✇♥ ❛s t❤❡ tr❛♥s✐t✐♦♥ ❢✉♥❝t✐♦♥ Q t ( y t , y t + ✶ ) = Pr♦❜ ( y t + ✶ | y t ) ◮ ❚✐♠❡✲✐♥✈❛r✐❛♥t ♣r♦❝❡ss✿ Q t ( y t , y t + ✶ ) = Q ( y t , y t + ✶ ) ◮ Q : Y × Y → [ ✵ , ✶ ] ✐s ❛ tr❛♥s✐t✐♦♥ ❢✉♥❝t✐♦♥ ✐❢ Q ( y t , y ) ✐s ❛ ♣❞❢✿ ❋♦r ❡❛❝❤ y t ∈ Y Q ( y t , y ) ≥ ✵ ❢♦r ❛❧❧ y ∈ Y � ❛♥❞ Q ( y t , y ) dy = ✶ Y

❙✉♣❡r ❜r✐❡❢ ✐♥tr♦❞✉❝t✐♦♥ t♦ st♦❝❤❛st✐❝ ▼❛r❦♦✈ ♣r♦❝❡ss❡s ■■■ ◮ ▼❛r❦♦✈ ♣r♦❝❡ss✿ st♦❝❤❛st✐❝ ♣r♦❝❡ss s❛t✐s❢②✐♥❣ t❤❡ ▼❛r❦♦✈ ♣r♦♣❡rt② ◮ ❈❤❛r❛❝t❡r✐s❡❞ ❜② ✸ ♦❜❥❡❝ts ◮ t❤❡ ❞♦♠❛✐♥ Y ◮ t❤❡ tr❛♥s✐t✐♦♥ ❢✉♥❝t✐♦♥ Q ◮ t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ t❤❡ ✐♥✐t✐❛❧ ✈❛❧✉❡ y ✶ ◮ ❚❤❡s❡ ❢✉❧❧② ❝❤❛r❛❝t❡r✐s❡ t❤❡ ❥♦✐♥t ❛♥❞ ♠❛r❣✐♥❛❧ ❞✐str✐❜✉t✐♦♥s ♦❢ y ❛t ❛❧❧ ♣♦✐♥ts ✐♥ t✐♠❡

❙✉♣❡r ❜r✐❡❢ ✐♥tr♦❞✉❝t✐♦♥ t♦ st♦❝❤❛st✐❝ ▼❛r❦♦✈ ♣r♦❝❡ss❡s ■❱ ◮ ❚❤❡ ✉♥❝♦♥❞✐t✐♦♥❛❧ ❞✐str✐❜✉t✐♦♥ ♦❢ y t ❝❛♥ ❜❡ ♦❜t❛✐♥❡❞ ✐t❡r❛t✐✈❡❧② ◮ ▲❡t π t − ✶ ❜❡ t❤❡ ♣❞❢ ♦❢ y ❛t t✐♠❡ t − ✶✳ ❚❤❡♥✱ ✐❢ π t − ✶ ✐s ❦♥♦✇♥ � π t ( y t ) = Q ( y , y t ) π t − ✶ ( y ) dy y ∈ Y ✇❤❡r❡ π t ❜❡ t❤❡ ♣❞❢ ♦❢ y ❛t t✐♠❡ t ◮ ❆ ▼❛r❦♦✈ ♣r♦❝❡ss ✐s st❛t✐♦♥❛r② ✐❢ π t ( y ) = π t ′ ( y ) = π ( y ) ◮ ■♥ t❤✐s ❝❛s❡✱ π ✐s t❤❡ ✜①❡❞ ♣♦✐♥t ✐♥ t❤❡ ❢✉♥❝t✐♦♥❛❧ ❡q✉❛t✐♦♥ � π ( y t ) = Q ( y , y t ) π ( y ) dy y ∈ Y

❙t♦❝❤❛st✐❝ ♦♣t✐♠✐s❛t✐♦♥ ■✐❞ ✐♥❝♦♠❡ ♣r♦❝❡ss

▼❡♠♦r②❧❡ss ✐♥❝♦♠❡ ♣r♦❝❡ss ✇✐t❤ ❞✐s❝r❡t❡ s✉♣♣♦rt � y ✶ , . . . , y n � ◮ ❚❛❦❡ ❛ ❞✐s❝r❡t❡ ✐♥❝♦♠❡ ♣r♦❝❡ss y t ∈ Y = ◮ ❋♦r ❛ ♠❡♠♦r②❧❡ss ♣r♦❜❧❡♠✱ t❤❡ tr❛♥s✐t✐♦♥ ❢✉♥❝t✐♦♥ ❡q✉❛❧s t❤❡ ✉♥❝♦♥❞✐t✐♦♥❛❧ ♣❞❢✿ � y t = y i � � y , y i � π i = Pr♦❜ = Q ❢♦r ❡❛❝❤ i = ✶ , . . . , n ◮ ❚❤❡ ❝♦♥s✉♠❡r✬s ♣r♦❜❧❡♠ ✐s    � a t + ✶ , y i �  � π i V t ( a t , y t ) = max  f ( a t , y t , a t + ✶ ) + β V t + ✶  a t + ✶ ∈D t ( a t , y t ) y i ∈ Y s✳t✳ y t ✐s ❛ r✈ ✇✐t❤ ♣❞❢ π ◮ ❚❤❡ ♣r♦❜❧❡♠ ✐s s❡t✉♣ ❛s ❛ ▼❛r❦♦✈ ♣r♦❝❡ss✿ ( a t + ✶ , y t + ✶ ) ❞❡♣❡♥❞s ♦♥❧② ♦♥ ( a t , y t )

▼❡♠♦r②❧❡ss ✐♥❝♦♠❡ ♣r♦❝❡ss ✇✐t❤ ❝♦♥t✐♥✉♦✉s s✉♣♣♦rt ◮ ❚❤❡ ♣r♦❜❧❡♠ ✐s V t ( a t , y t ) � � � = max f ( a t , y t , a t + ✶ ) + β V t + ✶ ( a t + ✶ , y ) π ( y ) dy a t + ✶ ∈D t ( a t , y t ) y ∈ Y ◮ ❋❡❛s✐❜✐❧✐t② s❡t✿ s❛✈✐♥❣s ❝❤♦✐❝❡s ❡♥s✉r✐♥❣ ♣♦s✐t✐✈❡ ❝♦♥s✉♠♣t✐♦♥ ✐s ❛✛♦r❞❛❜❧❡ ❡✈❡♥ ✐♥ ✇♦rst ♣♦ss✐❜❧❡ s❝❡♥❛r✐♦ D t ( a t , y t ) � � � T a t + ✶ : a t + y t − a t + ✶ R ( t + ✶ ) − τ y min > ✵ = > ✵ , a t + ✶ + R τ = t + ✶

❙✉♣♣♦rt ❛♥❞ ❢❡❛s✐❜✐❧✐t② s❡t ✐♥ ♣r❛❝t✐❝❡ ◮ ❋❡❛s✐❜✐❧✐t② s❡t ❢♦r a t + ✶ ✐s D t ( a t , y t ) ◮ ❙❡t ♦❢ ♣♦ss✐❜❧❡ ❝❤♦✐❝❡s a t + ✶ ❣✐✈❡♥ ❝✉rr❡♥t ✈❛❧✉❡ ♦❢ st❛t❡ ✈❛r✐❛❜❧❡s ◮ ❈♦♠♣✉t❛t✐♦♥❛❧ ✐♠♣❧❡♠❡♥t❛t✐♦♥✿ ♦♣t✐♠❛❧ s❛✈✐♥❣s ❝❤♦s❡♥ ✐♥ D t ( a t , y t ) ◮ ❙✉♣♣♦rt ♦❢ a t + ✶ ✐s A t + ✶ ◮ ❘❛♥❣❡ ♦❢ ❛❧❧ ♣♦ss✐❜❧❡ ✈❛❧✉❡s ♦❢ a t + ✶ ✱ ✐♥❞❡♣❡♥❞❡♥t❧② ♦❢ ❝✉rr❡♥t ✈❛❧✉❡ ♦❢ st❛t❡ ✈❛r✐❛❜❧❡s ◮ ❈♦♠♣✉t❛t✐♦♥❛❧ ✐♠♣❧❡♠❡♥t❛t✐♦♥✿ ❣r✐❞ ✐♥ a t + ✶ ❞r❛✇♥ t♦ r❡♣r❡s❡♥t A t + ✶ ◮ ❈❧❡❛r❧② D t ( a t , y t ) ⊆ A t + ✶ ❢♦r ❛❧❧ ( a t , y t ) ◮ ❙✉♣♣♦s❡ ✇❡ ❜♦✉♥❞ ❝♦♥s✉♠♣t✐♦♥ ❝❤♦✐❝❡s ❢r♦♠ ❜❡❧♦✇✿ ❡♥s✉r❡ c min ❛❧✇❛②s ❛✛♦r❞❛❜❧❡ ◮ ❆♥❞ ✉s❡ ❜♦✉♥❞❡❞ s✉♣♣♦rt ♦❢ ✐♥❝♦♠❡ ✐s Y = [ y min , y max ]

❙✉♣♣♦rt ❛♥❞ ❢❡❛s✐❜✐❧✐t② s❡t ✐♥ ♣r❛❝t✐❝❡✿ s✉♣♣♦rt ◮ ❯♣♣❡r ❜♦✉♥❞ ♦❢ A t + ✶ ✿ ♠❛①✐♠✉♠ s❛✈✐♥❣s r❡❛❝❤❡❞ ✐❢ y t = y max ❛♥❞ c t = c min ✐♥ t❤❡ ♣❛st � t � t a t + ✶ ≤ R t a ✶ + R τ y max − R τ c min τ = ✶ τ = ✶ ❯❇ t + ✶ = R t a ✶ + R ✶ − R t ⇒ ✶ − R ( y max − c min ) ◮ ▲♦✇❡r ❜♦✉♥❞ ♦❢ A t + ✶ ✿ ❡♥s✉r❡s c min ❛❧✇❛②s ❛✛♦r❞❛❜❧❡ ✐♥ ❢✉t✉r❡ � T � T R ( t + ✶ ) − τ y min ≥ R ( t + ✶ ) − τ c min a t + ✶ + τ = t + ✶ τ = t + ✶ ▲❇ t + ✶ = ✶ − R t − T ⇒ ✶ − R − ✶ ( c min − y min ) ◮ ❙♦ A t + ✶ = [ ▲❇ t + ✶ , ❯❇ t + ✶ ]

❙✉♣♣♦rt ❛♥❞ ❢❡❛s✐❜✐❧✐t② s❡t ✐♥ ♣r❛❝t✐❝❡✿ ❢❡❛s✐❜✐❧✐t② s❡t ◮ ❯♣♣❡r ❜♦✉♥❞ ♦❢ D t ❝♦♥❞✐t✐♦♥❛❧ ♦♥ ( a t , y t ) ❡♥s✉r❡s c t ≥ c min a t + y t − a t + ✶ R − ✶ ≥ c min ⇒ ❯❇ t + ✶ ( a t , y t ) = R ( a t + y t − c min ) ◮ ▲♦✇❡r ❜♦✉♥❞ ♦❢ D t ❡q✉❛❧s ❧♦✇❡r ❜♦✉♥❞ ♦❢ A t + ✶ ✿ LB t + ✶ ❝❛♥ ❛❧✇❛②s ❜❡ r❡❛❝❤❡❞ ♦r ♦t❤❡r✇✐s❡ ♣r♦❜❧❡♠ ❤❛s ♥♦ s♦❧✉t✐♦♥ ◮ ❙♦ D t ( a t , y y ) = [ ▲❇ t + ✶ , ❯❇ t + ✶ ( a t , y y )]

▼❡♠♦r②❧❡ss ✐♥❝♦♠❡ ♣r♦❝❡ss✿ ♦♣t✐♠❛❧✐t② ❝♦♥❞✐t✐♦♥s ◮ ❋♦❝ ❛t t✐♠❡ t ✿ ❞❡r✐✈❛t✐✈❡ ♦❢ ♦❜❥❡❝t✐✈❡ ❢✉♥❝t✐♦♥ ❛t t✐♠❡ t ✐s ③❡r♦ � ∂ V t + ✶ ( a t + ✶ , y ) f ✸ ( a t , y t , a t + ✶ ) + β π ( y ) dy = ✵ ∂ a t + ✶ y ∈ Y ◮ ❲♦r❦ ♦✉t ♠❛r❣✐♥❛❧ ✈❛❧✉❡ ♦❢ a t ✿   �   ∂ V t ( a t , y t ) ∂ V t + ✶ ∂ a t + ✶   = f ✶ +  f ✸ + β ∂ a t + ✶ π ( y ) dy = f ✶ ( a t , y t , a t + ✶ )   ∂ a t  ∂ a t y ∈ Y � �� � = ✵ ◮ ❙♦ ❛♥ ✐♥t❡r✐♦r ♦♣t✐♠✉♠ s❛t✐s✜❡s � f ✸ ( a t , y t , a t + ✶ ) + β f ✶ ( a t + ✶ , y , a t + ✷ ) π ( y ) dy = ✵ y ∈ Y � � u ′ ( c t ) − β R ❊ t u ′ ( c t + ✶ ) ⇔ = ✵

❙t♦❝❤❛st✐❝ ♦♣t✐♠✐s❛t✐♦♥ ■✐❞ ✐♥❝♦♠❡ ♣r♦❝❡ss✿ ◆✉♠❡r✐❝❛❧ s♦❧✉t✐♦♥

❈♦♠♣✉t❛t✐♦♥❛❧ ❛❧❣♦r✐t❤♠ ✶✳ P❛r❛♠❡t❡r✐s❡ ♠♦❞❡❧ ❛♥❞ s❡❧❡❝t ❣r✐❞s ( A , Y ) ❛♥❞ ❝♦♠♣✉t❡ ✇❡✐❣❤ts π j ✷✳ ❈❤♦♦s❡ st♦♣♣✐♥❣ ❝r✐t❡r✐♦♥ ǫ > ✵ � � a i ✸✳ ❙t♦r❡ ❊ V T + ✶ = ✵ ❢♦r ❛❧❧ i = ✶ , . . . , n a t + ✶ ✹✳ ▲♦♦♣ ♦✈❡r t ❜❛❝❦✇❛r❞s✿ t = T , . . . , ✶ ▲♦♦♣ ♦✈❡r i = ✶ , . . . , n a ✹✳✶ ❈♦♠♣✉t❡ ❢♦r j = ✶ , . . . , n y � � � � t + y j − a t + ✶ g ij a i + β � t = arg max u ❊ V t + ✶ ( a t + ✶ ) R a t + ✶ ∈D ij t ✹✳✷ ❈♦♠♣✉t❡ t❤❡ ❝♦♥t✐♥✉❛t✐♦♥ ✈❛❧✉❡ � � � �� � � t + y j − g ij + β � ❊ V i a i t g ij π j t = ❊ V t + ✶ u t R j = ✶ ,..., n y

Pr❛❝t✐❝❛❧ ✐ss✉❡s ■ ◮ ❙t❛t❡ s♣❛❝❡ ✐s ✷✲❞✐♠✿ ( a , y ) ◮ ❚❤❡ ✐♥❝♦♠❡ ♣r♦❝❡ss ❝♦✉❧❞ ❤❛✈❡ ❛ ❝♦♥t✐♥✉♦✉s s✉♣♣♦rt✿ ❞✐s❝r❡t✐s❡ Y ❛♥❞ s♦❧✈❡ ♣r♦❜❧❡♠ ✐♥ n a × n y ♣♦✐♥ts ❢♦r ❡❛❝❤ t ◮ ❇♦✉♥❞s ✐♥ Y ✿ ❡♥s✉r❡ ❢❡❛s✐❜✐❧✐t② ❛♥❞ ♠❡❛s✉r❛❜✐❧✐t② ◮ ●r✐❞ ✐♥ a t♦ ❛❝❝♦✉♥t ❢♦r t❤❡ ♠❛♥② ♣♦ss✐❜❧❡ ❢✉t✉r❡ ❝✐r❝✉♠st❛♥❝❡s ◮ ❋❡❛s✐❜✐❧✐t② ❛♠♦✉♥ts t♦ ❡♥s✉r❡ c min r❡♠❛✐♥s ❛✛♦r❞❛❜❧❡ ◮ ■♠♣♦s❡❞ ♦♥ ✇♦rst ❝❛s❡ s❝❡♥❛r✐♦ ♦❢ ❢✉t✉r❡ ✐♥❝♦♠❡ s♦ ✐t ❤♦❧❞s ✉♥❞❡r ❛❧❧ ♣♦ss✐❜❧❡ ❢✉t✉r❡ ❝✐r❝✉♠st❛♥❝❡s ◮ ❈♦♥t✐♥✉❛t✐♦♥ ✈❛❧✉❡✿ ❊ t V t + ✶ ◮ ▼❡❛s✉r❡❞ ❛t t ❝♦♥❞✐t✐♦♥❛❧ ♦♥ ❡①✐st✐♥❣ ✐♥❢♦r♠❛t✐♦♥ ◮ ❖♥❧② ❛r❣✉♠❡♥t ✐♥ ❊ t V t + ✶ ✐s a t + ✶ ◮ ❈❤♦✐❝❡ ♦❢ ❣r✐❞ ✐♥ y t♦ s✉♣♣♦rt ✐♥t❡❣r❛t✐♦♥ ◮ ◆❡❡❞ s❡t ♦❢ ✇❡✐❣❤ts t♦ ❝❛❧❝✉❧❛t❡ ✐♥t❡❣r❛❧ ♥✉♠❡r✐❝❛❧❧②✱ π j

Pr❛❝t✐❝❛❧ ✐ss✉❡s ■■ ❲❡ ❝❤♦♦s❡ t♦ st♦r❡ ❊ V ✐♥st❡❛❞ ♦❢ V ◮ ▼♦r❡ ❡✣❝✐❡♥t✿ s❛✈❡s ❝♦♠♣✉t❛t✐♦♥s ✐♥ s♦❧✉t✐♦♥ ◮ ❈❛♥ ❜❡ ✉s❡❞ t♦ r❡❝♦✈❡r V t ❛t ( a , y ) � � a + y − � g t ( a , y ) + β � V t ( a , y ) = u ❊ V t + ✶ ( � g t ( a , y )) R ◮ ■❢ ❤❛❞ st♦r❡❞ V t ✱ st❡♣ ✹✳✶ ✇♦✉❧❞ ❝♦♠♣✉t❡ ✭❢♦r ❡❛❝❤ ( i , j , t ) ✮ � � � � n y � � a t + ✶ , y l � t + y j − a t + ✶ � g ij a i π l t = arg max u + β V t + ✶ R a t + ✶ ∈D ij l = ✶ t ✐♥✈♦❧✈✐♥❣ n y ✐♥t❡r♣♦❧❛t✐♦♥s ❢♦r ❡❛❝❤ a t + ✶ ❝❛❧❧❡❞ ❜② ♠❛①✐♠✐s❛t✐♦♥ r♦✉t✐♥❡

◆✉♠❡r✐❝❛❧ ✐♥t❡❣r❛t✐♦♥ ■ � b ◮ ❙✉♣♣♦s❡ ✇❡ ✇❛♥t t♦ ❝♦♠♣✉t❡ a f ( y ) π y ( y ) dy ✇❤❡r❡ ◮ π y ✐s t❤❡ ♣❞❢ ♦❢ ② ◮ t❤❡ ✈❛❧✉❡ ♦❢ f ✐s ❦♥♦✇♥ ✐♥ ♣♦✐♥ts y i ✐♥ ❣r✐❞ Y ◮ ❚❤❡ ♥✉♠❡r✐❝❛❧ ✐♥t❡❣r❛❧ ✐s ❛ s✐♠♣❧❡ ✇❡✐❣❤t❡❞ ❛✈❡r❛❣❡ ♦❢ f ♦✈❡r ❛ ❞✐s❝r❡t❡ s❡❧❡❝t❡❞ ❣r✐❞ � b n y � f ( y i ) w i f ( y ) π y ( y ) dy ≃ a i = ✶ ◮ ❚❤❡ s✐♠♣❧❡st ♣r♦❝❡❞✉r❡ ✭❚❛✉❝❤❡♥✮ ✶✳ ❉✐✈✐❞❡ t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ y ✐♥t♦ n y ❡q✉❛❧✲♣r♦❜❛❜✐❧✐t② ✐♥t❡r✈❛❧s✱ Y i ✷✳ ❈♦♠♣✉t❡ t❤❡ ❣r✐❞ ♣♦✐♥ts y i = ❊ ( y | Y i ) ✸✳ ❚❤❡ ✇❡✐❣❤ts ❛r❡ ✉♥✐❢♦r♠✿ w i = n − ✶ y � � n y Y f ( y ) π y ( y ) dy ≃ n − ✶ i = ✶ f ( y i ) ✹✳ ❚❤❡♥ y

◆✉♠❡r✐❝❛❧ ✐♥t❡❣r❛t✐♦♥ ■■ ❆❧t❡r♥❛t✐✈❡ ♣r♦❝❡❞✉r❡s � � ◮ ●❛✉ss✐❛♥ q✉❛❞r❛t✉r❡✿ ●❛✉ss✐❛♥ ♥♦❞❡s ❛♥❞ ✇❡✐❣❤ts ( y i , w i ) ❛r❡ s❡❧❡❝t❡❞ t♦ ♠❛❦❡ ❡①❛❝t t❤❡ ♥✉♠❡r✐❝❛❧ ✐♥t❡❣r❛❧ ♦❢ ♣♦❧②♥♦♠✐❛❧s ♦❢ ❞❡❣r❡❡ ✷ n y + ✶ ♦r ❧❡ss ◮ ●♦♦❞ ♦♣t✐♦♥ ✐❢ f ❝❛♥ ❜❡ ❝❧♦s❡❧② ❛♣♣r♦①✐♠❛t❡❞ ❜② ❛ ♣♦❧②♥♦♠✐❛❧ ◮ ❲❡✐❣❤ts ❛♥❞ ♥♦❞❡s ❞❡♣❡♥❞ ♦♥ t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ y ✿ ●❛✉ss✲▲❛❣✉❡rr❡ ❢♦r ♥♦r♠❛❧✱ ●❛✉ss✲❍❡r♠✐t❡ ❢♦r ❧♦❣✲♥♦r♠❛❧✱ ✳✳✳ � y i � ◮ ▼♦♥t❡✲❈❛r❧♦ s✐♠✉❧❛t✐♦♥s✿ ❞r❛✇ r❛♥❞♦♠❧② ❢r♦♠ ✐ts ❞✐str✐❜✉t✐♦♥ ❛♥❞ ❝♦♠♣✉t❡ s✐♠♣❧❡ ❛✈❡r❛❣❡ ♦❢ f ( y ) ❛t r❛♥❞♦♠ ♣♦✐♥ts

Pr❛❝t✐❝❛❧ ✐ss✉❡s ■■■ ◮ ❚❤❡ ❛❧❣♦r✐t❤♠ ✇❡ s♣❡❝✐✜❡❞ ✐s ✐♠♣❧✐❝✐t❧② ❞❡s✐❣♥❡❞ t♦ ✉s❡ ✇✐t❤ ❛ s❡❛r❝❤ ♠❡t❤♦❞ ◮ ❇✉t ❛❣❛✐♥ ✐t ❝❛♥ ❜❡ ♠♦r❡ ❡✣❝✐❡♥t ❛♥❞ ❛❝❝✉r❛t❡ t♦ ✉s❡ ❢♦❝

❋✐♥❞ r♦♦t ♦❢ ❊✉❧❡r ❡q✉❛t✐♦♥✿ ❈❘❘❆ ✉t✐❧✐t② ◮ ❆t ❡❛❝❤ ( a i t , y j , t ) ✜♥❞ r♦♦t ✭ a t + ✶ ✮ ♦❢ u ′ � � t + y j − a t + ✶ − β R � a i dV t + ✶ ( a t + ✶ ) = ✵ R ◮ ■♥✈❡rs❡ ♠❛r❣✐♥❛❧ ✉t✐❧✐t② r❡❞✉❝❡s ♥♦♥✲❧✐♥❡❛r✐t② ✐♥ ♠❛r❣✐♥❛❧ ✈❛❧✉❡ ◮ ❈❛♥ s♦❧✈❡ ❊✉❧❡r ❡q✉❛t✐♦♥ ✐♥ ✐ts q✉❛s✐✲❧✐♥❡❛r✐s❡❞ ✈❡rs✐♦♥ � � t + y j − a t + ✶ − ( β R ) − ✶ γ � a i ❧ dV t + ✶ ( a t + ✶ ) = ✵ R ✇❤❡r❡ t❤❡ q✉❛s✐✲❧✐♥❡❛r ❡①♣❡❝t❡❞ ♠❛r❣✐♥❛❧ ✈❛❧✉❡ ✭❧ dV ✮ ✐s st♦r❡❞   � � − γ − ✶ n y γ � t + ✶ + y j − g ij t + ✶ = ( u ′ ) − ✶ � �  t + ✶  ❧ dV i dV i a i π j = t + ✶ R j = ✶

❙t♦❝❤❛st✐❝ ♦♣t✐♠✐s❛t✐♦♥ ❆✉t♦❝♦rr❡❧❛t❡❞ ✐♥❝♦♠❡ ♣r♦❝❡ss

❆✉t♦❝♦rr❡❧❛t❡❞ ✐♥❝♦♠❡ ♣r♦❝❡ss ◮ ▼♦r❡ ✐♥t❡r❡st✐♥❣ ♠♦❞❡❧ ♦❢ ✐♥❝♦♠❡✿ ❆❘✭✶✮ ♣r♦❝❡ss ◮ ❲❡ ❛ss✉♠❡ ln y t = α + ρ ln y t − ✶ + e t ◮ y t ✐s ❛ ▼❛r❦♦✈ ♣r♦❝❡ss✿ ▼❛r❦♦✈ str✉❝t✉r❡ ♦❢ ❞②♥❛♠✐❝ ♣r♦❜❧❡♠ ♥♦t ❝♦♠♣r♦♠✐s❡❞ ◮ ❙t❛t✐♦♥❛r✐t② r❡q✉✐r❡s t❤❛t ✉♥❝♦♥❞✐t✐♦♥❛❧ ♣❞❢ ♦❢ y ✐s t✐♠❡✲✐♥✈❛r✐❛♥t ◮ ❙t❛t✐♦♥❛r✐t② ✉♥❞❡r ❧♦❣✲♥♦r♠❛❧✐t② r❡q✉✐r❡s | ρ | < ✶ ❛♥❞✱ ❢♦r ❛❧❧ t ◮ ❊ (ln y t ) = α ( ✶ − ρ ) − ✶ � ✶ − ρ ✷ � − ✶ ◮ ❱❛r (ln y t ) = σ ✷ e

❆✉t♦❝♦rr❡❧❛t❡❞ ✐♥❝♦♠❡ ♣r♦❝❡ss✿ ♠♦❞❡❧ ❚❤❡ ❝♦♥s✉♠♣t✐♦♥✲s❛✈✐♥❣s ♣r♦❜❧❡♠ ✐s ✭ D t ( a , y ) ❛s ❞❡✜♥❡❞ ❡❛r❧✐❡r✮ V t ( a t , y t ) � � � = max f ( a t , y t , a t + ✶ ) + β V t + ✶ ( a t + ✶ , y ρ t exp { α + e } ) dF e ( e ) a t + ✶ ∈D t ◮ ●❡♥❡r❛❧❧② ♥❡❡❞ t♦ ❜♦✉♥❞ ❞♦♠❛✐♥ ♦❢ e t♦ ❡♥s✉r❡ ❢❡❛s✐❜✐❧✐t② ❛♥❞ ♠❡❛s✉r❛❜✐❧✐t② ❛t ❛❧❧ ♣♦✐♥ts ◮ ❚❤❡ ❊✉❧❡r ❡q✉❛t✐♦♥ ✐s u ′ ( c t ) = β R ❊ t [ u ′ ( c t + ✶ ) | y t ]

❙✐♠♣❧❡ ❡①❛♠♣❧❡ ■ ◆♦t ♠♦st ❛♣♣❡❛❧✐♥❣ ✷✲♣❡r✐♦❞ ♠♦❞❡❧✳✳✳ ❜✉t ❝❛♥ ❜❡ s♦❧✈❡❞ ❡①♣❧✐❝✐t❧② ◮ P❡r✐♦❞ ✶✿ ❝♦♥s✉♠❡r ❡♥❞♦✇❡❞ ✇✐t❤ ( a ✶ , y ✶ ) ✱ ❝♦♥s✉♠❡s c ✶ ◮ P❡r✐♦❞ ✷✿ ◮ a ✷ = R ( a ✶ + y ✶ − c ✶ ) ◮ y ✷ = ρ y ✶ + e ✷ ◮ c ✷ = R ( a ✶ + y ✶ − c ✶ ) + ( ρ y ✶ + e ✷ ) ✇❤❡r❡ e ✷ ✐s ❛ r✈ ♦❢ ♠❡❛♥ ③❡r♦✱ ✉♥❦♥♦✇♥ ❢r♦♠ ♣❡r✐♦❞ ✶ ❛♥❞ ✉♥r❡❧❛t❡❞ t♦ ♦t❤❡r ♠♦❞❡❧ ✈❛r✐❛❜❧❡s ◮ ❯t✐❧✐t② ❢✉♥❝t✐♦♥✿ u ( c ) = δ ✵ + δ ✶ c + δ ✷ c ✷ ◮ ❈♦♥s✉♠❡rs ♣r♦❜❧❡♠✿ max { u ( c ✶ ) + β ❊ ✶ u [ R ( a ✶ + y ✶ − c ✶ ) + ( ρ y ✶ + e ✷ )] } c ✶

❙✐♠♣❧❡ ❡①❛♠♣❧❡ ■■ ◮ ❚❤❡ ❊✉❧❡r ❡q✉❛t✐♦♥ ✐s ✭✇✐t❤ β R = ✶✮ δ ✶ + δ ✷ ❊ [ R ( a ✶ + y ✶ − c ✶ ) + ( ρ y ✶ + e ✷ )] δ ✶ + δ ✷ c ✶ = δ ✶ + δ ✷ [ R ( a ✶ + y ✶ − c ✶ ) + ρ y ✶ ] = ◮ ❲✐t❤ s♦❧✉t✐♦♥ ✶ + R a ✶ + ρ + R R = c ✶ ✶ + R y ✶ ◮ ■❢ ρ = ✵✿ ✐♥❝♦♠❡ s❤♦❝❦s ❞♦ ♥♦t ♣❡rs✐st ❛♥❞ ❝♦♥s✉♠♣t✐♦♥ r❡s♣♦♥❞s ❧❡ss t♦ s❤♦❝❦s ◮ ■❢ ρ = ✶✿ ♣❡r♠❛♥❡♥t ✐♥❝♦♠❡ s❤♦❝❦s ❛♥❞ ❝♦♥s✉♠♣t✐♦♥ r❡s♣♦♥❞s ❢✉❧❧② t♦ s❤♦❝❦s

❙♦❧✉t✐♦♥ ❛❧❣♦r✐t❤♠ ✶✳ P❛r❛♠❡t❡r✐s❡ ♠♦❞❡❧ ❛♥❞ s❡❧❡❝t ❣r✐❞s ( A , Y ) ❛♥❞ ❝♦♠♣✉t❡ ✇❡✐❣❤ts Q jl ✷✳ ❈❤♦♦s❡ st♦♣♣✐♥❣ ❝r✐t❡r✐♦♥ ǫ > ✵ � t + ✶ , y j � a i ✸✳ ❙t♦r❡ ❊ V T + ✶ = ✵ ❢♦r ❛❧❧ i = ✶ , . . . , n a ❛♥❞ j = ✶ , . . . , n y ✹✳ ▲♦♦♣ ♦✈❡r t ❜❛❝❦✇❛r❞s✿ t = T , . . . , ✶ ▲♦♦♣ ♦✈❡r i = ✶ , . . . , n a ✹✳✶ ❈♦♠♣✉t❡ ❢♦r j = ✶ , . . . , n y � � � � a t + ✶ , y j �� t + y j − a t + ✶ + β � g ij a i t = arg max u ❊ V t + ✶ R a t + ✶ ∈D ij t ✹✳✷ ❈♦♠♣✉t❡ t❤❡ ❝♦♥t✐♥✉❛t✐♦♥ ✈❛❧✉❡ ❛t ♣♦✐♥t ( a t , y t − ✶ ) = ( a i t , y l ) � � � t , y j �� � � t + y j − g ij + β � ❊ V il a i t g ij Q lj = u ❊ V t + ✶ t R j = ✶ ,..., n y

Pr❛❝t✐❝❛❧ ✐ss✉❡s ◮ ❚❤❡ ❝♦♥t✐♥✉❛t✐♦♥ ✈❛❧✉❡ ❛t t✐♠❡ t ✐s ❊ t [ V t + ✶ ( a t + ✶ , y t + ✶ ) | y t ] ✱ ❛ ❢✉♥❝t✐♦♥ ♦❢ ( a t + ✶ , y t ) ◮ ■❢ t❤❡ ❢♦❝ ✇❡r❡ t♦ ❜❡ ✉s❡❞ ✐♥ t❤❡ s♦❧✉t✐♦♥✱ t❤❡ ❧✐♥❡❛r✐s❡❞ ❡①♣❡❝t❡❞ ♠❛r❣✐♥❛❧ ✈❛❧✉❡ ✐♥ t✐♠❡ t ❊✉❧❡r ❡q✉❛t✐♦♥ ✇♦✉❧❞ ❛❧s♦ ❜❡ ❛ ❢✉♥❝t✐♦♥ ♦❢ ( a t + ✶ , y t ) ◮ P❡rs✐st❡♥❝② ✐♥ y t ✐♠♣❧✐❡s t❤❛t t❤❡ ✐♥t❡❣r❛t✐♦♥ ✇❡✐❣❤ts Q ♥❡❡❞ t♦ ❜❡ ❝♦♥❞✐t✐♦♥❛❧ ♦♥ t❤❡ ♣❛st r❡❛❧✐s❛t✐♦♥ ♦❢ y

❚r❛♥s✐t✐♦♥ ❢✉♥❝t✐♦♥✿ s✐♠♣❧❡ ♣r♦❝❡❞✉r❡ t♦ ❞❡t❡r♠✐♥❡ Q ji ◮ ❈♦♥s✐❞❡r ❛ st❛t✐♦♥❛r② ▼❛r❦♦✈ ♣r♦❝❡ss e ∼ N ( ✵ , σ ✷ ) x t = α + ρ x t − ✶ + e t ✇❤❡r❡ ◮ ❆ s✐♠♣❧❡ ♣r♦❝❡❞✉r❡ t♦ ❝♦♠♣✉t❡ Q jl � x i , x i � ✶✳ ❉✐✈✐❞❡ t❤❡ ❞♦♠❛✐♥ X ✐♥ n x ✐♥t❡r✈❛❧s { X i = } ✷✳ ❈♦♠♣✉t❡ t❤❡ ❣r✐❞ ♣♦✐♥ts x i = ❊ ( x i | x i ∈ X i ) ✸✳ ❚❤❡♥ � x t ∈ X i | x t − ✶ = x j � Q ji = Pr♦❜ � x i ≤ α + ρ x j + e t ≤ x i � = Pr♦❜ � x i − α − ρ x j ≤ e t ≤ x i − α − ρ x j � = Pr♦❜ � x i − α − ρ x j � � x i − α − ρ x j � = Φ − Φ σ σ

Pr❛❝t✐❝❛❧ s❡ss✐♦♥ ✸

■♥✜♥✐t❡ ❤♦r✐③♦♥ ❚❤❡ ♣r♦❜❧❡♠