An introduction to the theory of SDDP algorithm V. Lecl` ere (ENPC) - PowerPoint PPT Presentation

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion An introduction to the theory of SDDP algorithm V. Lecl` ere (ENPC) August 1, 2014 V. Lecl` ere Introduction to SDDP August 1, 2014 1 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Introduction Large scale stochastic problem are hard to solve. Two ways of attacking such problems : decompose (spatially) the problem and coordinate solutions, construct easily solvable approximations (Linear Programming). Behind the name SDDP there is three different things: a class of algorithms, a specific implementation of the algorithm, a software implementing this method develloped by PSR. The aim of this talk is to give you an idea of how the class of algorithm is working and give a convergence result. V. Lecl` ere Introduction to SDDP August 1, 2014 2 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Introduction Large scale stochastic problem are hard to solve. Two ways of attacking such problems : decompose (spatially) the problem and coordinate solutions, construct easily solvable approximations (Linear Programming). Behind the name SDDP there is three different things: a class of algorithms, a specific implementation of the algorithm, a software implementing this method develloped by PSR. The aim of this talk is to give you an idea of how the class of algorithm is working and give a convergence result. V. Lecl` ere Introduction to SDDP August 1, 2014 2 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Introduction Large scale stochastic problem are hard to solve. Two ways of attacking such problems : decompose (spatially) the problem and coordinate solutions, construct easily solvable approximations (Linear Programming). Behind the name SDDP there is three different things: a class of algorithms, a specific implementation of the algorithm, a software implementing this method develloped by PSR. The aim of this talk is to give you an idea of how the class of algorithm is working and give a convergence result. V. Lecl` ere Introduction to SDDP August 1, 2014 2 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion An introductive application A dam is seen as a stock of energy with uncertain inflows, valorized at an uncertain price. The objective is to minimize the expected cumulated cost. V. Lecl` ere Introduction to SDDP August 1, 2014 3 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Contents Technical Preliminaries 1 Problem Formulation and Dynamic Programming Duality Approach SDDP Algorithm 2 The SDDP algorithm Miscellaneous Convergence and Numerical Results 3 Conclusion 4 V. Lecl` ere Introduction to SDDP August 1, 2014 3 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Problem Formulation � � � � c T c T min 0 x 0 + E min 1 ( ξ 1 ) x 1 + E · · · A 0 x 0 = b 0 B 1 ( ξ 1 ) x 0 + A 1 ( ξ 1 ) x 1 = b 1 x 0 ≥ 0 x 1 ≥ 0 ����� � c T + E min T ( ξ T ) x T B T ( ξ T ) x T − 1 + A T ( ξ T ) x T = b T ( ξ T ) x T ≥ 0 � � Key “practical” assumption: the noise ξ 1 , . . . , ξ T is independent from time to time. If it is not the case we can often extend the state x to obtain an independent noise V. Lecl` ere Introduction to SDDP August 1, 2014 4 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Problem Formulation � � � � c T c T min 0 x 0 + E min 1 ( ξ 1 ) x 1 + E · · · A 0 x 0 = b 0 B 1 ( ξ 1 ) x 0 + A 1 ( ξ 1 ) x 1 = b 1 x 0 ≥ 0 x 1 ≥ 0 ����� � c T + E min T ( ξ T ) x T B T ( ξ T ) x T − 1 + A T ( ξ T ) x T = b T ( ξ T ) x T ≥ 0 � � Key “practical” assumption: the noise ξ 1 , . . . , ξ T is independent from time to time. If it is not the case we can often extend the state x to obtain an independent noise V. Lecl` ere Introduction to SDDP August 1, 2014 4 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Dynamic Programming Approach This problem can be solved by Dynamic Programming with: c T Q T − 1 ( x T − 1 , ξ T ) = min T ( ξ T ) x T B T ( ξ T ) x T − 1 + A T ( ξ T ) x T = b T ( ξ T ) x T ≥ 0 � � V T − 1 = E Q T − 1 ( x T − 1 , ξ T ) c T Q t − 1 ( x t − 1 , ξ t ) = min t x t + V t ( x t ) B t ( ξ t ) x t − 1 + A t ( ξ t ) x t = b t ( ξ t ) x t ≥ 0 � � V t − 1 ( x t − 1 ) = E Q t − 1 ( x t − 1 , ξ t ) Note that: the resolution can be done backward in time but require a priori discretization; functions Q t and V t are convex in x ; functions Q t and V t are pieciewise linear, hence above problems are linear programming problems. V. Lecl` ere Introduction to SDDP August 1, 2014 5 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Dynamic Programming Approach This problem can be solved by Dynamic Programming with: c T Q T − 1 ( x T − 1 , ξ T ) = min T ( ξ T ) x T B T ( ξ T ) x T − 1 + A T ( ξ T ) x T = b T ( ξ T ) x T ≥ 0 � � V T − 1 = E Q T − 1 ( x T − 1 , ξ T ) c T Q t − 1 ( x t − 1 , ξ t ) = min t x t + V t ( x t ) B t ( ξ t ) x t − 1 + A t ( ξ t ) x t = b t ( ξ t ) x t ≥ 0 � � V t − 1 ( x t − 1 ) = E Q t − 1 ( x t − 1 , ξ t ) Note that: the resolution can be done backward in time but require a priori discretization; functions Q t and V t are convex in x ; functions Q t and V t are pieciewise linear, hence above problems are linear programming problems. V. Lecl` ere Introduction to SDDP August 1, 2014 5 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Contents Technical Preliminaries 1 Problem Formulation and Dynamic Programming Duality Approach SDDP Algorithm 2 The SDDP algorithm Miscellaneous Convergence and Numerical Results 3 Conclusion 4 V. Lecl` ere Introduction to SDDP August 1, 2014 5 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Subgradient of a Value Function 1/2 Consider the following problem: c T Q t ( x , ξ ) = min t ( ξ ) x t + V t +1 ( x t +1 ) x t , x t +1 s . t . x t = x � λ t ( ξ t ) B t ( ξ ) x t − 1 + A t ( ξ ) x t = b t ( ξ ) Note λ t ( ξ t ) the Lagrangian multiplier associated to the constraint x t = x . Marginalist interpretation of the multiplier (and convexity of Q t ) implies that λ t ( ξ t ) ∈ ∂ Q t ( x t , ξ ) is a subgradient of Q t ( · , ξ ). In other words � � Q t ( · , ξ ) ≥ Q t ( x t , ξ ) + λ t , · − x t . V. Lecl` ere Introduction to SDDP August 1, 2014 6 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Subgradient of a Value Function 2/2 We have: � � ∀ x , Q t ( x , ξ ) ≥ Q t ( x t , ξ ) + λ t , x − x t . Recall that � � V t ( x ) = E Q t ( x , ξ ) . By linearity we obtain � � � � � � V t ( · ) ≥ E Q t ( x t , ξ ) + λ t ( ξ t ) , · − x t . E V. Lecl` ere Introduction to SDDP August 1, 2014 7 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Constructing a Cut from a Lower Approximation 1/2 V ( k ) t +1 ≤ V t +1 . Choose a point x ( k ) Assume that we have ˇ . t Recall that c T Q t ( x t , ξ ) = min t +1 x t + V t +1 ( x t +1 ) B t +1 ( ξ ) x t + A t +1 ( ξ ) x t +1 = b t +1 ( ξ ) x t +1 ≥ 0 For any possible ξ , consider V ( k ) β ( k ) ( ξ ) = min c T t ( ξ ) x t + ˇ t +1 ( x t +1 ) x t , x t +1 x t = x ( k ) � λ ( k ) ( ξ ) s . t . t t B t ( ξ ) x t − 1 + A t ( ξ ) x t = b t ( ξ ) Hence we have, β ( k ) λ t ( ξ ) , · − x ( k ) � � ( ξ ) + ≤ Q t ( · , ξ ) . t t then, β ( k ) , · − x ( k ) � � � � � � ( ξ t ) + λ t ( ξ t ) ≤ V t ( · ) . E E t t V. Lecl` ere Introduction to SDDP August 1, 2014 8 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Constructing a Cut from a Lower Approximation 2/2 Recall that, � � V t ( x ) = E Q t ( x , ξ t +1 ) . Hence β ( k ) λ t ( ξ ) , · − x ( k ) � � ( ξ ) + ≤ Q t ( · , ξ ) , t t leads to β ( k ) , · − x ( k ) � � � � � � E ( ξ t ) + E λ t ( ξ t ) ≤ V t ( · ) t t V. Lecl` ere Introduction to SDDP August 1, 2014 9 / 21

Technical Preliminaries SDDP Algorithm Convergence and Numerical Results Conclusion Contents Technical Preliminaries 1 Problem Formulation and Dynamic Programming Duality Approach SDDP Algorithm 2 The SDDP algorithm Miscellaneous Convergence and Numerical Results 3 Conclusion 4 V. Lecl` ere Introduction to SDDP August 1, 2014 9 / 21