An asynchronous parallel derivative-free algorithm for handling general constraints Josh Griffin Computational Sciences and Mathematics Research Sandia National Laboratories Livermore, California USA Second International Congress on Mathematical Software Castro Urdiales, SPAIN September 1–3, 2006 Joint work with Tammy Kolda, Robert Michael Lewis, and Virginia Torczon Computational Sciences and Mathematics Research Slide 1 September 2, 2006
Talk outline 1. Problems of interest 2. Generating set search background 3. Linear constraints 4. Nonlinear equality constraints 5. Numerical results Computational Sciences and Mathematics Research Slide 2 September 2, 2006
Why use derivative-free? Should I ? or the take the Derivative-based if ... Derivative-free if ... • Function evaluations quick • Function evaluations slow • All points finite/defined • Points may be undefined • Continous and smooth • Discontinous, nonsmooth, okay • Little to no noise • Noise okay Punchline Derivative-free methods more reliable, less restrictive Computational Sciences and Mathematics Research Slide 3 September 2, 2006
Problems we are interested in • Function evaluations are CPU-intensive • Simulation-based objective function can periodically crash • Noise limits ability to estimate derivatives • No analytic formula for objective function Computational Sciences and Mathematics Research Slide 4 September 2, 2006
Optimization in nuclear safety studies 1. Question: Could accidental drop jeopardize integrity of internal components? 2. Plan: Drop model from different angles 3. Goal: Find angle that maximizes damage Resulting Problem: maximize D ( x ) x ∈ R 2 subject to 0 ≤ x i ≤ π • D ( x ) is measure of damage • Time per evaluation: 1-15 hrs • Software entities: 3 Computational Sciences and Mathematics Research Slide 5 September 2, 2006
Generating Set Search and APPSPACK Computational Sciences and Mathematics Research Slide 6 September 2, 2006
Problem types for APPSPACK minimize f ( x ) x ∈ R n subject to c ( x ) = 0 Ax ≤ b Here f : R n → R , c : R n → R p , and A is an m × n matrix. • linear equalities permitted • derivatives for f ( x ) and c ( x ) unavailable • number of variables relatively small ( ≤ 100) Computational Sciences and Mathematics Research Slide 7 September 2, 2006
Generating set search algorithms General idea: Use set of positively spanning search directions Two examples: Guaranteed: Search direction within 90 ◦ of steepest descent direction Punchline: Always have a descent direction if one exists Computational Sciences and Mathematics Research Slide 8 September 2, 2006
Generating set search algorithms General idea: Use set of positively spanning search directions Two examples: Guaranteed: Search direction within 90 ◦ of steepest descent direction Punchline: Always have a descent direction if one exists Computational Sciences and Mathematics Research Slide 8 September 2, 2006
Generating set search algorithms General idea: Use set of positively spanning search directions Two examples: Guaranteed: Search direction within 90 ◦ of steepest descent direction Punchline: Always have a descent direction if one exists Computational Sciences and Mathematics Research Slide 8 September 2, 2006
Synchronous framework (unconstrained) while ( ∆ > ∆ tol ) 1. Generate trial-points and send to evaluation queue: X = { x + ∆ d ( i ) : d ( i ) ∈ search pattern } 2. Collect evaluated trial points: Y = X 3. Update: Is there a point y ∈ Y better than x ? Yes: x ← y (successful) No: ∆ ← . 5∆ (unsuccessful) end Computational Sciences and Mathematics Research Slide 9 September 2, 2006
Synchronous framework (unconstrained) while ( ∆ > ∆ tol ) 1. Generate trial-points and send to evaluation queue: X = { x + ∆ d ( i ) : d ( i ) ∈ search pattern } 2. Collect evaluated trial points: Y = X 3. Update: Is there a point y ∈ Y better than x ? ■ ❅ Yes: x ← y (successful) ❅ ❅ No: ∆ ← . 5∆ (unsuccessful) We enforce a sufficient decrease end conditions based on step size ∆ f ( y ) ≤ f ( x ) − α ∆ 2 Computational Sciences and Mathematics Research Slide 9 September 2, 2006
Synchronous framework (unconstrained) while ( ∆ > ∆ tol ) 1. Generate trial-points and send to evaluation queue: X = { x + ∆ d ( i ) : d ( i ) ∈ search pattern } ✛ 2. Collect evaluated trial points: Y = X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3. Update: Is there a point y ∈ Y better than x ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yes: x ← y (successful) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step where asynchronous No: ∆ ← . 5∆ (unsuccessful) algorithms wins in parallel end Y � = X Computational Sciences and Mathematics Research Slide 9 September 2, 2006
Asynchronous framework (unconstrained) while ( ∆ ( i ) > ∆ tol ) 1. Generate trial-points and send to evaluation queue: X = { x + ∆ ( i ) d ( i ) : d ( i ) ∈ search pattern and inactive } 2. Collect nonempty set of evaluated trial points: Y � = X 3. Update: Is there a point y ∈ Y better than x ? Yes: x ← y , ∆ ( i ) = max(∆ y , ∆ min ) Successful: may prune queue No: ∆ ( i ) ← . 5∆ ( i ) for ”evaluated” indices Unsuccessful: may not prune queue Computational Sciences and Mathematics Research Slide 10 September 2, 2006
Asynchronous framework (unconstrained) while ( ∆ ( i ) > ∆ tol ) 1. Generate trial-points and send to evaluation queue: X = { x + ∆ ( i ) d ( i ) : d ( i ) ∈ search pattern and inactive } 2. Collect nonempty set of evaluated trial points: Y � = X 3. Update: Is there a point y ∈ Y better than x ? Yes: x ← y , ∆ ( i ) = max(∆ y , ∆ min ) Successful: may prune queue No: ∆ ( i ) ← . 5∆ ( i ) for ”evaluated” indices Unsuccessful: may not prune queue Computational Sciences and Mathematics Research Slide 11 September 2, 2006
Asynchronous framework (unconstrained) while ( ∆ ( i ) > ∆ tol ) 1. Generate trial-points and send to evaluation queue: X = { x + ∆ ( i ) d ( i ) : d ( i ) ∈ search pattern and inactive } 2. Collect nonempty set of evaluated trial points: Y � = X 3. Update: Is there a point y ∈ Y better than x ? Yes: x ← y , ∆ ( i ) = max(∆ y , ∆ min ) Successful: may prune queue No: ∆ ( i ) ← . 5∆ ( i ) for ”evaluated” indices Unsuccessful: may not prune queue Computational Sciences and Mathematics Research Slide 12 September 2, 2006
Unconstrained optimization demo Trial points ✄ ❆ best: a ✄ ❆ ✄ ❆ ❆ ❯ pending: b c d e ✄ Current best point ✄ � evaluated: ✄ � pruned: ✄ � � ✠ ✎ ✄ � �� � ∆ ❅ ■ ❅ Step size Computational Sciences and Mathematics Research Slide 13 September 2, 2006
Unconstrained optimization demo best: a pending: b c d e evaluated: pruned: Computational Sciences and Mathematics Research Slide 14 September 2, 2006
Unconstrained optimization demo best: a pending: c d evaluated: b e pruned: Computational Sciences and Mathematics Research Slide 15 September 2, 2006
Unconstrained optimization demo best: a pending: f g c d evaluated: pruned: Computational Sciences and Mathematics Research Slide 16 September 2, 2006
Unconstrained optimization demo best: a pending: c d evaluated: f g pruned: Computational Sciences and Mathematics Research Slide 17 September 2, 2006
Unconstrained optimization demo best: f pending: h i j k c d evaluated: pruned: Computational Sciences and Mathematics Research Slide 18 September 2, 2006
Unconstrained optimization demo best: f pending: i k evaluated: c j h pruned: d ① Computational Sciences and Mathematics Research Slide 19 September 2, 2006
Unconstrained optimization demo best: c pending: l m n o i k evaluated: pruned: Computational Sciences and Mathematics Research Slide 20 September 2, 2006
Unconstrained optimization demo best: c pending: n k evaluated: l m o i pruned: Computational Sciences and Mathematics Research Slide 21 September 2, 2006
Unconstrained optimization demo best: l pending: p q r s n k evaluated: pruned: Computational Sciences and Mathematics Research Slide 22 September 2, 2006
Unconstrained optimization demo best: l pending: p q r s evaluated: n k pruned: Computational Sciences and Mathematics Research Slide 23 September 2, 2006
Unconstrained optimization demo best: l pending: p q r s evaluated: pruned: Computational Sciences and Mathematics Research Slide 24 September 2, 2006
Recommend
More recommend
