MINLP: Undecidability and Hardness A tutorial Leo Liberti, CNRS LIX - PowerPoint PPT Presentation

MINLP: Undecidability and Hardness A tutorial Leo Liberti, CNRS LIX Ecole Polytechnique liberti@lix.polytechnique.fr The Aussois COW, 2017 1 / 49

Section 1 Introduction 2 / 49

Mixed-Integer Nonlinear Programming MINLP ◮ Formal declarative language sentences describe optimization problems ◮ Can encode pure feasibility problems by minimizing a constant function e.g. min { 0 | g ( x ) ≤ 0 } ◮ Includes most other MP classes e.g. LP, MILP, NLP ◮ Interpreter = solver shifts focus from algorithmics to modelling ◮ Only consider single-objective MP 3 / 49

Syntax Given functions f, g 1 , . . . , g m : Q n → Q and Z ⊆ { 1 , . . . , n }  min f ( x )  ∀ i ≤ m g i ( x ) ≤ 0 ∀ j ∈ Z x j ∈ Z  ◮ f, g i represented by expression DAGs E.g. min { x 1 + 2 x 2 − log( x 1 x 2 ) | x 1 x 2 2 ≥ 1 ∧ x 1 ≥ 0 ∧ x 2 ∈ N } 4 / 49

Semantics P ≡ min { x 1 + 2 x 2 − log( x 1 x 2 ) | x 1 x 2 2 ≥ 1 ∧ x 1 ≥ 0 ∧ x 2 ∈ N } � P � = ( opt ( P ) , val ( P )) opt ( P ) = (1 , 1) val ( P ) = 3 5 / 49

“Solving an MP” ◮ Given an MP P , there are three possibilities: 1. � P � exists 2. P is unbounded 3. P is infeasible ◮ P has a feasible solution iff � P � exists or is unbounded otherwise it is infeasible ◮ P has an optimum iff � P � exists otherwise it is infeasible or unbounded ◮ Asymmetry between optimization and feasibility � YES ∃ � P � ∨ unbnd ( P ) P ≡ min { 0 | g ( x ) ≤ 0 ∧ x ∈ X } NO infeas ( P ) � YES ∃ � Q � Q ≡ min { f ( x ) | g ( x ) ≤ 0 ∧ x ∈ X } NO unbnd ( Q ) ∨ infeas ( Q ) 6 / 49

Section 2 Undecidability 7 / 49

Formal systems (FS) ◮ Formal System F ◮ alphabet and formal grammar well-formed formulæ and sentences ◮ Axioms A (recursive 1 consistent set of sentences) ◮ Inference rules R derive new sentences from old ones ◮ Language L set of all sentences of F ◮ Theory T sentences obtained by iterated application of R to A 1 M recursive if ∃ alg. solving “given a , is a ∈ M or not?” 8 / 49

Some FSs ◮ Peano Arithmetic ( PA ): →↔ , ∧ , ∨ , ¬ , ∀ , ∃ , + , × , = and variable names; 1st order sentences about N ; A = PL + induction; modus ponens and generalization ◮ T : provable sentences about N ◮ Real-closed Fields ( RLF ): like above and > ; polynomials over R ; field axioms for R , “basic operations on polynomials” ◮ T : polynomial systems over R with solution in R ◮ Diophantine Equations ( DE ): existentially quantified subset of PA ◮ T : polynomial systems over Z with solution in N ◮ { [ ∃ x ∈ N n p ( x ) = 0] | p ∈ Z [ x ] } ≡ { [ f ∧ x ∈ N n ] | f ∈ L ( RLF ) } ◮ “between PA and RLF ” 9 / 49

What is decidability? FS F is decidable if ∃ algorithm A : L → { 0 , 1 } � 1 if f ∈ T ∀ f ∈ L A ( f ) = otherwise 0 ◮ PA : does a sentence have a proof in PA or not? ◮ RLF : does a polynomial over R have a solution in R or not? ◮ DE : does a polynomial over Z have a solution in N or not? Only YES/NO answer required (rather than explicit proof) 10 / 49

Subsection 1 Polynomial systems in integer variables 11 / 49

DE : Relevance to MINLP ◮ DE ⊆ [MINLP feasibility] obvious ◮ DE ⊆ [MINLP optimality] � ∀ i ≤ m g i ( x ) = 0 is feasible N n ∈ x u ∗ = min u      ( g i ( x )) 2  (1 − u ) � = 0 ⇔ = 0 i ≤ m   N n +1 ( x, u ) ∈   ◮ if u ∗ > 1 get − 1 = 0 (contradiction) ◮ if u ∗ = 1 get g ( x ) � = 0 (infeasible) ◮ if u ∗ = 0 get g ( x ) = 0 (feasible) Suppose u ∗ = 1 and g ( x ) = 0 feas., then u = 0 would also satisfy constr. and contradict minimality of u ∗ 12 / 49

Goal: MINLP is undecidable ◮ MINLP contains DE ◮ show DE is undecidable ◮ any r.e. 2 subset of N can be encoded by a DE ◮ { a ∈ N | a ∈ Halting } is r.e. ◮ so Halting can be represented by a DE ◮ if every DE were decidable, we could solve Halting 2 M ⊆ N is r.e. if ∃ alg. terminating on input a iff a ∈ M (nonterm. for a �∈ M ) 13 / 49

DE and set membership ◮ DE sentence [ p ( y ) = 0 ∧ y ∈ N n +1 ] y = ( a, x ) where a ∈ N “encodes the instance” ◮ DEs define subsets M ⊆ N : ∃ x ∈ N n p ( a, x ) = 0 a ∈ M ↔ ( † ) ◮ Conversely, given M ⊆ N , is there p ( a, x ) ∈ Z [ a, x ] s.t. ( † )? ◮ Focus on r.e. sets M ⊆ N ◮ Matyiasevich-Davis-Putnam-Robinson thm. (MDPR, 1970): For each r.e. set M there is a DE p ( a, x ) s.t. ( † ) 14 / 49

MDPR implies undecidability ◮ Suppose DE is decidable ◮ ⇒ given DE, can decide feasibility/infeas. ◮ ⇒ given r.e. set M , can decide a ∈ M and a �∈ M ◮ Halting : given TM T and input ι , will T ( ι ) terminate? undecidable by [Turing 1936] (diagonal argument) ◮ Let H = { ( T, ι ) | T ( ι ) ↓} H is r.e.: simulate T with input ι , terminates iff T ( ι ) ↓ M H = encoding of H in N ⇒ M H is r.e. ◮ ⇒ can decide M H and solve Halting , contradiction ◮ Hence DE undecidable ◮ ∃ Universal Diophantine Equations (UDE) encoding the dynamics of a UTM ∃ UDE deg d and n vars where ( d, n ) ∈ { (4 , 58) , (1 . 6 × 10 45 , 9) } 15 / 49

Structure of the MDPR theorem ◮ Proof of Gödel’s 1st incompleteness thm. r.e. sets ≡ DE with < ∞ ∃ and bounded ∀ quantifiers ◮ Davis’ normal form one bounded quantifier suffices : ∃ x 0 ∀ a ≤ x 0 ∃ x p ( a, x ) = 0 ◮ (2 bnd qnt ≡ 1 bnd qnt on pairs) and induction ◮ Robinson’s idea get rid of universal quantifier by using exponent vars � � ◮ idea : [ ∃ x 0 ∀ a ≤ x 0 ∃ x p ( a, x ) = 0] “ → ” ∃ x � p ( a, x ) a ≤ x 0 ◮ precise encoding needs variables in exponents ◮ Matyiasevic’s contribution express c = b a using polynomials ◮ use Pell’s equation x 2 − dy 2 = 1 √ √ ◮ solutions ( x n , y n ) satisfy x n + y n d ) n d = ( x 1 + y 1 ◮ x n , y n grow exponentially with n 16 / 49

Subsection 2 Polynomial systems in continuous variables 17 / 49

RLF : Relevance to MINLP ◮ RLF ⊇ [poly NLP feasibility] obvious ◮ RLF ⊆ [poly NLP optimality] is feasible ∀ i ≤ m g i ( x ) = 0 α ∗ = min u 2 � � ⇔ = 0 (1 − u 2 ) � ( g i ( x )) 2 = 0 i ≤ m ◮ if α ∗ > 1 get − 1 = 0 (contradiction) ◮ if α ∗ > 0 get g ( x ) � = 0 (infeasible) ◮ if α ∗ = 0 get g ( x ) = 0 (feasible) 18 / 49

Goal 1: [poly NLP feas.] is decidable ◮ RLF contains [poly NLP feasibility] ◮ RLF decidable ◮ ⇒ [poly NLP feasibility] is decidable 19 / 49

Goal 2: [poly NLP] is decidable ◮ [poly NLP]: we need to tell optimality and unboundedness apart ◮ RLF also includes universal quantifiers ◮ P ≡ min { f ( x ) g ( x ) ≤ 0 } unbounded: ∀ y f ( x ) = y ∧ g ( x ) ≤ 0 ◮ � P � exists: f ( x ) = y ∧ g ( x ) ≤ 0 ∧ ¬ unbounded ( P ) ∃ y ◮ P infeasible: ¬∃ y f ( x ) = y ∧ g ( x ) ≤ 0 20 / 49

Decidability of RLF ◮ RLF sentence [ p ( x ) R 0] : p ∈ R [ x ] , x ∈ R n , R ∈ { = , < } p ( x ) < 0 ← → p ( x ) − y = 0 ∧ y < 0 p ( x ) − y 2 = 0 p ( x ) ≤ 0 ← → ( p i ( x )) 2 = 0 ∀ i ≤ m p i ( x ) = 0 ← → � i ≤ m open constraints y < 0 invalid in MP, need not bother ◮ ∃ ? alg. for deciding if any p ( x ) = 0 solves R or not? ◮ RLF is decidable by quantifier elimination [Tarski 1948] ◮ Quantifier elimination: ◮ constructs solution sets (YES) or derives contradictions (NO) ◮ ⇒ RLF is complete, too ◮ think of Fourier-Motzkin elimination for linear RLF 21 / 49

Example: quantifier elimination in R 1 ◮ DLO (Dense Linear Order): 0 , 1 , ¬ , ∨ , ∧ , ∃ , ∀ , <, = , vars; quantifiers over R ◮ Reduce to form ∃ x � q i i ≤ m where all q i ’s have form x = v , x < v , or x > v ( v var/const) ◮ x = x can be removed from conjunction ◮ x < x : sentence is false ( and there’s a proof! ) ◮ if v x differ, rewrite ∃ x x = v ∧ r ( x, v ) ↔ ∃ x r ( x, x ) back to previous case ◮ remaining case: � q i is � � ( u i < x ) ∧ ( x < v i ) i i rewrite as � i u i < v i ◮ get [ ∃ x ¯ where ¯ q does not involve x ] or contradiction q ◮ repeat until only constants in R left get proof of YES and proof of NO DLO is decidable and complete 22 / 49

Rationals ◮ [Robinson 1949]: RT (1st ord. theory over Q ) is undecidable ◮ [Pheidas 2000]: existential theory of Q ( ERT ) is open can we decide wether p ( x ) = 0 has solutions in Q ? Boh! ◮ [Matyiasevich 1993]: ◮ equivalence between DEH and ERT ◮ DEH = [ DE restricted to homogeneous polynomials] ◮ but we don’t know whether DEH is decidable Note that Diophantus solved DE in positive rationals 23 / 49

Subsection 3 Digressions 24 / 49

Proof complexity bounds with UDEs The following surprising bound is due to [Jones 1982] For any axiomatizable theory T in PA1 and any sentence p ∈ T , if p has a proof in T , then it has a proof consisting of 100 additions and multiplications of integers ◮ Gödel numbering: T − → r.e. subset of N ◮ Search for proofs ← → search for DE solutions solution encodes whole proof ◮ ∃ UDE the evaluation of which takes 100 + , × operations ◮ Any solution of the UDE can be verified in at most 100 operations 25 / 49