Computational Complexity (a quick introduction) Tiling puzzles Can - - PowerPoint PPT Presentation
Computational Complexity (a quick introduction) Tiling puzzles Can you write a computer program which, given any finite set T of tile types, . e.g., . decides whether T tiles the infinite N N grid ? the finite n n grid ? what is
Can you write a computer program which, given any finite set T of tile types, e.g., . . decides whether T tiles – the infinite N × N grid ? . . – the finite n × n grid ?
what is the run-time of the algorithm (in terms of |T | and n)? is there a faster algorithm?
Computational complexity 1
Garey & Johnson, 1979 Computers and Intractability: A Guide to the Theory of NP-Completeness
Computational complexity 2
Garey & Johnson, 1979 Computers and Intractability: A Guide to the Theory of NP-Completeness
Computational complexity 3
Garey & Johnson, 1979 Computers and Intractability: A Guide to the Theory of NP-Completeness
Computational complexity 4
two-way infinite read/write tape
1 1 q Σ tape alphabet (e.g., {0, 1}) Q states (initial state q0, accepting state qa, rejecting state qr) δ : Σ × Q → 2Σ×Q×{−1,0,1} transition relation (a′, q′, d) ∈ δ(a, q): when in state q scanning symbol a, write a′ on that tape cell, move the head in direction d and enter state q′ nondeterministic!
NDTM
deterministic if |δ(a, q)| ≤ 1
DTM
configuration = contents of the tape + state
e.g., 01q1
Computational complexity 5
TMs recognise languages, i.e., sets of words
(a word is a sequence of letters)
initial configuration: initial state q0 and input word w written in the first |w| cells
(the rest is blank)
c0 c′
1
c′′
1
δ(a, q) = {(a′, q′, d′), (a′′, q′′, d′′)}
branches may be infinite, lead to the accepting state qa
M accepts w if there is a path in the computation tree leading to an accepting configuration
a configuration with the accepting state qa
L(M) is the set of all inputs accepted by M (the language of M)
Computational complexity 6
– NDTMs are not more powerful than DTMs (they accept they same languages)
DTMs can simulate breadth-first search of NDTMs trees
– but NDTMs can be exponentially more efficient than DTMs “Whether this exponential loss is inherent or an artefact of our limited understanding of nondeterminism is the famous P = NP ? problem”
(Christos Papadimitriou)
NDTMs are not a ‘true’ model of computation; it is a very useful abstraction
– All other known deterministic models of computation are polynomially reducible to DTMs
Not clear for quantum computers, though
Computational complexity 7
The Church-Turing thesis says that any algorithmic procedure that can be carried out by a human being or by a computer can also be carried out by a Turing machine This is not a theorem that can be proved. Why is that? The Church-Turing thesis establishes a connection between the precise, formal notion of a Turing machine and the intuitive, informal notion of an algorithm. It cannot be proved because it is not, in fact can’t be, stated in a precise, formal way. In can rather be considered as a definition of the term ‘algorithm’: an algorithmic procedure is what can be carried out by a Turing machine
Computational complexity 8
Why should we believe that the thesis is true? Here are some arguments: (1) Turing machines can do many different kinds of algorithmic procedures: compute functions, decide languages, do iterations, simulate finite au- tomata, simulate Turing machines, etc. In fact, no one has yet found an algorithm that happened to be unimplementable by a Turing machine (2) Extensions with multiple tapes and/or heads, non-determinism, counters,
Turing machines. (3) All other suggested theoretical models of computation have turned out to be provably equivalent to Turing machines.
Computational complexity 9
In principle, yes. It can happen in the future that somebody finds some kind of a procedure
else), and
mented by a Turing machine. So far no one has yet found a procedure that happened to be widely ac- cepted as an algorithm and unimplementable by a Turing machine at the same time.
Computational complexity 10
M runs in time T (n) if no computation path takes > T (|w|) steps before halting M runs in space S(n) if no computation path uses > S(|w|) worktape cells DTIME(T (n)) = {L(M) | DTM M runs in time T (n)} NTIME(T (n)) = {L(M) | NDTM M runs in time T (n)} DSPACE(S(n)) = {L(M) | DTM M runs in space S(n)} NSPACE(S(n)) = {L(M) | NDTM M runs in space S(n)} basic inclusions, for T (n) ≥ log n DTIME(T (n)) ⊆ NTIME(T (n)) ⊆ DSPACE(T (n)) ⊆ NSPACE(T (n)) ⊆ DTIME(2O(T (n)))
DTIME(T (n)) DTIME(T (n)·log2(T (n)), DSPACE(S(n)) DSPACE(S(n)·log(S(n))
Savitch, 1970: NSPACE(S(n)) ⊆ DSPACE(S(n)2), for S(n) ≥ log n
Computational complexity 11
P =
DTIME(nc) = DTIME(nO(1)) NP = NTIME(nO(1)) PSPACE = DSPACE(nO(1)) = NSPACE(nO(1)) EXPTIME = DTIME(2nO(1)) NEXPTIME = NTIME(2nO(1)) EXPSPACE = DSPACE(2nO(1)) = NSPACE(2nO(1)) . . . DTIME(T (n)) = CODTIME(T (n))
(i.e., DTIME(T (n)) is closed under complement)
CODTIME(T (n)) = {L | L ∈ DTIME(T (n))}
where L = {0, 1}∗ \ L
Immerman and Szelepcs´ enyi, 1987: NSPACE(S(n)) = CONSPACE(S(n)),
for S(n) ≥ log n
What about NTIME(T (n)) and CONTIME(T (n))?
Computational complexity 12
Log Time Log Space P N P NPC
C O
N P PSPACE EXPTIME EXPSPACE . . .
ELEMENTARY
. . . 2EXPTIME R
Computational complexity 13
Cobham-Edmonds thesis: – P captures decision problems with ‘feasible’ decision procedures
is n100 feasible? In practice, problems in P always have ≤ n3 or ≤ n5 time algorithms
n2 typical running time
Problems in P – linear programming – calculating the greatest common divisor – circuit value problem – determining if a given number is prime (Agrawal, Kayal, Saxena, 2004)
not known whether P-complete, in NC
Complexity of factorisation is not known (it is in NP ∩ CONP):
hundreds of machines factored a 232-digit number (RSA-768) over 2 years
Computational complexity 14
Boolean formulas are constructed from variables p1, p2, . . . using Boolean connectives ∧ (and), ∨ (or) and ¬ (not) Example: (p1 ∧ p2) ∨ ¬p1 interpretation I maps variables pi onto the truth-values T and F – I | = pi if, and only if, pI
i = T
– I | = ϕ ∧ ψ if, and only if, I | = ϕ and I | = ψ – I | = ϕ ∨ ψ if, and only if, I | = ϕ or I | = ψ – I | = ¬ϕ if, and only if, I | = ϕ a formula ϕ is satisfiable if there is an interpretation I such that I | = ϕ
(NDTM guesses an interpretation I and checks in polynomial time that I | = ϕ)
the first of the 7 Millennium Prize Problems ($1,000,000)
Computational complexity 15
A language L ⊆ {0, 1}∗ is in NP if there exists a polynomial p: N → N and a polynomial-time DTM M (called the verifier for L) such that, for every word x ∈ {0, 1}∗, x ∈ L iff ∃y ∈ {0, 1}p(|x|) such that M(x, y) = 1
such y is called a certificate for x
Exercise: Given a graph G and a number k, decide whether it contains a k-clique It is hard to find a solution to a problem (or to prove a theorem), but it is easy to check whether a solution is correct What if P = NP? – Mathematicians and computer scientists would not be needed – Artificial Intelligence software would be perfect – No privacy in the digital domain (encryption schemes would be easily decodable)
Computational complexity 16
Language L is reducible to language L′ if there is a function f : L → L′ such that w ∈ L iff f(w) ∈ L′ f has to be effectively computable, e.g., P-computable L is NP-complete if L ∈ NP and every L′ in NP is P-reducible to L Cook 1971, Levin 1973: SAT is NP-complete
for some c (and uses ≤ nc cells). s(i, j, a) — cell i contains symbol a at step j h(i, j, q) — head is in state q while scanning cell i at step j consider the formulas, for 0 ≤ i, j < nc, a ∈ Σ q ∈ Q, s(i, j, a) ∧ h(i, j, q) →
(s(i, j + 1, a′) ∧ h(i + d, j + 1, q′)) s(i, j, a) ∧ h(i′, j, q) → s(i, j + 1, a),
for all i = i′
s(i, j, a) ∧ s(i, j, a′) → ⊥,
for all a′ = a
h(i, j, q) ∧ h(i′, j, q′) → ⊥,
for all (i′, q) = (i, q)
Computational complexity 17
– Travelling salesman problem: given a list of cities and their pairwise distances, find the shortest route that visits each city exactly once – Clique problem: given a graph G and a number k, decide whether G contains a k-clique – Graph colouring problem: is it possible to colour the vertices of a given graph in 3 colours so that no two adjacent vertices share the same colour – Subgraph isomorphism problem: is graph G1 isomorphic to a subgraph of graph G2? NB: Graph isomorphism is in NP
, but nobody knowns whether it is NP-complete
best algorithm: 2O(√n log n) for graphs with n vertices Ladner: if P = NP then there exists a language in NP that is not NP-complete Exercise: Reduce SAT to 3SAT = {ϕ | ϕ is a satisfiable 3CNF}
Computational complexity 18
CONP = {L | L ∈ NP}
where L = {0, 1}∗ \ L Equivalent definition: A language L ⊆ {0, 1}∗ is in CONP if there exists a polynomial p: N → N and a polynomial-time DTM M such that, for every word x ∈ {0, 1}∗, x ∈ L iff ∀y ∈ {0, 1}p(|x|) M(x, y) = 1 – Boolean validity is CONP-complete – If P = NP then NP = CONP = P – Not known whether NP = CONP (it is hard to believe that this is the case) Exercise: Show that P ⊆ NP ∩ CONP
Computational complexity 19
what’s wrong with polynomial reductions within P?
a LOGSPACE transducer is a DTM that halts on all inputs and has three tapes: a two-way read-only input tape a two-way read-write logspace-bounded worktape (initially blank) a write-only left-to-right output tape (initially blank) LOGSPACE reduction = reduction computable by a LOGSPACE transducer Intuitively, LOGSPACE is equivalent to — counting up to n, i.e., automata with a two-way input head and a fixed number
— or “finite fingers”, i.e., automata with a fixed number of two-way input heads
that may not move outside the input
L is P-complete if L ∈ P and every L′ in P is LOGSPACE-reducible to L
Computational complexity 20
a circuit is an acyclic graph of AND-, OR- and NOT-gates
(with many inputs and a single output)
p1 p2 p5 p3 := p1 ∨ p2 p4 := ¬(p1 ∧ p2) p5 := p3 ∧ p4 CVP = given a circuit and a truth assignment to its inputs, decide whether the output is true or not Ladner 1975: CVP is P-complete Proof is similar to the Cook-Levin theorem Horn-SAT = satisfiability of sets of Horn clauses, i.e., formulas of the form (¬p1 ∨ · · · ∨ ¬pn ∨ p) and (¬p1 ∨ · · · ∨ ¬pn)
(sets of clauses of ≤ 3 and ≤ 2 literals, respectively)
Computational complexity 21
s v u t a directed graph G = (V, E), where V is a set of vertices and E ⊆ V × V a set of edges MAZE = given a graph G = (V, E) and two vertices s, t ∈ V , decide whether there is a directed path from s to t in G Jones, Lien and Laaser 1976: MAZE is NLOGSPACE-complete
The number of configurations of M is bounded by an exponential function in log n, i.e., by a polynomial in n. Construct a graph G over all configurations of M such that its edges are the next-configuration relation and let s and t be the (unique) initial and accepting configurations. Reingold 2005: Reachability in undirected graphs is in LOGSPACE
Computational complexity 22
quantified Boolean formulas are constructed from variables p1, p2, . . . using Boolean connectives ∧, ∨, ¬ and quantifiers ∃ and ∀ Example: ∃p1∀p2((p1 ∧ p2) ∨ ¬p1) – I | = ∃p ϕ if, and only if, I[p → F] | = ϕ
I[p → T] | = ϕ – I | = ∀p ϕ if, and only if, I[p → F] | = ϕ and I[p → T] | = ϕ
a compact way of writing disjunctions and conjunctions—exponentially more succinct
QBF = given a closed (no free variables) quantified Boolean formula, decide whether it is true or not
w.l.o.g. prenex form Q1p1Q2p2 . . . Qnpnϕ, where ϕ is quantifier-free
– ‘guesses’ T or F for existentially quantified variables and – tries out both T and F for universally quantified variables and then computes ϕ in the constructed interpretations runs in exponential time (in n) but in linear space!
Computational complexity 23
states Q of TM are partitioned into existential
and universal
states a run is a subtree that contains – all successors for AND-states – one successor for OR-states an accepting run contains an accepting state in each branch
Computational complexity 24
ATIME(T (n)) = {L(M) | ATM M runs in time T (n)} ASPACE(S(n)) = {L(M) | ATM M runs in space S(n)} Chandra, Kozen and Stockmeyer 1981: ATIME(T (n)O(1)) = DSPACE(T (n)O(1)), for T (n) ≥ n ASPACE(S(n)) = DTIME(2O(S(n))), for S(n) ≥ log n LOGSPACE ⊆ P ⊆ PSPACE ⊆ EXPTIME ⊆ EXPSPACE ⊆ . . . = = = = ALOGSPACE ⊆ APTIME ⊆ APSPACE ⊆ AEXPTIME ⊆ . . .
Computational complexity 25
L is PSPACE-complete if it is in PSPACE and every L′ in PSPACE is P-reducible to L Stockmeyer and Meyer 1973: QBF is PSPACE-complete
to recognise L (and uses nc cells on the tape). Use formulas similar to the Cook-Levin theorem to encode computations. But instead of satisfiability, we consider the QBF of the form Q1p1, . . . Qncpncϕ so that the alternation of the quantifiers exactly reflects the alternation of AND- and OR-configurations in the computation tree
polynomial-time hierarchy Σp
0 = Πp 0 = P
no prefix
Σp
1 = NP
∃
Πp
1 = CONP
∀
Σp
2
∃∀
Πp
2
∀∃
Σp
3
∃∀∃
Πp
3
∀∃∀
. . . . . . PSPACE
all prefixes
Computational complexity 26
– NP-complete problems: a YES answer has a short certificate – PSPACE-complete problems: a winning strategy for a two-player game with perfect information (such as Chess, Go)
(no short certificate)
Computational complexity 27
Let A be a subset of a plane, i.e., A ⊆ Z × Z Given a finite set T of tile types t = (left(t), right(t), up(t), down(t)) decide whether there exists a tiling τ : A → T such that, for all (i, j) ∈ A, up(τ(i, j)) = down(τ(i, j + 1))
and
right(τ(i, j)) = left(τ(i + 1, j)).
Computational complexity 28
configuration head position state +
step
Bounded tiling: Given a set T of tile types and an n × n square region A with a given colouring along the edge, does there exist a tiling of A extending the colouring of the edge? Levin 1973, Lewis 1978: Bounded tiling is NP-complete. Corridor tiling: Given a set T of tile types and a pair of edges of length n with a given colouring, does there exist a height m and a tiling of the n × m re- gion A such that its top and bottom colouring are as defined? Theorem. Corridor tiling is PSPACE- complete. Quadrant Tiling: Given a set T of tile types, does there exist a tiling of N × N? Berger 1966: Quadrant tiling is undecidable.
Computational complexity 29
constants a1, a2, . . . variables x1, x2, . . . functions f1, f2, . . . each function is of fixed arity terms: variables, constants, or functions with terms
e.g., x, a, f(g(a), h(x, b))
predicates P1, P2, . . . each predicate is of fixed arity formulas are built from predicates with terms as their arguments using Boolean connectives and quantifiers Example: ∀x¬K(x, x), ∀x∀y(K(x, y) ∨ (x = y) ∨ K(y, x)) interpretation I = (∆I, aI
1, . . . , f I 1 , . . . , P I 1 , . . . ):
∆I is the domain of I (non-empty set) aI
i ∈ ∆I
(constants are elements of the domain)
f I
i : (∆I)ki → ∆I
(functions are functions of the respective arity on the domain)
P I
i ⊆ (∆I)ni
(predicates are relations of the respective arity on the domain)
assignment a: V → ∆I
(assigns a domain element to each variable)
aI,a
i
= aI
i ,
xI,a
i
= a(xi), f(t1, . . . , tk)I,a = f I(tI,a
1 , . . . tI,a k ) Birkbeck 5.11.19 22
=a P (t1, . . . , tn)
if and only if
(tI,a
1 , . . . , tI,a n ) ∈ P I
=a ¬ϕ
if and only if
I | =a ϕ
=a ϕ ∨ ψ
if and only if
I | =a ϕ or I | =a ψ
=a ϕ ∧ ψ
if and only if
I | =a ϕ and I | =a ψ
=a ∃x ϕ
if and only if
I | =a′ ϕ, for some a′ = a[x → d] with d ∈ ∆I
=a ∀x ϕ
if and only if
I | =a′ ϕ, for all a′ = a[x → d] with d ∈ ∆I formula ϕ is satisfiable if there is an interpretation I and an assignment a such that I | =a ϕ Example: ∀x∀y∀z (K(x, y) ∧ K(y, z) → K(x, z)) can be satisfied only in an interpretation with K transitive Example: ∀x∀y (K(x, y) ∨ (x = y) ∨ K(y, x)) can be satisfied only in an interpretation with K linear
Birkbeck 5.11.19 23
Let T be a set of tile types. It is undecidable whether it tiles the quadrant. K is a transitive linear irreflexive order: ∀x ¬K(x, x) ∀x∀y (K(x, y) ∨ (x = y) ∨ K(y, x)) ∀x∀y∀z (K(x, y) ∧ K(y, z) → K(x, z)) S is the total successor relation: ∀x∃y S(x, y) ∀x∀y (S(x, y) ↔ (K(x, y) ∧ ¬∃z (K(x, z) ∧ K(z, y)))) T tiles the quadrant
(Ti is a binary predicate for tile type ti ∈ T ):
∀x∀y
ti∈T Ti(x, y)
∀x∀y∀z (S(x, y) ∧ Ti(x, z) ∧ Tj(y, z) → ⊥) if right(ti) = left(tj) ∀x∀y∀z (S(x, y) ∧ Ti(z, x) ∧ Tj(z, y) → ⊥) if up(ti) = down(tj)
Birkbeck 5.11.19 24
Mortimer 1975, Gr¨ adel, Kolaitis and Vardi 1996: Satisfiability in FO with 2 variables is decidable and NEXPTIME-complete guarded formulas built predicates using Boolean connectives and guarded quantifiers ∀x (P (x, y, z) → ϕ(x, y)) and ∃x (P (x, y, z) ∧ ϕ(x, y)) Example: ∀x∀y (R(x, y) → R(y, x)) — guarded ∃y (R(x, y) ∧ ψ(y) ∧ ∀z ((R(x, z) ∧ R(z, y) → ϕ(z)))) — not guarded
description logic axioms (ontologies) Andr´ eka, van Benthem & N´ emeti 1996, Gr¨ adel 1998: Satisfiability in the Guarded Fragment is decidable and 2NEXPTIME-complete various extensions, . . .
Birkbeck 5.11.19 25