Introduction to Complexity Theory Winter 2005/06 Dr. Axel Gromann - - PowerPoint PPT Presentation

Introduction to Complexity Theory

Winter 2005/06

• Dr. Axel Groÿmann

Knowledge Representation and Reasoning Group Artificial Intelligence Institute

1

Computational complexity theory

Complexity theory is part of the theory of computation dealing with the resources required during computation to solve a given problem. The most common resources

• time (how many steps it takes to solve a problem)
• space (how much memory it takes)

Other resources, e.g.,

• how many parallel processors are needed to solve a problem in parallel

Computability theory deals with whether a problem can be solved at all, regardless of the resources required.

Axel Großmann, CT 2005/06 2

Problems and instances

A single problem is an entire set of related questions, where each question is a finite-length string. The FACTORIZE problem

• given an integer written in binary
• return all of the prime factors of that number

A particular question is called an instance. Instance of the FACTORIZE problem

• give the factors of the number 15

Axel Großmann, CT 2005/06 3

Decision problems

Much of complexity theory deals with decision problems. A decision problem is a problem where the answer is always YES/NO. The IS-PRIME problem

• given an integer written in binary
• return whether it is a prime number or not

Decision problems are often considered because an arbitrary problem can always be reduced to a decision problem.

Axel Großmann, CT 2005/06 4

Complexity classes

The complexity class P is the set of decision problems that can be solved by a deterministic machine in polynomial time. This class corresponds to an intuitive idea of the problems which can be effectively solved in the worst cases. The complexity class NP is the set of decision problems that can be solved by a non-deterministic machine in polynomial time. This class contains many problems that people would like to be able to solve effectively, including the Boolean satisfiability problem, All the problems in this class have the property that their solutions can be checked effectively. The question of whether P is the same set as NP is the most important open question in theoretical computer science.

Axel Großmann, CT 2005/06 5

History of computational complexity

The ancient Greeks

• The idea of `efficient algorithms'

1936

• Turing developed hist theoretical computational model based on how he perceived

mathematicians think.

1940s/50s

• Digital computers were developed.
• Turing machine proved itself as the right theoretical model for computation, but

fails to account for the amount of time or memory needed by a computer.

Axel Großmann, CT 2005/06 6

History of computational complexity (2)

Mid of 1960's

• Hartmanis and Stearns [1965] introduced the idea to measure time and space as

a function of the length of the input.

• The birth of computational complexity
• Edmonds [1965] argued that polynomial-time gives good formalisation of efficient

computation.

• NP-completeness at the most important concept and its most fundamental ques-

tion, whether P = NP

1970's

• Cook and Karp showed a large number of combinatorial and logical problems were

NP-complete.

• Growth of complexity classes
• Probabilistic computation

Axel Großmann, CT 2005/06 7

History of computational complexity (3)

1980's

• Finite models like circuits

1990's

• Quantum computers and propositional proof systems

Axel Großmann, CT 2005/06 8

Questions and answers that make up complexity theory

Complexity theory is concerned with the study of the intrinsic complexity of computational tasks. Its final goals:

• absolute answers about specific computational phenomena

(determining the complexity of any well-defined class)

• answers about the relation between computational phenomena

Failure to resolve questions of the absolute type led to flourish- ing of methods for coping with questions of the relative type. Establishing relations between phenomena seems more revealing than making statements about each phenomenon.

Axel Großmann, CT 2005/06 9

Questions and answers that make up complexity theory (2)

Complexity theory failed to determine the intrinsic complexity

• f task such as finding a satisfying assignment to a given

(satisfiable) propositional formula of finding a 3-colouring of a given (3-colourable) graph. But is has established that these two seemingly different com- putational tasks are computationally equivalent.

Axel Großmann, CT 2005/06 10

The P versus NP question

Our daily experience is that it is harder to solve a problem than it is to check the correctness of a solution. Is this experience merely a coincidence or does it represent a fundamental fact of life (or a property of the world)? Could you imagine a world in which solving any problem is not significantly harder than checking a solution to it? Would the term `solving a problem' not lose its meaning in such a hypothetical (and impossible in our opinion) world? The denial of the plausibility of such a hypothetical world (in which "solving" is not harder than "checking") is what `P different than NP' actually means, where P represents tasks that are efficiently solvable and NP represents tasks for which solutions can be efficiently checked.

Axel Großmann, CT 2005/06 11

References

Recommended text:

• Christos H. Papadimitriou. Computational Complexity,

Addison Wesley, 1994

Other useful texts:

• Michael Sipser. Introduction to the Theory of Computation,

PWS, 1997

• Michael R. Garey and David S. Johnson. Computers and Intractability,
• W. H. Freeman, 1979

Axel Großmann, CT 2005/06 12

List of topics arranges by weeks

Week 1 (18th Nov.): 2 lectures, 1 tutorial

• Introduction (Slides)
• Languages for specification and the need for formalism. Algorithms, polynomials

and polynomial time algorithms. (Ch. 1) ⊲ Graph reachability ⊲ Maximum flow and matching ⊲ Travelling Salesman problem

Week 3 (2nd Dec.): 2 lectures, 1 tutorial

• Turing machines as formalised algorithm.

Complexity classes. Space complexity and nondeterminism (Ch. 2, except for 2.6) ⊲ TM basics ⊲ TM as algorithms ⊲ TM with multiple strings ⊲ Linear speedup ⊲ Space bounds ⊲ Nondeterministic machines

Axel Großmann, CT 2005/06 13

List of topics arranges by weeks (2)

Week 5 (16th Dec.): 3 lectures, 1 tutorial

• Universal machines and undecidability. (Ch. 3)
• Propositional logic.

The complexity of satisfiability and validity. (Sec. 4.1 and 4.2) ⊲ Universal TM ⊲ Halting problem ⊲ More undecidability ⊲ Boolean logic ⊲ Boolean expressions ⊲ Satisfiability and validity (except for Horn clauses)

Week 8 (20th Jan.): 3 lectures, 1 tutorial

• Complexity classes. The Hierarchy theorem. (Sec. 7.1 and 7.2)
• Reductions and completeness. (Sec. 8.1 and 8.2)

⊲ Complexity classes ⊲ Hierarchy theorem ⊲ Reductions ⊲ Completeness (Cook's theorem)

Axel Großmann, CT 2005/06 14

List of topics arranges by weeks (3)

Exam period

• Written exam on all courses within the Foundations modules

Axel Großmann, CT 2005/06 15