15-251 Great Theoretical Ideas in Computer Science Lecture 9: - PowerPoint PPT Presentation

15-251 Great Theoretical Ideas in Computer Science Lecture 9: Boolean Circuits September 29th, 2015

Where we are, where we are going Computer science is no more about computers than astronomy is about telescopes.

? P = NP

P vs NP is on the horizon 1 million dollar question (or maybe 6 million dollar question) P = NP ???

Computational complexity of an algorithm Recall: Definition: The running time of an algorithm A is defined as worst-case T A ( n ) = max { # steps A takes on I } instances I of size n

Computational complexity of a problem The intrinsic complexity of a problem: Complexity of the best algorithm computing the problem. How to show an upper bound on the intrinsic complexity? > Give an algorithm that solves the problem. How to show a lower bound on the intrinsic complexity? > Argue against all possible algorithms that solve the problem. The dream : Get a matching upper and lower bound.

What is P ? P The set of languages that can be decided in steps for some constant . O ( n k ) k The theoretical divide between efficient and inefficient: efficiently solvable. L ∈ P not efficiently solvable. L 62 P

What is P ? In practice: Awesome! Like really awesome! O ( n ) Great! O ( n log n ) Kind of efficient. O ( n 2 ) Barely efficient. (???) O ( n 3 ) Would not call it efficient. O ( n 5 ) O ( n 10 ) Definitely not efficient! O ( n 100 ) WTF?

Why P ? - P is not meant to mean “efficient in practice” - It means “You have done something extraordinarily better than brute force (exhaustive) search.” - So P is about mathematical insight into a problem’s structure. - Robust to notion of what is an elementary step, what model we use, reasonable encoding of input, implementation details. - Wouldn’t make sense to cut it off at some specific exponent. - Plus, big exponents don’t really arise. - If it does arise, usually can be brought down.

Why P ? Summary : Being in P vs not being in P is a qualitative difference, not a quantitative one.

What is NP ? EXP The set of languages that can be decided in steps for some constant . O ( k n ) k > 1 DECIDABLE LANGUAGES EXP NP : NP A class between P and . P EXP

What is NP ? NP ? P = NP P asks whether these two sets are equal. How would you show ? P = NP Show that every problem in NP You have to argue against can be solved in poly-time. all possible poly-time TMs. How would you show ? P 6 = NP Show that there is a problem in NP which cannot be solved in poly-time.

Boolean Circuits

Some preliminary questions What is a Boolean circuit? - It is a computational model for computing decision problems (or computational problems). We already have TMs. Why Boolean circuits? - The definition is simpler. - Easier to understand, usually easier to reason about. - Boolean circuits can efficiently simulate TMs. (efficient decider TM efficient/small circuits.) ⇒ = - Circuits are good models to study parallel computation . - Real computers are built with digital circuits.

Sounds awesome! So why didn’t we just learn about circuits first? There is a small catch. An algorithm is a finite answer to infinite number of questions. Stephen Kleene (1909 - 1994)

Sounds awesome! So why didn’t we just learn about circuits first? There is a small catch. Circuits are an infinite answer to infinite number of questions. Anil Ada (???? - 2077)

Dividing a problem according to length of input Σ = { 0 , 1 } f : { 0 , 1 } ∗ → { 0 , 1 } L ⊆ { 0 , 1 } ∗ { 0 , 1 } n = all strings of length n f n : { 0 , 1 } n → { 0 , 1 } L n = { w ∈ L : | w | = n } for x ∈ { 0 , 1 } n , f n ( x ) = f ( x ) f = ( f 0 , f 1 , f 2 , . . . ) L = L 0 ∪ L 1 ∪ L 2 ∪ · · ·

Dividing a problem according to length of input A TM is a finite object (finite number of states) but can handle any input length. TM input output computes L Imagine a model where we allow the TM to grow with input length. … TM0 TM1 TM2 TM3 … L 3 L 1 L 2 L 0

Dividing a problem according to length of input So one machine does not compute . L You use a family of machines: ( M 0 , M 1 , M 2 , . . . ) (Imagine having a different Python function for each input length.) Is this a reasonable/realistic model of computation? Boolean circuits work this way. Need a separate circuit for each input length.

Picture of a circuit ∨ ∧ ∧ ∨ ¬ ¬ … x 1 x 2 x n x 3

Picture of a circuit gates ∨ ∧ ∧ ∨ ¬ ¬ … x 1 x 2 x n x 3

Picture of a circuit binary OR gate ∨ ∨ binary AND gate ∧ ∧ ∧ unary NOT gate ¬ ∨ ¬ ¬ input gate x i … x 1 x 2 x n x 3 output gate

Picture of a circuit binary OR gate ∨ wires ∨ binary AND gate ∧ ∧ ∧ unary NOT gate ¬ ∨ ¬ ¬ input gate x i … x 1 x 2 x n x 3 output gate

Picture of a circuit ∨ ∧ ∧ No feedback loops ∨ ¬ ¬ allowed! … x 1 x 2 x n x 3 Information flows from input gates to the output gate.

Picture of a circuit binary OR gate ∨ ∨ binary AND gate ∧ ∧ ∧ unary NOT gate ¬ ∨ ¬ ¬ input gate x i … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit binary OR gate ∨ ∨ binary AND gate ∧ ∧ ∧ unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit binary OR gate ∨ ∨ binary AND gate ∧ ∧ ∧ unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 1 1 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit binary OR gate ∨ ∨ binary AND gate ∧ ∧ ∧ unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 1 1 1 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit binary OR gate ∨ ∨ binary AND gate ∧ ∧ ∧ 1 unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 1 1 1 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit binary OR gate ∨ ∨ binary AND gate ∧ ∧ ∧ 1 0 0 unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 1 1 1 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit binary OR gate ∨ ∨ binary AND gate ∧ ∧ ∧ 1 0 0 1 unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 1 1 1 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit binary OR gate ∨ ∨ 0 binary AND gate ∧ ∧ ∧ 1 0 0 1 unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 1 1 1 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit binary OR gate ∨ ∨ 0 0 binary AND gate ∧ ∧ ∧ 1 0 0 1 unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 1 1 1 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Picture of a circuit 0 binary OR gate ∨ ∨ 0 0 binary AND gate ∧ ∧ ∧ 1 0 0 1 unary NOT gate ¬ ∨ ¬ ¬ input gate x i 0 1 1 1 … x 1 x 2 x n x 3 output gate 0 1 1 0 Computes a function . f : { 0 , 1 } n → { 0 , 1 } So how does it compute ? f ( x 1 , x 2 , . . . , x n )

Poll: What does this circuit compute ? (sometimes circuits are drawn upside down) ¬ ¬ ¬ ¬ ∧ ∧ ∧ ∧ ∨ ∨ ¬ ¬ ∧ ∧ ∨

Poll: What does this circuit compute ? (sometimes circuits are drawn upside down) ¬ ¬ ¬ ¬ ∧ ∧ ∧ ∧ ∨ ∨ ¬ ¬ ∧ ∧ ∨

Poll: What does this circuit compute ? (sometimes circuits are drawn upside down) ¬ ¬ ¬ ¬ parity of parity of x + x x + x 1 2 3 4 ∧ ∧ ∧ ∧ x 1 ⊕ x 2 x 3 ⊕ x 4 ∨ ∨ ¬ ¬ ∧ ∧ ∨ x 1 ⊕ x 2 ⊕ x 3 ⊕ x 4

How does a circuit decide/compute a language? How do we measure the complexity of a circuit?