Introduction to Computer Science CSCI 109 China Tianhe-2 Andrew - - PowerPoint PPT Presentation

Introduction to Computer Science

CSCI 109

Andrew Goodney

Fall 2019

China – Tianhe-2

Lecture 11: Abstract Machines and Theory Nov. 18th, 2019

Schedule

1

Abstract machines and theory

uWhat is theoretical computer

science?

uFinite state machines uThe Turing machine

u The Church-Turing thesis u The halting problem

uThe analysis of algorithms uThe analysis of problems

2

Reading:

• St. Amant Ch. 8

What is theoretical computer science?

uThe study of what can be computed and

how efficiently it can be computed

uBased on abstract mathematical models uInsight into the behavior of a real

computer… …any real computer

3

Finite state machines

4

On Off Up Down Down Up

Light switch

Crying Asleep

Awake, not crying

Fed Hungry More hungry Hungry

1 day old infant 10 day old infant

Finite state machines

5

M=F M+1 M+2 M+3 M+4 F+1 F+2 F+3 F+4 Add M Add M Add M Add F Add F Add F Add M Add M Add M Add M Add F Add F Add F Add M Add F Add F

• St. Amant pp. 152

Add M Add F

Finite State Machines

u Why look at FSMs? u They “compute”

v Sort-of… lots of limitations

u They are simple

v Can be modeled mathematically

6

Finite state machine limitations

uIf a finite state machine with n states is given

n+1 symbols, it will revisit one state at least once

uNo way to tell if revisited state is being visited for

the 1st time or 5th time or nth time

uThe pigeonhole principle uProblems an FSM cannot solve:

v Given a string of As and Bs, tell if the number of As

and Bs are the same

v Tell if a string is a palindrome

7

FSM -> TM

u FSM can’t do general computation u Their simplicity is alluring u Can we do better?

v Combine FSM + Memory

u Several forms

v Most famous is Turing Machine v Memory is a “tape”

8

The Turing machine

9

u An infinite tape, with squares marked on it u Each square can be blank or hold a symbol u A machine moves over the tape u When positioned over a square,

the machine can

v Read a symbol from the tape v Erase a symbol from the tape v Write a symbol onto the tape (erasing what was

• n the square)

v Do nothing

u The machine can move either one square

to right or left

Simple Turing Machine with 1 symbol

10

https://vimeo.com/46913004

Alan Turing

11

Turing Machines…WTF?

u “Alan Turing and his Ph.D. advisor Church studied the

universality of computation (among other things) in the

• 1930s. Turing formulated the Turing Machine in order to

capture everything that computers can do…” Igor Markov, EECS Prof

at Michigan

u “As others have correctly pointed out that the purpose ‘was

to answer the halting problem, which Turing published in his 1937 paper On Computable Numbers With an Application to the Entscheidungsproblem’. The idea that a machine can perform the tasks of any other machine is at the centre of the concept of a Universal (Turing) machine…” Hector Zenil, Lab

Leader, Karolinska - Senior Researcher, Oxford

12

More quotes re: Turing Machines

u “A Turing machine can perform any algorithm -- it can always

be configured to give a particular output for a particular

• input. But this simple idealized device requires infinite

memory so it's not possible to build a physical Turing

• machine. However, a physical device is said to be Turing

complete if it can simulate any algorithm that could be run by the idealized Turing machine.” Jeff Lamattery, B.S. Physics, UCSD

13

More quotes re: Turing Machines

u “…Newman mentioned Hilbert's Entscheidungsproblem which

dates back to 1928. This problem is that of finding an algorithm that can decide whether a given statement is provable within a logical system. If this decision problem were solvable, we could use algorithms to carry out mathematical research… Turing set out to understand the Entscheidungsproblem, and for this he needed to develop a mathematical model of algorithms. This model is what we now call the Turing machine model…The main theorem of the paper is that the Entscheidungsproblem is not solvable – put informally, there can be no algorithm that can "automatize" mathematics – and the proof relies on being able to express the behaviour of a Turing machine in first-order logic.” Hans Hyttel, Associate

Professor

14

Turing Machine Representation: Table

15

This machine has 4 symbols: 0, 1, _ and * L means the head moves to the left R means the head moves to the right

Turing Machine Representation: Table

16

Turing Machine Representation: Table

17

Turing Machine Representation: Table

18

write_symbol, head_move_direction, state_update

Turing Machine Representation: Table

19

write_symbol, head_move_direction, state_update

Same Machine as a State Diagram

20

read_symbol,write_symbol,head_position_move

Same Machine as a State Diagram

21

read_symbol,write_symbol,head_position_move

22

Example execution

u Assume the machine always begins in state 0 with the head

positioned on the rightmost non-blank square

u The machine stops when it is in state 4 (called an accept state) u The machine stops when it is in state 5 (called a reject state) u How does the execution look if the tape has an input of 10

written on it at the beginning ?

23

Behavior of the machine on input 10

24

25

Behavior of the machine

• n input 10

26

Behavior of the machine

• n input 1010

What does this Turing machine do?

uChecks if a binary string has the same

number of 1’s and 0’s and enters the accept state if it does, otherwise it enters the reject state.

27

Turing Machine Programming

uWriting programs for a Turing machine is

very tedious

uHowever…

28

The Church-Turing thesis

Turing machines are capable of solving any effectively solvable algorithmic problem Turing machines are thus excellent models for what (all) real computers are capable of doing

29

Problems not ‘effectively computable’

uAre there problems that are not effectively

computable ?

uYes, infinite such problems uCould these be computable someday ? uPerhaps, but not on a Turing-equivalent

computer

30

A halting question

uDeciding whether a program will halt (terminate) uDoes the following program halt ?

Input number While number is not 0 Print number number = number – 1

31

The halting problem

uDeciding whether a given program will halt

(terminate) on an arbitrary input

uIn 1936, Turing proved that a general algorithm

to the halting problem for all possible programs

• ver all possible inputs cannot exist

The halting problem is undecidable

32

Undecidable Problem

u Undecidable Problem:

v Problem requires a yes/no answer v But no program exists that will always give the correct answer

33

Proof sketch for halting problem

u Want to decide if program A halts on input a u Need to write a program, lets call it halter

v Inputs: A, a v Output: Yes (if program A halts on input a) or No (if program A

does not halt on input a)

halter(A,a) if(some complex computation)is true Return Yes else Return No

34

i.e analyze A for input a

The halting problem is undecidable

uIn 1936, Turing proved that a general

algorithm to the halting problem for all possible programs over all possible inputs cannot exist

uIn other words, a general version of

halter (one that works for all programs and their inputs) does not exist

35

Proof by contradiction

uAssume that a general version of halter exists uShow that this assumption leads to a contradiction uInvent a new program called clever

clever(program,input) result = halter(program,input) if result is No return Yes else loop forever

36

Proof by contradiction

uWhat does clever do when given itself as

input?

clever(program,input) result=halter(program,input) if result is No return Yes else loop forever

37

Proof by contradiction

u If halter says

that clever halts, then clever loops forever (which means it doesn’t halt)

u If halter says

that clever does not halt then clever halts and returns Yes

38

clever(program,input) result=halter(program,input) if result is No return Yes else loop forever

Conclusion: The program halter cannot exist

Russell’s Paradox

u In a hypothetical small

town:

v There is one male barber v The barber shaves all those

and only those men in town who do not shave themselves

v All men stay clean shaven

either by shaving themselves

• r going to the barber

u Who shaves the barber?

39

u More generally:

v Let R be the set of all sets

that are not members of themselves

v If R is not a member of itself,

then its definition dictates that it must contain itself, and if it contains itself, then it contradicts its own definition as the set of all sets that are not members of themselves

https://plato.stanford.edu/entries/russell-paradox/

Other Undecidable Problems

u Both of these proved by showing an implementation on a Turing machine

would just be the halting problem

u Matrix mortality: given a finite set of n x n sized matrices, does a sequence of

matrix multiplications (repetitions allowed) exist that yields the zero matrix?

v Undecidable for a set of six or more 3 x 3 matrices

u Post’s Correspondence Problem

v Given two equal length lists of strings written in the same alphabet, (list A, list B) where

the strings are indexed 1….|A|, does a sequence of indexes exist that when applied to each list, generates the same compound word?

40

Analysis of problems

u Study of algorithms illuminates the study of classes of

problems

u If a polynomial time algorithm exists to solve a problem

then the problem is called tractable

u If a problem cannot be solved by a polynomial time

algorithm then it is called intractable

u This divides problems into three groups:

v Problems with known polynomial time algorithms v Problems that are proven to have no polynomial-time algorithm v Problems with no known polynomial time algorithm but not yet

proven to be intractable

41

Tractable and Intractable

u Tractable problems (P)

v Sorting a list v Searching an unordered list v Finding a minimum spanning tree

in a graph

42

u Intractable

v Listing all permutations (all

possible orderings) of n numbers

u Might be (in)tractable

v Subset sum: given a set of

numbers, is there a subset that adds up to a given number?

v Travelling salesperson: n cities, n!

routes, find the shortest route These problems have no known polynomial time solution However no one has been able to prove that such a solution does not exist

Tractability, Intractability, Undecidability

u ‘Properties of problems’ (NOT ‘properties of algorithms’) u Tractable: problem can be solved by a polynomial time algorithm (or

something more efficient)

u Intractable: problem cannot be solved by a polynomial time algorithm

(all solutions are proven to be more inefficient than polynomial time)

u Unknown: not known if the problem is tractable or intractable (no

known polynomial time solution, no proof that a polynomial time solution does not exist)

u Undecidable: decision problem for which there is (provably) no

algorithmic solution on a Turing machine

v Non-existence v Not a matter of efficiency 43