Turings Legacy What is computation ? What is an algorithm ? How can - - PowerPoint PPT Presentation

▶
turing s legacy what is computation what is an algorithm
SMART_READER_LITE
LIVE PREVIEW

Turings Legacy What is computation ? What is an algorithm ? How can - - PowerPoint PPT Presentation

Oct 27, 2023 147 likes •916 views

15-251: Great Theoretical Ideas in Computer Science Lecture 5 Turings Legacy What is computation ? What is an algorithm ? How can we mathematically define them? Quick Recap Mathematical definition of a (computational) problem: Input /

slide-1
SLIDE 1

15-251: Great Theoretical Ideas in Computer Science

Turing’s Legacy

Lecture 5

slide-2
SLIDE 2

What is computation? What is an algorithm?

How can we mathematically define them?

slide-3
SLIDE 3

Mathematical definition of a (computational) problem: A simple mathematical model for algorithms: q0 q1 1 1 Quick Recap Input / output function: Language: DFAs

slide-4
SLIDE 4

PRIMALITY 0n1m 0n1n

Regular Languages (Solvable with DFAs) Decidable Languages (Solvable with “algorithms”) ???

HALF(AWESOME)

All Languages

slide-5
SLIDE 5

# Determines if string S is of form 0^n 1^n def Solution( S ): i = 0 j = len(S)-1 while j >= i: if S[i] != '0' or S[j] != '1': return False i = i + 1 j = j - 1 return True

Solving 0n1n with Python

slide-6
SLIDE 6

/* Determines if string S is of form 0^n 1^n */ int Solution(char S[]) { int i = 0, j; while (S[j] != NULL) /* NULL is end-of-string char */ j++; j--; while (j >= i) { if (S[i] != '0' || S[j] != '1') return 0; /* Reject */ i++; j--; } return 1; /* Accept */ }

Solving 0n1n with C

slide-7
SLIDE 7

PRIMALITY 0n1n

Regular Languages (Solvable with DFAs) Decidable Languages (Solvable with “algorithms”) All Languages

0n1m HALF(AWESOME)

slide-8
SLIDE 8

PRIMALITY 0n1n

Regular Languages (Solvable with DFAs) All Languages Decidable Languages (Solvable with Python)

0n1m HALF(AWESOME)

slide-9
SLIDE 9

Question: Should we just define “algorithm” to mean a function written in Python?

(allowed access to unlimited memory)

Answer: Actually, we’ll see that this would be OK!

slide-10
SLIDE 10
  • Why choose Python?

Why not C, or Java, or SML, or…?

Downsides as a formal definition:

  • Extremely complicated to rigorously define.

E.g., official 2011 ISO definition of C requires a 701-page PDF file!

  • A “philosophical” justification would be nice…
slide-11
SLIDE 11
  • Why choose Python?

Why not C, or Java, or SML, or…?

Downsides as a formal definition:

  • Extremely complicated to rigorously define.

E.g., official 2011 ISO definition of C requires a 701-page PDF file!

  • A “philosophical” justification would be nice…
slide-12
SLIDE 12

PRIMALITY 0n1n …

Solvable with Python Solvable with C

Claim:

=

slide-13
SLIDE 13

“Proof:”

Solvable with Python Solvable with C

Claim:

=

Proof intuition: Our shared experience with programming. Proof:

  • 1. Solvable with Python

Solvable with C. The standard Python interpreter is written in C.

  • 2. Solvable with C

Solvable with Python. It’s pretty clear one can write a C interpreter in Python.

slide-14
SLIDE 14

Interpreters

A Python function is (representable by) a string. A Python interpreter is an algorithm M that takes two inputs: P, a Python function;

x, a string;

and step-by-step simulates P(x). In particular, M(P,x) accepts if and only if P(x) accepts.

slide-15
SLIDE 15

Interpreters

You can write a Python interpreter in C. You can write a C interpreter in Python. You can write a Python interpreter in Java. You can write a Java interpreter in Python. You can write a Python interpreter in SML. You can write an SML interpreter in Python. You can write a Python interpreter in Python!! The last one is called a “Universal Python Program”

slide-16
SLIDE 16

Solvable with Python = Solvable with C = Solvable with Java = Solvable with SML

What we want to define to be “computable”.

PRIMALITY 0n1n

Regular Languages (Solvable with DFAs)

0n1m HALF(AWESOME)

slide-17
SLIDE 17
  • Why choose Python?

Why not C, or Java, or SML, or…?

Downsides as a formal definition:

  • Extremely complicated to rigorously define.

E.g., official 2011 ISO definition of C requires a 701-page PDF file!

  • A “philosophical” justification would be nice…
slide-18
SLIDE 18
  • Why choose Python?

Why not C, or Java, or SML, or…?

Downsides as a formal definition:

  • Extremely complicated to rigorously define.

E.g., official 2011 ISO definition of C requires a 701-page PDF file!

  • A “philosophical” justification would be nice…
slide-19
SLIDE 19

It would be nice to have a totally minimal (“TM”) programming language such that: a) can simulate Python, C, Java, SML, etc.; b) is simple enough to reason about rigorously completely mathematically.

slide-20
SLIDE 20

Turing Machine Inspired by

™

slide-21
SLIDE 21

Turing’s mathematical abstraction of a computer

  • A (human) computer writes symbols on paper
  • WLOG, the paper is a sequence of squares
  • No upper bound on the number of squares
  • At most finitely many kinds of symbols
  • Human observes one square at a time
  • Human has only finitely many mental states
  • Human can change symbols and change

focus to a neighboring square, but only based on its state and the symbol it observes

  • Human acts deterministically
slide-22
SLIDE 22

tape (paper) 1 1 1 input the “finite control” read/write“head” (focus) Illustration of a TM trying to decide {0n1n : n } We’ll explain the finite control later

slide-23
SLIDE 23

# 1 1 1 the “finite control” the “tape alphabet” in this example is {0,1,#, } (blank)

slide-24
SLIDE 24

# 1 1 1 the “finite control”

slide-25
SLIDE 25

# 1 1 1 the “finite control”

slide-26
SLIDE 26

# 1 1 1 the “finite control”

slide-27
SLIDE 27

# 1 1 1 the “finite control”

slide-28
SLIDE 28

# 1 1 1 the “finite control”

slide-29
SLIDE 29

# 1 1 1 the “finite control”

slide-30
SLIDE 30

# 1 1 1 the “finite control”

slide-31
SLIDE 31

# 1 1 1 the “finite control”

slide-32
SLIDE 32

# 1 1 the “finite control” #

slide-33
SLIDE 33

# 1 1 the “finite control” #

slide-34
SLIDE 34

# 1 1 the “finite control” #

slide-35
SLIDE 35

# 1 1 the “finite control” #

slide-36
SLIDE 36

# 1 1 the “finite control” #

slide-37
SLIDE 37

# 1 1 the “finite control” #

slide-38
SLIDE 38

# 1 1 the “finite control” #

slide-39
SLIDE 39

# 1 1 the “finite control” #

slide-40
SLIDE 40

# 1 1 the “finite control” # #

slide-41
SLIDE 41

# 1 1 the “finite control” # #

slide-42
SLIDE 42

# 1 1 the “finite control” # #

slide-43
SLIDE 43

# 1 1 the “finite control” # #

slide-44
SLIDE 44

# 1 1 the “finite control” # #

slide-45
SLIDE 45

# 1 1 the “finite control” # #

slide-46
SLIDE 46

# 1 1 the “finite control” # #

slide-47
SLIDE 47

# 1 the “finite control” # # #

slide-48
SLIDE 48

# 1 the “finite control” # # #

slide-49
SLIDE 49

# 1 the “finite control” # # #

slide-50
SLIDE 50

# 1 the “finite control” # # #

slide-51
SLIDE 51

# 1 the “finite control” # # #

slide-52
SLIDE 52

# 1 the “finite control” # # #

slide-53
SLIDE 53

# 1 the “finite control” # # # #

slide-54
SLIDE 54

# 1 the “finite control” # # # #

slide-55
SLIDE 55

# 1 the “finite control” # # # #

slide-56
SLIDE 56

# 1 the “finite control” # # # #

slide-57
SLIDE 57

# 1 the “finite control” # # # # REJECT

slide-58
SLIDE 58

tape (paper) 1 1 1 input the “finite control” read/write head

slide-59
SLIDE 59

Turing’s mathematical abstraction of a computer

  • A (human) computer writes symbols on paper
  • WLOG, the paper is a sequence of squares
  • No upper bound on the number of squares
  • At most finitely many kinds of symbols
  • Human observes one square at a time
  • Human has only finitely many mental states
  • Human can change its state, change symbols,

and change focus to a neighboring square, but only based on its state and the symbol it observes

  • Human acts deterministically
slide-60
SLIDE 60

q0 qreject qaccept qend qright qleft qdone?

0,1 R ,# L # 0,1 L

The finite control (aka transition rules)

slide-61
SLIDE 61

Formal definition of Turing Machines

A Turing Machine is a 7-tuple M = (Q, q0, qaccept, qreject, Σ, Γ, δ): Q is a finite set of states, q0 Q is the start state, qaccept Q is the accept state, qreject Q is the reject state, qreject ≠ qaccept. Σ is a finite input alphabet (with Σ), Γ is a finite tape alphabet (with Γ, Σ Γ), δ : Q’×Γ → Q×Γ×{L,R} is the transition function

(here Q’ = Q \ {qaccept, qreject})

slide-62
SLIDE 62

Input alphabet: Σ = {0,1} Tape alphabet: Σ = {0,1,#, }

q0 qreject qaccept qend qright qleft qdone?

0,1 R ,# L # 0,1 L

slide-63
SLIDE 63

Formal definition of Turing Machines

Rules of computation:

Tape starts with input x Σ*, followed by infinite ’s. Control starts in state q0, head starts in leftmost square. If the current state is q and head is reading symbol s Γ, the machine transitions according to δ(q,s), which gives: the next state, what tape symbol to overwrite the current square with, and whether the head moves Left or Right. Technicality: moving left from the leftmost square ≡ staying put. Continues until either the accept state or reject state reached. When accept/reject state is reached, M halts. M might also never halt, in which case we say it loops.

slide-64
SLIDE 64

Decidable languages

Definition: A language L Σ* is decidable if there is a Turing Machine M which:

  • 1. Halts on every input x

Σ*.

  • 2. Accepts inputs x L and rejects inputs x L.

Such a Turing Machine is called a decider. It ‘decides’ the language L.

We like deciders. We don’t like TM’s that sometimes loop.

slide-65
SLIDE 65

Computable functions

Definition: A function f: Σ* → (Γ \ { })* is computable if there is a Turing Machine M which: Halts on every input x Σ* with the tape containing f(x) followed by ’s. A function f : Σ* → {0,1} is computable if L = {x Σ* : f(x) = 1} is decidable

slide-66
SLIDE 66

Decidable languages

Examples: Hopefully you’re convinced that {0n1n : n } is decidable. (Recall it’s not “regular”.) The language {02n : n } {0}*, i.e. {0, 00, 0000, 00000000, …}, is decidable. Proof: I’ll show you a decider TM for it…

slide-67
SLIDE 67
  • fig. by Mike Sipser,

using slightly different notation:

slide-68
SLIDE 68

Describing Turing Machines

Low Level: Explicitly describing all states and transitions. Medium Level: Carefully describing in English how the TM

  • perates. Should be ‘obvious’ how to

translate into a Low Level description. High Level: Skips ‘standard’ details, just highlights ‘tricky’ details. For experts only!

slide-69
SLIDE 69

{02n : n } is decidable

Medium Level description:

  • 1. Sweep from left to right across the tape,
  • verwriting a # over top of every other 0.
  • 2. If you saw one 0 on the sweep, accept.
  • 3. If you saw an odd number of 0’s, reject.
  • 4. Move back to the leftmost square.

(Say you write

  • n the leftmost square at the

very beginning so that you can recognize it later.)

  • 5. Go back to step 1.
slide-70
SLIDE 70

TM programming exercises & tricks

  • 1. Move right (or left) until first

encountered.

  • 2. Shift entire input string one cell to the right
  • 3. Convert input x1x2x3···xn to x1 x2 x3 ··· xn
  • 4. Simulate large tape alphabet Γ with just {0,1, }
  • 5. Ability to “mark” cells (e.g., replace symbol a by å)
  • 6. Copy a stretch of tape between two marked cells into

another marked section

  • 7. Increment or Decrement an input in binary.
  • 8. Implement basic string and arithmetic operations
slide-71
SLIDE 71

TM programming exercises & tricks

  • 9. Simulate a TM with 2 tapes and read/write heads
  • 10. Implement a dictionary data structure
  • 11. Simulate “random access memory”
  • 12. ….
  • 13. Simulate an assembly language interpreter
  • 14. Simulate a C interpreter
  • 15. Create a Turing Machine interpreter or Universal TM, i.e.,

a Turing Machine U whose input is M , the encoding of a TM M, x, a string and which simulates the execution of M on x.

slide-72
SLIDE 72

Universal Turing Machine

If you get stuck on the last exercise, you can look up the answer in Turing’s 1936 paper!

slide-73
SLIDE 73

Solvable with Python = Solvable with C = Solvable with Java = Solvable with SML

PRIMALITY 0n1n

Regular Languages (Solvable with DFAs)

Decidable Languages (decidable by Turing Machienes) =

0n1m HALF(AWESOME)

slide-74
SLIDE 74

Church–Turing Thesis:

“Any natural / reasonable notion of computation can be simulated by a TM.”

slide-75
SLIDE 75

Describing Turing Machines

Low Level: Medium Level: High Level: Super-high Level: Just describe an algorithm / pseudocode. Assuming the Church–Turing Thesis there exists a TM which executes that algorithm.

slide-76
SLIDE 76

Definitions:

Turing Machines Decidable languages/ computable functions Universal TM Church−Turing Thesis

Theorems/proofs: {02n : n } is decidable {0n1n : n } is decidable Equivalence of Solvability

(between Python, C, TM)

Practice:

Programming with TM’s

Study Guide