CPSC 121: Models of Computation Module 11: Sets, Functions and DFAs. - - PowerPoint PPT Presentation

CPSC 121 – 2016W T1 1

CPSC 121: Models of Computation

Module 11: Sets, Functions and DFAs.

CPSC 121 – 2016W T1 2

Module 11: Sets, Functions and DFAs

By the start of class, you should be able to:

Define the set operations union, intersection, complement and difference, and the logical

• perations subset and set equality in terms of

predicate logic and set membership. Translate between sets represented explicitly (possibly using ellipses, e.g., { 4, 6, 8, … }) and using "set builder" notation (e.g., { x in Z+ | x2 > 10 and x is even }). Execute set operations on sets expressed explicitly, using set builder notation, or a combination of these.

CPSC 121 – 2016W T1 3

Module 11: Sets, Functions and DFAs

By the start of class, you should be able to:

Interpret the empty set symbol, including the fact that the empty set has no members and that it is a subset

• f any set.

Define the terms domain, co-domain, range, image, and pre-image Use appropriate function syntax to relate these terms (e.g., f : A → B indicates that f is a function mapping domain A to co-domain B). Determine whether f : A → B is a function given a definition for f as an equation or arrow diagram.

CPSC 121 – 2016W T1 4

Module 11: Sets, Functions and DFAs.

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

CPSC 121: the BIG questions:

• 1. How does the computer (e.g. Dr. Racket) decide if

the characters of your program represent a name, a number, or something else? How does it figure

• ut if you have mismatched " " or ( )?

More generally:

What can we compute? Are there problems we can not solve?

CPSC 121 – 2016W T1 5

Module 11: Sets, Functions and DFAs

By the end of this module, you should be able to:

Define the terms injective (one-to-one), surjective (onto), bijective (one-to-one correspondence), and inverse. Determine whether a given function is injective, surjective, and/or bijective. Relate DFAs to sets and functions. Better understand the limitations of DFAs, that is, what they can and can not do.

CPSC 121 – 2016W T1 6

Final Exam information

Announcements:

Final exam: Monday December 19th, 8:30 in OSBO. Questions & Answers Sessions (locations TBD):

Alice: Wednesday December 14th, 14:00 to 16:00 Patrice: Thursday December 15th, 14:00 to 16:00

There will be additional office hours; they will be posted on the course web site.

CPSC 121 – 2016W T1 7

Final Exam Information

Final exam details:

2.5 hours Covers everything discussed in the course

includes labs, although you don't need to memorize all of the solutions.

Slightly more emphasis on later topics. The following will be provided if necessary

Logical equivalences, rules of inference, powers of 2, regular expression syntax

You can bring one 8.5 x 11in 2-sided sheet of paper.

CPSC 121 – 2016W T1 8

Teaching/TA Evaluations

Teaching assistants:

Adam Jozefiak; Angad Kalra; Arshdeep Ganda; Cindy Tu Christine Liu; Elad Michaeli; Hannah Lin; Harsh Patel; Jastine Irwan; Zara Lim; Jessica Que; Jingwei Zhang Jonathan Budiardjo; Leo Sporn; Linh Phan; Luke Schuster Minnie Liu; Pinia Chandra; Sherry Yuan; Willis Peng; Xuejie Tang Yi Ren; Yves Chan

CPSC 121 – 2016W T1 9

Teaching/TA Evaluations

Labs:

CPSC 121 – 2016W T1 10

Module 11: Sets, Functions and DFAs

Module Summary

Sets. Cardinality, Power sets and Cartesian Products. Injections, surjections, bijections and inverses. DFAs and regular expressions. Computations that we are unable to perform.

CPSC 121 – 2016W T1 11

Module 11.1: Sets

How does Set Theory fit in with what we have been doing so far in the course?

Every set operation is defined in terms of predicate logic and the basic predicate ∈. So we prove theorems about sets by

figuring out which predicate logic statement corresponds to the terms and operations used in the theorem. applying already known proof techniques.

No new proof techniques are needed!

CPSC 121 – 2016W T1 12

Module 11.1: Sets

Confusion: ∈versus ⊆.

The ∈ predicate takes in an element x and a set S, and returns true if the element x belongs to S. The ⊆ predicate takes in two sets S and T, and returns true if every element of S is also an element of T. That is, if ∀x∈D, x∈S → x∈T

CPSC 121 – 2016W T1 13

Module 11.1: Sets

Historical Notes:

Mathematicians formalized set theory to create a foundation for all of mathematics. Essentially all mathematical constructs can be defined in terms of sets. Hence sets are a powerful means of formalizing new ideas. But we have to be careful!

CPSC 121 – 2016W T1 14

Module 11.1: Sets

Russel's paradox (part 1).

Let S be the set of all sets that contain themselves. Does S contain itself?

a) Yes, definitely. b) No, certainly not. c) Maybe (either way is fine). d) None of the above.

▷

CPSC 121 – 2016W T1 16

Module 11.1: Sets

Russel's paradox (part 2).

Let S be the set of all sets that do not contain themselves (that is, S ∉S). Does S contain itself?

a) Yes, definitely. b) No, certainly not. c) Maybe (either way is fine). d) None of the above.

▷

CPSC 121 – 2016W T1 18

Module 11.1: Sets

Despite this, sets (and functions) are incredibly useful. E.g. We can definite valid DFAs formally: a DFA is a 5-tuple (Σ, S, s0, F, δ) where

Σ is a finite set of characters (input alphabet). S is a finite set of states. s0 ∈ S is the initial state. F ⊆ S is the set of accepting states. δ: S x Σ → S is the transition function.

CPSC 121 – 2016W T1 19

Module 11: Sets, Functions and DFAs

Module Summary

Sets. Cardinality, Power sets and Cartesian Products. Injections, surjections, bijections and inverses. DFAs and regular expressions. Computations that we are unable to perform.

CPSC 121 – 2016W T1 20

Module 11.2: Cardinality, Power sets, etc

Cardinality: the number of elements of a set S, denoted by |S|. What is the cardinality of the following set: { 1, 2, 3, { a, b, c }, snow, rain } ?

a) 3 b) 6 c) 8 d) Some other integer e) The cardinality of the set is undefined.

▷

CPSC 121 – 2016W T1 22

Module 11.2: Cardinality, Power sets, etc

This gets trickier with infinite sets. OPTIONAL Consider A = Z+, B = Z, C = Q (the rationals). Which of these sets have the smaller and larger cardinality?

a) Smaller: A, larger: C b) Smaller: A and B, larger: C c) All three sets have the same cardinality d) None of the above.

▷

CPSC 121 – 2016W T1 24

Module 11.2: Cardinality, Power sets, etc

Suppose we have a set S with n students, and we want to form a group to go watch a movie.

What are the different groups that can we form?

OR: we have a circuit with n inputs (so S is the set of inputs).

What combinations of inputs might be true?

In both cases, we are looking for all the subsets

• f S. That is, we want to find { T | T ⊆ S }

CPSC 121 – 2016W T1 25

Module 11.2: Cardinality, Power sets, etc

The power set of a set S, denoted P (S), is the set whose elements are all subsets of S. Given the definitions A = { a, b, f }, B = { b, c }, what are:

a) P (A) b) P (A - B) c) |P (A ∩ B)| d) |P (A ∪ B)| e) Ac

CPSC 121 – 2016W T1 26

Module 11.2: Cardinality, Power sets, etc

The Cartesian product of two sets S and T, denoted S x T, is the set { (s, t) | s S ∈ ∧ t T } ∈ .

Each element of S x T is called an ordered tuple, or simply a tuple (or, in this case, a 2-tuple or a pair). Two ordered tuples are equal if the elements in corresponding positions are the same.

Examples:

(3, 7) = (4 – 1, 14 / 2) (3, 7) ≠ (7, 3)

CPSC 121 – 2016W T1 27

Module 11.2: Cardinality, Power sets, etc

We can visualize Z+ x Z+ as follows: What is { a, b } x { 8, 5, 1 } ?

1 2 3 4 5 1 2 3 4 5 6 7 8

CPSC 121 – 2016W T1 28

Module 11: Sets, Functions and DFAs

Module Summary

Sets. Cardinality, Power sets and Cartesian Products. Injections, surjections, bijections and inverses. DFAs and regular expressions. Computations that we are unable to perform.

CPSC 121 – 2016W T1 29

Module 11.3: Special types of functions

You are already familiar with functions:

Every combinational circuit implements a function.

e.g. m : B x B x B → B m(a, b, c) = a if c is false, b if c is true

Which function is this?

Sequential circuits and Java methods are not always functions. Why not?

booleans

CPSC 121 – 2016W T1 30

Module 11.3: Special types of functions

You are already familiar with functions:

Every combinational circuit implements a function.

e.g. m : B x B x B → B m(a, b, c) = a if c is false, b if c is true

Which function is this?

Sequential circuits and Java methods are not always functions. Why not?

booleans

CPSC 121 – 2016W T1 31

Module 11.3: Special types of functions

Some special types of functions:

A function f : A → B is injective (one-to-one) if x A, y A, x ∀ ∈ ∀ ∈ ≠ y → f(x) ≠ f(y).

In the arrow diagram: at most one arrow points to each element of B.

George CPSC 121/202 Patrice CPSC 121/203 Neil CPSC 210/BCS Kimberly CPSC 310 Gail CPSC 319 CPSC 110/201 George CPSC 110 Patrice CPSC 121 Gail CPSC 210 Kimberly CPSC 310 Neil CPSC 319 Injective Not injective: f(George) = f(Patrice)

CPSC 121 – 2016W T1 32

Module 11.3: Special types of functions

Some special types of functions (continued)

A function f : A → B is surjective (onto) if y B, x A, ∀ ∈ ∃ ∈

f(x) = y. In the arrow diagram: at least one arrow points to each element of B.

George CPSC 121/202 Patrice CPSC 121/203 Neil CPSC 210/BCS Kimberly CPSC 310 Gail CPSC 319 CPSC 110/201 George Patrice CPSC 121 Gail CPSC 210 Kimberly CPSC 310 Neil CPSC 319 Not Surjective Surjective

CPSC 121 – 2016W T1 33

Module 11.3: Special types of functions

Some special types of functions (continued)

A function f : A → B is bijective if it is both injective and surjective

In the arrow diagram: exactly one arrow points to each element of B.

George CPSC 121/202 Patrice CPSC 121/203 Neil CPSC 210/BCS Kimberly CPSC 310 Gail CPSC 319 CPSC 110/201 George Patrice CPSC 121 Gail CPSC 210 Kimberly CPSC 310 Neil CPSC 319 Not Bijective Not Bijective either

CPSC 121 – 2016W T1 34

Module 11.3: Special types of functions

Some special types of functions (continued)

Patrice CPSC 121 Gail CPSC 210 Kimberly CPSC 310 Neil CPSC 319 Bijective

CPSC 121 – 2016W T1 35

Module 11.3: Special types of functions

The inverse of a function f: A → B, denoted f -1, is the set { (y, x) ∈ B x A | y = f(x) }. In other words:

If we think of a function as a list of pairs.

E.g. f(x) = x2: { (1, 1), (2, 4), (3, 9), (4, 16), ... }

Then f -1 is obtained by swapping the elements of each pair:

f-1 = { (1, 1), (4, 2), (9, 3), (16, 4), ... }

CPSC 121 – 2016W T1 36

Module 11.3: Special types of functions

Is f -1 a function?

a) Yes, always. b) No, never. c) Yes, but only if f is injective. d) Yes, but only if f is surjective. e) None of the above.

▷

CPSC 121 – 2016W T1 38

Module 11: Sets, Functions and DFAs

Module Summary

Sets. Cardinality, Power sets and Cartesian Products. Injections, surjections, bijections and inverses. DFAs and regular expressions. Computations that we are unable to perform.

CPSC 121 – 2016W T1 39

Module 11.4: DFAs and regular expressions

DFAs and Regular expressions.

In lab 8, you wrote regular expressions matching the patterns we gave you. Regular expressions are very useful when you need to read and validate input. Many modern programming languages provide functions that allow you to manipulate regular expressions:

• Dr. Racket, Java, Python, Perl, etc.

CPSC 121 – 2016W T1 40

Module 11.4: DFAs and regular expressions

How does a program determine if an input string matches a regular expression?

Theorem: every set of strings matched by a regular expression can be recognized by a DFA. Hence the functions build a DFA corresponding to the regular expression. And then this DFA is given the input string one character at a time.

CPSC 121 – 2016W T1 41

Module 11.4: DFAs and regular expressions

This result goes both ways:

if a set of strings is accepted by a DFA, then there is a regular expression for this set of strings. In this sense, DFAs are exactly as “powerful” as regular expressions (no more, no less).

How do we build the DFA given a regular expression?

First we build a NFA (non-deterministic finite-state automaton) for the regular expression. Then we convert the NFA into a DFA.

CPSC 121 – 2016W T1 42

Module 11.4: DFAs and regular expressions

What is a NFA?

It is like a DFA but

There can be multiple arrows with the same label leaving from a state There can be arrows labelled that we can take without reading the next input character. So we can sometimes choose which state to go to.

A NFA accepts a string if at least one sequence of choices leads to an accepting state. ε

CPSC 121 – 2016W T1 43

Module 11.4: DFAs and regular expressions

Example:

ε ε

CPSC 121 – 2016W T1 44

Module 11.4: DFAs and regular expressions

What regular expression corresponds to the strings that this NFA accepts?

a) ε | ab b) ε | abaa c) ε | ab | abaa d) ε | ab | (aba)+a e) None of the above.

empty string

▷

CPSC 121 – 2016W T1 46

Module 11.4: DFAs and regular expressions

Theorem: we can transform every regular expression into a NFA with

Exactly one accepting state No arcs pointing to the initial state No arcs leaving the accepting state

Fact: every regular expression can be rewritten to use only the following:

The empty string Individual characters The operators |, *, and string concatenation. ε.

CPSC 121 – 2016W T1 47

Module 11.4: DFAs and regular expressions

Exercise: rewrite the following expressions to use only the options listed on the previous slide:

a? a+ a{3,5} \d

CPSC 121 – 2016W T1 48

Module 11.4: DFAs and regular expressions

Proof: by induction on the structure of the regular expression. Base cases:

The expression that matches the empty string: The expression that matches no string:

ε

CPSC 121 – 2016W T1 49

Module 11.4: DFAs and regular expressions

Base cases (continued)

The expression that matches a single character a:

Induction step:

The expression E1|E2 where E1, E2 are regular expressions:

ε ε ε ε

CPSC 121 – 2016W T1 50

Module 11.4: DFAs and regular expressions

Induction step (continued and finished):

The expression E1E2 where E1, E2 are regular expressions: The expression E* where E is a regular expression:

ε ε ε ε ε

CPSC 121 – 2016W T1 51

Module 11.4: DFAs and regular expressions

Example: (a|b)*c

a b a | b (a | b)*

CPSC 121 – 2016W T1 52

Module 11.4: DFAs and regular expressions

How do we transform a NFA into a DFA?

A DFA that reads a string with n characters ends up in exactly one state. A NFA that reads a string with n characters may end up in many different stages. Can we figure out which states?

CPSC 121 – 2016W T1 53

Module 11.4: DFAs and regular expressions

Which state(s) will the following NFA end up in after reading the string ab?

a) S1 only b) S6 only c) S3 or S7 d) S4 or S6 e) None of the above

ε ε

▷

CPSC 121 – 2016W T1 55

Module 11.4: DFAs and regular expressions

So we build the DFA as follows:

Each state of the DFA is a subset of the states of the NFA. If the DFA is in state { Si1, Si2, ..., Sik }, and it sees a character x, then the new state is the state that contains every NFA state that we can get to from

• ne of Si1, Si2, ..., Sik upon reading x.

A state of the DFA is accepting if it contains the accepting state of the NFA.

CPSC 121 – 2016W T1 56

Module 11: Sets, Functions and DFAs

Module Summary

Sets. Cardinality, Power sets and Cartesian Products. Injections, surjections, bijections and inverses. DFAs and regular expressions. Computations that we are unable to perform.

CPSC 121 – 2016W T1 57

Module 11.5: Computations we are unable to perform

We have discussed several models of computation in the course:

Combinational circuits Sequential circuits (the working computer). DFAs

One thing computer scientists (we) like to know about their (our) computational models is:

What can they do? What can they not do?

CPSC 121 – 2016W T1 58

Module 11.5: Computations we are unable to perform

Example: DFAs

Intuition:

DFAs have no memory apart from the current state. So a DFA with n states can only “count” up to n before it gets confused.

How do we formalize this?

We need to define a set L of strings (language). And show that no DFA can accept exactly the strings in L.

CPSC 121 – 2016W T1 59

Module 11.5: Computations we are unable to perform

Definition: we denote by anbn a string that consists of

n copies of the letter a, followed by n copies of the letter b.

Theorem: no DFA can recognize the language { anbn | n Z ∈

+ }.

CPSC 121 – 2016W T1 60

Module 11.5: Computations we are unable to perform

Proof: we use a proof by contradiction.

Suppose there is such a DFA. This DFA has k states for some positive integer k. Now look at what happens when the DFA is looking at the input akbk. While it's reading the input string, it goes through a number of states:

CPSC 121 – 2016W T1 61

Module 11.5: Computations we are unable to perform

q0 (initial state) q1 (after reading the string a) q2 (after reading the string aa) ... qk (after reading the string ak).

The DFA only has k different states, so two of q0, q1, ..., qk must be the same state. Let us call these two qi and qj, with i < j.

CPSC 121 – 2016W T1 62

Module 11.5: Computations we are unable to perform

Observation 1: If the DFA is in state qi and it sees j – i copies of a then it ends up in state

a) q0 b) qi c) qk d) Some other state.

▷

CPSC 121 – 2016W T1 64

Module 11.5: Computations we are unable to perform

Observation 2: If the DFA is in state qi and it sees k – j copies of a then it ends up in state .

a) q0 b) qi c) qk d) Some other state.

▷

CPSC 121 – 2016W T1 66

Module 11.5: Computations we are unable to perform

What happens if we give the DFA the string ak+(j-i)bk?

While reading the first i copies of a, it goes through states q0 .. qi. Then it reads the next j – i copies of a, and ends up in state qi again. From qi it reads the next k – j copies of a, and ends up in state qk. Then it reads the k copies of b, and terminates in the same state it terminated when it was reading akbk.

CPSC 121 – 2016W T1 67

Module 11.5: Computations we are unable to perform

But akbk should be accepted, and ak+(j-i)bk should be rejected! So the DFA will make a mistake on one of these two strings, which means it's not recognizing { anbn | n ∈ Z+ } correctly. This contradictions our initial assumption, and hence no DFA can recognize this language. QED

CPSC 121 – 2016W T1 68

Module 11.5: Computations we are unable to perform

But what about sequential circuits/Java/Racket?

They are more powerful than DFAs. For instance, you can easily write a Racket function

• r a Java method that will recognize { anbn | n

Z ∈

+ }.

Can they solve every problem?

No: there are problems that can not be solved.

Halting Problem: given a program P and an input I, will P halt if we run it on input I?

CPSC 121 – 2016W T1 69

Module 11.5: Computations we are unable to perform

Theorem: It is not possible to write a program that solves the halting problem.

Aside: remember Russel's paradox? We will use a similar trick to prove this theorem.

Proof 1: proof by video:

https://www.youtube.com/watch?v=92WHN-pAFCs

CPSC 121 – 2016W T1 70

Module 11.5: Computations we are unable to perform

Proof 2: we use a proof by contradiction.

Suppose this program exists. Let us call it will-halt (Racket) or willHalt (Java). We use this function or method to write the following function or method:

CPSC 121 – 2016W T1 71

Module 11.5: Computations we are unable to perform

Racket version:

(define (paradox input) (if (will-halt input input) (paradox input) ; go into an infinite recursion true))

Java version:

public static void main(String[ ] args) { if (willHalt(args[0], args[0])) while(true) ; else

return;

}

CPSC 121 – 2016W T1 72

Module 11.5: Computations we are unable to perform

What happens when we call this program with itself as input?

If it halts, then will-halt/willHalt returns true, and so it won't halt. But if it doesn't halt, then will-halt/willHalt returns false, and so it will halt.

So whether the program halts or not, we end up with a contradiction. Therefore this program does not exist. QED

CPSC 121 – 2016W T1 73

Module 11: Sets, Functions and DFAs

 The End   The End 