Taaltheorie en Taalverwerking BSc Artificial Intelligence Raquel - - PowerPoint PPT Presentation

Taaltheorie en Taalverwerking

BSc Artificial Intelligence

Raquel Fernández Institute for Logic, Language, and Computation

Winter 2014, lecture 1a

Raquel Fernández TTTV 2014 - lecture 1a 1

TTTV: Practical Matters

• Lecturer:

∗ Raquel Fernández – raquel.fernandez@uva.nl Institute for Logic, Language & Computation (ILLC) ∗ Building SP 107, room F1.07 (meeting by appointment) ∗ http://www.illc.uva.nl/~raquel

Raquel Fernández TTTV 2014 - lecture 1a 2

TTTV: Practical Matters

• Lecturer:

∗ Raquel Fernández – raquel.fernandez@uva.nl Institute for Logic, Language & Computation (ILLC) ∗ Building SP 107, room F1.07 (meeting by appointment) ∗ http://www.illc.uva.nl/~raquel

• Teaching Assistants: meet them today at the practical session

∗ Group A: Lewis Zwart ∗ Group B: Mart van Baalen ∗ Group C: Nick de Wolf ∗ Group D: Ysbrand Galama

Raquel Fernández TTTV 2014 - lecture 1a 2

TTTV: Practical Matters

• Lecturer:

∗ Raquel Fernández – raquel.fernandez@uva.nl Institute for Logic, Language & Computation (ILLC) ∗ Building SP 107, room F1.07 (meeting by appointment) ∗ http://www.illc.uva.nl/~raquel

• Teaching Assistants: meet them today at the practical session

∗ Group A: Lewis Zwart ∗ Group B: Mart van Baalen ∗ Group C: Nick de Wolf ∗ Group D: Ysbrand Galama

• Course information: visit the Blackboard site for the course

∗ studiewijzer ∗ weekly materials such as readings and lecture slides ∗ exercises, assignments, including submissions

• Book: Jurafsky & Martin, Speech and Language Processing

Raquel Fernández TTTV 2014 - lecture 1a 2

What is this course about?

Raquel Fernández TTTV 2014 - lecture 1a 3

What is this course about?

Taaltheorie en Taalverwerking ≈ Computational Linguistics

Raquel Fernández TTTV 2014 - lecture 1a 3

What is this course about?

Taaltheorie en Taalverwerking ≈ Computational Linguistics ≈ Natural Language Processing

Raquel Fernández TTTV 2014 - lecture 1a 3

What is this course about?

Taaltheorie en Taalverwerking ≈ Computational Linguistics ≈ Natural Language Processing

“Computational linguistics (CL) is a discipline at the interface of linguistics and computer science which is concerned with the computational aspects of the human language faculty. It belongs to the cognitive sciences and overlaps with the field of artificial intelligence, a branch of computer science aiming at computational models of human cognition.” Hans Uszkoreit

Raquel Fernández TTTV 2014 - lecture 1a 3

Levels of Language Processing

Language is very easy for us, but very difficult for computers. What kind of knowledge do we need to be able to process, understand, and properly react to language?

Raquel Fernández TTTV 2014 - lecture 1a 4

Levels of Language Processing

Language is very easy for us, but very difficult for computers. What kind of knowledge do we need to be able to process, understand, and properly react to language?

• phonetics and phonology

the sequences of sounds that make up a language

• morphology

the meaningful components of words

• syntax

the structural relationships between words in a sentence

• semantics

the meaning of words and sentences

• pragmatics

the relation between meaning and the intentions of the speakers

Raquel Fernández TTTV 2014 - lecture 1a 4

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president”

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what (to whom)

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what (to whom) (when

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what (to whom) (when, where

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what (to whom) (when, where ...why)

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what (to whom) (when, where ...why) ∃x[President(x) ∧ Phone(Mary, x)]

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what (to whom) (when, where ...why) ∃x[President(x) ∧ Phone(Mary, x)]

morphology & syntax help us to get to the meaning we are after

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what (to whom) (when, where ...why) ∃x[President(x) ∧ Phone(Mary, x)]

morphology & syntax help us to get to the meaning we are after

• Sometimes, we need less than deep semantic representations

bag of words approach: {phone, president} ≈ {call, leader}

Raquel Fernández TTTV 2014 - lecture 1a 5

Levels of Language Processing

• Most often, we are interested in semantics:

“Mary phoned the president” who did what (to whom) (when, where ...why) ∃x[President(x) ∧ Phone(Mary, x)]

morphology & syntax help us to get to the meaning we are after

• Sometimes, we need less than deep semantic representations

bag of words approach: {phone, president} ≈ {call, leader}

• Sometimes we need more. . . pragmatics

“Mary has a gun” a statement? a warning? “Can you plass me the salt?” a question? a command?

Raquel Fernández TTTV 2014 - lecture 1a 5

In this course we’ll focus on. . .

• syntax, with a bit of morphology (part 1)
• semantics (part 2)

See the list of readings in BB and the learning objectives in the studiewijzer.

Raquel Fernández TTTV 2014 - lecture 1a 6

Related courses in the AI curriculum

We will build on knowledge and skills you have acquired during the first semester of the 1st year:

• Logisch Programmeren en Zoektechnieken
• Inleiding Logica

Raquel Fernández TTTV 2014 - lecture 1a 7

Related courses in the AI curriculum

We will build on knowledge and skills you have acquired during the first semester of the 1st year:

• Logisch Programmeren en Zoektechnieken
• Inleiding Logica

Other language-related courses in subsequent years:

• 2nd year: Natuurlijke Taalmodellen en Interfaces
• 3rd year: Discourse

Possibility to do your Bachelor’s thesis on language-related topics.

Raquel Fernández TTTV 2014 - lecture 1a 7

Let’s get started

We will start by looking into the structural properties of language. In particular, this week we’ll look into Formal Language Theory, which studies the structural regularities of “languages” as abstract entities.

Raquel Fernández TTTV 2014 - lecture 1a 8

Formal Languages: strings and alphabets

A formal language is a set of strings, each string composed of symbols from a finite set called an alphabet (or a vocabulary).

Raquel Fernández TTTV 2014 - lecture 1a 9

Formal Languages: strings and alphabets

A formal language is a set of strings, each string composed of symbols from a finite set called an alphabet (or a vocabulary).

Examples

• Let Σ1 = {0, 1} be an alphabet. Then all binary numbers are strings over Σ1.

For instance: 01101, 000001, 1101.

Raquel Fernández TTTV 2014 - lecture 1a 9

Formal Languages: strings and alphabets

A formal language is a set of strings, each string composed of symbols from a finite set called an alphabet (or a vocabulary).

Examples

• Let Σ1 = {0, 1} be an alphabet. Then all binary numbers are strings over Σ1.

For instance: 01101, 000001, 1101.

• Let Σ2 = {a, b, c, d, e, f , g} be an alphabet.

Raquel Fernández TTTV 2014 - lecture 1a 9

Formal Languages: strings and alphabets

A formal language is a set of strings, each string composed of symbols from a finite set called an alphabet (or a vocabulary).

Examples

• Let Σ1 = {0, 1} be an alphabet. Then all binary numbers are strings over Σ1.

For instance: 01101, 000001, 1101.

• Let Σ2 = {a, b, c, d, e, f , g} be an alphabet. Then bee, dad, cabbage, and face are

strings over Σ2, as are fffff and agagag.

Raquel Fernández TTTV 2014 - lecture 1a 9

Formal Languages: strings and alphabets

A formal language is a set of strings, each string composed of symbols from a finite set called an alphabet (or a vocabulary).

Examples

• Let Σ1 = {0, 1} be an alphabet. Then all binary numbers are strings over Σ1.

For instance: 01101, 000001, 1101.

• Let Σ2 = {a, b, c, d, e, f , g} be an alphabet. Then bee, dad, cabbage, and face are

strings over Σ2, as are fffff and agagag.

• Let Σ3 = {ba, ca, fa, ce, fe, ge} be an alphabet.

Raquel Fernández TTTV 2014 - lecture 1a 9

Formal Languages: strings and alphabets

A formal language is a set of strings, each string composed of symbols from a finite set called an alphabet (or a vocabulary).

Examples

• Let Σ1 = {0, 1} be an alphabet. Then all binary numbers are strings over Σ1.

For instance: 01101, 000001, 1101.

• Let Σ2 = {a, b, c, d, e, f , g} be an alphabet. Then bee, dad, cabbage, and face are

strings over Σ2, as are fffff and agagag.

• Let Σ3 = {ba, ca, fa, ce, fe, ge} be an alphabet. Then face is a string over Σ3 but

bee, dad or cabbage are not.

Raquel Fernández TTTV 2014 - lecture 1a 9

Formal Languages: strings and alphabets

A formal language is a set of strings, each string composed of symbols from a finite set called an alphabet (or a vocabulary).

Examples

• Let Σ1 = {0, 1} be an alphabet. Then all binary numbers are strings over Σ1.

For instance: 01101, 000001, 1101.

• Let Σ2 = {a, b, c, d, e, f , g} be an alphabet. Then bee, dad, cabbage, and face are

strings over Σ2, as are fffff and agagag.

• Let Σ3 = {ba, ca, fa, ce, fe, ge} be an alphabet. Then face is a string over Σ3 but

bee, dad or cabbage are not.

• Let Σ4 = {♠, △, ♣} be an alphabet. Then ♠♠ and ♣△♣ are strings over Σ4.

Raquel Fernández TTTV 2014 - lecture 1a 9

Strings and Substrings

The length of a string is the number of token symbols from the alphabet it contains.

Raquel Fernández TTTV 2014 - lecture 1a 10

Strings and Substrings

The length of a string is the number of token symbols from the alphabet it contains.

Examples

• the length of face over Σ2 = {a, b, c, d, e, f , g} is 4

Raquel Fernández TTTV 2014 - lecture 1a 10

Strings and Substrings

The length of a string is the number of token symbols from the alphabet it contains.

Examples

• the length of face over Σ2 = {a, b, c, d, e, f , g} is 4
• the length of face over Σ3 = {ba, ca, fa, ce, fe, ge} is

Raquel Fernández TTTV 2014 - lecture 1a 10

Strings and Substrings

The length of a string is the number of token symbols from the alphabet it contains.

Examples

• the length of face over Σ2 = {a, b, c, d, e, f , g} is 4
• the length of face over Σ3 = {ba, ca, fa, ce, fe, ge} is 2

Raquel Fernández TTTV 2014 - lecture 1a 10

Strings and Substrings

The length of a string is the number of token symbols from the alphabet it contains.

Examples

• the length of face over Σ2 = {a, b, c, d, e, f , g} is 4
• the length of face over Σ3 = {ba, ca, fa, ce, fe, ge} is 2

The string of length 0 is called the empty string, denoted ǫ

Raquel Fernández TTTV 2014 - lecture 1a 10

Strings and Substrings

The length of a string is the number of token symbols from the alphabet it contains.

Examples

• the length of face over Σ2 = {a, b, c, d, e, f , g} is 4
• the length of face over Σ3 = {ba, ca, fa, ce, fe, ge} is 2

The string of length 0 is called the empty string, denoted ǫ Given a string s, a substring of s is a string formed by taking contiguous symbols of s in the order in which they occurr in s. An initial substring is called a prefix and a final substring, a suffix.

Raquel Fernández TTTV 2014 - lecture 1a 10

Strings and Substrings

The length of a string is the number of token symbols from the alphabet it contains.

Examples

• the length of face over Σ2 = {a, b, c, d, e, f , g} is 4
• the length of face over Σ3 = {ba, ca, fa, ce, fe, ge} is 2

The string of length 0 is called the empty string, denoted ǫ Given a string s, a substring of s is a string formed by taking contiguous symbols of s in the order in which they occurr in s. An initial substring is called a prefix and a final substring, a suffix.

Examples Let unthinkable be a string over Σ = {a, b, c . . . x, y, z}

Raquel Fernández TTTV 2014 - lecture 1a 10

Strings and Substrings

The length of a string is the number of token symbols from the alphabet it contains.

Examples

• the length of face over Σ2 = {a, b, c, d, e, f , g} is 4
• the length of face over Σ3 = {ba, ca, fa, ce, fe, ge} is 2

The string of length 0 is called the empty string, denoted ǫ Given a string s, a substring of s is a string formed by taking contiguous symbols of s in the order in which they occurr in s. An initial substring is called a prefix and a final substring, a suffix.

Examples Let unthinkable be a string over Σ = {a, b, c . . . x, y, z} Then, ǫ, un, unth, unthinkable are prefixes, while ǫ, e, able, thinkable, and unthinkable are suffixes. Other substrings include nthi, inka, bl.

Raquel Fernández TTTV 2014 - lecture 1a 10

Some Operations on Strings

Raquel Fernández TTTV 2014 - lecture 1a 11

Some Operations on Strings

• Concatenation: two string s1 and s2 over Σ can be concatenated

(written one after the other) to form a new string s1 · s2 over Σ.

Σ = {a, b} a · b = ab

Raquel Fernández TTTV 2014 - lecture 1a 11

Some Operations on Strings

• Concatenation: two string s1 and s2 over Σ can be concatenated

(written one after the other) to form a new string s1 · s2 over Σ.

Σ = {a, b} a · b = ab

• Exponent: we can apply an exponent operator n to a string s.

The resulting string sn is obtained by concatenating s with itself n times.

a0 = ǫ, a1 = a, a2 = aa, a3 = aaa. . .

Raquel Fernández TTTV 2014 - lecture 1a 11

Some Operations on Strings

• Concatenation: two string s1 and s2 over Σ can be concatenated

(written one after the other) to form a new string s1 · s2 over Σ.

Σ = {a, b} a · b = ab

• Exponent: we can apply an exponent operator n to a string s.

The resulting string sn is obtained by concatenating s with itself n times.

a0 = ǫ, a1 = a, a2 = aa, a3 = aaa. . .

• Kleene star: a special exponent operator ∗ which applied to a

string s denotes any string obtained by concatenating s with itself any number of times.

a∗ = ǫ or a or aa or aaa . . .

Raquel Fernández TTTV 2014 - lecture 1a 11

Formal Languages

Σ∗

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

We may now define a formal language over an alphabet Σ as any subset of Σ∗

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

We may now define a formal language over an alphabet Σ as any subset of Σ∗

Examples of formal languages Let Σ = {a, b, c . . . x, y, z}. Then Σ∗ is the set of strings over the Latin alphabet and the following subsets of Σ∗ are possible formal languages:

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

We may now define a formal language over an alphabet Σ as any subset of Σ∗

Examples of formal languages Let Σ = {a, b, c . . . x, y, z}. Then Σ∗ is the set of strings over the Latin alphabet and the following subsets of Σ∗ are possible formal languages:

• the set of strings consisting of consonants only

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

We may now define a formal language over an alphabet Σ as any subset of Σ∗

Examples of formal languages Let Σ = {a, b, c . . . x, y, z}. Then Σ∗ is the set of strings over the Latin alphabet and the following subsets of Σ∗ are possible formal languages:

• the set of strings consisting of consonants only
• the set of strings containing at least one vowel and one consonant

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

We may now define a formal language over an alphabet Σ as any subset of Σ∗

Examples of formal languages Let Σ = {a, b, c . . . x, y, z}. Then Σ∗ is the set of strings over the Latin alphabet and the following subsets of Σ∗ are possible formal languages:

• the set of strings consisting of consonants only
• the set of strings containing at least one vowel and one consonant
• the set of strings whose length is less than 9 symbols

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

We may now define a formal language over an alphabet Σ as any subset of Σ∗

Examples of formal languages Let Σ = {a, b, c . . . x, y, z}. Then Σ∗ is the set of strings over the Latin alphabet and the following subsets of Σ∗ are possible formal languages:

• the set of strings consisting of consonants only
• the set of strings containing at least one vowel and one consonant
• the set of strings whose length is less than 9 symbols
• the set {one, two, three, four, five, six, seven, eight, nine, ten}

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

We may now define a formal language over an alphabet Σ as any subset of Σ∗

Examples of formal languages Let Σ = {a, b, c . . . x, y, z}. Then Σ∗ is the set of strings over the Latin alphabet and the following subsets of Σ∗ are possible formal languages:

• the set of strings consisting of consonants only
• the set of strings containing at least one vowel and one consonant
• the set of strings whose length is less than 9 symbols
• the set {one, two, three, four, five, six, seven, eight, nine, ten}
• the set of all English words

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

Σ∗ denotes the set of all strings over an alphabet Σ.

→ note that Σ∗ is always infinite, regardless of the number of symbols Σ contains.

We may now define a formal language over an alphabet Σ as any subset of Σ∗

Examples of formal languages Let Σ = {a, b, c . . . x, y, z}. Then Σ∗ is the set of strings over the Latin alphabet and the following subsets of Σ∗ are possible formal languages:

• the set of strings consisting of consonants only
• the set of strings containing at least one vowel and one consonant
• the set of strings whose length is less than 9 symbols
• the set {one, two, three, four, five, six, seven, eight, nine, ten}
• the set of all English words
• the empty set

Raquel Fernández TTTV 2014 - lecture 1a 12

Formal Languages

How can we characterise the language(s) we are interested in?

• given an alphabet Σ and the infinite set Σ∗ of formal languages it can

give rise to, how can we select a particular formal language?

Raquel Fernández TTTV 2014 - lecture 1a 13

Formal Languages

How can we characterise the language(s) we are interested in?

• given an alphabet Σ and the infinite set Σ∗ of formal languages it can

give rise to, how can we select a particular formal language? For instance: ∗ can we distinguish the set of strings of letters that constitute proper English words? ∗ can we distinguish the set of strings of words that count as well-formed sentences of Dutch?

Raquel Fernández TTTV 2014 - lecture 1a 13

Formal Languages

How can we characterise the language(s) we are interested in?

• given an alphabet Σ and the infinite set Σ∗ of formal languages it can

give rise to, how can we select a particular formal language? For instance: ∗ can we distinguish the set of strings of letters that constitute proper English words? ∗ can we distinguish the set of strings of words that count as well-formed sentences of Dutch?

We have two formal mechanisms at our disposal:

• formalisms (formal expressions and grammars): sets of rules
• automata: computational devices for computing languages

Raquel Fernández TTTV 2014 - lecture 1a 13

Formalisms and Automata

• Formalisms and automata allow us to distinguish a formal

language of interest (a set of strings) from other possible languages over a given alphabet.

∗ they capture the patterns that characterise a language ∗ as such, they act as a definition of the language they capture

• From an abstract point of view, a natural language – like Dutch
• r English – is a set of strings (of sounds/letters, of words, etc.)
• Therefore, formalisms and automata can help us to model

aspects of natural languages.

Raquel Fernández TTTV 2014 - lecture 1a 14

Formalisms and Automata

• Formalisms and automata allow us to distinguish a formal

language of interest (a set of strings) from other possible languages over a given alphabet.

∗ they capture the patterns that characterise a language ∗ as such, they act as a definition of the language they capture

• From an abstract point of view, a natural language – like Dutch
• r English – is a set of strings (of sounds/letters, of words, etc.)
• Therefore, formalisms and automata can help us to model

aspects of natural languages. This week: we’ll look into one formalism to define formal languages, regular expressions, and into one type of automaton, finite state automata.

Raquel Fernández TTTV 2014 - lecture 1a 14

Regular Expressions

Regular expressions are a formal notation for characterising sets of strings that follow a fairly simple regular pattern.

Raquel Fernández TTTV 2014 - lecture 1a 15

Regular Expressions

Regular expressions are a formal notation for characterising sets of strings that follow a fairly simple regular pattern. We can construct regular expressions over an alphabet Σ as follows:

Regular expression Languages empty set: ∅ {} empty string: ǫ {ǫ} symbol (∀a ∈ Σ): a {a} If a and b are reg exp, so are: concatenation: a · b {ab} disjunction (or union): (a|b) {a, b} Kleene star (or closure): a∗ {ǫ, a, aa, aaa, aaaaa, . . .}

Raquel Fernández TTTV 2014 - lecture 1a 15

Regular Expressions

Regular expressions are a formal notation for characterising sets of strings that follow a fairly simple regular pattern. We can construct regular expressions over an alphabet Σ as follows:

Regular expression Languages empty set: ∅ {} empty string: ǫ {ǫ} symbol (∀a ∈ Σ): a {a} If a and b are reg exp, so are: concatenation: a · b {ab} disjunction (or union): (a|b) {a, b} Kleene star (or closure): a∗ {ǫ, a, aa, aaa, aaaaa, . . .}

• we often ignore the dot in concatenation (a · b) and simply write ab
• disjunction (or union) may be written as a|b, a + b or a ∪ b

Raquel Fernández TTTV 2014 - lecture 1a 15

Regular Expressions

Regular expressions are a formal notation for characterising sets of strings that follow a fairly simple regular pattern. We can construct regular expressions over an alphabet Σ as follows:

Regular expression Languages empty set: ∅ {} empty string: ǫ {ǫ} symbol (∀a ∈ Σ): a {a} If a and b are reg exp, so are: concatenation: a · b {ab} disjunction (or union): (a|b) {a, b} Kleene star (or closure): a∗ {ǫ, a, aa, aaa, aaaaa, . . .}

• we often ignore the dot in concatenation (a · b) and simply write ab
• disjunction (or union) may be written as a|b, a + b or a ∪ b
• a+ is the set of a-strings with at least one a;

Raquel Fernández TTTV 2014 - lecture 1a 15

Regular Expressions

Regular expressions are a formal notation for characterising sets of strings that follow a fairly simple regular pattern. We can construct regular expressions over an alphabet Σ as follows:

Regular expression Languages empty set: ∅ {} empty string: ǫ {ǫ} symbol (∀a ∈ Σ): a {a} If a and b are reg exp, so are: concatenation: a · b {ab} disjunction (or union): (a|b) {a, b} Kleene star (or closure): a∗ {ǫ, a, aa, aaa, aaaaa, . . .}

• we often ignore the dot in concatenation (a · b) and simply write ab
• disjunction (or union) may be written as a|b, a + b or a ∪ b
• a+ is the set of a-strings with at least one a; can be used to abbreviate a∗a or aa∗

Raquel Fernández TTTV 2014 - lecture 1a 15

Regular Expressions

Regular expressions are a formal notation for characterising sets of strings that follow a fairly simple regular pattern. We can construct regular expressions over an alphabet Σ as follows:

Regular expression Languages empty set: ∅ {} empty string: ǫ {ǫ} symbol (∀a ∈ Σ): a {a} If a and b are reg exp, so are: concatenation: a · b {ab} disjunction (or union): (a|b) {a, b} Kleene star (or closure): a∗ {ǫ, a, aa, aaa, aaaaa, . . .}

• we often ignore the dot in concatenation (a · b) and simply write ab
• disjunction (or union) may be written as a|b, a + b or a ∪ b
• a+ is the set of a-strings with at least one a; can be used to abbreviate a∗a or aa∗
• the notation Σ∗ can be seen as abbreviating (a|b|...)∗ for any symbol a, b, . . . in Σ.

Raquel Fernández TTTV 2014 - lecture 1a 15

Regular Expressions

Regular expressions are a formal notation for characterising sets of strings that follow a fairly simple regular pattern. We can construct regular expressions over an alphabet Σ as follows:

Regular expression Languages empty set: ∅ {} empty string: ǫ {ǫ} symbol (∀a ∈ Σ): a {a} If a and b are reg exp, so are: concatenation: a · b {ab} disjunction (or union): (a|b) {a, b} Kleene star (or closure): a∗ {ǫ, a, aa, aaa, aaaaa, . . .}

• we often ignore the dot in concatenation (a · b) and simply write ab
• disjunction (or union) may be written as a|b, a + b or a ∪ b
• a+ is the set of a-strings with at least one a; can be used to abbreviate a∗a or aa∗
• the notation Σ∗ can be seen as abbreviating (a|b|...)∗ for any symbol a, b, . . . in Σ.
• Σn can be seen as abbreviating the concatenation of (a|b|...) with itself n times

Raquel Fernández TTTV 2014 - lecture 1a 15

Regular Expressions

Regular expressions are a formal notation for characterising sets of strings that follow a fairly simple regular pattern. We can construct regular expressions over an alphabet Σ as follows:

Regular expression Languages empty set: ∅ {} empty string: ǫ {ǫ} symbol (∀a ∈ Σ): a {a} If a and b are reg exp, so are: concatenation: a · b {ab} disjunction (or union): (a|b) {a, b} Kleene star (or closure): a∗ {ǫ, a, aa, aaa, aaaaa, . . .}

• we often ignore the dot in concatenation (a · b) and simply write ab
• disjunction (or union) may be written as a|b, a + b or a ∪ b
• a+ is the set of a-strings with at least one a; can be used to abbreviate a∗a or aa∗
• the notation Σ∗ can be seen as abbreviating (a|b|...)∗ for any symbol a, b, . . . in Σ.
• Σn can be seen as abbreviating the concatenation of (a|b|...) with itself n times
• an can be used to abbreviate the concatenation of a with itself n times

Raquel Fernández TTTV 2014 - lecture 1a 15

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .}

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . }

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c)

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .}

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .} me(o)∗w

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .} me(o)∗w {mew, meow, meoow, meooow, meooooooooow, . . .}

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .} me(o)∗w {mew, meow, meoow, meooow, meooooooooow, . . .} ba(a)+

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .} me(o)∗w {mew, meow, meoow, meooow, meooooooooow, . . .} ba(a)+ {baa, baaaaaa, baaaaaaaaa, . . .}

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .} me(o)∗w {mew, meow, meoow, meooow, meooooooooow, . . .} ba(a)+ {baa, baaaaaa, baaaaaaaaa, . . .} sunΣ∗

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .} me(o)∗w {mew, meow, meoow, meooow, meooooooooow, . . .} ba(a)+ {baa, baaaaaa, baaaaaaaaa, . . .} sunΣ∗ {sun, sunglasses, sunset, sunz, sunaaaaa, sunyxjshiksr . . .}

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .} me(o)∗w {mew, meow, meoow, meooow, meooooooooow, . . .} ba(a)+ {baa, baaaaaa, baaaaaaaaa, . . .} sunΣ∗ {sun, sunglasses, sunset, sunz, sunaaaaa, sunyxjshiksr . . .} (co)2Σ∗

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions: Examples

What kind of strings would the language defined by each of the following regular expressions contain?

Let Σ = {a, b, c, d...x, y, z} Regular expression Language (ab)∗c {c, abc, ababc, abababc, . . .} (a|b)∗c {c, ac, bc, aac, abc, bac, bbc, babc . . . } (a∗c)|(b∗c) {c, ac, aac, aaac, . . . bc, bbc, bbbc, . . .} me(o)∗w {mew, meow, meoow, meooow, meooooooooow, . . .} ba(a)+ {baa, baaaaaa, baaaaaaaaa, . . .} sunΣ∗ {sun, sunglasses, sunset, sunz, sunaaaaa, sunyxjshiksr . . .} (co)2Σ∗ {coco, cocoa, coconut, cocoz, coconjsbfx, cocococovuyfvco . . .}

Raquel Fernández TTTV 2014 - lecture 1a 16

Regular Expressions in Programming Languages

Many programming languages such as Perl, Python, Java, or Unix tools like grep include ways to specify regular expressions. For instance:

Perl notation underlying regular expression ranges: [a-z] (a|b|c|d| . . . |z)

• ptionality:

colo(u)?r colo(u|ǫ)r digits: \d (0|1|2|3|4|5|6|7|8|9) There are many other options, such as negation, upper-/lower-case, white spaces, etc.

Raquel Fernández TTTV 2014 - lecture 1a 17

Regular Expressions in Programming Languages

Many programming languages such as Perl, Python, Java, or Unix tools like grep include ways to specify regular expressions. For instance:

Perl notation underlying regular expression ranges: [a-z] (a|b|c|d| . . . |z)

• ptionality:

colo(u)?r colo(u|ǫ)r digits: \d (0|1|2|3|4|5|6|7|8|9) There are many other options, such as negation, upper-/lower-case, white spaces, etc.

These operators are “syntactic sugar”: all regular expressions can be constructed with the basic operations we have seen earlier: concatenation, disjunction, and kleene star, plus the empty string.

Raquel Fernández TTTV 2014 - lecture 1a 17

Regular Expressions in Programming Languages

Many programming languages such as Perl, Python, Java, or Unix tools like grep include ways to specify regular expressions. For instance:

Perl notation underlying regular expression ranges: [a-z] (a|b|c|d| . . . |z)

• ptionality:

colo(u)?r colo(u|ǫ)r digits: \d (0|1|2|3|4|5|6|7|8|9) There are many other options, such as negation, upper-/lower-case, white spaces, etc.

These operators are “syntactic sugar”: all regular expressions can be constructed with the basic operations we have seen earlier: concatenation, disjunction, and kleene star, plus the empty string. This syntactic sugar, however, is very useful! regex are used all

• ver the place for string search. This won’t be covered in class –

see the book and practice in the werkcolleges.

Raquel Fernández TTTV 2014 - lecture 1a 17

Regular Expressions and Automata

• A regular expression allows us characterise a formal language

declaratively.

• We can characterise the same language procedurally by means of

an automaton that specifies how the language is computed.

Raquel Fernández TTTV 2014 - lecture 1a 18

Regular Expressions and Automata

• A regular expression allows us characterise a formal language

declaratively.

• We can characterise the same language procedurally by means of

an automaton that specifies how the language is computed.

regular expression: meo∗w regular expression: (b|c)ar

Raquel Fernández TTTV 2014 - lecture 1a 18

Regular Expressions and Automata

• A regular expression allows us characterise a formal language

declaratively.

• We can characterise the same language procedurally by means of

an automaton that specifies how the language is computed.

regular expression: meo∗w regular expression: (b|c)ar language: {mew, meow, meoow, meooow...} language: {bar, car}

Raquel Fernández TTTV 2014 - lecture 1a 18

Regular Expressions and Automata

• A regular expression allows us characterise a formal language

declaratively.

• We can characterise the same language procedurally by means of

an automaton that specifies how the language is computed.

regular expression: meo∗w regular expression: (b|c)ar language: {mew, meow, meoow, meooow...} language: {bar, car} q0 q1 q2 q3 m e

• w

q0 q1 q2 q3 c b a r

Raquel Fernández TTTV 2014 - lecture 1a 18

Finite State Automata: Formal Definition

Raquel Fernández TTTV 2014 - lecture 1a 19

Finite State Automata: Formal Definition

We can formally specify an FSA by the following 5 parameters:

• Q: a finite set of states
• Σ: a finite input alphabet
• q0 ∈ Q: the start state
• F: a set of final or accepting states (F ⊆ Q)
• δ: a transition function between states that maps pairs of states

and input symbols q, i to a new state q′ (Q × Σ → Q).

Raquel Fernández TTTV 2014 - lecture 1a 19

Finite State Automata: Formal Definition

We can formally specify an FSA by the following 5 parameters:

• Q: a finite set of states
• Σ: a finite input alphabet
• q0 ∈ Q: the start state
• F: a set of final or accepting states (F ⊆ Q)
• δ: a transition function between states that maps pairs of states

and input symbols q, i to a new state q′ (Q × Σ → Q).

q0 q1 q2 q3 c b a r

Raquel Fernández TTTV 2014 - lecture 1a 19

Finite State Automata: Formal Definition

We can formally specify an FSA by the following 5 parameters:

• Q: a finite set of states
• Σ: a finite input alphabet
• q0 ∈ Q: the start state
• F: a set of final or accepting states (F ⊆ Q)
• δ: a transition function between states that maps pairs of states

and input symbols q, i to a new state q′ (Q × Σ → Q).

q0 q1 q2 q3 c b a r Q = {q0, q1, q2, q3} Σ = {a, b, c, r} start state: q0 F = {q3} δ = {(q0, c, q1), (q0, b, q1), (q1, a, q2), (q2, r, q3)}

Raquel Fernández TTTV 2014 - lecture 1a 19

Finite State Automata with ǫ-transitions

We may extend the transition function δ with the empty string symbol ǫ, so that δ = Q × Σ ∪ {ǫ} → Q

regular expression: colo(u|ǫ)r

Raquel Fernández TTTV 2014 - lecture 1a 20

Finite State Automata with ǫ-transitions

We may extend the transition function δ with the empty string symbol ǫ, so that δ = Q × Σ ∪ {ǫ} → Q

regular expression: colo(u|ǫ)r q0 q1 q2 q3 colo u ǫ r

Raquel Fernández TTTV 2014 - lecture 1a 20

Finite State Automata with ǫ-transitions

We may extend the transition function δ with the empty string symbol ǫ, so that δ = Q × Σ ∪ {ǫ} → Q

regular expression: colo(u|ǫ)r q0 q1 q2 q3 colo u ǫ r

. . . but note that every FSA with ǫ-transitions is equivalent to one without them:

Raquel Fernández TTTV 2014 - lecture 1a 20

Finite State Automata with ǫ-transitions

We may extend the transition function δ with the empty string symbol ǫ, so that δ = Q × Σ ∪ {ǫ} → Q

regular expression: colo(u|ǫ)r q0 q1 q2 q3 colo u ǫ r

. . . but note that every FSA with ǫ-transitions is equivalent to one without them:

regular expression: colo(u|ǫ)r q0 q1 q2 q3 colo u r r

(more on this when we discuss non-deterministic FSAs)

Raquel Fernández TTTV 2014 - lecture 1a 20

Regular Expressions and FSAs

Regular expressions and FSAs capture the same class of languages: any language that is the denotation of some regular expression can be computed by an FSA, and vice-versa.

Raquel Fernández TTTV 2014 - lecture 1a 21

Regular Expressions and FSAs

Regular expressions and FSAs capture the same class of languages: any language that is the denotation of some regular expression can be computed by an FSA, and vice-versa. Let’s see how we can build an FSA from any regular expression.

• Reg. exp. Languages

empty set: ∅ {} empty string: ǫ {ǫ} symbol (∀a ∈ Σ): a {a} If a and b are reg exp, so are: concatenation: ab {ab} disjunction (or union): (ab|ba) {ab, ba} Kleene star (or closure): a∗ {ǫ, a, aa, aaa, aaaaa, . . .}

Strategy:

• Base case: build an automaton for simple expressions
• Inductive step: show how to reproduce each of the operations on

regular expressions with an automaton

Raquel Fernández TTTV 2014 - lecture 1a 21

From Reg Exp to FSA: Base Case

Regular expression Corresponding FSAs

a q0 q1 a

Raquel Fernández TTTV 2014 - lecture 1a 22

From Reg Exp to FSA: Base Case

Regular expression Corresponding FSAs

a q0 q1 a ǫ q0 q1 ǫ

Raquel Fernández TTTV 2014 - lecture 1a 22

From Reg Exp to FSA: Base Case

Regular expression Corresponding FSAs

a q0 q1 a ǫ q0 q1 ǫ ∅ q0 q1

Raquel Fernández TTTV 2014 - lecture 1a 22

From Reg Exp to FSA: Base Case

Regular expression Corresponding FSAs

a q0 q1 a ǫ q0 q1 ǫ ∅ q0 q1

• r simply

q0

Raquel Fernández TTTV 2014 - lecture 1a 22

From Reg Exp to FSA: Concatenation

Regular expression:

• If a and b are regular expressions, so is ab.

Raquel Fernández TTTV 2014 - lecture 1a 23

From Reg Exp to FSA: Concatenation

Regular expression:

• If a and b are regular expressions, so is ab.

Corresponding FSA:

• build FSA for a (A1), build FSA for b (A2)
• link all final states in A1 to the initial state of A2 with ǫ-transitions.

Raquel Fernández TTTV 2014 - lecture 1a 23

From Reg Exp to FSA: Concatenation

Regular expression:

• If a and b are regular expressions, so is ab.

Corresponding FSA:

• build FSA for a (A1), build FSA for b (A2)
• link all final states in A1 to the initial state of A2 with ǫ-transitions.

This gives us a general procedure regardless of the properties of A1 and

• A2. We may be able to simplify the ǫ-transitions at the end.

Raquel Fernández TTTV 2014 - lecture 1a 23

From Reg Exp to FSA: Disjunction

Regular expression:

• If a and b are regular expressions, so is (a|b).

Raquel Fernández TTTV 2014 - lecture 1a 24

From Reg Exp to FSA: Disjunction

Regular expression:

• If a and b are regular expressions, so is (a|b).

Corresponding FSA:

• build FSA for a (A1), build FSA for b (A2)
• add a new initial state and link it to A1 ’s initial state and to A2’s

initial state with ǫ-transitions. Again: this gives us a general procedure regardless of the properties of A1 and A2. We may be able to simplify the ǫ-transitions at the end.

Raquel Fernández TTTV 2014 - lecture 1a 24

From Reg Exp to FSA: Kleene * or Closure

Regular expression:

• a∗: the concatenation of a with itself any number of times.

Raquel Fernández TTTV 2014 - lecture 1a 25

From Reg Exp to FSA: Kleene * or Closure

Regular expression:

• a∗: the concatenation of a with itself any number of times.

Corresponding FSA:

• build FSA for a (A)
• add a new final state and link all of A’s final states to it with

ǫ-transitions

• link A’s initial state to the new final state with an ǫ-transition in both

directions.

Raquel Fernández TTTV 2014 - lecture 1a 25

From Reg Exp to FSA: Kleene * or Closure

Regular expression:

• a∗: the concatenation of a with itself any number of times.

Corresponding FSA:

• build FSA for a (A)
• add a new final state and link all of A’s final states to it with

ǫ-transitions

• link A’s initial state to the new final state with an ǫ-transition in both

directions.

In the simplest case (a), after simplification this amounts to: q0 a

Raquel Fernández TTTV 2014 - lecture 1a 25

Summig Up

Raquel Fernández TTTV 2014 - lecture 1a 26

Summig Up

• Engaging in linguistic behaviour requeires different kinds of

knowledge about language: syntax, semantics, pragmatics...

Raquel Fernández TTTV 2014 - lecture 1a 26

Summig Up

• Engaging in linguistic behaviour requeires different kinds of

knowledge about language: syntax, semantics, pragmatics...

• We started to look into how to formaly characterise structural

properties of languages using tools from formal language theory

Raquel Fernández TTTV 2014 - lecture 1a 26

Summig Up

• Engaging in linguistic behaviour requeires different kinds of

knowledge about language: syntax, semantics, pragmatics...

• We started to look into how to formaly characterise structural

properties of languages using tools from formal language theory

∗ a formal language is a set of strings built up from a finite alphabet

• r vocabulary

Raquel Fernández TTTV 2014 - lecture 1a 26

Summig Up

• Engaging in linguistic behaviour requeires different kinds of

knowledge about language: syntax, semantics, pragmatics...

• We started to look into how to formaly characterise structural

properties of languages using tools from formal language theory

∗ a formal language is a set of strings built up from a finite alphabet

• r vocabulary

∗ regular expressions are a formalisms for characterising languages that follow simple regular patterns

Raquel Fernández TTTV 2014 - lecture 1a 26

Summig Up

• Engaging in linguistic behaviour requeires different kinds of

knowledge about language: syntax, semantics, pragmatics...

• We started to look into how to formaly characterise structural

properties of languages using tools from formal language theory

∗ a formal language is a set of strings built up from a finite alphabet

• r vocabulary

∗ regular expressions are a formalisms for characterising languages that follow simple regular patterns ∗ many programming languages include regex functions and notation

Raquel Fernández TTTV 2014 - lecture 1a 26

Summig Up

• Engaging in linguistic behaviour requeires different kinds of

knowledge about language: syntax, semantics, pragmatics...

• We started to look into how to formaly characterise structural

properties of languages using tools from formal language theory

∗ a formal language is a set of strings built up from a finite alphabet

• r vocabulary

∗ regular expressions are a formalisms for characterising languages that follow simple regular patterns ∗ many programming languages include regex functions and notation ∗ regular languages can be characterised procedurally by means of finite state automata

Raquel Fernández TTTV 2014 - lecture 1a 26

Summig Up

• Engaging in linguistic behaviour requeires different kinds of

knowledge about language: syntax, semantics, pragmatics...

• We started to look into how to formaly characterise structural

properties of languages using tools from formal language theory

∗ a formal language is a set of strings built up from a finite alphabet

• r vocabulary

∗ regular expressions are a formalisms for characterising languages that follow simple regular patterns ∗ many programming languages include regex functions and notation ∗ regular languages can be characterised procedurally by means of finite state automata ∗ regex and FSAs capture the same class of languages

Raquel Fernández TTTV 2014 - lecture 1a 26

To do by tomorrow

• Read the Studiewijzer
• Familiarise yourself with the BB site
• In particular, see “Weekly Materials > Week 1”

Raquel Fernández TTTV 2014 - lecture 1a 27