Finite State Automata Stephan Busemann Thanks to Anette Frank, on - - PowerPoint PPT Presentation

▶
finite state automata
SMART_READER_LITE
LIVE PREVIEW

Finite State Automata Stephan Busemann Thanks to Anette Frank, on - - PowerPoint PPT Presentation

Oct 06, 2023 102 likes •833 views

Finite State Automata Stephan Busemann Thanks to Anette Frank, on whose materials this lecture is based Course Computational Linguistics, Summer 2011 Overview of the lecture Background Chomsky hierarchy of languages Basic definitions,

slide-1
SLIDE 1

Course “Computational Linguistics”, Summer 2011

Finite State Automata

Stephan Busemann Thanks to Anette Frank, on whose materials this lecture is based

slide-2
SLIDE 2

Lecture “Computational Linguistics”, Summer 2011 (2)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-3
SLIDE 3

Lecture “Computational Linguistics”, Summer 2011 (3)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-4
SLIDE 4

Lecture “Computational Linguistics”, Summer 2011 (4)

Finite-state automata: what for?

Chomsky Hierarchy of Languages Regular languages (type-3) Context-free languages (type-2) Context-sensitive languages (type-1) Unconstrained languages (type-0) Hierarchy of Grammars & Automata Regular PS grammar Finite-state automata Context-free PS grammar Push-down automata Tree adjoining grammars Linear bounded automata General PS grammars Turing machine

More expressivity Less computational efficiency

slide-5
SLIDE 5

Lecture “Computational Linguistics”, Summer 2011 (5)

FSAs and regular expressions

Regular expressions Finite-state automata Regular languages

describe / specify recognise

slide-6
SLIDE 6

Lecture “Computational Linguistics”, Summer 2011 (6)

Some basic definitions (1)

Alphabet Σ: finite set of symbols String: sequence x1...xn of symbols xi taken from the alphabet Σ Language over Σ: set of strings that can be generated from Σ

Sigma star Σ*: set of all possible strings over the alphabet Σ,

For instance, if Σ = {a,b}, then Σ* = {ε, a, b, aa, ab, ba, bb, aaa, aab, ... }

Sigma plus Σ+ removes the empty element: Σ+ = Σ* - {ε} Special language ∅ = {}, called the empty language

Attention: note the difference with {ε}: language with one element, the empty string

Formal language: a subset of Σ*

slide-7
SLIDE 7

Lecture “Computational Linguistics”, Summer 2011 (7)

Some basic definitions (2)

A formal grammar is a tuple G= < Σ, Φ, S, R >, where Σ is an alphabet of terminal symbols Φ is an alphabet of non-terminal symbols S is the start symbol R is a finite set of rules, with R ⊆ Γ+ × Γ*.

Γ is the union of terminal and non-terminal symbols: Γ = Σ ∪ Φ Each rule ∈ R is of the form α → β

slide-8
SLIDE 8

Lecture “Computational Linguistics”, Summer 2011 (8)

Some basic definitions (3)

Derivation: Assume a grammar G= < Σ, Φ, S, R > and two arbitrary strings u and v ∈ Γ* = (Σ ∪ Φ)* A direct derivation u ⇒G v holds iff there exists s1, s2 ∈ Γ* such that u = (s1 α s2) and v = (s1 β s2) and there is a rule α → β in R A general derivation u ⇒G* v holds iff either u = v or there exists a string z ∈ Γ* such that u ⇒G* z and z ⇒G v The language L(G) generated by a grammar G is defined as the set of strings w ⊆ Σ* that can be derived from the start symbol S according to the grammar G In other words: L(G) = {w : S ⇒G* w ∧ w ∈ Σ*}

slide-9
SLIDE 9

Lecture “Computational Linguistics”, Summer 2011 (9)

Some basic definitions (4)

Basic operation on strings: concatenation • Assume two strings a and b, defined by a = xi... xm and b=xm+1 ... xn Then the concatenation a • b = xi ... xm xm+1 ... xn Concatenation is associative, but not commutative ε is the identity element: a • ε = ε • a = a

slide-10
SLIDE 10

Lecture “Computational Linguistics”, Summer 2011 (10)

Chomsky Hierarchy of grammars (1)

Classification of languages generated by formal grammars A language is of type i (i = 0,1,2,3) iff it is generated by a type-i grammar Classification according to increasingly restricted types of production rules:

L-type-0 ⊃ L-type1 ⊃ L-type-2 ⊃ L-type-3

Every grammar generates a unique language, but a language can be generated by several different grammars. Two grammars are (Weakly) equivalent if they generate the same string language Strongly equivalent if they generate both the same string language and the same tree language

slide-11
SLIDE 11

Lecture “Computational Linguistics”, Summer 2011 (11)

Chomsky Hierarchy of grammars (2)

Type - 0 languages: general phrase structure grammars No restrictions on the form of production rules: arbitrary strings on both the left-hand and right-hand side of rules A grammar G= < Σ, Φ, S, R > generates a language L-type-0 iff:

All rules R are of the form α→β, where α ∈ Γ+ and β ∈ Γ* In other words, the LHS must be a nonempty sequence of non-terminal

  • r terminal symbols

And RHS a (possibly empty) sequence of non-terminal or terminal symbols

Example:

G = < {S, A, B, C, D, E}, {a}, S, R >, with the following production rules: Question: what is the language generated by G?

S → ACaB CB → E aE → Ea Ca → aaC aD → Da AE → ε CB →DB AD → AC

slide-12
SLIDE 12

Lecture “Computational Linguistics”, Summer 2011 (12)

Chomsky Hierarchy of grammars (3)

Type-1 languages: context-sensitive grammars a grammar G= < Σ, Φ, S, R > generates a language L-type-1 iff

all rules are of the form αAϒ → αβϒ, where A is a non-terminal (∈ Φ) and α,β,ϒ ∈ Γ* In other words, the LHS is a non-empty sequence of NT and T symbols, with at least one NT symbol The RHS is a non-empty sequence of NT or T symbols

Example:

G = < {S,B,C}, {a,b,c}, S, R >, with the following production rules: Question: what is the language generated by G?

S → a S B C a B → a b S → a B C b B → b b C B→B C b C → b c c C → c c

slide-13
SLIDE 13

Lecture “Computational Linguistics”, Summer 2011 (13)

Chomsky Hierarchy of grammars (4)

Type-2 languages: context-free grammars a grammar G= < Σ, Φ, S, R > generates a language L-type-2 iff

all rules are of the form A → α, where A is a non-terminal (∈ Φ) and α ∈ Γ* In other words, the LHS is a single NT symbol The RHS is a non-empty sequence of NT or T symbols

Example:

G = < {S, A}, {a, b}, S, R >, with the following production rules: Question: what is the language generated by G?

S → A S A A → a S → b

slide-14
SLIDE 14

Lecture “Computational Linguistics”, Summer 2011 (14)

Chomsky Hierarchy of grammars (5)

Type-3 languages: regular or finite-state grammars a grammar G= < Σ, Φ, S, R > generates a language L-type-2 iff

all rules are of the form A → wB or A →w , where A, B are non- terminals (∈ Φ) and w ∈ Σ* In other words, the LHS is a single NT symbol, and the RHS is a possibly empty sequence of T symbols, optionally followed by a single NT symbol The definition above is right linear. Left linear grammars have rules of the form A → Bw, and function similarly

Example:

G = < {S, A, B}, {a, b}, S, R >, with the following production rules: Question: what is the language generated by G?

S → a A B → b B a → b b B A → a A B → b

slide-15
SLIDE 15

Lecture “Computational Linguistics”, Summer 2011 (15)

Operations on languages

slide-16
SLIDE 16

Lecture “Computational Linguistics”, Summer 2011 (16)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-17
SLIDE 17

Lecture “Computational Linguistics”, Summer 2011 (17)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-18
SLIDE 18

Lecture “Computational Linguistics”, Summer 2011 (18)

FSAs and regular expressions

Regular expressions Finite-state automata Regular languages

describe / specify recognise

Executable!

slide-19
SLIDE 19

Lecture “Computational Linguistics”, Summer 2011 (19)

Regular languages and expressions

slide-20
SLIDE 20

Lecture “Computational Linguistics”, Summer 2011 (20)

Finite-state automata (FSA)

slide-21
SLIDE 21

Lecture “Computational Linguistics”, Summer 2011 (21)

FSA transition graphs (1)

slide-22
SLIDE 22

Lecture “Computational Linguistics”, Summer 2011 (22)

FSA transition graphs (2)

slide-23
SLIDE 23

Lecture “Computational Linguistics”, Summer 2011 (23)

Traversal and acceptance

slide-24
SLIDE 24

Lecture “Computational Linguistics”, Summer 2011 (24)

Regular grammars and FSAs

slide-25
SLIDE 25

Lecture “Computational Linguistics”, Summer 2011 (25)

Deterministic finite-state automata

slide-26
SLIDE 26

Lecture “Computational Linguistics”, Summer 2011 (26)

Multiple equivalent FSAs

slide-27
SLIDE 27

Lecture “Computational Linguistics”, Summer 2011 (27)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-28
SLIDE 28

Lecture “Computational Linguistics”, Summer 2011 (28)

Defining FSAs through regexps

slide-29
SLIDE 29

Lecture “Computational Linguistics”, Summer 2011 (29)

29

Defining FSAs through regexps

slide-30
SLIDE 30

Lecture “Computational Linguistics”, Summer 2011 (30)

Defining FSAs through regexps

slide-31
SLIDE 31

Lecture “Computational Linguistics”, Summer 2011 (31)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-32
SLIDE 32

Lecture “Computational Linguistics”, Summer 2011 (32)

Non-deterministic FSA

slide-33
SLIDE 33

Lecture “Computational Linguistics”, Summer 2011 (33)

Non-determinism in FSAs

slide-34
SLIDE 34

Lecture “Computational Linguistics”, Summer 2011 (34)

Non-determinism in FSAs

slide-35
SLIDE 35

Lecture “Computational Linguistics”, Summer 2011 (35)

35

Non-determinism in FSAs

slide-36
SLIDE 36

Lecture “Computational Linguistics”, Summer 2011 (36)

Non-determinism in FSAs

slide-37
SLIDE 37

Lecture “Computational Linguistics”, Summer 2011 (37)

Non-determinism in FSAs

slide-38
SLIDE 38

Lecture “Computational Linguistics”, Summer 2011 (38)

Equivalence of DFSAs and NFSAs

slide-39
SLIDE 39

Lecture “Computational Linguistics”, Summer 2011 (39)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-40
SLIDE 40

Lecture “Computational Linguistics”, Summer 2011 (40)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-41
SLIDE 41

Lecture “Computational Linguistics”, Summer 2011 (41)

Determinization by subset construction

slide-42
SLIDE 42

Lecture “Computational Linguistics”, Summer 2011 (42)

Determinization by subset construction

slide-43
SLIDE 43

Lecture “Computational Linguistics”, Summer 2011 (43)

Determinization by subset construction

slide-44
SLIDE 44

Lecture “Computational Linguistics”, Summer 2011 (44)

Determinization by subset construction

slide-45
SLIDE 45

Lecture “Computational Linguistics”, Summer 2011 (45)

Determinization by subset construction

slide-46
SLIDE 46

Lecture “Computational Linguistics”, Summer 2011 (46)

Determinization by subset construction

slide-47
SLIDE 47

Lecture “Computational Linguistics”, Summer 2011 (47)

ε-transitions and ε-closure

slide-48
SLIDE 48

Lecture “Computational Linguistics”, Summer 2011 (48)

Example

slide-49
SLIDE 49

Lecture “Computational Linguistics”, Summer 2011 (49)

Algorithm for subset construction

slide-50
SLIDE 50

Lecture “Computational Linguistics”, Summer 2011 (50)

Algorithm for subset construction

slide-51
SLIDE 51

Lecture “Computational Linguistics”, Summer 2011 (51)

Algorithm for subset construction

slide-52
SLIDE 52

Lecture “Computational Linguistics”, Summer 2011 (52)

Algorithm for subset construction

slide-53
SLIDE 53

Lecture “Computational Linguistics”, Summer 2011 (53)

Algorithm for subset construction

slide-54
SLIDE 54

Lecture “Computational Linguistics”, Summer 2011 (54)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-55
SLIDE 55

Lecture “Computational Linguistics”, Summer 2011 (55)

55

Minimization of FSAs

slide-56
SLIDE 56

Lecture “Computational Linguistics”, Summer 2011 (56)

56

Minimization of FSAs

slide-57
SLIDE 57

Lecture “Computational Linguistics”, Summer 2011 (57)

Partitioning in equivalence classes

slide-58
SLIDE 58

Lecture “Computational Linguistics”, Summer 2011 (58)

Minimization of a DFSA

slide-59
SLIDE 59

Lecture “Computational Linguistics”, Summer 2011 (59)

Example

slide-60
SLIDE 60

Lecture “Computational Linguistics”, Summer 2011 (60)

Example

slide-61
SLIDE 61

Lecture “Computational Linguistics”, Summer 2011 (61)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski‘s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-62
SLIDE 62

Lecture “Computational Linguistics”, Summer 2011 (62)

Brzozowski‘s algorithm

slide-63
SLIDE 63

Lecture “Computational Linguistics”, Summer 2011 (63)

Why does it yield a minimal DFSA?

slide-64
SLIDE 64

Lecture “Computational Linguistics”, Summer 2011 (64)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-65
SLIDE 65

Lecture “Computational Linguistics”, Summer 2011 (65)

Applications: string matching

slide-66
SLIDE 66

Lecture “Computational Linguistics”, Summer 2011 (66)

Applications: string matching

slide-67
SLIDE 67

Lecture “Computational Linguistics”, Summer 2011 (67)

Applications: replacement

slide-68
SLIDE 68

Lecture “Computational Linguistics”, Summer 2011 (68)

From automata to transducers

slide-69
SLIDE 69

Lecture “Computational Linguistics”, Summer 2011 (69)

Overview of the lecture

Background

Chomsky hierarchy of languages Basic definitions, generic operations on languages

Generalities about Finite-State Automata (FSA)

Regular languages, regular expressions and FSAs Constructing a FSA from a regular expression Non-deterministic FSAs

Optimization algorithms for FSAs

Determinization of a FSA via subset construction Minimization of a FSA: equivalence classes, Brzozowski„s algorithm

Applications of FSAs & extensions to finite-state transducers Conclusions, exercises

slide-70
SLIDE 70

Lecture “Computational Linguistics”, Summer 2011 (70)

Conclusions

In this lecture, we presented finite-state automata and their algorithms

FSAs and regular expressions have the same expressive power: they both define a regular language, type-3 in the Chomsky hierarchy FSAs can be automatically constructed from a given regular expression FSAs can be deterministic or non-deterministic

We also saw two algorithms used to improve the (runtime) efficiency of a finite-state automata:

FSA determinization, via subset construction FSA minimization, via either equivalence classes, or Brzozowski

Finite-state automata can be extended to finite-state transducers, which define relations between languages Due to their simplicity and efficiency, FSAs are pervasive in computational linguistics (morphology, parsing, dialogue management, etc.)

slide-71
SLIDE 71

Lecture “Computational Linguistics”, Summer 2011 (71)

Exercises

{C,D,E}>