Appetizer: Simultaneous Translation ACL 2019 Invited Talk - - PowerPoint PPT Presentation

▶
appetizer simultaneous translation
SMART_READER_LITE
LIVE PREVIEW

Appetizer: Simultaneous Translation ACL 2019 Invited Talk - - PowerPoint PPT Presentation

Sep 26, 2022 196 likes •1.29k views

LinearFold: Linear-Time Parsing Meets RNA Folding x GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA y (((((((..((((........)))).(((((.......))))).....(((((.......)))))))))))).... 1 G C U C C A C G G C 70

slide-1
SLIDE 1

LinearFold: Linear-Time Parsing Meets RNA Folding

Liang Huang

Oregon State University & Baidu Research USA

Joint work with He Zhang, Dezhong Deng, Kai Zhao, 
 Kaibo Liu, David Hendrix and David Mathews

G C G G G A A U A G C U C A G U U G G U A G A G C A C G A C C U U G C C A A G G U C G G G G U C G C G A G U U C G A G U C U C G U U U C C C G C U C C A

1 10 20 30 40 50 60 70 76

x y

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

Southern California NLP Symposium, September 2019

slide-2
SLIDE 2

LinearFold: Linear-Time Parsing Meets RNA Folding

Liang Huang

Oregon State University & Baidu Research USA

Joint work with He Zhang, Dezhong Deng, Kai Zhao, 
 Kaibo Liu, David Hendrix and David Mathews

G C G G G A A U A G C U C A G U U G G U A G A G C A C G A C C U U G C C A A G G U C G G G G U C G C G A G U U C G A G U C U C G U U U C C C G C U C C A

1 10 20 30 40 50 60 70 76

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

x y

Southern California NLP Symposium, September 2019

slide-3
SLIDE 3

Appetizer: Simultaneous Translation

ACL 2019 Invited Talk Simultaneous Translation: Recent Advances and Remaining Challenges

slide-4
SLIDE 4

Appetizer: Simultaneous Translation

  • ur

work

Baidu World Conference, Nov. 2017 full-sentence translation (latency: 10+ secs) Baidu World Conference, Nov. 2018 low-latency simultaneous translation (latency: ~3 secs)

slide-5
SLIDE 5

Appetizer: Simultaneous Translation

  • ur

work

Baidu World Conference, Nov. 2017 full-sentence translation (latency: 10+ secs) Baidu World Conference, Nov. 2018 low-latency simultaneous translation (latency: ~3 secs)

slide-6
SLIDE 6

Appetizer: Simultaneous Translation

  • ur

work

Baidu World Conference, Nov. 2017 full-sentence translation (latency: 10+ secs) Baidu World Conference, Nov. 2018 low-latency simultaneous translation (latency: ~3 secs)

slide-7
SLIDE 7

Appetizer: Simultaneous Translation

  • ur

work

Baidu World Conference, Nov. 2017 full-sentence translation (latency: 10+ secs) Baidu World Conference, Nov. 2018 low-latency simultaneous translation (latency: ~3 secs)

slide-8
SLIDE 8

Appetizer: Simultaneous Translation

  • ur

work

Baidu World Conference, Nov. 2017 full-sentence translation (latency: 10+ secs) Baidu World Conference, Nov. 2018 low-latency simultaneous translation (latency: ~3 secs)

slide-9
SLIDE 9

Appetizer: Simultaneous Translation

  • ur

work

Baidu World Conference, Nov. 2017 full-sentence translation (latency: 10+ secs) Baidu World Conference, Nov. 2018 low-latency simultaneous translation (latency: ~3 secs)

slide-10
SLIDE 10

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation
slide-11
SLIDE 11

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation

President

Bùshí

布什茶

Bush zǒngtǒng

总统

President

slide-12
SLIDE 12

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation

President Bush

Bùshí

布什茶

Bush zǒngtǒng

总统

President zài

在

in

slide-13
SLIDE 13

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation

President Bush meets

Bùshí

布什茶

Bush zǒngtǒng

总统

President zài

在

in Mòsīkē

莫斯科

Moscow

slide-14
SLIDE 14

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation

President Bush meets with

Bùshí

布什茶

Bush zǒngtǒng

总统

President zài

在

in Mòsīkē

莫斯科

Moscow yǔ

与

with

slide-15
SLIDE 15

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation

President Bush meets with Russian

Bùshí

布什茶

Bush zǒngtǒng

总统

President zài

在

in Mòsīkē

莫斯科

Moscow yǔ

与

with Éluósī

俄罗斯

Russian

slide-16
SLIDE 16

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation

President Bush meets with Russian President

Bùshí

布什茶

Bush zǒngtǒng

总统

President zài

在

in Mòsīkē

莫斯科

Moscow yǔ

与

with zǒngtǒng

总统

President Éluósī

俄罗斯

Russian

slide-17
SLIDE 17

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation

President Bush meets with Russian President

Bùshí

布什茶

Bush zǒngtǒng

总统

President zài

在

in Mòsīkē

莫斯科

Moscow yǔ

与

with zǒngtǒng

总统

President Éluósī

俄罗斯

Russian Pǔjīng

普京

Putin

Putin

slide-18
SLIDE 18

Key Idea: Prefix-to-Prefix, not Seq-to-Seq

…

… wait whole source sentence …

1 2

source: target:

4 1 2 3 5

seq-to-seq

4 1 2 3

…

wait k words

1 2

source: target:

5

prefix-to-prefix
 (wait-k)

  • seq-to-seq is only suitable for

conventional full-sentence MT

  • we propose prefix-to-prefix, tailed to

simultaneous MT

  • special case: wait-k policy: translation is

always k words behind source sentence

  • training in this way enables anticipation

President Bush meets with Russian President

Bùshí

布什茶

Bush zǒngtǒng

总统

President zài

在

in Mòsīkē

莫斯科

Moscow yǔ

与

with zǒngtǒng

总统

President Éluósī

俄罗斯

Russian Pǔjīng

普京

Putin

Putin in Moscow

huìwù

会晤

meet

slide-19
SLIDE 19

Entreé: Linear-Time RNA Folding

x y

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

slide-20
SLIDE 20

Entreé: Linear-Time RNA Folding

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

x y

slide-21
SLIDE 21

Background: RNAs and RNA folding

6

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

RNA sequence

RNA has dual roles: informational (DNA=>RNA=>protein) functional (non-coding RNAs) knowing structures can infer function

slide-22
SLIDE 22

Background: RNAs and RNA folding

6

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

RNA sequence secondary structure structure prediction
 (“RNA folding”)

RNA has dual roles: informational (DNA=>RNA=>protein) functional (non-coding RNAs) knowing structures can infer function

slide-23
SLIDE 23

Background: RNAs and RNA folding

6

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

RNA sequence secondary structure structure prediction
 (“RNA folding”)

RNA has dual roles: informational (DNA=>RNA=>protein) functional (non-coding RNAs) knowing structures can infer function

slide-24
SLIDE 24

RNA Secondary Structure Prediction

7 GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

allowed pairs: G-C A-U G-U assume no crossing pairs (no pseudoknots)

x

7

input example: transfer RNA (tRNA)

slide-25
SLIDE 25

RNA Secondary Structure Prediction

7 GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

allowed pairs: G-C A-U G-U assume no crossing pairs (no pseudoknots)

x y

7

input

  • utput

example: transfer RNA (tRNA)

slide-26
SLIDE 26

RNA Secondary Structure Prediction

7 GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

allowed pairs: G-C A-U G-U assume no crossing pairs (no pseudoknots)

x y

7

G C G G G A A U A G C U C A G U U G G U A G A G C A C G A C C U U G C C A A G G U C G G G G U C G C G A G U U C G A G U C U C G U U U C C C G C U C C A

1 10 20 30 40 50 60 70 76

input

  • utput

example: transfer RNA (tRNA)

slide-27
SLIDE 27

parse tree

RNA Secondary Structure Prediction

7 GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

allowed pairs: G-C A-U G-U assume no crossing pairs (no pseudoknots)

x y

7

G C G G G A A U A G C U C A G U U G G U A G A G C A C G A C C U U G C C A A G G U C G G G G U C G C G A G U U C G A G U C U C G U U U C C C G C U C C A

1 10 20 30 40 50 60 70 76

input

  • utput

example: transfer RNA (tRNA)

slide-28
SLIDE 28

parse tree

RNA Secondary Structure Prediction

7 GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

allowed pairs: G-C A-U G-U assume no crossing pairs (no pseudoknots)

x y

7

problem: standard structure prediction algorithms are way too slow: O(n3)

G C G G G A A U A G C U C A G U U G G U A G A G C A C G A C C U U G C C A A G G U C G G G G U C G C G A G U U C G A G U C U C G U U U C C C G C U C C A

1 10 20 30 40 50 60 70 76

input

  • utput

example: transfer RNA (tRNA)

. . .

O(n3)

S NP DT the NN man VP VB bit NP DT the NN dog

slide-29
SLIDE 29

parse tree

RNA Secondary Structure Prediction

7 GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

allowed pairs: G-C A-U G-U assume no crossing pairs (no pseudoknots)

x y

7

problem: standard structure prediction algorithms are way too slow: O(n3) solution: adapt my linear-time dynamic programming algorithms from parsing

G C G G G A A U A G C U C A G U U G G U A G A G C A C G A C C U U G C C A A G G U C G G G G U C G C G A G U U C G A G U C U C G U U U C C C G C U C C A

1 10 20 30 40 50 60 70 76

input

  • utput

example: transfer RNA (tRNA)

. . .

O(n3)

S NP DT the NN man VP VB bit NP DT the NN dog

slide-30
SLIDE 30

Fast Structure Prediction Enables RNA Design

8

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

RNA sequence RNA secondary structure RNA 3D structure design structure prediction
 (“folding”)

Professor Rhiju Das
 Stanford Medical School EteRNA game
 (RNA design)

detecting active TB using RNA design which needs our fast RNA folding

slide-31
SLIDE 31

From Linguistics to Biology

x y

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

slide-32
SLIDE 32

From Linguistics to Biology

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

x y

slide-33
SLIDE 33

Background: CKY for RNA Folding: O(n3)

C A A G U

( )

i j i+1 j-1 i j k

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-34
SLIDE 34

Background: CKY for RNA Folding: O(n3)

C A A G U

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-35
SLIDE 35

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . .

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-36
SLIDE 36

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . ..

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-37
SLIDE 37

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . .. ..

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-38
SLIDE 38

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . .. .. .. ()

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-39
SLIDE 39

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . .. .. .. () ...

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-40
SLIDE 40

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . .. .. .. () ... (.)

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-41
SLIDE 41

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . .. .. .. () ... (.) (.)

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-42
SLIDE 42

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . .. .. .. () ... (.) (.) .(.)

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-43
SLIDE 43

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . .. .. .. () ... (.) (.) .(.) (.).

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-44
SLIDE 44

Background: CKY for RNA Folding: O(n3)

C A A G U . . . . . .. .. .. () ... (.) (.) .(.) (.). ((.))

( )

i j i+1 j-1 i j k

. . .

  • Dynamic Programming — O(n3)
  • bottom-up CKY parsing
  • example: maximize # of pairs (A-U, G-C, or G-U)

10

slide-45
SLIDE 45

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

11

1964 Cocke \ bottom-up 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs)

S NP DT the NN man VP VB bit NP DT the NN dog

…

O(n3)

dynamic programming

slide-46
SLIDE 46

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

11

1964 Cocke \ bottom-up 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs)

S NP DT the NN man VP VB bit NP DT the NN dog

…

O(n3)

:= id x + id y const 3

x = y + 3;

O(n)

dynamic programming

slide-47
SLIDE 47

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1986 Tomita: Generalized LR
 for all CFGs: O(n3) 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

11

1964 Cocke \ bottom-up 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs)

S NP DT the NN man VP VB bit NP DT the NN dog

…

O(n3)

:= id x + id y const 3

x = y + 3;

O(n)

O(n3) dynamic programming

slide-48
SLIDE 48

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1986 Tomita: Generalized LR
 for all CFGs: O(n3) 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

11

1964 Cocke \ bottom-up 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs) 2010: Huang & Sagae: O(n)
 (approx.) DP for all CFGs

S NP DT the NN man VP VB bit NP DT the NN dog

…

O(n3)

:= id x + id y const 3

x = y + 3;

O(n)

O(n3) O(n) dynamic programming

slide-49
SLIDE 49

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1986 Tomita: Generalized LR
 for all CFGs: O(n3) 2019: LinearFold O(n) 
 (approx.) RNA folding 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

11

1964 Cocke \ bottom-up 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs) 2010: Huang & Sagae: O(n)
 (approx.) DP for all CFGs

S NP DT the NN man VP VB bit NP DT the NN dog

…

O(n3)

:= id x + id y const 3

x = y + 3;

O(n)

O(n3) O(n) dynamic programming

slide-50
SLIDE 50

5’ 3’ GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

How to Fold RNAs in Linear-Time?

  • idea 0: tag each nucleotide from left to right
  • maintain a stack: push “(”, pop “)”, skip “.”
  • naive: O(3n)

12

( . )

slide-51
SLIDE 51

5’ 3’ GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

How to Fold RNAs in Linear-Time?

  • idea 0: tag each nucleotide from left to right
  • maintain a stack: push “(”, pop “)”, skip “.”
  • naive: O(3n)

12

( . )

slide-52
SLIDE 52

5’ 3’ GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

How to Fold RNAs in Linear-Time?

  • idea 1: DP by packing “equivalent states”
  • maintain graph-structured stacks
  • DP: O(n3)

(((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

13

( . )

slide-53
SLIDE 53

5’ 3’ GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

How to Fold RNAs in Linear-Time?

  • idea 1: DP by packing “equivalent states”
  • maintain graph-structured stacks
  • DP: O(n3)

14

( . )

slide-54
SLIDE 54

5’ 3’ GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

  • idea 2: approximate search: beam pruning
  • keep only top b states per step
  • DP+beam: O(n)

How to Fold RNAs in Linear-Time?

(((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

15

slide-55
SLIDE 55

5’ 3’ GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

  • idea 2: approximate search: beam pruning
  • keep only top b states per step
  • DP+beam: O(n)

How to Fold RNAs in Linear-Time?

(((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

15

each DP state corresponds to
 exponentially many non-DP states

slide-56
SLIDE 56

5’ 3’ GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

  • idea 2: approximate search: beam pruning
  • keep only top b states per step
  • DP+beam: O(n)

How to Fold RNAs in Linear-Time?

(((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

15

each DP state corresponds to
 exponentially many non-DP states

slide-57
SLIDE 57

5’ 3’ GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

  • idea 2: approximate search: beam pruning
  • keep only top b states per step
  • DP+beam: O(n)

How to Fold RNAs in Linear-Time?

(((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

15

b e a m s e a r c h

each DP state corresponds to
 exponentially many non-DP states

slide-58
SLIDE 58

On to details...

x y

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

slide-59
SLIDE 59

On to details...

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA (((((((..((((........)))).(((((.......))))).....(((((.......))))))))))))....

x y

slide-60
SLIDE 60

An Example (Optimal) Path

17

step: 1 2 3 4 5 C CC CCA CCAG CCAGG ✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

( ( . ) ) push push skip pop pop

A Example state:

Example path: ((.) structure

+0.9

score

1 0

stack

Nussinov-style scoring model: +1.0 for each pair


  • 0.1 for each unpaired nucleotide
slide-61
SLIDE 61

An Example (Optimal) Path

17

step: 1 2 3 4 5 C CC CCA CCAG CCAGG ✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

( ( . ) ) push push skip pop pop

A Example state:

Example path: ((.) structure

+0.9

score

1 0

stack

Nussinov-style scoring model: +1.0 for each pair


  • 0.1 for each unpaired nucleotide
slide-62
SLIDE 62

An Example (Optimal) Path

17

step: 1 2 3 4 5 C CC CCA CCAG CCAGG ✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

( ( . ) ) push push skip pop pop

A Example state:

Example path: ((.) structure

+0.9

score

1 0

stack

Nussinov-style scoring model: +1.0 for each pair


  • 0.1 for each unpaired nucleotide
slide-63
SLIDE 63

An Example (Optimal) Path

17

step: 1 2 3 4 5 C CC CCA CCAG CCAGG ✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

( ( . ) ) push push skip pop pop

A Example state:

Example path: ((.) structure

+0.9

score

1 0

stack

Nussinov-style scoring model: +1.0 for each pair


  • 0.1 for each unpaired nucleotide
slide-64
SLIDE 64

An Example (Optimal) Path

17

step: 1 2 3 4 5 C CC CCA CCAG CCAGG ✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

( ( . ) ) push push skip pop pop

A Example state:

Example path: ((.) structure

+0.9

score

1 0

stack

Nussinov-style scoring model: +1.0 for each pair


  • 0.1 for each unpaired nucleotide
slide-65
SLIDE 65

Version 1: Naive Exhaustive Search: O(3n)

18

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2

+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3

+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2

+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3

+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . )

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

slide-66
SLIDE 66

Version 1: Naive Exhaustive Search: O(3n)

18

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2

+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3

+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2

+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3

+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . )

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

slide-67
SLIDE 67

Version 1: Naive Exhaustive Search: O(3n)

18

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2

+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3

+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2

+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3

+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . )

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

slide-68
SLIDE 68

Version 1: Naive Exhaustive Search: O(3n)

18

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2

+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3

+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2

+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3

+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . )

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

slide-69
SLIDE 69

Version 1: Naive Exhaustive Search: O(3n)

18

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2

+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3

+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2

+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3

+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . )

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

slide-70
SLIDE 70

Idea 1: Merge Identical Stacks => Version 2: O(2n)

19

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2

+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3

+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2

+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3

+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . )

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

slide-71
SLIDE 71

Idea 1: Merge Identical Stacks => Version 2: O(2n)

20

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2

+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

( . . ( . ( . . . . . . ) . ) ) ) . )

E

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

slide-72
SLIDE 72

Details: Pack “Equivalent” States => O(n3)

21

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2

+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

( . . ( . ( . . . . . . ) . ) ) ) . )

E

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

?(

0.0

... 2

?(.

  • 0.1

2 ... 2

. .

✏ (

0.0

0 0 ... 1

.

  • 0.1

0 0

. ( ( (

((.) ((.))

+0.9 +1.9

... 1 0 0

.(.)

+0.8

0 0

. )

two states are equivalent if they share the same stack top

( .(.)

) )

unpacking packing

slide-73
SLIDE 73

Details: Pack “Equivalent” States => O(n3)

21

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2

+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

( . . ( . ( . . . . . . ) . ) ) ) . )

E

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

?(

0.0

... 2

?(.

  • 0.1

2 ... 2

. .

✏ (

0.0

0 0 ... 1

.

  • 0.1

0 0

. ( ( (

((.) ((.))

+0.9 +1.9

... 1 0 0

.(.)

+0.8

0 0

. )

two states are equivalent if they share the same stack top

( .(.)

) )

unpacking packing

slide-74
SLIDE 74

Details: Pack “Equivalent” States => O(n3)

21

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2

+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

( . . ( . ( . . . . . . ) . ) ) ) . )

E

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

?(

0.0

... 2

?(.

  • 0.1

2 ... 2

. .

✏ (

0.0

0 0 ... 1

.

  • 0.1

0 0

. ( ( (

((.) ((.))

+0.9 +1.9

... 1 0 0

.(.)

+0.8

0 0

. )

two states are equivalent if they share the same stack top

( .(.)

) )

unpacking packing

slide-75
SLIDE 75

Details: Pack “Equivalent” States => O(n3)

21

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2

+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

( . . ( . ( . . . . . . ) . ) ) ) . )

E

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

?(

0.0

... 2

?(.

  • 0.1

2 ... 2

. .

✏ (

0.0

0 0 ... 1

.

  • 0.1

0 0

. ( ( (

((.) ((.))

+0.9 +1.9

... 1 0 0

.(.)

+0.8

0 0

. )

two states are equivalent if they share the same stack top

( .(.)

) )

unpacking packing

slide-76
SLIDE 76

Details: Pack “Equivalent” States => O(n3)

21

✏ ( (( ((. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2

+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

( . . ( . ( . . . . . . ) . ) ) ) . )

E

step: 1 2 3 4 5 C CC CCA CCAG CCAGG

D

?(

0.0

... 2

?(.

  • 0.1

2 ... 2

. .

✏ (

0.0

0 0 ... 1

.

  • 0.1

0 0

. ( ( (

((.) ((.))

+0.9 +1.9

... 1 0 0

.(.)

+0.8

0 0

. )

two states are equivalent if they share the same stack top

( .(.)

) )

unpacking packing

slide-77
SLIDE 77

Details: Packing “Temporarily Equivalent” States

  • two states are “temporarily equivalent” if they share the same stack top index
  • because they are looking for the same nucleotide(s) to match this stack-top nucleotide
  • we pack them as a single state until a matching is found, when they unpack
  • this is how we can explore exponentially many structures in linear time

22

‘ ( (( ((. ((.) ((.)) . .( .(. .(.) .(.).

. ( . ) . ( ( . ) )

b e a m s e a r c h

slide-78
SLIDE 78

Details: Packing “Temporarily Equivalent” States

  • two states are “temporarily equivalent” if they share the same stack top index
  • because they are looking for the same nucleotide(s) to match this stack-top nucleotide
  • we pack them as a single state until a matching is found, when they unpack
  • this is how we can explore exponentially many structures in linear time

22

‘ ( (( ((. ((.) ((.)) . .( .(. .(.) .(.).

. ( . ) . ( ( . ) )

‘ ( ?( ?(. ((.) ((.)) . .(.)

. ( . . ) . ( ( ) )

b e a m s e a r c h

slide-79
SLIDE 79

Details: Packing “Temporarily Equivalent” States

  • two states are “temporarily equivalent” if they share the same stack top index
  • because they are looking for the same nucleotide(s) to match this stack-top nucleotide
  • we pack them as a single state until a matching is found, when they unpack
  • this is how we can explore exponentially many structures in linear time

22

‘ ( (( ((. ((.) ((.)) . .( .(. .(.) .(.).

. ( . ) . ( ( . ) )

‘ ( ?( ?(. ((.) ((.)) . .(.)

. ( . . ) . ( ( ) )

b e a m s e a r c h

packing unpacking

slide-80
SLIDE 80

Idea 3: Beam Search => Version 4: O(n), approx.

23

C

(low energy) high score (high energy) low score

Idea 3

b

✏ ( ?( ?(. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

(. (.. .(.)

  • 0.1
  • 0.2

+0.8

... 1 ... 1 0 0

. .. ... ?(..

  • 0.1
  • 0.2
  • 0.3
  • 0.2

0 0 0 0 0 0 ... 2

✏ ( ?( ?(. ((.) ((.))

0.0 0.0

  • 0.1

+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

. (. (.. .(.)

  • 0.1
  • 0.1
  • 0.2

+0.8

0 0 ... 1 ... 1 0 0

. ( . ( ( . . . . . . . ) ) . ) ) . ) . ( ( ( . . . . ) ) ) . )

F G

slide-81
SLIDE 81

From O(n3) to O(3n) back to O(n3) finally to O(n)

  • v0: conventional bottom-up DP: O(n3)
  • 4 versions of incremental (5’-to-3’) search
  • v1: exhaustive: O(3n)
  • v2: DP

, merge by full stack: O(2n)

  • v3: DP

, pack by stack top: O(n3)

  • v4: approx. DP via beam search: O(n)
  • this is a simple illustration on the toy

“Nussinov-like” scoring model

  • our real systems have full scoring functions

24

step: 1 2 3 4 5 C CC CCA CCAG CCAGG no DP ✏ ( (( ((. ((.) ((.))

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2
+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3
+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2
+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3
+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

✏ ( (( ((. ((.) ((.)) naive DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

✏ ( ?( ?(. ((.) ((.)) stacktop DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

(. (.. .(.)

  • 0.1
  • 0.2
+0.8

... 1 ... 1 0 0

. .. ... ?(..

  • 0.1
  • 0.2
  • 0.3
  • 0.2

0 0 0 0 0 0 ... 2

✏ ( ?( ?(. ((.) ((.)) LinearFold

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

. (. (.. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 ... 1 ... 1 0 0

(approx.)

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . ) ( . . ( . ( . . . . . . ) . ) ) ) . ) . ( . ( ( . . . . . . . ) ) . ) ) . ) . ( ( ( . . . . ) ) ) . )

D E F G

O(3n) O(2n) O(n3) O(n) +identical stack merging +Idea 1 +stack-top packing +Idea 2 +beam prune +Idea 3

slide-82
SLIDE 82

From O(n3) to O(3n) back to O(n3) finally to O(n)

  • v0: conventional bottom-up DP: O(n3)
  • 4 versions of incremental (5’-to-3’) search
  • v1: exhaustive: O(3n)
  • v2: DP

, merge by full stack: O(2n)

  • v3: DP

, pack by stack top: O(n3)

  • v4: approx. DP via beam search: O(n)
  • this is a simple illustration on the toy

“Nussinov-like” scoring model

  • our real systems have full scoring functions

24

step: 1 2 3 4 5 C CC CCA CCAG CCAGG no DP ✏ ( (( ((. ((.) ((.))

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2
+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3
+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2
+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3
+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

✏ ( (( ((. ((.) ((.)) naive DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

✏ ( ?( ?(. ((.) ((.)) stacktop DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

(. (.. .(.)

  • 0.1
  • 0.2
+0.8

... 1 ... 1 0 0

. .. ... ?(..

  • 0.1
  • 0.2
  • 0.3
  • 0.2

0 0 0 0 0 0 ... 2

✏ ( ?( ?(. ((.) ((.)) LinearFold

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

. (. (.. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 ... 1 ... 1 0 0

(approx.)

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . ) ( . . ( . ( . . . . . . ) . ) ) ) . ) . ( . ( ( . . . . . . . ) ) . ) ) . ) . ( ( ( . . . . ) ) ) . )

D E F G

O(3n) O(2n) O(n3) O(n) +identical stack merging +Idea 1 +stack-top packing +Idea 2 +beam prune +Idea 3

. . .

v0: O(n3)

slide-83
SLIDE 83

From O(n3) to O(3n) back to O(n3) finally to O(n)

  • v0: conventional bottom-up DP: O(n3)
  • 4 versions of incremental (5’-to-3’) search
  • v1: exhaustive: O(3n)
  • v2: DP

, merge by full stack: O(2n)

  • v3: DP

, pack by stack top: O(n3)

  • v4: approx. DP via beam search: O(n)
  • this is a simple illustration on the toy

“Nussinov-like” scoring model

  • our real systems have full scoring functions

24

step: 1 2 3 4 5 C CC CCA CCAG CCAGG no DP ✏ ( (( ((. ((.) ((.))

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2
+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3
+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2
+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3
+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

✏ ( (( ((. ((.) ((.)) naive DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

✏ ( ?( ?(. ((.) ((.)) stacktop DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

(. (.. .(.)

  • 0.1
  • 0.2
+0.8

... 1 ... 1 0 0

. .. ... ?(..

  • 0.1
  • 0.2
  • 0.3
  • 0.2

0 0 0 0 0 0 ... 2

✏ ( ?( ?(. ((.) ((.)) LinearFold

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

. (. (.. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 ... 1 ... 1 0 0

(approx.)

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . ) ( . . ( . ( . . . . . . ) . ) ) ) . ) . ( . ( ( . . . . . . . ) ) . ) ) . ) . ( ( ( . . . . ) ) ) . )

D E F G

O(3n) O(2n) O(n3) O(n) +identical stack merging +Idea 1 +stack-top packing +Idea 2 +beam prune +Idea 3

v1: O(3n)

. . .

v0: O(n3)

slide-84
SLIDE 84

From O(n3) to O(3n) back to O(n3) finally to O(n)

  • v0: conventional bottom-up DP: O(n3)
  • 4 versions of incremental (5’-to-3’) search
  • v1: exhaustive: O(3n)
  • v2: DP

, merge by full stack: O(2n)

  • v3: DP

, pack by stack top: O(n3)

  • v4: approx. DP via beam search: O(n)
  • this is a simple illustration on the toy

“Nussinov-like” scoring model

  • our real systems have full scoring functions

24

step: 1 2 3 4 5 C CC CCA CCAG CCAGG no DP ✏ ( (( ((. ((.) ((.))

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2
+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3
+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2
+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3
+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

✏ ( (( ((. ((.) ((.)) naive DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

✏ ( ?( ?(. ((.) ((.)) stacktop DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

(. (.. .(.)

  • 0.1
  • 0.2
+0.8

... 1 ... 1 0 0

. .. ... ?(..

  • 0.1
  • 0.2
  • 0.3
  • 0.2

0 0 0 0 0 0 ... 2

✏ ( ?( ?(. ((.) ((.)) LinearFold

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

. (. (.. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 ... 1 ... 1 0 0

(approx.)

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . ) ( . . ( . ( . . . . . . ) . ) ) ) . ) . ( . ( ( . . . . . . . ) ) . ) ) . ) . ( ( ( . . . . ) ) ) . )

D E F G

O(3n) O(2n) O(n3) O(n) +identical stack merging +Idea 1 +stack-top packing +Idea 2 +beam prune +Idea 3

v1: O(3n) v3: O(n3) v4: O(n)

. . .

v0: O(n3)

slide-85
SLIDE 85

From O(n3) to O(3n) back to O(n3) finally to O(n)

  • v0: conventional bottom-up DP: O(n3)
  • 4 versions of incremental (5’-to-3’) search
  • v1: exhaustive: O(3n)
  • v2: DP

, merge by full stack: O(2n)

  • v3: DP

, pack by stack top: O(n3)

  • v4: approx. DP via beam search: O(n)
  • this is a simple illustration on the toy

“Nussinov-like” scoring model

  • our real systems have full scoring functions

24

step: 1 2 3 4 5 C CC CCA CCAG CCAGG no DP ✏ ( (( ((. ((.) ((.))

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (.. (..) (..).

  • 0.1
  • 0.2
+0.8 +0.7

1 0 1 0 0 0 0 0

(... (...)

  • 0.3
+0.7

1 0 0 0

. .( .(. .(.) .(.).

  • 0.1
  • 0.1
  • 0.2
+0.8 +0.7

0 0 2 0 2 0 0 0 0 0

.(.. .(..)

  • 0.3
+0.7

2 0 0 0

.. ... .... .....

  • 0.2
  • 0.3
  • 0.4
  • 0.5

0 0 0 0 0 0 0 0

✏ ( (( ((. ((.) ((.)) naive DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 1 0 1 2 1 2 1 0 0 0

(. (..

  • 0.1
  • 0.2

1 0 1 0

. .( .(. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 2 0 2 0 0 0

.. ... .(..

  • 0.2
  • 0.3
  • 0.3

0 0 0 0 2 0

✏ ( ?( ?(. ((.) ((.)) stacktop DP

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

(. (.. .(.)

  • 0.1
  • 0.2
+0.8

... 1 ... 1 0 0

. .. ... ?(..

  • 0.1
  • 0.2
  • 0.3
  • 0.2

0 0 0 0 0 0 ... 2

✏ ( ?( ?(. ((.) ((.)) LinearFold

0.0 0.0
  • 0.1
+0.9 +1.9

0 0 ... 1 ... 2 ... 2 ... 1 0 0

. (. (.. .(.)

  • 0.1
  • 0.1
  • 0.2
+0.8

0 0 ... 1 ... 1 0 0

(approx.)

. ( ( . . ( . . . . . . ) . ) ) . ) . ) . ) ( . . ( . ( . . . . . . ) . ) ) ) . ) . ( . ( ( . . . . . . . ) ) . ) ) . ) . ( ( ( . . . . ) ) ) . )

D E F G

O(3n) O(2n) O(n3) O(n) +identical stack merging +Idea 1 +stack-top packing +Idea 2 +beam prune +Idea 3

v1: O(3n) v3: O(n3) v4: O(n)

. . .

v0: O(n3)

slide-86
SLIDE 86

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1986 Tomita: Generalized LR
 for all CFGs: O(n3) 2019: LinearFold O(n) 
 (approx.) RNA folding 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

25

1964 Cocke \ 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs) 2010: Huang & Sagae: O(n)
 (approx.) DP for all CFGs 2019: 5’-to-3’ O(n3) RNA folding (LinearFold with +∞ beam)

slide-87
SLIDE 87

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1986 Tomita: Generalized LR
 for all CFGs: O(n3) 2019: LinearFold O(n) 
 (approx.) RNA folding 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

25

1964 Cocke \ 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs) 2010: Huang & Sagae: O(n)
 (approx.) DP for all CFGs v0 v0 2019: 5’-to-3’ O(n3) RNA folding (LinearFold with +∞ beam)

. . .

v0: O(n3)

slide-88
SLIDE 88

v3: O(n3)

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1986 Tomita: Generalized LR
 for all CFGs: O(n3) 2019: LinearFold O(n) 
 (approx.) RNA folding 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

25

1964 Cocke \ 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs) 2010: Huang & Sagae: O(n)
 (approx.) DP for all CFGs v0 v0

v4

v3 2019: 5’-to-3’ O(n3) RNA folding (LinearFold with +∞ beam) v3

v4 v4: O(n)

. . .

v0: O(n3)

slide-89
SLIDE 89

Connections to Incremental Parsing

  • shared key observation: local ambiguity packing
  • pack non-crucial local ambiguities along the way
  • unpack (in a reduce action) only when needed

(Huang and Sagae, 2010)

psycholinguistic evidence 
 (eye-tracking experiments): delayed disambiguation

John and Mary had 2 papers John and Mary had 2 papers

Frazier and Rayner (1990), Frazier (1999)

26

slide-90
SLIDE 90

Connections to Incremental Parsing

  • shared key observation: local ambiguity packing
  • pack non-crucial local ambiguities along the way
  • unpack (in a reduce action) only when needed

(Huang and Sagae, 2010)

psycholinguistic evidence 
 (eye-tracking experiments): delayed disambiguation

John and Mary had 2 papers John and Mary had 2 papers

Frazier and Rayner (1990), Frazier (1999)

each together

26

slide-91
SLIDE 91

Connections to Incremental Parsing

  • shared key observation: local ambiguity packing
  • pack non-crucial local ambiguities along the way
  • unpack (in a reduce action) only when needed

27

psycholinguistic evidence 
 (eye-tracking experiments): delayed disambiguation

John and Mary had 2 papers John and Mary had 2 papers

Frazier and Rayner (1990), Frazier (1999)

‘ ( (( ((. ((.) ((.)) . .( .(. .(.) .(.).

. ( . ) . ( ( . ) )

‘ ( ?( ?(. ((.) ((.)) . .(.)

. ( . . ) . ( ( ) )

packing unpacking

slide-92
SLIDE 92

Connections to Incremental Parsing

  • shared key observation: local ambiguity packing
  • pack non-crucial local ambiguities along the way
  • unpack (in a reduce action) only when needed

27

psycholinguistic evidence 
 (eye-tracking experiments): delayed disambiguation

John and Mary had 2 papers John and Mary had 2 papers

Frazier and Rayner (1990), Frazier (1999)

each together

‘ ( (( ((. ((.) ((.)) . .( .(. .(.) .(.).

. ( . ) . ( ( . ) )

‘ ( ?( ?(. ((.) ((.)) . .(.)

. ( . . ) . ( ( ) )

packing unpacking

slide-93
SLIDE 93

Connections to Incremental Parsing

  • shared key observation: local ambiguity packing
  • pack non-crucial local ambiguities along the way
  • unpack (in a reduce action) only when needed

27

psycholinguistic evidence 
 (eye-tracking experiments): delayed disambiguation

John and Mary had 2 papers John and Mary had 2 papers

Frazier and Rayner (1990), Frazier (1999)

each together

John and Mary … had 2 papers … together … each

‘ ( (( ((. ((.) ((.)) . .( .(. .(.) .(.).

. ( . ) . ( ( . ) )

‘ ( ?( ?(. ((.) ((.)) . .(.)

. ( . . ) . ( ( ) )

packing unpacking

slide-94
SLIDE 94

Results

slide-95
SLIDE 95

LinearFold with SOTA Prediction Models

  • models from two widely-used folding engines
  • CONTRAfold MFE (machine-learned)
  • Vienna RNAfold (thermodynamic)
  • we linearized both systems from O(n3) to O(n)

efficiency systems time space machine-learned thermo-dynamic

baselines O(n3) O(n2) CONTRAfold Vienna RNAfold

  • ur work

O(n) O(n) LinearFold-C LinearFold-V

29

slide-96
SLIDE 96

LinearFold with SOTA Prediction Models

  • models from two widely-used folding engines
  • CONTRAfold MFE (machine-learned)
  • Vienna RNAfold (thermodynamic)
  • we linearized both systems from O(n3) to O(n)

efficiency systems time space machine-learned thermo-dynamic

baselines O(n3) O(n2) CONTRAfold Vienna RNAfold

  • ur work

O(n) O(n) LinearFold-C LinearFold-V

29

slide-97
SLIDE 97

Efficiency & Scalability: O(n) time, O(n) memory

30

Archive II data set


(~3,000 seqs, max len: ~3,000 nt)

A

2 4 6 8 10 1000nt 2000nt 3000nt running time (seconds)

Vienna RNAfold: ~n2.4 CONTRAfold MFE: ~n2.2 LinearFold-V: ~n1.2 LinearFold-C: ~n1.1

B

10 10

C

V i e n n a R N A f

  • l

d CONTRAfold L i n e a r F

  • l

d

  • C

L i n e a r F

  • l

d

  • V

RNAcentral data set


(sampled, max len: ~250,000 nt)

LinearFold-V LinearFold-C V i e n n a R N A f

  • l

d C O N T R A f

  • l

d

10,000nt (~HIV) 4min 7s 244,296nt 
 (longest in RNAcentral) ~200hrs 120s

slide-98
SLIDE 98

Accuracy

  • Tested on Archive II dataset 


(on a family-by-family basis)

  • significantly better on 3 long families
  • biggest boost on the longest families: 


16S/23S rRNAs

  • LinearFold-V vs.

Vienna is similar

31

40 50 60 70 80

tRNA 5S rRNA SRP RNaseP tmRNA Group I Intron telomerase RNA 16S rRNA 23S rRNA *

Precision

Standard O(n3) search LinearFold: O(n) search * *

40 50 60 70 80

tRNA 5S rRNA SRP RNaseP tmRNA Group I Intron telomerase RNA 16S rRNA 23S rRNA **

Recall

Standard O(n3) search LinearFold: O(n) search * * * *

(precision) (recall)

slide-99
SLIDE 99

Accuracy

  • Tested on Archive II dataset 


(on a family-by-family basis)

  • significantly better on 3 long families
  • biggest boost on the longest families: 


16S/23S rRNAs

  • LinearFold-V vs.

Vienna is similar

31

40 50 60 70 80

tRNA 5S rRNA SRP RNaseP tmRNA Group I Intron telomerase RNA 16S rRNA 23S rRNA *

Precision

Standard O(n3) search LinearFold: O(n) search * *

40 50 60 70 80

tRNA 5S rRNA SRP RNaseP tmRNA Group I Intron telomerase RNA 16S rRNA 23S rRNA **

Recall

Standard O(n3) search LinearFold: O(n) search * * * *

(precision) (recall)

slide-100
SLIDE 100

Improvements on Long-Range Base Pairs

  • long-distance pairs are well-known to be hard to predict
  • LinearFold outputs less long-range base pairs, but more correct ones

32

10 20 30 40 50 60 70 1-200 201-500 >500 PPV base pair distance

CONTRAfold MFE LinearFold-C b=100

10 20 30 40 50 60 70 1-200 201-500 >500 Sensitivity base pair distance

CONTRAfold MFE LinearFold-C b=100

slide-101
SLIDE 101

Improvements on Long-Range Base Pairs

  • long-distance pairs are well-known to be hard to predict
  • LinearFold outputs less long-range base pairs, but more correct ones

32

10 20 30 40 50 60 70 1-200 201-500 >500 PPV base pair distance

CONTRAfold MFE LinearFold-C b=100

10 20 30 40 50 60 70 1-200 201-500 >500 Sensitivity base pair distance

CONTRAfold MFE LinearFold-C b=100

C

200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800

5’ 3’

I

200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800

5’ 3’

CONTRAfold Vienna RNAfold

  • E. coli 23S rRNA

(2,904 nt)

slide-102
SLIDE 102

Improvements on Long-Range Base Pairs

  • long-distance pairs are well-known to be hard to predict
  • LinearFold outputs less long-range base pairs, but more correct ones

32

10 20 30 40 50 60 70 1-200 201-500 >500 PPV base pair distance

CONTRAfold MFE LinearFold-C b=100

10 20 30 40 50 60 70 1-200 201-500 >500 Sensitivity base pair distance

CONTRAfold MFE LinearFold-C b=100

L

200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800

5’ 3’

2901 582

F

200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800

5’ 3’

582

LinearFold-C LinearFold-V

C

200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800

5’ 3’

I

200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800

5’ 3’

CONTRAfold Vienna RNAfold

  • E. coli 23S rRNA

(2,904 nt)

slide-103
SLIDE 103

Incremental Parsing <=> Incremental Folding

  • humans process sentences incrementally
  • human language sentences evolve to be

incrementally parsable

33

  • RNAs & proteins fold while being assembled
  • RNA & protein sequences evolve to be

incrementally foldable

  • these might explain why linear-time search performs better than exact search

Chinese
 speech English
 text Chinese
 text

slide-104
SLIDE 104

Fast Structure Prediction Enables RNA Design

34

GCGGGAAUAGCUCAGUUGGUAGAGCACGACCUUGCCAAGGUCGGGGUCGCGAGUUCGAGUCUCGUUUCCCGCUCCA

RNA sequence RNA secondary structure RNA 3D structure design structure prediction
 (“folding”)

Professor Rhiju Das
 Stanford Medical School EteRNA game
 (RNA design)

detecting active TB using RNA design which needs our fast RNA folding

slide-105
SLIDE 105

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1986 Tomita: Generalized LR
 for all CFGs: O(n3) 2019: LinearFold O(n) 
 (approx.) RNA folding 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

35

1964 Cocke \ bottom-up 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs) 2010: Huang & Sagae: O(n)
 (approx.) DP for all CFGs

S NP DT the NN man VP VB bit NP DT the NN dog

…

O(n3)

dynamic programming

slide-106
SLIDE 106

Computational Linguistics => Computational Biology

linguistics computational biology compiler theory

1958 Backus & Naur: CFGs for program, lang. 1986 Tomita: Generalized LR
 for all CFGs: O(n3) 1978: Nussinov O(n3) RNA folding 1981: Zuker & Siegler

36

1964 Cocke \ bottom-up 1965 Kasami - CKY O(n3) 1967 Younger / for all CFGs 1965 Knuth: LR parsing for 
 restricted CFGs: O(n)

  • comp. linguistics

1955 Chomsky: context-free 
 grammars (CFGs) 2010: Huang & Sagae: O(n)
 (approx.) DP for all CFGs

S NP DT the NN man VP VB bit NP DT the NN dog

…

O(n3)

dynamic programming 1985 Shieber: NL is not CF 1970s Joshi: 1990s TAGs for RNA pseudoknots

slide-107
SLIDE 107

first linear-time (approximate) RNA folding algorithm better in accuracy (esp. long seqs and long-range pairs) http://linearfold.org

slide-108
SLIDE 108

Thank you very much !

first linear-time (approximate) RNA folding algorithm better in accuracy (esp. long seqs and long-range pairs) http://linearfold.org