practical fast on line exact pattern matching algorithms
play

Practical fast on-line exact pattern matching algorithms for highly - PowerPoint PPT Presentation

Feb 29, 2024 •228 likes •615 views

Practical fast on-line exact pattern matching algorithms for highly similar sequences Nadia Ben Nsira Thierry Lecroq Elise Prieur-Gaston LITIS EA 4108, Normastic FR3638, IRIB, Universit e de Rouen Normandie, Normandie Universit e,

  1. Practical fast on-line exact pattern matching algorithms for highly similar sequences ´ Nadia Ben Nsira Thierry Lecroq Elise Prieur-Gaston LITIS EA 4108, Normastic FR3638, IRIB, Universit´ e de Rouen Normandie, Normandie Universit´ e, France Workshop SeqBio 2018, November 19th, 2018 Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 1 / 26

  2. Table of contents Introduction and notations 1 Search in highly similar sequences 2 Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 2 / 26

  3. Table of contents Introduction and notations 1 Search in highly similar sequences 2 Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 3 / 26

  4. Big data NGS technologies output numerous individual genomes of the same species More than 99% similar Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 4 / 26

  5. Highly similar sequences Differ from the reference by: SNVs (SNPs), indels, CNVs, translocations, ... Common and non-common parts Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 5 / 26

  6. Efficient solutions Strong need for efficient indexing and pattern matching Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 6 / 26

  7. Pattern matching Find one(all the) position(s) of a pattern of length m in a sequence of length n : with index → O ( m ) without index → O ( n ) Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 7 / 26

  8. Notations finite alphabet Σ string x [0 . . m − 1] on Σ ∗ length | x | = m x is the reverse of x ( x [ m − 1] x [ m − 2] · · · x [1] x [0] ) ˜ x [ i . . j ] is a factor (substring) of x from position i to position j (both inclusive) x [0 . . i ] is a prefix x [ i . . m − 1] is a suffix u is a border of x if u is both a prefix and a suffix of x Border ( x ) is the longest border of x Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 8 / 26

  9. Sliding window n y x m y x y x Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 9 / 26

  10. Knuth-Morris-Pratt algorithm (1977) comparisons j y u b � = x u a � = z c k = min { ℓ | x [ | Border ℓ ( u ) | ] � = a } and z = Border k ( u ) Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 10 / 26

  11. Boyer-Moore algorithm (1977) comparisons y v b x a v x c v . Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 11 / 26

  12. Table of contents Introduction and notations 1 Search in highly similar sequences 2 Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 12 / 26

  13. Off-line sith an index Huang et al. 2010: O ( n + N log N ) bits where n is the total length of common parts in one string and N is the total length of non-common parts in all sequences Kuruppu et al. 2010: Relative Lempel-Ziv index Na et al. 2018: FM-index of an alignment BWBBLE, Huang et al. 2013: practical solution Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 13 / 26

  14. Highly similar sequences r sequences 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 y 0 A T G C T A G C A A G A T A C A G y 1 A T G C T A G C A A C A T A C A G y 2 A T G C G A G C A A G A T A C A G y 3 A T G C T A G C A A C A T A C A T Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 14 / 26

  15. Highly similar sequences r sequences 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 y 0 A T G C T A G C A A G A T A C A G y 1 A T G C T A G C A A C A T A C A G y 2 A T G C G A G C A A G A T A C A G y 3 A T G C T A G C A A C A T A C A T y { G , T } { C , G } { G , T } A T G C A G C A A A T A C A Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 14 / 26

  16. Highly similar sequences r sequences 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 y 0 A T G C T A G C A A G A T A C A G y 1 A T G C T A G C A A C A T A C A G y 2 A T G C G A G C A A G A T A C A G y 3 A T G C T A G C A A C A T A C A T y { G , T } { C , G } { G , T } A T G C A G C A A A T A C A G A G C A A C Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 14 / 26

  17. Highly similar sequences r sequences 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 y 0 A T G C T A G C A A G A T A C A G y 1 A T G C T A G C A A C A T A C A G y 2 A T G C G A G C A A G A T A C A G y 3 A T G C T A G C A A C A T A C A T y { G , T } { C , G } { G , T } A T G C A G C A A A T A C A G A G C A A C R. Grossi, C. S. Iliopoulos, C. Liu, N. Pisanti, S. P. Pissis, A. Retha, G. Rosone, F. Vayani, L. Versari On-Line Pattern Matching on Similar Texts 28th Combinatorial Pattern Matching (CPM) , Warsaw, Poland (2017) 9:1–9:14 Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 14 / 26

  18. Highly similar sequences r sequences 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 y 0 A T G C T A G C A A G A T A C A G y 1 A T G C T A G C A A C A T A C A G y 2 A T G C G A G C A A G A T A C A G y 3 A T G C T A G C A A C A T A C A T y 0 et Z = (( { 2 } , 4 , G ) , ( { 1 , 3 } , 10 , C ) , ( { 3 } ) , 16 , T ) Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 14 / 26

  19. For highly similar sequences Hamming distance For u, v ∈ A ∗ such that | u | = | v | : Ham ( u, v ) = ♯ { i | u [ i ] � = v [ i ] } Longest Common Extension For x ∈ A ∗ and 0 ≤ i ≤ j ≤ | x | − 1 : LCE k x ( i, j ) = max { ℓ | Ham ( x [ i . . i + ℓ − 1] , x [ j . . j + ℓ − 1]) ≤ k } Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 15 / 26

  20. Kangaroo jumps Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 16 / 26

  21. Kangaroo jumps i j Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 16 / 26

  22. Kangaroo jumps i j Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 16 / 26

  23. Kangaroo jumps 1 i j Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 16 / 26

  24. Kangaroo jumps 1 2 i j Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 16 / 26

  25. Kangaroo jumps 1 2 3 i j Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 16 / 26

  26. Kangaroo jumps 1 2 3 i j LCE k x ( i, j ) can be computed in O ( k ) time after O ( n ) preprocessing time Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 16 / 26

  27. References Restriction: 1 variation on a window of size m Adaptations of KMP and BM without LCE by adapting the shift functions N. Ben Nsira, T. Lecroq and M. Elloumi A fast Boyer-Moore type pattern matching algorithm for highly similar sequences International Journal of Data Mining and Bioinformatics 13 (3) (2015) 266-288 N. Ben Nsira, T. Lecroq and M. Elloumi On-line String Matching in Highly Similar DNA Sequences Mathematics in Computer Science 11 (2) (2017) 113–126 Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 17 / 26

  28. 2 variants relaxing the restriction from 1 to k variations in a window of size m searching for a finite set of patterns (still with 1 variation in a window of size m Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 18 / 26

  29. Single pattern with at most k variations Applying the Landau-Vishkin algorithm as a filter Searching with k mismatches in O ( kn ) When Ham ( x, y 0 [ j . . j + ℓ − 1]) = ℓ ≤ k ℓ = 0 : an exact occurrence of the pattern has been found in y 0 and all the other sequence that do not have a variation comparing to y 0 between position j and position j + m − 1 both included. ℓ > 0 : let W = { i 0 , . . . , i ℓ − 1 } be the set of the ℓ positions such that y 0 [ j + i p ] � = x [ i p ] with 0 ≤ p < ℓ . Then x occurs exactly in y h if: ◮ ( G , j + i p , x [ i p ]) ∈ Z with g ∈ G for all 0 ≤ p < ℓ ; ◮ � ∃ ( G , h, c ) ∈ Z such that h �∈ W . Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 19 / 26

  30. Single pattern with at most k variations r = 2 and k = 2 0 1 2 3 4 5 6 7 8 9 10 y 0 A C C T A C G A C T A x C T A C T T j = 2 and W = (4 , 5) x C T A C T T j = 5 and W = (1 , 5) y 1 A C C T A C T A C T T Z = (( { 1 } , 6 , T ) , ( { 1 } , 10 , T )) Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 20 / 26

  31. Single pattern with at most k variations r = 2 and k = 2 0 1 2 3 4 5 6 7 8 9 10 y 0 A C C T A C G A C T A x C T A C T T j = 2 and W = (4 , 5) x C T A C T T j = 5 and W = (1 , 5) y 1 A C C T A C T A C T T Z = (( { 1 } , 6 , T ) , ( { 1 } , 10 , T )) Our solution runs in time O ( knr ) Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 20 / 26

  32. Multiple patterns with at most 1 variation Build a classical trie of the patterns Scan the highly similar sequences with at most 2 active states Ben Nsira, Lecroq, Prieur (LITIS) Similar Sequences SeqBio 2018 21 / 26

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern

Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern

Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern Matching 1 Outline and Reading Strings (11.1) Pattern matching algorithms Brute-force algorithm (11.2.1) Boyer-Moore algorithm (11.2.2)

612 views • 14 slides
Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p

Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p

Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p 1 p n and text t = t 1 t m Output: All positions 1&lt; i &lt; ( m n + 1) such that the n - letter substring of t starting at i matches p

486 views • 24 slides
Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p

Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p

Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p 1 p n and text t = t 1 t m Output: All positions 1&lt; i &lt; ( m n + 1) such that the n - letter substring of t starting at i matches p

657 views • 19 slides
Quantum pattern matching fast on average Ashley Montanaro Department of Computer Science,

Quantum pattern matching fast on average Ashley Montanaro Department of Computer Science,

Quantum pattern matching fast on average Ashley Montanaro Department of Computer Science, University of Bristol, UK 12 January 2015 Pattern matching In the traditional pattern matching problem, we seek to find a pattern P : [ m ] within

1.25k views • 53 slides
CSE182-L6 P-value and E-value Dicitionary matching Pattern matching October 09 CSE182 Why is

CSE182-L6 P-value and E-value Dicitionary matching Pattern matching October 09 CSE182 Why is

CSE182-L6 P-value and E-value Dicitionary matching Pattern matching October 09 CSE182 Why is BLAST fast? Assume that keyword searching does not consume any time and that alignment computation the expensive step. Query m=1000, random

433 views • 18 slides
Pattern matching with the su ffi x tree Zsuzsanna Lipt ak Masters in Medical Bioinformatics

Pattern matching with the su ffi x tree Zsuzsanna Lipt ak Masters in Medical Bioinformatics

Bioinformatics Algorithms (Fundamental Algorithms, module 2) Pattern matching with the su ffi x tree Zsuzsanna Lipt ak Masters in Medical Bioinformatics academic year 2018/19, II. semester Su ffi x Trees 2 2 / 18 Recall: Pattern matching

522 views • 3 slides
Pattern matching algorithms Vineet Bafna October 4, 2004 1 Algorithms for keyword search

Pattern matching algorithms Vineet Bafna October 4, 2004 1 Algorithms for keyword search

Pattern matching algorithms Vineet Bafna October 4, 2004 1 Algorithms for keyword search These are meant to supplement your class notes, but do not substitute for class lectures. Try these algorithms on examples and make sure that you

230 views • 3 slides
Pattern matching algorithms Vineet Bafna April 23, 2004 1 Algorithms for keyword search

Pattern matching algorithms Vineet Bafna April 23, 2004 1 Algorithms for keyword search

Pattern matching algorithms Vineet Bafna April 23, 2004 1 Algorithms for keyword search These are meant to supplement your class notes, but do not substitute for class lectures. Try these algorithms on examples and make sure that you

63 views • 3 slides
Bioinformatics Algorithms (Fundamental Algorithms, module 2) Zsuzsanna Lipt ak Masters in

Bioinformatics Algorithms (Fundamental Algorithms, module 2) Zsuzsanna Lipt ak Masters in

Bioinformatics Algorithms (Fundamental Algorithms, module 2) Zsuzsanna Lipt ak Masters in Medical Bioinformatics academic year 2018/19, II. semester Suffix Trees 2 Pattern matching with the suffix tree 2 / 18 Recall: Pattern matching

580 views • 27 slides
Co-Evaluation of Pattern Matching Algorithms on IoT Devices with Embedded GPUs Charalampos

Co-Evaluation of Pattern Matching Algorithms on IoT Devices with Embedded GPUs Charalampos

Co-Evaluation of Pattern Matching Algorithms on IoT Devices with Embedded GPUs Charalampos Stylianopoulos Simon Kindstrm Magnus Almgren Olaf Landsiedel Marina Papatriantafilou Distributed Computing and Systems Motivation IoT security

908 views • 31 slides
Concurrent Pattern Matching: combining discovery, privacy and symmetry using pattern matching

Concurrent Pattern Matching: combining discovery, privacy and symmetry using pattern matching

Introduction Background Deliverables &amp; Methodology State of research Concurrent Pattern Matching: combining discovery, privacy and symmetry using pattern matching Thomas Given-Wilson, University of Technology, Sydney Supervisor:

559 views • 21 slides
sed: Stream Oriented, Non- Interactive, Text Editor ! Line-oriented tool for pattern matching and

sed: Stream Oriented, Non- Interactive, Text Editor ! Line-oriented tool for pattern matching and

sed: Stream Oriented, Non- Interactive, Text Editor ! Line-oriented tool for pattern matching and replacement (stream editor) CSCI: 4500/6500 Programming Looks for patterns one line at a time, like grep Languages Change lines of

228 views • 6 slides
Exscind: Fast Pattern Matching for Intrusion Detection Using Exclusion and Inclusion Filters

Exscind: Fast Pattern Matching for Intrusion Detection Using Exclusion and Inclusion Filters

Exscind: Fast Pattern Matching for Intrusion Detection Using Exclusion and Inclusion Filters Monther Aldwairi and Duaa Alansari Department of Network Engineering and Security Jordan University of Science and Technology Irbid, Jordan

212 views • 7 slides
Grep Find all lines matching some pattern No need to combine anything Reduce is not

Grep Find all lines matching some pattern No need to combine anything Reduce is not

9/22/2011 Grep Find all lines matching some pattern No need to combine anything Reduce is not needed, i.e., just identity function Map takes line and outputs it if it matches the pattern Map could also take an entire document

227 views • 6 slides
Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern

Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern

CSC209 Fall 2001 What is AWK? Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern matches, do something Many details handled automatically very easy to one off (write and throw away)

673 views • 12 slides
Minimal absent words in a sliding window &amp; applications to on-line pattern matching Maxime

Minimal absent words in a sliding window &amp; applications to on-line pattern matching Maxime

Minimal absent words in a sliding window &amp; applications to on-line pattern matching Maxime Crochemore 1 , 2 , Alice Hliou 3 , Gregory Kucherov 2 , Laurent Mouchard 4 , Solon Pissis 1 , Yann Ramusat 5 1 Department of Informatics, Kings

1.59k views • 83 slides
Complete Sequent Calculi for Induction and Infinite Descent James Brotherston and Alex Simpson

Complete Sequent Calculi for Induction and Infinite Descent James Brotherston and Alex Simpson

Complete Sequent Calculi for Induction and Infinite Descent James Brotherston and Alex Simpson Dept. of Computing, Imperial College / LFCS, University of Edinburgh LICS-22, 1014 July, Wroclaw, Poland Overview Our interest: inductive

164 views • 15 slides
Productively (and non-productively) Menger spaces Piotr Szewczak Cardinal Stefan Wyszy nski

Productively (and non-productively) Menger spaces Piotr Szewczak Cardinal Stefan Wyszy nski

Productively (and non-productively) Menger spaces Piotr Szewczak Cardinal Stefan Wyszy nski University, Poland, and Bar-Ilan University, Israel joint work with Boaz Tsaban Toposym 2016 Supported by National Science Center Poland

1.29k views • 111 slides
Generalized Cayley graphs for fundamental groups of one-dimensional spaces Hanspeter Fischer

Generalized Cayley graphs for fundamental groups of one-dimensional spaces Hanspeter Fischer

Motivation Approach Word Sequences Generalized Cayley graphs for fundamental groups of one-dimensional spaces Hanspeter Fischer (Ball State University, USA) joint work with Andreas Zastrow (University of Gda nsk, Poland) Dubrovnik VII

871 views • 31 slides
Recent ATLAS results in the field of meson physics Rui Wang University of New Mexico On behalf

Recent ATLAS results in the field of meson physics Rui Wang University of New Mexico On behalf

Recent ATLAS results in the field of meson physics Rui Wang University of New Mexico On behalf of ATLAS collaboration 13 th International Workshop on Meson Production, Properties and Interaction KRAKW, POLAND, 29 th May - 3 rd June 2014

537 views • 34 slides
Banach Contraction Mapping Principle Oksana Bihun March 2, 2010 Department of Mathematics and

Banach Contraction Mapping Principle Oksana Bihun March 2, 2010 Department of Mathematics and

Banach Contraction Mapping Principle Oksana Bihun March 2, 2010 Department of Mathematics and Computer Science Concordia College, Moorhead, MN Stefan Banach Metric Spaces Banach Contraction Mapping Principle Applications to Ordinary

737 views • 17 slides
Inverse scattering for reflectionless Schrdinger operators and generalized KdV solitons

Inverse scattering for reflectionless Schrdinger operators and generalized KdV solitons

Introduction Reflectionless potentials Main results Inverse scattering for reflectionless Schrdinger operators and generalized KdV solitons Rostyslav Hryniv 1 Yaroslav Mykytyuk 2 1 Ukrainian Catholic University, Lviv, Ukraine University of

630 views • 24 slides
On Polynomial-Time Congestion-Free Software-Defined Network Updates Saeed Akhoondian Amiri (MPI,

On Polynomial-Time Congestion-Free Software-Defined Network Updates Saeed Akhoondian Amiri (MPI,

On Polynomial-Time Congestion-Free Software-Defined Network Updates Saeed Akhoondian Amiri (MPI, Saarland, Germany) Szymon Dudycz ( 6 University of Wroclaw, Poland) Mahmoud Parham and Stefan Schmid (University of Vienna, Austria) Sebastian

653 views • 40 slides
p -adic valuations of certain colored partition functions Maciej Ulas Institute of Mathematics,

p -adic valuations of certain colored partition functions Maciej Ulas Institute of Mathematics,

p -adic valuations of certain colored partition functions Maciej Ulas Institute of Mathematics, Jagiellonian University, Krak ow, Poland September 7, 2018 Maciej Ulas p -adic valuations ... Short plan of the presentation The general

1.43k views • 114 slides

More recommend