est clustering
play

EST clustering Swiss Institute of Bioinformatics (SIB) 26-30 - PowerPoint PPT Presentation

Sep 02, 2022 •2.49k likes •2.74k views

EST clustering Swiss Institute of Bioinformatics (SIB) 26-30 November 2000 Course 2001 EST clustering Expressed sequence tags (ESTs) ESTs represent the most extensive available survey of the transcribed portion of the genome. ESTs are

  1. EST clustering Swiss Institute of Bioinformatics (SIB) 26-30 November 2000

  2. Course 2001 EST clustering Expressed sequence tags (ESTs) ESTs represent the most extensive available survey of the transcribed portion of the genome. ESTs are single-pass reads from the 5’ and/or 3’ end of cDNA clones. ESTs represent partial sequences of cDNA clones ( ∼ 300 bp). High-volume and high-throughput data production. ESTs are used extensively and are indispensable for gene discovery and genomic mapping. ESTs are often associated with tissues. There are 9,372,718 of EST entries in GeneBank (October 26, 2001): • 3,859,807 entries of human ESTs; • 2,328,188 entries of mouse ESTs; • ... 1

  3. Course 2001 EST clustering Expressed sequence tags (ESTs) ESTs are difficult to use effectively: • partial gene sequences; • high error rates ( ∼ 1 / 100 ) because of the sequence single-pass; • not a defined protein product; • not curate in a highly annotated form; • high redundancy in the data. The value of ESTs is greatly enhanced if they are used to construct high-fidelity set of non-redundant transcripts: • fewer sequences to analyze; • solving redundancy can help to correct errors; • longer sequences; • better annotated; • easier association to mRNAs and proteins; • detection of splice variants; • they can be used for extensive functional annotation; • facilitates gene expression studies. 2

  4. Course 2001 EST clustering EST clustering The goal of the clustering process is to incorporate overlapping ESTs which tag the same transcript of the same gene in a single cluster. Once clustering is completed, one or more consensus assemblies for each individual cluster can be produced. EST clustering and assembling presents a number of distinct computational prob- lems: • can be extremely time consuming due to the intrinsic need for all pairs of ESTs to be tested for overlap; • ESTs derive from a wide variety of sources representing the polymorphism in the original samples; • high sequencing errors; • high rate of insertions and deletions; • contaminations by vector and linker sequences; ⇒ the degree of identity in overlapping sequences from the same gene will be lower than in genomic projects. 3

  5. Course 2001 EST clustering EST clustering Patterns of overlapping sequences caused by alternative splicing will be different from the ones observed in the genomics shotgun projects. A big number of EST sequences lack the base-calling quality values or the associated chromatograms. ⇒ Although the clustering and assembling of ESTs is a problem similar to clustering and assembling of genome shotgun data, software and parameters have to be adapted to solve the ESTs specific problems. Different clustering/assembly procedures have been proposed with associated result- ing database, also called gene indices: • UniGene (http://www.ncbi.nlm.nih.gov/UniGene) • TIGR Gene Indices (http://www.tigr.org/tdb/tgi.shtml) • STACK (http://www.sanbi.ac.za/Dbases.html) • trEST (ftp://ftp.isrec.isb-sib.ch/pub/databases/trest) 4

  6. Course 2001 EST clustering Pipeline for EST clustering The steps for EST clustering: • Obtain EST sequences of interest (sequencing project, dbEST, ...) and/or chromatograms. • Process chromatograms for low-quality regions (Phred). • Mask repeats (pairwise comparison programs, RepBase). • Mask/delete contaminants (pairwise comparison programs, mitochondrial DNA, ribosomal DNA, ...). • Clustering (time expensive): ⊲ ”Strict” cluster: pairwise alignment programs are used to compare each sequence of the dataset. Only sequences sharing good homology of the same regions are clustered together. ⊲ ”Loose” cluster: accept sequences in the cluster if they have a region of homology with at least one sequence of the cluster. • Assembling of the clusters to generate consensus sequences and contigs. ⊲ This step is performed by programs like Phrap, CAP3, ... ⊲ Each cluster can produce ≥ 1 contigs, representing chimeras, splice variants, se- quencing errors, ... • Some post-processing can be done. 5

  7. Course 2001 EST clustering UniGene UniGene characteristics: • contains clusters of sequences deriving from GeneBank; • each cluster represents a unique gene as well as tissue types where the gene is expressed and map locations; • ESTs are included; • no attempts to produce contigs or consensus sequences; • all splice variants for a gene are put into the same cluster. UniGene uses pairwise sequence comparison at various levels of stringency to group related sequences, placing closely related and alternatively spliced transcripts into clusters. UniGene data can be downloaded from ftp://ftp.ncbi.nlm.nih.gov/repository/UniGene. 6

  8. Course 2001 EST clustering UniGene UniGene build procedure: • Screen for contaminants, repeats, and low-complexity regions in GeneBank: ⊲ Low-complexity are detected using Dust ; ⊲ Contaminants (vector, linker, bacterial, mitochondrial, ribosomal sequences) are detected using pairwise alignment programs; ⊲ Repeat masking of repeated regions (RepeatMasker). • Clustering procedure, which results in clusters called anchored clusters: ⊲ Build clusters of genes and mRNAs. ⊲ Add ESTs to previous clusters (megablast). ⊲ ESTs that join two clusters of genes/mRNAs are discarded. ⊲ Any resulting cluster without a polyadenilation signal or two 3’ ESTs is discarded. • Ensures 5’ and 3’ ESTs from the same clone belongs to the same cluster. • ESTs that have not been clustered, are reprocessed with lower level of stringency. ESTs added during this step are called guest members. • Clusters of size 1 are compared against the rest of the clusters with a lower level of stringency and merged with the cluster containing the most similar sequence. 7

  9. Course 2001 EST clustering TIGR Gene Indices TIGR Gene Indices uses assembly algorithms, rather than clustering, to produce tentative consensus (TC) sequences that represent the underlying mRNA transcripts. The TIGR Gene Indices building method tightly groups highly related sequences and discard under-represented, divergent, or noisy sequences. TIGR Gene Indices characteristics: • separate closely related genes into distinct consensus sequences; • separate splice variants into separate clusters; • low level of contamination. TC sequences can be used for genome annotation, genome mapping, and identifica- tion of orthologs/paralogs genes. References: • Quackenbush et al. (2000) Nucleic Acid Research , 28 , 141-145. • Quackenbush et al. (2001) Nucleic Acid Research , 29 , 159-164. 8

  10. Course 2001 EST clustering TIGR Gene Indices Construction process of the TIGR Gene Indexes (identical process for each gene index): • EST sequences recovered form dbEST (http://www.ncbi.nlm.nih.gov/dbEST); • Sequences are trimmed to remove: ⊲ vectors ⊲ polyA/T tails ⊲ adaptor sequences ⊲ bacterial sequences ⊲ mitochondrial sequences ⊲ ribosomal sequences ⊲ low quality sequences 9

  11. Course 2001 EST clustering TIGR Gene Indices • Get expressed transcripts (ETs) from EGAD (http://www.tigr.org/tdb/egad/egad.shtml): ⊲ EGAD (Expressed Gene Anatomy Database) is based on mRNA and CDS (coding sequences) from GeneBank. • Cleaned ESTs and ETs are compared using FLAST (a rapid pairwise comparison program). Sequences are grouped in the same cluster if both conditions are true: ⊲ they share ≥ 95% identity over 40 nt or longer regions ⊲ < 20 bases of mismatch • Each cluster is assembled using CAP3 assembling program to produce tentative consensus (TC) sequences. ⊲ CAP3 can generate multiple consensus sequences for each cluster ⊲ CAP3 rejects chimeric, low-quality and non-overlapping sequences. • Builded TCs are loaded in the TIGR Gene Indices database and annotated. 10

  12. Course 2001 EST clustering STACK Based on ”loose” clustering, followed by strict assembly and analysis tools to identify, characterize, view and isolate sequence divergence. The ”loose” clustering approach, d2 cluster, is not based on alignments, but per- forms comparisons via non-contextual assessment of the composition and multiplicity of words within each sequence. STACK produces longer consensus sequences than TIGR Gene Indices. STACK concentrates on human data. References: • Miller et al. (1999) Genome Research , 9 , 1143-1155. • Christoffels et al. (2001) Nucleic Acid Research , 29 , 234-238. 11

  13. Course 2001 EST clustering STACK The STACK procedure: • Sub-partitioning. ⊲ Select human ESTs from GeneBank; ⊲ Sequences are grouped in tissue-based categories. • Masking using cross-match. ⊲ Parameters: minmatch=12 minscore=20; ⊲ Human repeat sequences (RepBase); ⊲ Vector sequences (ftp://ncbi.nlm.nih.gov/blast/db/vector.Z); ⊲ Ribosomal and mitochondrial DNA. • ”Loose” clustering (d2 cluster). ⊲ Parameters: word size=6 similarity cutoff=0.96 minimum sequence size=50 win- dow size=100; ⊲ The algorithm looks for a window of size 100 bases having at least 96% identity. ⊲ Clusters highly related sequences; ⊲ Clusters also sequences related by rearrangements or alternative splicing. 12

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Many

768 views • 43 slides
Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find a finite set of representatives that provides good coverage of a complex, possibly infinite, high-dimensional space. Finding meaningful

396 views • 36 slides
Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides
Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering Mixture models Hierarchical clustering Dimensionality reduction Principal component analysis Multidimensional scaling Isomap

1.61k views • 17 slides
Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods 1 2 What is Clustering? What is Clustering? Typical

647 views • 18 slides
Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

LUDWIG- MAXIMILIANS- INSTITUTE DATABASE UNIVERSITT FOR SYSTEMS MNCHEN MNCHEN INFORMATICS INFORMATICS GROUP GROUP Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative Clustering, Multiview

513 views • 9 slides
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides
Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K- means clustering What is clustering?

1.32k views • 66 slides
PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin

PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin

PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin Motivation Example Clustering cannot be analyzed without specifying what it will be used for! be used for! Example Cluster then pack

293 views • 28 slides
Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means clustering Hierarchical clustering Incremental clustering Probability-based clustering Self-Organising Maps Classification vs. Clustering

1k views • 67 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management of web data Outline

607 views • 41 slides
Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean clustering, hierarchical clustering Adaptive learning (online learning) CL, FSCL, RPCL Gaussian Mixture Models (GMM)

805 views • 34 slides
Solid State Batteries for Medical Devices and Sensors Denis Pasero, Product Commercialisation

Solid State Batteries for Medical Devices and Sensors Denis Pasero, Product Commercialisation

Solid State Batteries for Medical Devices and Sensors Denis Pasero, Product Commercialisation Manager Medical Battery Conference 17 Nov 2017 Dsseldorf, Germany Presentation structure 1. Introduction 2. Challenges for Powering Medical

706 views • 31 slides
Coherence and Synchronization of Noisy-Driven Oscillators Denis S. Goldobin Institut fr

Coherence and Synchronization of Noisy-Driven Oscillators Denis S. Goldobin Institut fr

Coherence and Synchronization of Noisy-Driven Oscillators Denis S. Goldobin Institut fr Physik, Universitt Potsdam Scientific advisor: Prof. Dr. Arkady Pikovsky Potsdam - Aug 27, 2007 2 Outline 1. Coherence of Oscillators with Delayed

787 views • 34 slides
Learning from Lived Experience This presentation was delivered by Sally Bromley, chair of the

Learning from Lived Experience This presentation was delivered by Sally Bromley, chair of the

Learning from Lived Experience This presentation was delivered by Sally Bromley, chair of the Oxford Branch of Parkinsons UK, at a conference in London, England on 16 th March 2018. The main focus of the conference was the implementation of

319 views • 11 slides
Public Realm Agenda Item 5 Spotlight on - The Essential Agenda Item 5 Tremor Awareness Day

Public Realm Agenda Item 5 Spotlight on - The Essential Agenda Item 5 Tremor Awareness Day

Appendix B 28 March 2019 Sutton Local Committee Neighbourhood Grants &amp; Page 19 Public Realm Agenda Item 5 Spotlight on - The Essential Agenda Item 5 Tremor Awareness Day Introducing Jackie Farrell from the Page 20 Essential Tremor

340 views • 8 slides
The Specialist Committee on Uncertainty Analysis Final Report and Recommendations to the 25th

The Specialist Committee on Uncertainty Analysis Final Report and Recommendations to the 25th

Proceedings of 25th ITTC Volume II 433 The Specialist Committee on Uncertainty Analysis Final Report and Recommendations to the 25th ITTC 1. INTRODUCTION Southern Europe appointed a new member to replace him. 1.1 Membership and Meetings

538 views • 39 slides
Accurate Methods for the Statistics of Surprise and Coincidence Ted Dunning Computing Research

Accurate Methods for the Statistics of Surprise and Coincidence Ted Dunning Computing Research

Accurate Methods for the Statistics of Surprise and Coincidence Ted Dunning Computing Research Laboratory New Mexico State University Las Cruces, NM 88003-0001 ABSTRACT Much work has been done on the statistical analysis of text. In some

472 views • 18 slides
IMPORTANT NOTICE IMPORTANT: You must read the following disclaimer before continuing. The following

IMPORTANT NOTICE IMPORTANT: You must read the following disclaimer before continuing. The following

IMPORTANT NOTICE IMPORTANT: You must read the following disclaimer before continuing. The following disclaimer applies to the Prospectus attached to this electronic transmission and you are therefore advised to read this disclaimer carefully

1.64k views • 142 slides
Web Intelligence for Improved Decision Making (WISDOM) Final Presentation January 22, 2014

Web Intelligence for Improved Decision Making (WISDOM) Final Presentation January 22, 2014

Web Intelligence for Improved Decision Making (WISDOM) Final Presentation January 22, 2014 Member of the University of Applied Sciences Eastern Switzerland (FHO) Agenda 1. Introduction 2. Key technologies 3. Project highlights &amp;

761 views • 65 slides

More recommend