For Publishers
DiscoveryMetadataPeer reviewHostingPublishing
For Researchers
JoinPublishReviewCollect
Register
Blog
About
Search
Advanced search
My ScienceOpen
Search
For Publishers
For Researchers
Blog
About
271
views
5
references
 Top references
cited by
2,222
    
0 reviews
 Review
0
comments
 Comment
0
recommends
+1 Recommend
0 collections
    0
    shares
      162
      similar
       All similar
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      CD-HIT: accelerated for clustering the next-generation sequencing data

      research-article
      Author(s): , , , , *
      Publication date (Electronic):
      Journal: Bioinformatics
      Publisher: Oxford University Press

      Read this article at

      ScienceOpenPublisherPMC
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Summary: CD-HIT is a widely used program for clustering biological sequences to reduce sequence redundancy and improve the performance of other sequence analyses. In response to the rapid increase in the amount of sequencing data produced by the next-generation sequencing technologies, we have developed a new CD-HIT program accelerated with a novel parallelization strategy and some other techniques to allow efficient clustering of such datasets. Our tests demonstrated very good speedup derived from the parallelization for up to ∼24 cores and a quasi-linear speedup for up to ∼8 cores. The enhanced CD-HIT is capable of handling very large datasets in much shorter time than previous versions.

          Availability: http://cd-hit.org.

          Contact: liwz@ 123456sdsc.edu

          Supplementary information: Supplementary data are available at Bioinformatics online.

          Related collections

          Most cited references5

          • Record: found
          • Abstract: found
          • Article: not found

          UniRef: comprehensive and non-redundant UniProt reference clusters.

          Baris Suzek, Hongzhan Huang, Peter McGarvey … (2007)
          Redundant protein sequences in biological databases hinder sequence similarity searches and make interpretation of search results difficult. Clustering of protein sequence space based on sequence similarity helps organize all sequences into manageable datasets and reduces sampling bias and overrepresentation of sequences. The UniRef (UniProt Reference Clusters) provide clustered sets of sequences from the UniProt Knowledgebase (UniProtKB) and selected UniProt Archive records to obtain complete coverage of sequence space at several resolutions while hiding redundant sequences. Currently covering >4 million source sequences, the UniRef100 database combines identical sequences and subfragments from any source organism into a single UniRef entry. UniRef90 and UniRef50 are built by clustering UniRef100 sequences at the 90 or 50% sequence identity levels. UniRef100, UniRef90 and UniRef50 yield a database size reduction of approximately 10, 40 and 70%, respectively, from the source sequence set. The reduced redundancy increases the speed of similarity searches and improves detection of distant relationships. UniRef entries contain summary cluster and membership information, including the sequence of a representative protein, member count and common taxonomy of the cluster, the accession numbers of all the merged entries and links to rich functional annotation in UniProtKB to facilitate biological discovery. UniRef has already been applied to broad research areas ranging from genome annotation to proteomics data analysis. UniRef is updated biweekly and is available for online search and retrieval at http://www.uniprot.org, as well as for download at ftp://ftp.uniprot.org/pub/databases/uniprot/uniref. Supplementary data are available at Bioinformatics online.
          Article 0 comments Cited 574 times – based on 0 reviews     Review now
            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Artificial and natural duplicates in pyrosequencing reads of metagenomic data

            Beifang Niu, Limin Fu, Shulei Sun … (2010)
            Background Artificial duplicates from pyrosequencing reads may lead to incorrect interpretation of the abundance of species and genes in metagenomic studies. Duplicated reads were filtered out in many metagenomic projects. However, since the duplicated reads observed in a pyrosequencing run also include natural (non-artificial) duplicates, simply removing all duplicates may also cause underestimation of abundance associated with natural duplicates. Results We implemented a method for identification of exact and nearly identical duplicates from pyrosequencing reads. This method performs an all-against-all sequence comparison and clusters the duplicates into groups using an algorithm modified from our previous sequence clustering method cd-hit. This method can process a typical dataset in ~10 minutes; it also provides a consensus sequence for each group of duplicates. We applied this method to the underlying raw reads of 39 genomic projects and 10 metagenomic projects that utilized pyrosequencing technique. We compared the occurrences of the duplicates identified by our method and the natural duplicates made by independent simulations. We observed that the duplicates, including both artificial and natural duplicates, make up 4-44% of reads. The number of natural duplicates highly correlates with the samples' read density (number of reads divided by genome size). For high-complexity metagenomic samples lacking dominant species, natural duplicates only make up <1% of all duplicates. But for some other samples like transcriptomic samples, majority of the observed duplicates might be natural duplicates. Conclusions Our method is available from http://cd-hit.org as a downloadable program and a web server. It is important not only to identify the duplicates from metagenomic datasets but also to distinguish whether they are artificial or natural duplicates. We provide a tool to estimate the number of natural duplicates according to user-defined sample types, so users can decide whether to retain or remove duplicates in their projects.
            Article 0 comments Cited 131 times – based on 0 reviews     Review now
              Bookmark
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              Gene identification and protein classification in microbial metagenomic sequence data via incremental clustering

              Shibu Yooseph, Weizhong Li, Granger Sutton (2008)
              Background The identification and study of proteins from metagenomic datasets can shed light on the roles and interactions of the source organisms in their communities. However, metagenomic datasets are characterized by the presence of organisms with varying GC composition, codon usage biases etc., and consequently gene identification is challenging. The vast amount of sequence data also requires faster protein family classification tools. Results We present a computational improvement to a sequence clustering approach that we developed previously to identify and classify protein coding genes in large microbial metagenomic datasets. The clustering approach can be used to identify protein coding genes in prokaryotes, viruses, and intron-less eukaryotes. The computational improvement is based on an incremental clustering method that does not require the expensive all-against-all compute that was required by the original approach, while still preserving the remote homology detection capabilities. We present evaluations of the clustering approach in protein-coding gene identification and classification, and also present the results of updating the protein clusters from our previous work with recent genomic and metagenomic sequences. The clustering results are available via CAMERA, (http://camera.calit2.net). Conclusion The clustering paradigm is shown to be a very useful tool in the analysis of microbial metagenomic data. The incremental clustering method is shown to be much faster than the original approach in identifying genes, grouping sequences into existing protein families, and also identifying novel families that have multiple members in a metagenomic dataset. These clusters provide a basis for further studies of protein families.
              Article 0 comments Cited 23 times – based on 0 reviews     Review now
                Bookmark
                All references

                Author and article information

                Journal
                Journal ID (nlm-ta): Bioinformatics
                Journal ID (iso-abbrev): Bioinformatics
                Journal ID (publisher-id): bioinformatics
                Journal ID (hwp): bioinfo
                Title: Bioinformatics
                Publisher: Oxford University Press
                ISSN (Print): 1367-4803
                ISSN (Electronic): 1367-4811
                Publication date (Print): 1 December 2012
                Publication date (Electronic): 11 October 2012
                Publication date PMC-release: 11 October 2012
                Volume: 28
                Issue: 23
                Pages: 3150-3152
                Affiliations
                Center for Research in Biological Systems, University of California San Diego, La Jolla, CA 92093, USA
                Author notes
                *To whom correspondence should be addressed.

                †Present address: Department of Genetics, University of Pennsylvania, Philadelphia, Pennsylvania.

                Associate Editor: Inanc Birol

                Article
                Publisher ID: bts565
                DOI: 10.1093/bioinformatics/bts565
                PMC ID: 3516142
                PubMed ID: 23060610
                SO-VID: 2f33043c-2d16-4157-b512-b3690d1a21d1
                Copyright © © The Author 2012. Published by Oxford University Press.
                License:

                This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/3.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

                History
                Date received : 27 June 2012
                Date revision received : 31 August 2012
                Date accepted : 12 September 2012
                Page count
                Pages: 3
                Categories
                Subject: Applications Note
                Subject: Sequence Analysis

                ScienceOpen disciplines: Bioinformatics & Computational biology
                Data availability:
                ScienceOpen disciplines: Bioinformatics & Computational biology

                Comments

                Comment on this article

                scite_

                Similar content162

                See all similar

                Cited by3,416

                See all cited by

                Most referenced authors497

                See all reference authors