15
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      conLSH: Context based Locality Sensitive Hashing for Mapping of noisy SMRT Reads

      Preprint
      ,

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Single Molecule Real-Time (SMRT) sequencing is a recent advancement of Next Gen technology developed by Pacific Bio (PacBio). It comes with an explosion of long and noisy reads demanding cutting edge research to get most out of it. To deal with the high error probability of SMRT data, a novel contextual Locality Sensitive Hashing (conLSH) based algorithm is proposed in this article, which can effectively align the noisy SMRT reads to the reference genome. Here, sequences are hashed together based not only on their closeness, but also on similarity of context. The algorithm has \(\mathcal{O}(n^{\rho+1})\) space requirement, where \(n\) is the number of sequences in the corpus and \(\rho\) is a constant. The indexing time and querying time are bounded by \(\mathcal{O}( \frac{n^{\rho+1} \cdot \ln n}{\ln \frac{1}{P_2}})\) and \(\mathcal{O}(n^\rho)\) respectively, where \(P_2 > 0\), is a probability value. This algorithm is particularly useful for retrieving similar sequences, a widely used task in biology. The proposed conLSH based aligner is compared with rHAT, popularly used for aligning SMRT reads, and is found to comprehensively beat it in speed as well as in memory requirements. In particular, it takes approximately \(24.2\%\) less processing time, while saving about \(70.3\%\) in peak memory requirement for H.sapiens PacBio dataset.

          Related collections

          Most cited references15

          • Record: found
          • Abstract: not found
          • Conference Proceedings: not found

          Approximate nearest neighbors

            Bookmark
            • Record: found
            • Abstract: not found
            • Conference Proceedings: not found

            Locality-sensitive hashing scheme based on p-stable distributions

              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Scalable Nearest Neighbor Algorithms for High Dimensional Data.

              For many computer vision and machine learning problems, large training sets are key for good performance. However, the most computationally expensive part of many computer vision and machine learning algorithms consists of finding nearest neighbor matches to high dimensional vectors that represent the training data. We propose new algorithms for approximate nearest neighbor matching and evaluate and compare them with previous algorithms. For matching high dimensional features, we find two algorithms to be the most efficient: the randomized k-d forest and a new algorithm proposed in this paper, the priority search k-means tree. We also propose a new algorithm for matching binary features by searching multiple hierarchical clustering trees and show it outperforms methods typically used in the literature. We show that the optimal nearest neighbor algorithm and its parameters depend on the data set characteristics and describe an automated configuration procedure for finding the best algorithm to search a particular data set. In order to scale to very large data sets that would otherwise not fit in the memory of a single machine, we propose a distributed nearest neighbor matching framework that can be used with any of the algorithms described in the paper. All this research has been released as an open source library called fast library for approximate nearest neighbors (FLANN), which has been incorporated into OpenCV and is now one of the most popular libraries for nearest neighbor matching.
                Bookmark

                Author and article information

                Journal
                11 March 2019
                Article
                1903.04925
                d7332a6a-7d4d-4e29-a9c1-6579ef75e250

                http://arxiv.org/licenses/nonexclusive-distrib/1.0/

                History
                Custom metadata
                arXiv admin note: text overlap with arXiv:1705.03933
                q-bio.GN cs.DS cs.LG stat.ML

                Data structures & Algorithms,Machine learning,Artificial intelligence,Genetics
                Data structures & Algorithms, Machine learning, Artificial intelligence, Genetics

                Comments

                Comment on this article