3
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Imputer: Sequence Modelling via Imputation and Dynamic Programming

      Preprint

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          This paper presents the Imputer, a neural sequence model that generates output sequences iteratively via imputations. The Imputer is an iterative generative model, requiring only a constant number of generation steps independent of the number of input or output tokens. The Imputer can be trained to approximately marginalize over all possible alignments between the input and output sequences, and all possible generation orders. We present a tractable dynamic programming training algorithm, which yields a lower bound on the log marginal likelihood. When applied to end-to-end speech recognition, the Imputer outperforms prior non-autoregressive models and achieves competitive results to autoregressive models. On LibriSpeech test-other, the Imputer achieves 11.1 WER, outperforming CTC at 13.0 WER and seq2seq at 12.5 WER.

          Related collections

          Author and article information

          Journal
          20 February 2020
          Article
          2002.08926
          c5da0989-8e48-4c1f-9ae8-db2331ffc22e

          http://arxiv.org/licenses/nonexclusive-distrib/1.0/

          History
          Custom metadata
          preprint
          eess.AS cs.CL cs.LG cs.SD

          Theoretical computer science,Artificial intelligence,Graphics & Multimedia design,Electrical engineering

          Comments

          Comment on this article