Blog
About

63
views
0
recommends
+1 Recommend
0 collections
    5
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Mutual Information and Diverse Decoding Improve Neural Machine Translation

      Preprint

      ,

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Sequence-to-sequence neural translation models learn semantic and syntactic relations between sentence pairs by optimizing the likelihood of the target given the source, i.e., \(p(y|x)\), an objective that ignores other potentially useful sources of information. We introduce an alternative objective function for neural MT that maximizes the mutual information between the source and target sentences, modeling the bi-directional dependency of sources and targets. We implement the model with a simple re-ranking method, and also introduce a decoding algorithm that increases diversity in the N-best list produced by the first pass. Applied to the WMT German/English and French/English tasks, the proposed models offers a consistent performance boost on both standard LSTM and attention-based neural MT architectures.

          Related collections

          Author and article information

          Journal
          2016-01-03
          2016-03-22
          1601.00372

          http://arxiv.org/licenses/nonexclusive-distrib/1.0/

          Custom metadata
          cs.CL cs.AI

          Theoretical computer science, Artificial intelligence

          Comments

          Comment on this article