Blog
About

  • Record: found
  • Abstract: found
  • Article: found
Is Open Access

The Effect Of Smoothing In Language Models For Novelty Detection

BCS IRSG Symposium: Future Directions in Information Access 2007 (FDIA)

Future Directions in Information Access

28-29 August 2007

Language Models, Smoothing, Novelty Detection, Kullback-Leibler Divergence

Read this article at

Bookmark
      There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

      Abstract

      The novelty task consists of finding relevant and novel sentences in a ranking of documents given a query. In the literature, different techniques have been applied to address this problem. Nevertheless, little is known about Language Models for novelty detection and, especially, the effect of smoothing on the selection of novel sentences. Language Models can be used to study novelty and relevance in a principled way. These statistical models have been shown to perform well empirically in many Information Retrieval tasks. In this work we study formally the effects of smoothing on novelty detection. To this aim, we compare different techniques based on the Kullback-Leibler divergence and we analyze the sensitivity of retrieval performance to the smoothing parameters. The ability of Language Modeling estimation methods to handle quantitatively the uncertainty associated to the use of natural language is a powerful tool that can drive the future development of novelty-based mechanisms.

      Related collections

      Most cited references 3

      • Record: found
      • Abstract: not found
      • Article: not found

      A language modeling approach to information retrieval

       Jay Ponte,  W. Croft (1998)
        Bookmark
        • Record: found
        • Abstract: not found
        • Article: not found

        A study of smoothing methods for language models applied to Ad Hoc information retrieval

          Bookmark
          • Record: found
          • Abstract: not found
          • Article: not found

          Retrieval and novelty detection at the sentence level

            Bookmark

            Author and article information

            Affiliations
            Departamento de Electrónica y Computación.

            Universidad de Santiago de Compostela

            Campus Sur, s/n.

            15782 Santiago de Compostela, SPAIN
            Contributors
            Conference
            August 2007
            August 2007
            : 1-6
            10.14236/ewic/FDIA2007.2
            © Ronald T. Fernández. Published by BCS Learning and Development Ltd. BCS IRSG Symposium: Future Directions in Information Access 2007, Glasgow

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            BCS IRSG Symposium: Future Directions in Information Access 2007
            FDIA
            Glasgow
            28-29 August 2007
            Electronic Workshops in Computing (eWiC)
            Future Directions in Information Access
            Product
            Product Information: 1477-9358 BCS Learning & Development
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Comments

            Comment on this article