Blog
About

88
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      The Effect Of Smoothing In Language Models For Novelty Detection

      BCS IRSG Symposium: Future Directions in Information Access 2007 (FDIA)

      Future Directions in Information Access

      28-29 August 2007

      Language Models, Smoothing, Novelty Detection, Kullback-Leibler Divergence

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          The novelty task consists of finding relevant and novel sentences in a ranking of documents given a query. In the literature, different techniques have been applied to address this problem. Nevertheless, little is known about Language Models for novelty detection and, especially, the effect of smoothing on the selection of novel sentences. Language Models can be used to study novelty and relevance in a principled way. These statistical models have been shown to perform well empirically in many Information Retrieval tasks. In this work we study formally the effects of smoothing on novelty detection. To this aim, we compare different techniques based on the Kullback-Leibler divergence and we analyze the sensitivity of retrieval performance to the smoothing parameters. The ability of Language Modeling estimation methods to handle quantitatively the uncertainty associated to the use of natural language is a powerful tool that can drive the future development of novelty-based mechanisms.

          Related collections

          Most cited references 3

          • Record: found
          • Abstract: not found
          • Article: not found

          A language modeling approach to information retrieval

           Jay Ponte,  W. Croft (1998)
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            A study of smoothing methods for language models applied to Ad Hoc information retrieval

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Retrieval and novelty detection at the sentence level

                Bookmark

                Author and article information

                Contributors
                Conference
                August 2007
                August 2007
                : 1-6
                Affiliations
                Departamento de Electrónica y Computación.

                Universidad de Santiago de Compostela

                Campus Sur, s/n.

                15782 Santiago de Compostela, SPAIN
                Article
                10.14236/ewic/FDIA2007.2
                © Ronald T. Fernández. Published by BCS Learning and Development Ltd. BCS IRSG Symposium: Future Directions in Information Access 2007, Glasgow

                This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

                BCS IRSG Symposium: Future Directions in Information Access 2007
                FDIA
                Glasgow
                28-29 August 2007
                Electronic Workshops in Computing (eWiC)
                Future Directions in Information Access
                Product
                Product Information: 1477-9358 BCS Learning & Development
                Self URI (journal page): https://ewic.bcs.org/
                Categories
                Electronic Workshops in Computing

                Comments

                Comment on this article