1,531
views
0
recommends
+1 Recommend
1 collections
    0
    shares

      Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65

      scite_
       
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      The Effect Of Smoothing In Language Models For Novelty Detection

      proceedings-article
      BCS IRSG Symposium: Future Directions in Information Access 2007 (FDIA)
      Future Directions in Information Access
      28-29 August 2007
      Language Models, Smoothing, Novelty Detection, Kullback-Leibler Divergence
      Bookmark

            Abstract

            The novelty task consists of finding relevant and novel sentences in a ranking of documents given a query. In the literature, different techniques have been applied to address this problem. Nevertheless, little is known about Language Models for novelty detection and, especially, the effect of smoothing on the selection of novel sentences. Language Models can be used to study novelty and relevance in a principled way. These statistical models have been shown to perform well empirically in many Information Retrieval tasks. In this work we study formally the effects of smoothing on novelty detection. To this aim, we compare different techniques based on the Kullback-Leibler divergence and we analyze the sensitivity of retrieval performance to the smoothing parameters. The ability of Language Modeling estimation methods to handle quantitatively the uncertainty associated to the use of natural language is a powerful tool that can drive the future development of novelty-based mechanisms.

            Content

            Author and article information

            Contributors
            Conference
            August 2007
            August 2007
            : 1-6
            Affiliations
            [0001]Departamento de Electrónica y Computación.

            Universidad de Santiago de Compostela

            Campus Sur, s/n.

            15782 Santiago de Compostela, SPAIN
            Article
            10.14236/ewic/FDIA2007.2
            dde3bc80-e8a8-41e3-aa68-889d9c755381
            © Ronald T. Fernández. Published by BCS Learning and Development Ltd. BCS IRSG Symposium: Future Directions in Information Access 2007, Glasgow

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            BCS IRSG Symposium: Future Directions in Information Access 2007
            FDIA
            Glasgow
            28-29 August 2007
            Electronic Workshops in Computing (eWiC)
            Future Directions in Information Access
            History
            Product

            1477-9358 BCS Learning & Development

            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/FDIA2007.2
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction
            Language Models,Smoothing,Novelty Detection,Kullback-Leibler Divergence

            Comments

            Comment on this article