1,642
views
0
recommends
+1 Recommend
1 collections
    4
    shares

      Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org

      scite_
       
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Reassessing and extending the Precision and Recall concepts

      Published
      proceedings-article
      MIRA '99 (MIRA)
      MIRA '99
      14th - 16th April 1999
      Bookmark

            Abstract

            The contrivances of ‘Recall’ and ‘Precision’ are customarily used to assess the effectiveness of document retrieval systems. Despite their extensive use in experiments, including the recent TREC experiments, and their dominance in mathematical discussions of system performance, there has been continual questioning of their validity since their introduction with the Cranfield experiments. This has largely centred on critical analysis of the (pre-mathematical) concept of ‘relevance’. Those analyses have now led to the near-consensual view amongst relevance theoreticians that two types of relevance require to be distinguished, namely (1) document ‘topicality’, where cognition acts as a largely passive receiving agent of knowledge that has an objective, a priori or public character, and (2) ‘psychological relevance’, where cognition is more actively and creatively involved in the knowledge encoded in the document, i.e. where relevance is non-public (subjective) and conditioned by the user’s context and experience (etc) at a particular time. (Various synonyms for these terms have been suggested.) The continued use of P and R in their initial, largely Cranfield, form in document retrieval experiments, especially the continued, uncritical, use of ‘Recall’, suggest that experimentalists have largely failed to internalise this distinction, since (1) Recall is meaningless under one of these viewpoints, and (2) Precision is ambiguous when the separate validities of each are recognised. Several ways in which this distinction should influence the design of future experiments on document-retrieval/cognition interactions are suggested, involving the choice of, and amendments to the definitions of, these basic measures. Lastly, a generic 3-valued vectorial approach is suggested as a means of integrating both perspectives on ‘relevance’ within a common evaluative framework.

            Content

            Author and article information

            Contributors
            Conference
            April 1999
            April 1999
            : 1-15
            Affiliations
            [0001]Department of Information and Library Management, University of Northumbria

            Newcastle upon Tyne, England
            Article
            10.14236/ewic/MIRA1999.4
            5e23a79d-3b8b-46c7-a454-34d879b095ed
            © M. H. Heine et al. Published by BCS Learning and Development Ltd. MIRA '99

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            MIRA '99
            MIRA
            Glasgow, Scotland
            14th - 16th April 1999
            Electronic Workshops in Computing (eWiC)
            MIRA '99
            History
            Product

            1477-9358 BCS Learning & Development

            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/MIRA1999.4
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction

            Comments

            Comment on this article