1,046
views
0
recommends
+1 Recommend
1 collections
    0
    shares

      Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65

      scite_
       
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      The Application of Work Tasks in connection with the Evaluation of Interactive Information Retrieval Systems: Empirical Results

      proceedings-article
      ,
      MIRA '99 (MIRA)
      MIRA '99
      14th - 16th April 1999
      Bookmark

            Abstract

            This paper reports on the empirical findings of the meta-evaluation of the application of work tasks in connection with the evaluation of interactive information retrieval (IIR) systems. The purpose of the meta-evaluation is to uncover if it is recommendable to apply work tasks to future evaluation of IIR systems. It is investigated if any search behavioural differences exist between test persons’ treatment of their own real information needs versus simulated information needs. The hypothesis is that if no difference exists one can correctly substitute real information needs with simulated information needs through the application of simulated work task situations. We are also interested in learning what defines a ’good’ work task situation. The empirical results of the meta-evaluation provide positive evidence of the application of simulated work task situations in connection with evaluation of IIR systems. The results also point to that tuning of work task situations towards the group of test persons is of importance due to motivation of the test persons. Furthermore, the results of the evaluation show that different versions of semantic openness of the simulated situations make no difference to the test persons’ search treatment. Finally, it is verified that there exists a general pattern of assessment behaviour for the test persons. This verification makes yet another experimental reason for permutation of work tasks between test persons, in order to avoid bias of the retrieval results.

            Content

            Author and article information

            Conference
            April 1999
            April 1999
            : 1-15
            Affiliations
            [0001]Department of Information Studies, Royal School of Library and Information Science

            Birketinget 6, DK-2300 Copenhagen S, Denmark
            Article
            10.14236/ewic/MIRA1999.1
            c4f5f495-a71e-4db4-8d13-127c33011132
            © Pia Borlund et al. Published by BCS Learning and Development Ltd. MIRA '99

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            MIRA '99
            MIRA
            Glasgow, Scotland
            14th - 16th April 1999
            Electronic Workshops in Computing (eWiC)
            MIRA '99
            History
            Product

            1477-9358 BCS Learning & Development

            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/MIRA1999.1
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction

            Comments

            Comment on this article