Blog
About

84
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      The Application of Work Tasks in connection with the Evaluation of Interactive Information Retrieval Systems: Empirical Results

      ,

      MIRA '99 (MIRA)

      MIRA '99

      14th - 16th April 1999

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          This paper reports on the empirical findings of the meta-evaluation of the application of work tasks in connection with the evaluation of interactive information retrieval (IIR) systems.

          The purpose of the meta-evaluation is to uncover if it is recommendable to apply work tasks to future evaluation of IIR systems. It is investigated if any search behavioural differences exist between test persons’ treatment of their own real information needs versus simulated information needs. The hypothesis is that if no difference exists one can correctly substitute real information needs with simulated information needs through the application of simulated work task situations. We are also interested in learning what defines a ’good’ work task situation.

          The empirical results of the meta-evaluation provide positive evidence of the application of simulated work task situations in connection with evaluation of IIR systems. The results also point to that tuning of work task situations towards the group of test persons is of importance due to motivation of the test persons. Furthermore, the results of the evaluation show that different versions of semantic openness of the simulated situations make no difference to the test persons’ search treatment. Finally, it is verified that there exists a general pattern of assessment behaviour for the test persons. This verification makes yet another experimental reason for permutation of work tasks between test persons, in order to avoid bias of the retrieval results.

          Related collections

          Author and article information

          Conference
          April 1999
          April 1999
          : 1-15
          Affiliations
          Department of Information Studies, Royal School of Library and Information Science

          Birketinget 6, DK-2300 Copenhagen S, Denmark
          10.14236/ewic/MIRA1999.1
          © Pia Borlund et al. Published by BCS Learning and Development Ltd. MIRA '99

          This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

          MIRA '99
          MIRA
          Glasgow, Scotland
          14th - 16th April 1999
          Electronic Workshops in Computing (eWiC)
          MIRA '99
          Product
          Product Information: 1477-9358 BCS Learning & Development
          Self URI (journal page): https://ewic.bcs.org/
          Categories
          Electronic Workshops in Computing

          Comments

          Comment on this article