Blog
About

  • Record: found
  • Abstract: found
  • Article: found
Is Open Access

The Application of Work Tasks in connection with the Evaluation of Interactive Information Retrieval Systems: Empirical Results

,

MIRA '99 (MIRA)

MIRA '99

14th - 16th April 1999

Read this article at

Bookmark
      There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

      Abstract

      This paper reports on the empirical findings of the meta-evaluation of the application of work tasks in connection with the evaluation of interactive information retrieval (IIR) systems.

      The purpose of the meta-evaluation is to uncover if it is recommendable to apply work tasks to future evaluation of IIR systems. It is investigated if any search behavioural differences exist between test persons’ treatment of their own real information needs versus simulated information needs. The hypothesis is that if no difference exists one can correctly substitute real information needs with simulated information needs through the application of simulated work task situations. We are also interested in learning what defines a ’good’ work task situation.

      The empirical results of the meta-evaluation provide positive evidence of the application of simulated work task situations in connection with evaluation of IIR systems. The results also point to that tuning of work task situations towards the group of test persons is of importance due to motivation of the test persons. Furthermore, the results of the evaluation show that different versions of semantic openness of the simulated situations make no difference to the test persons’ search treatment. Finally, it is verified that there exists a general pattern of assessment behaviour for the test persons. This verification makes yet another experimental reason for permutation of work tasks between test persons, in order to avoid bias of the retrieval results.

      Related collections

      Author and article information

      Affiliations
      Department of Information Studies, Royal School of Library and Information Science

      Birketinget 6, DK-2300 Copenhagen S, Denmark
      Conference
      April 1999
      April 1999
      : 1-15
      10.14236/ewic/MIRA1999.1
      © Pia Borlund et al. Published by BCS Learning and Development Ltd. MIRA '99

      This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

      MIRA '99
      MIRA
      Glasgow, Scotland
      14th - 16th April 1999
      Electronic Workshops in Computing (eWiC)
      MIRA '99
      Product
      Product Information: 1477-9358 BCS Learning & Development
      Self URI (journal page): https://ewic.bcs.org/
      Categories
      Electronic Workshops in Computing

      Comments

      Comment on this article