+1 Recommend
1 collections
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      The Application of Work Tasks in connection with the Evaluation of Interactive Information Retrieval Systems: Empirical Results


      MIRA '99 (MIRA)

      MIRA '99

      14th - 16th April 1999

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          This paper reports on the empirical findings of the meta-evaluation of the application of work tasks in connection with the evaluation of interactive information retrieval (IIR) systems.

          The purpose of the meta-evaluation is to uncover if it is recommendable to apply work tasks to future evaluation of IIR systems. It is investigated if any search behavioural differences exist between test persons’ treatment of their own real information needs versus simulated information needs. The hypothesis is that if no difference exists one can correctly substitute real information needs with simulated information needs through the application of simulated work task situations. We are also interested in learning what defines a ’good’ work task situation.

          The empirical results of the meta-evaluation provide positive evidence of the application of simulated work task situations in connection with evaluation of IIR systems. The results also point to that tuning of work task situations towards the group of test persons is of importance due to motivation of the test persons. Furthermore, the results of the evaluation show that different versions of semantic openness of the simulated situations make no difference to the test persons’ search treatment. Finally, it is verified that there exists a general pattern of assessment behaviour for the test persons. This verification makes yet another experimental reason for permutation of work tasks between test persons, in order to avoid bias of the retrieval results.

          Related collections

          Author and article information

          April 1999
          April 1999
          : 1-15
          Department of Information Studies, Royal School of Library and Information Science

          Birketinget 6, DK-2300 Copenhagen S, Denmark
          © Pia Borlund et al. Published by BCS Learning and Development Ltd. MIRA '99

          This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit

          MIRA '99
          Glasgow, Scotland
          14th - 16th April 1999
          Electronic Workshops in Computing (eWiC)
          MIRA '99
          Product Information: 1477-9358BCS Learning & Development
          Self URI (journal page):
          Electronic Workshops in Computing


          Comment on this article