Integrating standard test collections in interactive IR instruction

Information retrieval experiments usually measure the average effectiveness of the IR methods developed. Direct use of an operational retrieval system has the shortcoming that the user is not given feedback about the effectiveness of queries. The Query Performance Analyser (QPA) for information retrieval systems is an interactive tool for the performance analysis of individual queries. On top of a standard test collection, it gives an instant visualisation of the performance achieved in a given search topic by any user-generated query. In addition to experimental IR research, the QPA can be used in user training to demonstrate the characteristics of and compare differences between IR systems and searching strategies. The experiences of using the tool in IR instruction are reported. The need to link instruction and research is underlined.