Toward a theoretical framework for information retrieval (IR) evaluation in an information seeking context

This paper presents work in progress toward the development of the information retrieval (IR) evaluation measure – Information Problem Shift - within a theoretical framework and model for conceptualizing and researching information retrieval (IR) within an information seeking context. The theoretical model consists of a set of situated actions by information-seekers within interactive search sessions with information retrieval (IR) systems over a period of time. First, the paper outlines the IR evaluation measure – Information Problem Shift – within the theoretical framework of the model. Second, the paper discusses work in progress in the form of two studies currently collecting data, including the proposed IR evaluation measure - one study of information-seekers using an IR system and another study of information-seekers using the Web. Overall, this research seeks to develop an integrated view of the interactive IR processes by information-seekers and an IR evaluation measure with utility for IR system researchers, designers and information-seekers themselves.