Improving Information Retrieval Evaluation via Markovian User Models and Visual Analytics

Abstract

To address the challenge of adapting experimental evaluation to the constantly evolving user tasks and needs, we develop a new family of Markovian Information Retrieval (IR) evaluation measures, called Markov Precision (MP), where the interaction between the user and the ranked result list is modelled via Markov chains, and which will be able to explicitly link lab-style and on-line evaluation methods. Moreover, since experimental results are often not so easy to understand, we will develop a Web-based Visual Analytics (VA) prototype where an animated state diagram of the Markov chain will explain how the user is interacting with the ranked result list in order to offer a support for a careful failure analysis.

Affiliations

Department of Information Engineering

University of Padua, Italy

Contributors

Maria Maistro

Conference

Publication date: September 2015

Publication date (Print): September 2015

Pages: 11-14

ID: 10.14236/ewic/FDIA2015.4

Copyright statement: © Maistro. Published by BCS Learning and Development Ltd. Proceedings of the 6 ^th Symposium on Future Directions in Information Access 2015

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Sixth BCS-IRSG Symposium on Future Directions in Information Access (FDIA 2015)

Conference acronym: FDIA 2015

Conference number: 6

Conference location: Thessaloniki, Greece

Conference date: 31 August - 4 September 2015

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Future Directions in Information Access

Product

Product Information: 1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/FDIA2015.4

Self URI (journal page): https://ewic.bcs.org/

Categories

Series title: Electronic Workshops in Computing