Which Entry is More Similar? A Non-linear Visualisation of Query Results in Image Retrieval and Image Recognition Problem

Content based image retrieval (CBIR) has been a subject of exploration in digital humanities since 1990’s (Gudivada 1995). Various descriptors were implemented to represent shape, texture and colour content of the image as sequences of numerical values (Zhang & Lu 2004, Veltkamp, Latecki 2006, Zha & Yang 2010). At the same time similarity measures and learning algorithms were designed to enable efficient image classification and retrieval (LeCun 1998). The issue, however, remains in a simple question: which descriptor and which similarity measure best reflects the human perception of similarity of visual objects? And is this the same one, that best responds to the ground truth in a retrieval query? In this paper, we move for a while away from the very technical issues of shape descriptors definition and verification and we focus on the question how visualisation of the computed data affects the final result of a visual query. We are replacing a traditional, linear presentation of n most similar outputs to a set of graph-like and scatterplot based visualisation modes. The research study is performed on a particular example of visual search in large databases of historical watermarks, trademarks and monograms. We believe, that the approach to search across a digital print room repository involving intuitive user interaction is a step toward fully making use of its potential. We state, that a modern interface, that allows the end user an intuitive navigation through options and partial results is a milestone on the way to fill a technological gap between users familiar with image processing issues and with computer science background and those for whom obtaining an answer for a particular research question is worth more, than understanding how the result was actually computed. Finally, we proof the concept with a proposition of a shape descriptor followed by a set of flexible interfaces designed to display and navigate through the results of a visual query.

Content

Author and article information

Contributors

Joanna Gancarczyk

Anna Olszewska

Conference

Publication date: July 2017

Publication date (Print): July 2017

Pages: 74-80

Affiliations

[0001]University of Bielsko-Biała

Blelsko-Biała, Poland

[0002]AGH University of Science and Technology

Kraków, Poland

Article

DOI: 10.14236/ewic/EVA2017.14

SO-VID: 80e2e6cc-51ac-4be9-abda-0318487f5c14

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Electronic Visualisation and the Arts (EVA 2017)

Conference acronym: EVA

Conference number:

Conference location: London, UK

Conference date: 11 – 13 July 2017

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Electronic Visualisation and the Arts

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2017.14

Self URI (journal page): https://ewic.bcs.org/

5. REFERENCES

C. M. Briquet 1923 Les filigranes dictionnaire historique des marques du papier des leurs apparition vers 1282 jusqu’en 1600
P. BürgisserF. Cucker 2013 Condition: The geometry of numerical algorithms 349 Springer Science & Business Media
J. P. EakinsJ. E. BrownJ. RileyR. Mulholland 2001 Evaluating a shape retrieval system for watermark images A. Bendkowska Digital Art History - A Subject in Transition: Opportunities and Problems CHArt Conference Proceedings
M. C. Ferreira de OliveiraH. Levkowitz 2003 From visual data exploration to visual data mining: a survey IEEE Transactions on Visualization and Computer Graphics 9 3 378 394 July
E. H. Gombrich 1979 The Sense of Order: A study in the psychology of decorative art Cornell University Press
V. N. GudivadaV. V. Raghavan 1995 Content based image retrieval systems Computer 28 9 18 22
Y. LeCunL. BottouY. BengioP. Haffner 1998 Gradient-based learning applied to document recognition Proceedings of the IEEE.
F. Lugt Les marques de collections de dessins & d’estampes. The new edition on-line http://www.fondationcustodia.fr/english/merkenboek/default.cfm?gr=menu2 24 February 2017
G. K. Nagler 1860 Die Monogrammisten und diejenigen bekannten und und unberkannten Künstler aller Schulen. G. Hirth’s Verlag
C. RauberT. PunP. F. Tschudin 1997 Retrieval of images from a library of water-marks for ancient paper identification Proc. Second ACM International Conference on Digital Libraries ACM New York, USA 123 130
L. J. P. Van der MaatenG. E. Hinton 2008 Visualizing High-Dimensional Data Using t-SNE Journal of Machine Learning Research 9 2579 2260 November
R. C. VeltkampL. J. Latecki 2006 Properties and performance of shape similarity measures Data Science and Classification 47 56 Springer Berlin Heidelberg
Z. J. ZhaL. YangT. MeiM. WangZ. WangT. S. ChuaX. S. Hua 2010 Visual query suggestion: Towards capturing user intent in internet image search ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 6 3 13
D. ZhangG. Lu 2004 Review of shape representation and description techniques Pattern Recognition 37 1 1 19

Comments

Comment on this article

[1] C. M. Briquet 1923 Les filigranes dictionnaire historique des marques du papier des leurs apparition vers 1282 jusqu’en 1600

[2] P. BürgisserF. Cucker 2013 Condition: The geometry of numerical algorithms 349 Springer Science & Business Media

[3] J. P. EakinsJ. E. BrownJ. RileyR. Mulholland 2001 Evaluating a shape retrieval system for watermark images A. Bendkowska Digital Art History - A Subject in Transition: Opportunities and Problems CHArt Conference Proceedings

[4] M. C. Ferreira de OliveiraH. Levkowitz 2003 From visual data exploration to visual data mining: a survey IEEE Transactions on Visualization and Computer Graphics 9 3 378 394 July

[5] E. H. Gombrich 1979 The Sense of Order: A study in the psychology of decorative art Cornell University Press

[6] V. N. GudivadaV. V. Raghavan 1995 Content based image retrieval systems Computer 28 9 18 22

[7] Y. LeCunL. BottouY. BengioP. Haffner 1998 Gradient-based learning applied to document recognition Proceedings of the IEEE.

[8] F. Lugt Les marques de collections de dessins & d’estampes. The new edition on-line http://www.fondationcustodia.fr/english/merkenboek/default.cfm?gr=menu2 24 February 2017

[9] G. K. Nagler 1860 Die Monogrammisten und diejenigen bekannten und und unberkannten Künstler aller Schulen. G. Hirth’s Verlag

[10] C. RauberT. PunP. F. Tschudin 1997 Retrieval of images from a library of water-marks for ancient paper identification Proc. Second ACM International Conference on Digital Libraries ACM New York, USA 123 130

[11] L. J. P. Van der MaatenG. E. Hinton 2008 Visualizing High-Dimensional Data Using t-SNE Journal of Machine Learning Research 9 2579 2260 November

[12] R. C. VeltkampL. J. Latecki 2006 Properties and performance of shape similarity measures Data Science and Classification 47 56 Springer Berlin Heidelberg

[13] Z. J. ZhaL. YangT. MeiM. WangZ. WangT. S. ChuaX. S. Hua 2010 Visual query suggestion: Towards capturing user intent in internet image search ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 6 3 13

[14] D. ZhangG. Lu 2004 Review of shape representation and description techniques Pattern Recognition 37 1 1 19

Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org