Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Demonstration of an open source framework for qualitative evaluation of CBIR systems

Duran, Paula Gomez, Mohedano, Eva, McGuinness, Kevin orcid logoORCID: 0000-0003-1336-6477, Giró-i-Nieto, Xavier orcid logoORCID: 0000-0002-9935-5332 and O'Connor, Noel E. orcid logoORCID: 0000-0002-4033-9135 (2018) Demonstration of an open source framework for qualitative evaluation of CBIR systems. In: ACM Multimedia, 22-26 Oct 2018, Seoul, Republic of Korea. ISBN 978-1-4503-5665-7

Abstract
Evaluating image retrieval systems in a quantitative way, for example by computing measures like mean average precision, allows for objective comparisons with a ground-truth. However, in cases where ground-truth is not available, the only alternative is to collect feedback from a user. Thus, qualitative assessments become important to better understand how the system works. Visualizing the results could be, in some scenarios, the only way to evaluate the results obtained and also the only opportunity to identify that a system is failing. This necessitates developing a User Interface (UI) for a Content Based Image Retrieval (CBIR) system that allows visualization of results and improvement via capturing user relevance feedback. A well-designed UI facilitates understanding of the performance of the system, both in cases where it works well and perhaps more importantly those which highlight the need for improvement. Our open-source system implements three components to facilitate researchers to quickly develop these capabilities for their retrieval engine. We present: a web-based user interface to visualize retrieval results and collect user annotations; a server that simplifies connection with any underlying CBIR system; and a server that manages the search engine data.
Metadata
Item Type:Conference or Workshop Item (Other)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:LATEX; Visualization tool; image retrieval; visual searching; relevance feedback; annotation tool
Subjects:Computer Science > Image processing
Computer Science > Information retrieval
Computer Science > Machine learning
Computer Science > Information storage and retrieval systems
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering
Research Initiatives and Centres > INSIGHT Centre for Data Analytics
Published in: Proceedings of the 26th ACM international conference on Multimedia. . ACM. ISBN 978-1-4503-5665-7
Publisher:ACM
Official URL:https://doi.org/10.1145/3240508.3241395
Copyright Information:© 2018 ACM
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:Science Foundation Ireland SFI/12/RC/2289 and SFI/15/SIRG/3283, Insight, Spanish Ministry of Economy and Competitivity and the European Regional Development Fund (ERDF) under contract TEC2016-75976-R.
ID Code:22816
Deposited On:23 Nov 2018 11:07 by Kevin Mcguinness . Last Modified 25 Jan 2019 10:03
Documents

Full text available as:

[thumbnail of DemonstrationofanOpen.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
11MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record