SmartSketcher: Sketch-based Image Retrieval with Dynamic Semantic Re-ranking

Portenier, Tiziano; Hu, Qiyang; Favaro, Paolo; Zwicker, Matthias (2017). SmartSketcher: Sketch-based Image Retrieval with Dynamic Semantic Re-ranking. In: EXPRESSIVE 2017 (pp. 1-12). ACM 10.1145/3092907.3092910

[img] Text
SmartSketcher- Sketch-based Image Retrieval with Dynamic Semantic Re-ranking.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (20MB) | Request a copy

We present a sketch-based image retrieval system, designed to answer arbitrary queries that may go beyond searching for prede ned object or scene categories. While sketching is fast and intuitive to formulate visual queries, pure sketch-based image retrieval often re- turns many outliers because it lacks a semantic understanding of the query. Our key idea is to combine sketch-based queries with interactive, semantic re-ranking of query results. We leverage progress in deep learning and use a feature representation learned for image classi cation for re-ranking. This allows us to cluster semantically similar images, re-rank based on the clusters, and present more meaningful query results to the user. We report on two large-scale benchmarks and demonstrate that our re-ranking approach leads to signi cant improvements over the state of the art. Finally, a user study designed to evaluate a practical use case con rms the bene ts of our approach.

Item Type:

Conference or Workshop Item (Paper)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF) > Computer Graphics Group (CGG)
08 Faculty of Science > Institute of Computer Science (INF) > Computer Vision Group (CVG)
08 Faculty of Science > Institute of Computer Science (INF)

UniBE Contributor:

Portenier, Tiziano; Hu, Qiyang; Favaro, Paolo and Zwicker, Matthias

Subjects:

000 Computer science, knowledge & systems
500 Science > 510 Mathematics

ISBN:

978-1-4503-5079-2

Publisher:

ACM

Language:

English

Submitter:

Xiaochen Wang

Date Deposited:

20 Apr 2018 12:47

Last Modified:

20 Apr 2018 12:47

Publisher DOI:

10.1145/3092907.3092910

BORIS DOI:

10.7892/boris.113237

URI:

https://boris.unibe.ch/id/eprint/113237

Actions (login required)

Edit item Edit item
Provide Feedback