FaceShop: Deep Sketch-based Image Editing

Portenier, Tiziano; Hu, Qiyang; Szabo, Attila; Arjomand, Siavash; Favaro, Paolo; Zwicker, Matthias (2018). FaceShop: Deep Sketch-based Image Editing. ACM transactions on graphics, 37(4), pp. 1-13. Association for Computing Machinery 10.1145/3197517.3201393

[img]
Preview
Text
portenier.pdf - Published Version
Available under License Creative Commons: Attribution-Noncommercial (CC-BY-NC).

Download (107MB) | Preview

We present a novel system for sketch-based face image editing, enabling users to edit images intuitively by sketching a few strokes on a region of interest. Our interface features tools to express a desired image manipulation by providing both geometry and color constraints as user-drawn strokes. As an alternative to the direct user input, our proposed system naturally supports a copy-paste mode, which allows users to edit a given image region by using parts of another exemplar image without the need of hand-drawn sketching at all. The proposed interface runs in real-time and facilitates an interactive and iterative workflow to quickly express the intended edits. Our system is based on a novel sketch domain and a convolutional neural network trained end-to-end to automatically learn to render image regions corresponding to the input strokes. To achieve high quality and semantically consistent results we train our neural network on two simultaneous tasks, namely image completion and image translation. To the best of our knowledge, we are the first to combine these two tasks in a unified framework for interactive image editing. Our results show that the proposed sketch domain, network architecture, and training procedure generalize well to real user input and enable high quality synthesis results without additional post-processing.

Item Type:

Journal Article (Original Article)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF)

UniBE Contributor:

Portenier, Tiziano; Hu, Qiyang; Szabo, Attila; Arjomand, Siavash; Favaro, Paolo and Zwicker, Matthias

Subjects:

000 Computer science, knowledge & systems
500 Science > 510 Mathematics

ISSN:

0730-0301

Publisher:

Association for Computing Machinery

Language:

English

Submitter:

Xiaochen Wang

Date Deposited:

28 May 2019 16:41

Last Modified:

02 Jun 2019 02:39

Publisher DOI:

10.1145/3197517.3201393

BORIS DOI:

10.7892/boris.126533

URI:

https://boris.unibe.ch/id/eprint/126533

Actions (login required)

Edit item Edit item
Provide Feedback