Self-Supervised Feature Learning by Learning to Spot Artifacts

Jenni, Simon; Favaro, Paolo (June 2018). Self-Supervised Feature Learning by Learning to Spot Artifacts. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, Utah, United States. June 18 - June 22, 2018.

[img] Text
Jenni_Self-Supervised_Feature_Learning_CVPR_2018_paper.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (2MB) | Request a copy

We introduce a novel self-supervised learning method based on adversarial training. Our objective is to train a discriminator network to distinguish real images from images with synthetic artifacts, and then to extract features from its intermediate layers that can be transferred to other data domains and tasks. To generate images with artifacts, we pre-train a high-capacity autoencoder and then we use a damage and repair strategy: First, we freeze the autoencoder and damage the output of the encoder by randomly dropping its entries. Second, we augment the decoder with a repair network, and train it in an adversarial manner against the discriminator. The repair network helps generate more realistic images by inpainting the dropped feature entries. To make the discriminator focus on the artifacts, we also make it predict what entries in the feature were dropped. We demonstrate experimentally that features learned by creating and spotting artifacts achieve state of the art performance in several benchmarks.

Item Type:

Conference or Workshop Item (Paper)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF)

UniBE Contributor:

Jenni, Simon and Favaro, Paolo

Subjects:

000 Computer science, knowledge & systems
500 Science > 510 Mathematics

Language:

English

Submitter:

Xiaochen Wang

Date Deposited:

28 May 2019 15:34

Last Modified:

26 Oct 2019 14:26

BORIS DOI:

10.7892/boris.126516

URI:

https://boris.unibe.ch/id/eprint/126516

Actions (login required)

Edit item Edit item
Provide Feedback