Self-Supervised Feature Learning by Learning to Spot Artifacts

Jenni, Simon; Favaro, Paolo (June 2018). Self-Supervised Feature Learning by Learning to Spot Artifacts. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, Utah, United States. June 18 - June 22, 2018.

Text Jenni_Self-Supervised_Feature_Learning_CVPR_2018_paper.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (2MB)

We introduce a novel self-supervised learning method based on adversarial training. Our objective is to train a discriminator network to distinguish real images from images with synthetic artifacts, and then to extract features from its intermediate layers that can be transferred to other data domains and tasks. To generate images with artifacts, we pre-train a high-capacity autoencoder and then we use a damage and repair strategy: First, we freeze the autoencoder and damage the output of the encoder by randomly dropping its entries. Second, we augment the decoder with a repair network, and train it in an adversarial manner against the discriminator. The repair network helps generate more realistic images by inpainting the dropped feature entries. To make the discriminator focus on the artifacts, we also make it predict what entries in the feature were dropped. We demonstrate experimentally that features learned by creating and spotting artifacts achieve state of the art performance in several benchmarks.

Interest & Impact

Downloads

3 since deposited on 28 May 2019
0 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item