Emergence of Object Segmentation in Perturbed Generative Models

Bielski, Adam; Favaro, Paolo (December 2019). Emergence of Object Segmentation in Perturbed Generative Models. Advances in Neural Information Processing Systems (NIPS), 32. MIT Press

[img]
Preview
Text
8946-emergence-of-object-segmentation-in-perturbed-generative-models.pdf - Published Version
Available under License Publisher holds Copyright.

Download (13MB) | Preview

We introduce a framework to learn object segmentation from a collection of images without any manual annotation. We build on the observation that the location of object segments can be perturbed locally relative to a given background without affecting the realism of a scene. First, we train a generative model of a layered scene. The layered representation consists of a background image, a foreground image and the mask of the foreground. A composite image is then obtained by overlaying the masked foreground image onto the background. The generative model is trained in an adversarial fashion against a discriminator, which forces the generative model to produce realistic composite images. To force the generator to learn a representation where the foreground layer corresponds to an object, we perturb the output of the generative model by introducing a random shift of both the foreground image and mask relative to the background. Because the generator is unaware of the shift before computing its output, it must produce layered representations that are realistic for any such random perturbation. Second, we learn to segment an image by defining an autoencoder consisting of an encoder, which we train, and the pretrained generator as the decoder, which we fix. The encoder maps an image to the input of the generator, which then outputs a composite image matching the original input image. Because the generator outputs an explicit layered representation of the scene, the encoder learns to detect and segment objects. We demonstrate this framework on real images of several object categories.

Item Type:

Conference or Workshop Item (Paper)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF)

UniBE Contributor:

Bielski, Adam Jakub and Favaro, Paolo

Subjects:

000 Computer science, knowledge & systems
500 Science > 510 Mathematics

ISSN:

1049-5258

Publisher:

MIT Press

Language:

English

Submitter:

Xiaochen Wang

Date Deposited:

18 Feb 2020 09:01

Last Modified:

01 Jul 2020 02:30

BORIS DOI:

10.7892/boris.140390

URI:

https://boris.unibe.ch/id/eprint/140390

Actions (login required)

Edit item Edit item
Provide Feedback