Learning Controllable Representations for Image Synthesis

Hu, Qiyang (2019). Learning Controllable Representations for Image Synthesis. (Dissertation, Universität Bern, Philosophisch-naturwissenschaftliche Fakultät)

[img] Text
PhDThesis_QiyangHu.pdf - Other
Restricted to registered users only
Available under License BORIS Standard License.

Download (39MB)

In this thesis, our focus is learning a controllable representation and applying the learned controllable feature representation on images synthesis, video generation, and even 3D reconstruction. We propose different methods to disentangle the feature representation in neural network and analyze the challenges in disentanglement such as reference ambiguity and shortcut problem when using the weak label. We use the disentangled feature representation to transfer attributes between images such as exchanging hairstyle between two face images. Furthermore, we study the problem of how another type of feature, sketch, works in a neural network. The sketch can provide shape and contour of an object such as the silhouette of the side-view face. We leverage the silhouette constraint to improve the 3D face reconstruction from 2D images. The sketch can also provide the moving directions of one object, thus we investigate how one can manipulate the object to follow the trajectory provided by a user sketch. We propose a method to automatically generate video clips from a single image input using the sketch as motion and trajectory guidance to animate the object in that image. We demonstrate the efficiency of our approaches on several synthetic and real datasets.

Item Type:

Thesis (Dissertation)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF) > Computer Vision Group (CVG)
08 Faculty of Science > Institute of Computer Science (INF)

UniBE Contributor:

Hu, Qiyang

Subjects:

000 Computer science, knowledge & systems
500 Science > 510 Mathematics

Language:

English

Submitter:

Llukman Cerkezi

Date Deposited:

31 Aug 2022 14:40

Last Modified:

05 Dec 2022 16:23

BORIS DOI:

10.48350/172511

URI:

https://boris.unibe.ch/id/eprint/172511

Actions (login required)

Edit item Edit item
Provide Feedback