Learning to Deblur and Rotate Motion-Blurred Faces

Meishvili, Givi; Szabo, Atilla; Jenni, Simon; Favaro, Paolo (2021). Learning to Deblur and Rotate Motion-Blurred Faces. In: The British Machine Vision Conference (BMVC). Proceedings. BMVA

Preview	Text Conference_Learning_to_Deblur_and_Rotate_Motion-Blurred_Faces.pdf - Published Version Available under License Publisher holds Copyright. Author holds Copyright Download (18MB) | Preview
Preview	Text Conference_Supp_Learning_to_Deblur_and_Rotate_Motion-Blurred_Faces.pdf - Supplemental Material Available under License Publisher holds Copyright. Author holds Copyright Download (31MB) | Preview

We propose a solution to the novel task of rendering sharp videos from new viewpoints from a single motion-blurred image of a face. Our method1 handles the complexity of face blur by implicitly learning the geometry and motion of faces through the joint training on three large datasets: FFHQ and 300VW, which are publicly available, and a new Bern Multi-View Face Dataset (BMFD) that we built. The first two datasets provide a large variety of faces and allow our model to generalize better. BMFD instead allows us to introduce multi-view constraints, which are crucial to synthesizing sharp videos from a new camera view. It consists of high frame rate synchronized videos from multiple views of several subjects displaying a wide range of facial expressions. We use the high frame rate videos to simulate realistic motion blur through averaging. Thanks to this dataset, we train a neural network to reconstruct a 3D video representation from a single image and the corresponding face gaze. We then provide a camera viewpoint relative to the estimated gaze and the blurry image as input to an encoder-decoder network to generate a video of sharp frames with a novel camera viewpoint. We demonstrate our approach on test subjects of our multi-view dataset and VIDTIMIT.

Interest & Impact

Downloads

99 since deposited on 10 May 2022
20 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item