Learning to Extract Flawless Slow Motion from Blurry Videos

Jin, Meiguang; Hu, Zhe; Favaro, Paolo (June 2019). Learning to Extract Flawless Slow Motion from Blurry Videos. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE Computer Society

[img]
Preview
Text
Jin_Learning_to_Extract_Flawless_Slow_Motion_From_Blurry_Videos_CVPR_2019_paper.pdf - Accepted Version
Available under License Publisher holds Copyright.
This CVPR paper is the Open Access version, provided by the Computer Vision Foundation. Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore.

Download (1MB) | Preview

In this paper, we introduce the task of generating a sharp slow-motion video given a low frame rate blurry video. We propose a data-driven approach, where the training data is captured with a high frame rate camera and blurry images are simulated through an averaging process. While it is possible to train a neural network to recover the sharp frames from their average, there is no guarantee of the temporal smoothness for the formed video, as the frames are estimated independently. To address the temporal smoothness requirement we propose a system with two networks: One, DeblurNet, to predict sharp keyframes and the second, InterpNet, to predict intermediate frames between the generated keyframes. A smooth transition is ensured by interpolating between consecutive keyframes using InterpNet. Moreover, the proposed scheme enables further increase in frame rate without retraining the network, by applying InterpNet recursively between pairs of sharp frames. We evaluate the proposed method on several datasets, including a novel dataset captured with a Sony RX V camera. We also demonstrate its performance of increasing the frame rate up to 20 times on real blurry videos.

Item Type:

Conference or Workshop Item (Paper)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF)

UniBE Contributor:

Jin, Meiguang and Favaro, Paolo

Subjects:

000 Computer science, knowledge & systems
500 Science > 510 Mathematics

Publisher:

IEEE Computer Society

Language:

English

Submitter:

Xiaochen Wang

Date Deposited:

18 Feb 2020 09:24

Last Modified:

18 Feb 2020 09:24

BORIS DOI:

10.7892/boris.140394

URI:

https://boris.unibe.ch/id/eprint/140394

Actions (login required)

Edit item Edit item
Provide Feedback