Learning to Reconstruct Confocal Microscopy Stacks From Single Light Field Images

Vizcaino, Josue Page; Saltarin, Federico; Belyaev, Yury; Lyck, Ruth; Lasser, Tobias; Favaro, Paolo (2021). Learning to Reconstruct Confocal Microscopy Stacks From Single Light Field Images. IEEE transactions on computational imaging, 7, pp. 775-788. IEEE 10.1109/TCI.2021.3097611

Text Learning_to_Reconstruct_Confocal_Microscopy_Stacks_From_Single_Light_Field_Images.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (3MB)

We present a novel deep learning approach to reconstruct confocal microscopy stacks from single light field images. To perform the reconstruction, we introduce LFMNet, a fully convolutional neural network architecture inspired by the U-Net design. A property of LFMNet is its ability to reconstruct with near confocal accuracy a 112 ×112 ×57.6 μm 3 volume (1287×1287×64 voxels) in 50 ms given a single light field image of 1287×1287 pixels, thus dramatically reducing 720-fold the capture time compared to confocal scanning of assays at the same volumetric resolution and 64-fold the required storage. To prove its applicability in life sciences, our approach is evaluated both quantitatively and qualitatively on mouse brain slices with fluorescently labeled blood vessels. Because of the drastic reduction in image acquisition time, our setup and method are suitable for imaging highly dynamic and light-sensitive events, and in closed-loop systems, where latency due to the reconstruction time is crucial. We provide empirical analysis of the optical design, of the network architecture and of our training procedure to reconstruct confocal-like volumes for a given target depth range. To train our network, we built a data set of 362 light field images of the mouse brain sample and the corresponding aligned set of 3D confocal scans, which we use as ground truth. The data set will be made available for research purposes.

Interest & Impact

Downloads

2 since deposited on 26 Jan 2022
0 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item