Partially Supervised Multi-Task Network for Single-View Dietary Assessment

Lu, Ya; Stathopoulou, Thomai; Mougiakakou, Stavroula (2021). Partially Supervised Multi-Task Network for Single-View Dietary Assessment (In Press). In: ICPR2020. Milano (Virtual). Jan 10-15, 2021.

Text 2008.00818.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (8MB)

Food volume estimation is an essential step in the pipeline of dietary assessment and demands the precise depth estimation of the food surface and table plane. Existing methods based on computer vision require either multi-image input or additional depth maps, reducing convenience of implementation and practical significance. Despite the recent advances in unsupervised depth estimation from a single image, the achieved performance in the case of large texture-less areas needs to be improved. In this paper, we propose a network architecture that jointly performs geometric understanding (i.e., depth prediction and 3D plane estimation) and semantic prediction on a single food image, enabling a robust and accurate food volume estimation regardless of the texture characteristics of the target plane. For the training of the network, only monocular videos with semantic ground truth are required, while the depth map and 3D plane ground truth are no longer needed. Experimental results on two separate food image databases demonstrate that our method performs robustly on texture-less scenarios and is superior to unsupervised networks and structure from motion based approaches, while it achieves comparable performance to fully-supervised methods.

Interest & Impact

Downloads

3 since deposited on 02 Jun 2021
0 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item