Sensors and processing for 3D passive imaging

3D imaging has many practical applications in professional fields, such as aerial cartography, industrial measurement and robotics, as well as in consumer technologies (3D video, games).

When using passive sensors, the traditional depth measurement method uses a stereoscopic system, image matching and the triangulation method. Since the mid-2000s, new techniques have emerged, driven by the convergence of on-board computing and digital sensors. This evolution, associated with the terms co-design or computational imaging, consists in designing hybrid imaging systems, combining sensors and processing, globally optimized for the final information delivered to the user. The aim of this article is to present the two main techniques for depth estimation using passive sensors that have emerged from this approach - the exploitation of defocus blur (often referred to by the acronym DFD for Depth from Defocus) and the use of plenoptic sensors - and to compare them with stereovision.

It should be noted that all three techniques are based on indirect measurement, in which depth is estimated numerically from an observable quantity in the image plane. The relationship between the observable and the depth includes parameters that have to be identified during a prior calibration operation. Performance modeling is therefore quite complex, since it depends on sensor configuration, calibration, the type of processing used and its parameterization.

For each technique, we present its implementation, the quantity observable in the images and its relationship with depth, the main algorithmic paths for estimation, and finally the elements available on the performance model. Illustrations on real data are proposed throughout the text, including experimental evaluation results of depth estimation uncertainty, compared with performance model predictions.

The final section summarizes passive depth estimation and outlines elements for comparison between the three approaches studied.

At the end of the article, readers will find a glossary of important terms and expressions, as well as a table of acronyms and symbols.