COL Cover Story: Experimental assessment of a 3D plenoptic endoscopic imaging system
At: 2017/3/31 16:09:00 by admin

A realistic three-dimensional (3D) reconstruction of a surgical site has become a highly needed imaging method in minimally invasive surgery. To date, common clinical and research efforts have employed 3D reconstruction methods based on stereoscopy, time-of-flight, or structured illumination to extract the concavity and convexity of the target specimen. Nevertheless, all of these techniques suffer from mediocre performance for obtaining high depth resolution unless some auxiliary measures are employed, which increase the system complications inevitably. 

Recently, an extension of stereoscopy technique called plenoptic imaging has been investigated to deduce 3D shape, which has been widely applied in industry for its multi-focus function and the simplified correspondence searching performance. However, its application in medicine has not been fully explored, as the current depth precision is limited to 1 mm, and no endoscopic plenoptic camera exists.  

To adopt the plenoptic technique in 3D endoscopic vision, the photonics and mechanical engineering groups led by Dr. Jin Ung Kang (Photonics and Optoelectronic Lab, Johns Hopkins University) and Dr. Axel Krieger (Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Health System) have proposed a plenoptic endoscope with a customized relay optical system to be used in minimally invasive setting. This work was published in Chinese Optics Letters, Volume 15, No. 5, 2017 (Hanh N. D. Le, et al., Experimental assessment of a 3-D plenoptic endoscopic imaging system).

The plenoptic imaging technique involves a microlens array integrated onto an imaging sensor, such that each point of the object can be viewed and imaged at different angles via adjacent microlenses, which is analogous to the stereoscopy approach. Besides, the presented setup compensates for the aperture mismatch between the endoscope and the microlens array fabricated on the plenoptic camera. The achieved depth accuracy error of about 1 mm and precision error of about 2 mm are recorded within a 25 mm × 25 mm field of view and the system operates at 11 frames per second.

Future development on the design will focus on facilitating multiple modalities to achieve an even higher level of surgical vision in minimally invasive surgery.

Graphic description: Spatial reconstruction of 3D-printed objects. (a-c) Microlens image of a plane and inhomogeneous objects and (d-f) its reconstructed depth maps.

Related Picture: