Enhanced imaging colonoscopy facilitates dense motion-based 3D reconstruction

We propose a novel approach for estimating a dense 3D model of neoplasia in colonoscopy using enhanced imaging endoscopy modalities. Estimating a dense 3D model of neoplasia is important to make 3D measurements and to classify the superficial lesions in standard frameworks such as the Paris classification. However, it is challenging to obtain decent dense 3D models using computer vision techniques such as Structure-from-Motion due to the lack of texture in conventional (white light) colonoscopy. Therefore, we propose to use enhanced imaging endoscopy modalities such as Narrow Band Imaging and chromoendoscopy to facilitate the 3D reconstruction process. Thanks to the use of these enhanced endoscopy techniques, visualization is improved, resulting in more reliable feature tracks and 3D reconstruction results. We first build a sparse 3D model of neoplasia using Structure-from-Motion from enhanced endoscopy imagery. Then, the sparse reconstruction is densified using a Multi-View Stereo approach, and finally the dense 3D point cloud is transformed into a mesh by means of Poisson surface reconstruction. The obtained dense 3D models facilitate classification of neoplasia in the Paris classification, in which the 3D size and the shape of the neoplasia play a major role in the diagnosis.