3D view reconstruction from endoscopic videos for gastrointestinal tract surgery planning
Conference paper
Gao, X., Rahmanti, A. and Braden, B. 2025. 3D view reconstruction from endoscopic videos for gastrointestinal tract surgery planning. Kim, J., Conceição, R., Yousef, M., Bhavsar, A., Pelayo, S., Fred, A. and Gamboa, H. (ed.) 18th International Joint Conference on Biomedical Engineering Systems and Technologies. Porto, Portugal 20 - 22 Feb 2025 SCITEPRESS - Science and Technology Publications. pp. 221-228 https://doi.org/10.5220/0013125000003911
Type | Conference paper |
---|---|
Title | 3D view reconstruction from endoscopic videos for gastrointestinal tract surgery planning |
Authors | Gao, X., Rahmanti, A. and Braden, B. |
Abstract | This paper investigates the application of neural radiance field (NeRF) to reconstruct a 3D model from 2D endoscopic videos for surgical planning and removal of gastrointestinal lesions. It comprises three stages. The first one is video preprocess to remove frames with artefact of colour misalignment based on a deep learning network. Then the remaining frames are converted into NeRF compatible format. This stage includes extraction of camera information regarding intrinsic, extrinsic and ray pathway parameters as well as conversion to NeRF format based on COLMAP library, a pipeline built upon structure-from-motion (SfM) with multi-view stereo (MVS). Finally the training takes place for establishment of NeRF model implemented upon Nerfstudio library. Initial results illustrate that this end-to-end, i.e. from 2D video input to 3D model output deep learning architecture presents great potentials for reconstruction of gastrointestinal tract. Base on the two sets of data containing 2600 i mages, the similarity measures of SSIM, PSNR and LPIPS between original (ground truth) and rendered images are 19.46 ± 2.56, 0.70 ± 0.054, and 0.49 ± 0.05 respectively. Future work includes enlarging dataset and removal of ghostly artefact from rendered images. |
Keywords | Deep Learning; 3D Reconstruction; Nerfs; Gastrointestinal Tract; SfM; 2D Endoscopic Video; Endoscopic Artefacts |
Sustainable Development Goals | 3 Good health and well-being |
Middlesex University Theme | Health & Wellbeing |
Conference | 18th International Joint Conference on Biomedical Engineering Systems and Technologies |
Page range | 221-228 |
Proceedings Title | Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 1: BIOIMAGING |
Editors | Kim, J., Conceição, R., Yousef, M., Bhavsar, A., Pelayo, S., Fred, A. and Gamboa, H. |
ISSN | 2184-4305 |
ISBN | 9789897587313 |
Publisher | SCITEPRESS - Science and Technology Publications |
Publication dates | |
20 Feb 2025 | |
Publication process dates | |
Accepted | 04 Dec 2024 |
Deposited | 14 Mar 2025 |
Output status | Published |
Publisher's version | License File Access Level Open |
Copyright Statement | Paper published under CC license (CC BY-NC-ND 4.0) |
Digital Object Identifier (DOI) | https://doi.org/10.5220/0013125000003911 |
Web address (URL) of conference proceedings | http://doi.org/10.5220/0000197500003911 |
https://repository.mdx.ac.uk/item/219q79
Download files
Publisher's version
2025-BIOSTEC_2025_paper-XG-AR-BB.pdf | ||
License: CC BY-NC-ND 4.0 | ||
File access level: Open |
1
total views2
total downloads1
views this month2
downloads this month