Fusion of colour contrasted images for early detection of oesophageal squamous cell dysplasia from endoscopic videos in real time
Article
Gao, X., Taylor, S., Pang, W., Hui, R., Lu, X., Oxford GI Investigators and Braden, B. 2023. Fusion of colour contrasted images for early detection of oesophageal squamous cell dysplasia from endoscopic videos in real time. Information Fusion. 92, pp. 64-79. https://doi.org/10.1016/j.inffus.2022.11.023
Type | Article |
---|---|
Title | Fusion of colour contrasted images for early detection of oesophageal squamous cell dysplasia from endoscopic videos in real time |
Authors | Gao, X., Taylor, S., Pang, W., Hui, R., Lu, X., Oxford GI Investigators and Braden, B. |
Abstract | Standard white light (WL) endoscopy often misses precancerous oesophageal changes due to their only subtle differences to the surrounding normal mucosa. While deep learning (DL) based decision support systems benefit to a large extent, they face two challenges, which are limited annotated data sets and insufficient generalisation. This paper aims to fuse a DL system with human perception by exploiting computational enhancement of colour contrast. Instead of employing conventional data augmentation techniques by alternating RGB values of an image, this study employs a human colour appearance model, CIECAM, to enhance the colours of an image. When testing on a frame of endoscopic videos, the developed system firstly generates its contrast-enhanced image, then processes both original and enhanced images one after another to create initial segmentation masks. Finally, fusion takes place on the assembled list of masks obtained from both images to determine the finishing bounding boxes, segments and class labels that are rendered on the original video frame, through the application of non-maxima suppression technique (NMS). This deep learning system is built upon real-time instance segmentation network Yolact. In comparison with the same system without fusion, the sensitivity and specificity for detecting early stage of oesophagus cancer, i.e. low-grade dysplasia (LGD) increased from 75% and 88% to 83% and 97%, respectively. The video processing/play back speed is 33.46 frames per second. The main contribution includes alleviation of data source dependency of existing deep learning systems and the fusion of human perception for data augmentation. |
Keywords | Early squamous cell cancer detection; Deep machine learning; Colour contrast enhancement; Endoscopic treatment; Surveillance; Oesophagus cancer; gastrointestinal endoscopy |
Sustainable Development Goals | 3 Good health and well-being |
Middlesex University Theme | Health & Wellbeing |
Research Group | Artificial Intelligence group |
Publisher | Elsevier |
Journal | Information Fusion |
ISSN | 1566-2535 |
Electronic | 1872-6305 |
Publication dates | |
Online | 24 Nov 2022 |
30 Apr 2023 | |
Publication process dates | |
Deposited | 23 Nov 2022 |
Submitted | 09 Sep 2022 |
Accepted | 20 Nov 2022 |
Output status | Published |
Publisher's version | License |
Accepted author manuscript | License File Access Level Restricted |
Copyright Statement | Published version: Copyright © 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/ |
Digital Object Identifier (DOI) | https://doi.org/10.1016/j.inffus.2022.11.023 |
Web of Science identifier | WOS:000891921400005 |
Language | English |
https://repository.mdx.ac.uk/item/8q29y
Download files
86
total views14
total downloads0
views this month0
downloads this month