Sparse-to-dense coarse-to-fine depth estimation for colonoscopy

doi:10.1016/j.compbiomed.2023.106983

科研成果详情

题名	Sparse-to-dense coarse-to-fine depth estimation for colonoscopy
作者	Liu, Ruyu 1,2; Liu, Zhengzhe 1; Lu, Jiaming 3; Zhang, Guodao 4; Zuo, Zhigui5; Sun, Bo 2; Zhang, Jianhua 3; Sheng, Weiguo 1; Guo, Ran 6; Zhang, Lejun 6,7,8; Hua, Xiaozhen9
发表日期	2023-06
发表期刊	Computers in biology and medicine 影响因子和分区
语种	英语
原始文献类型	Journal Article
关键词	Deep learning Depth estimation Endoscopic SLAM Medical metaverse
其他关键词	INVASIVE SURGERY ; SLAM ; RECONSTRUCTION
摘要	Colonoscopy, as the golden standard for screening colon cancer and diseases, offers considerable benefits to patients. However, it also imposes challenges on diagnosis and potential surgery due to the narrow observation perspective and limited perception dimension. Dense depth estimation can overcome the above limitations and offer doctors straightforward 3D visual feedback. To this end, we propose a novel sparse-to-dense coarse-to-fine depth estimation solution for colonoscopic scenes based on the direct SLAM algorithm. The highlight of our solution is that we utilize the scattered 3D points obtained from SLAM to generate accurate and dense depth in full resolution. This is done by a deep learning (DL)-based depth completion network and a reconstruction system. The depth completion network effectively extracts texture, geometry, and structure features from sparse depth along with RGB data to recover the dense depth map. The reconstruction system further updates the dense depth map using a photometric error-based optimization and a mesh modeling approach to reconstruct a more accurate 3D model of colons with detailed surface texture. We show the effectiveness and accuracy of our depth estimation method on near photo-realistic challenging colon datasets. Experiments demonstrate that the strategy of sparse-to-dense coarse-to-fine can significantly improve the performance of depth estimation and smoothly fuse direct SLAM and DL-based depth estimation into a complete dense reconstruction system.
资助项目	National Natural Science Foundation of China[62172353,62202137];Natural Science Foundation of Zhejiang Province[LQ22F030004];
出版者	Elsevier Ltd
ISSN	0010-4825
EISSN	1879-0534
卷号	160
DOI	10.1016/j.compbiomed.2023.106983
页数	10
WOS类目	Biology ; Computer Science, Interdisciplinary Applications ; Engineering, Biomedical ; Mathematical & Computational Biology
WOS研究方向	Life Sciences & Biomedicine - Other Topics ; Computer Science ; Engineering ; Mathematical & Computational Biology
WOS记录号	WOS:001005401300001
收录类别	PUBMED ; EI ; SCIE ; SCOPUS
在线发表日期	2023-05
EI入藏号	20232014092194
EI主题词	Three dimensional computer graphics
EI分类号	461.4 Ergonomics and Human Factors Engineering ; 461.6 Medicine and Pharmacology ; 717.1 Optical Communication Systems ; 723.2 Data Processing and Image Processing ; 723.5 Computer Applications
URL	查看原文
PubMed ID	37187133
SCOPUSEID	2-s2.0-85159173713
通讯作者地址	[Guo, Ran]Cyberspace Institute Advanced Technology,Guangzhou University,Guangzhou,510006,China
Scopus学科分类	Health Informatics;Computer Science Applications
TOP期刊	TOP期刊
引用统计
文献类型	期刊论文
条目标识符	https://kms.wmu.edu.cn/handle/3ETUA0LF/180511
专题	附属第一医院第二临床医学院，附属第二医院、育英儿童医院_儿科学
通讯作者	Guo, Ran
作者单位	1.School of Information Science and Technology,Hangzhou Normal University,Hangzhou,311121,China; 2.Haixi Institutes,Chinese Academy of Sciences Quanzhou Institute of Equipment Manufacturing,Quanzhou,362000,China; 3.School of Computer Science and Engineering,Tianjin University of Technology,Tianjin,300384,China; 4.Department of Digital Media Technology,Hangzhou Dianzi University,Hangzhou,310018,China; 5.Department of Colorectal Surgery,the First Affiliated Hospital of Wenzhou Medical University,Wenzhou,325035,China; 6.Cyberspace Institute Advanced Technology,Guangzhou University,Guangzhou,510006,China; 7.College of Information Engineering,Yangzhou University,Yangzhou,225127,China; 8.Research and Development Center for E-Learning,Ministry of Education,Beijing,100039,China; 9.Department of Pediatrics,Cangnan Affiliated Hospital of Wenzhou Medical University,Wenzhou,325800,China
推荐引用方式 GB/T 7714	Liu, Ruyu,Liu, Zhengzhe,Lu, Jiaming,et al. Sparse-to-dense coarse-to-fine depth estimation for colonoscopy[J]. Computers in biology and medicine,2023,160.
APA	Liu, Ruyu., Liu, Zhengzhe., Lu, Jiaming., Zhang, Guodao., Zuo, Zhigui., ... & Hua, Xiaozhen. (2023). Sparse-to-dense coarse-to-fine depth estimation for colonoscopy. Computers in biology and medicine, 160.
MLA	Liu, Ruyu,et al."Sparse-to-dense coarse-to-fine depth estimation for colonoscopy".Computers in biology and medicine 160(2023).