Automatic Content Analysis Of Endoscopy Video (Endoscopic Multimedia Information System)

Advances in video technology are being incorporated into today's healthcare practice. For example, various types of endoscopes are used for colonoscopy, upper gastrointestinal endoscopy, enteroscopy, bronchoscopy, cystoscopy, laparoscopy, and some minimal invasive surgeries (i.e., video endoscopic neurosurgery). These endoscopes come in various sizes, but all have a tiny video camera at the tip of the endoscopes. During an endoscopic procedure, the tiny video camera generates a video signal of the interior of the human organ, for example, the internal mucosa of the colon. The video data are displayed on a monitor for real-time analysis by the physician. Diagnosis, biopsy and therapeutic operations can be performed during the procedure. We define endoscopy videos as digital videos captured during endoscopic procedures. Despite a large body of knowledge in medical image analysis, endoscopy videos are not systematically captured for real-time or post-procedure reviews and analyses. No hardware and software tools have been developed to capture, analyze, and provide user-friendly and efficient access to important content on such videos. To address this problem, a project has been proposed to develop an Endoscopic Multimedia Information System (EMIS) which captures high quality endoscopy videos, analyzes the captured videos for important contents, and provides efficient access to these contents. In this dissertation, we focus on the automatic analysis techniques of endoscopy videos for important contents by presenting (1) object & frame recognition, (2) multi-level endoscopy video segmentation and (3) application for endoscopy video analysis (Measurement of Endoscopy Quality). To analyze the contents of endoscopy videos, we first propose three object & frame recognition algorithm: Endoscopy Video Frame Classification, Lumen Identification and Polyp Detection. The problem of segmenting visual data into smaller chunks is a basic problem in multimedia analysis, and its solution helps in problems such as video indexing and retrieval. However, traditional video segmentation techniques are not suitable for segmenting endoscopy video because endoscopy videos are generated by a single camera operation without shot, which makes it difficult to manage and analyze them. To address this problem, I propose a novel algorithm of multi-level segmentation for endoscopy video, which represents the semantic structure of endoscopy video: Video, Phase, Piece, and Objective Shot. Based on the information obtained by object & frame recognition and multi-level endoscopy video segmentation, we develop software tool to measure the quality of endoscopic procedure. The development of software tool will directly benefit endoscopic research, education, and training: especially for the research-based advanced training of students in graduate and undergraduate programs in medical informatics.