Automatic segmentation and classification of multiplex-fluorescence in-situ hybridization chromosome images

Multicolor fluorescence in-situ hybridization (M-FISH) techniques provide color karyotyping that allows simultaneous analysis of numerical and structural abnormalities of whole human chromosomes. Chromosomes are stained combinatorially in M-FISH. By analyzing the intensity combinations of each pixel, all chromosome pixels in an image are classified. Often, the intensity distributions between different images are found to be considerably different and the difference becomes the source of misclassifications of the pixels. Improved pixel classification accuracy is the most important task to ensure the success of the M-FISH technique. Along with a reliable pixel classification method, automation of the karyotyping process is another important goal. The automation requires segmentation of chromosomes, which not only involves object/background separation but also involves separating touching and overlapping chromosomes. While automating the segmentation of partially occluded chromosomes is an extremely challenging problem, a pixel classification method that satisfies both high accuracy and minimum human intervention has not been realized. The main contributions of this dissertation include development of a new feature normalization method for M-FISH images that reduces the difference in the feature distributions among different images, and development of a new decomposition method for clusters of overlapping and touching chromosomes. A significant improvement was achieved in pixel classification accuracy after the new feature normalization. The overall pixel classification accuracy improved by 40% after normalization. Given a cluster, a number of hypotheses was formed utilizing the geometry of a cluster, pixel classification results, and chromosome sizes, and a hypotheis that maximized the likelihood function was chosen as the correct decomposition. Superior decomposition results were obtained using the new method compared to the previous methods. Contributions also include development of a color compensation method for combinatorially stained FISH images (including M-FISH images) based on a new signal model for multicolor/multichannel FISH images. The true signal was recovered based on the signal model after color compensation. The resulting true signal does not have color spreading (channel crosstalk) among different color channels. Two new unsupervised nonparametric classification methods for M-FISH images are also introduced in this dissertation: a fuzzy logic classifier and a template matching method (a minimum distance clasvii sifier). While both methods produce an equivalent accuracy compared to a supervised classification method, their computation time is significantly less than a Bayes classifier. Highly sophisticated and practical algorithms have been developed through this research. Using the developed methods, the amount of human intervention required will be significantly reduced: chromosomes are reliably and accurately segmented from the background, pixels are accurately classified, and clusters of overlapping and touching chromosomes are automatically decomposed.