Welcome to Biomedical Imaging Informatics (BII) Laboratory

-- Retinal Pigment Epithelial (RPE) cells serve as a supporter for the metabolism and visual function of photoreceptors and a barrier for photoreceptor protection. Morphology dynamics, spatial organization, distribution density, and growth patterns of RPE cells are important for further research on these RPE main functions. To enable such investigations within the authentic eyeball structure, we present a new method to estimate the three-dimension (3D) eyeball sphere from two-dimension (2D) tissue flat-mount microscopy images. An error-correction term is formulated to compensate for the reconstruction error due to tissue distortions. We evaluate the effect of the tissue-distortion error by excluding partial data points from the low- and high-latitude zones. The error-correction parameter is learned automatically using a set of samples with the ground truth eyeball diameters gauged by noncontact LED micrometry at sub-micron accuracy and precision. Our analysis demonstrates that the error-correction term in the reconstruction model is a valid way to model tissue distortions in the tissue flat-mount preparation steps. We have developed a new method to enable RPE morphometry analysis with respect to locations on an eyeball sphere, an important step to further enhance RPE research and eye disease diagnosis.

-- In spite of serving as the "gold-standard" for diagnosis of hepatic fibrosis and cirrhosis, the liver needle biopsy is known to include inaccurate diagnostic information, primarily due to the inexorable sampling bias. However, there is lack of method to gauge such sampling bias as it is infeasible to take a large number of needle biopsies from the same patient. To develop a computational analysis pipeline to enable needle biopsy sampling bias investigations, we construct a 3D virtual liver tissue volume by spatially registered high resolution Whole Slide Images (WSIs) of serial liver tissue sections with a novel dynamic registration method. With spatially aligned serial liver WSIs, we further develop a Virtual Needle Biopsy Sampling function (VNBS) that mimics the needle biopsy sampling process with a physical needle in current clinical practice. Such a virtual sampling function is applied to the reconstructed digital liver image volume for multiple times, each at a distinct tissue location and angle. Additionally, we develop an analysis pipeline to quantitate Collagen Proportionate Area (CPA) in all resulting needle biopsies. All sampled biopsies are manually graded by the Scheuer and Ishak staging methods. With such domain annotations and machine-based collagen fiber quantification results, we investigate the inter- and intra-biopsy variability by the pathology stage score and CPA with a set of 3D virtual needle biopsies and their internal 2D cross sections. Our experimental results suggest the developed computational pipeline can recover the substantial presence of needle biopsy sampling bias in both pathology stage scores and liver CPAs.The developed virtual liver needle biopsy sampling method provides a new avenue for investigating needle biopsy sampling bias with 3D virtual tissue volumes. Although the developed working pipeline specifically investigates the sampling bias in liver needle biopsy in this study, the presented analysis methods can be tailored to a large scope of other tissue based disease diagnoses where the needle biopsy sampling bias substantially affects the diagnostic results.

-- In the era of precision medicine, human tumor atlas oriented studies have been significantly facilitated by high-resolution, multi-modal tissue based microscopic pathology image analytics. To better support such tissue-based investigations, we develop Digital Pathology Laboratory (DPLab), a publicly available web-based platform, to assist biomedical research groups, non-technical end users, and clinicians for pathology Whole-Slide Image (WSI) visualization, annotation, analysis, and sharing via web browsers. A major advance of this work is the easy-to-follow methods to reconstruct three-dimension (3D) tissue image volumes by registering two-dimension (2D) whole-slide pathology images of serial tissue sections stained by hematoxylin and eosin (H&E), and immunohistochemistry (IHC). The integration of these serial slides stained by different methods provides cellular phenotype and pathophysiologic states in the context of a 3D tissue micro-environment. DPLab is hosted on a publicly accessible server and connected to a backend computational cluster for intensive image analysis computations, with results visualized, downloaded, and shared via a web interface. Equipped with an analysis toolbox of numerous image processing algorithms, DPLab supports continued integration of community-contributed algorithms and presents an effective solution to improve the accessibility and dissemination of image analysis algorithms by research communities. It represents the first step in making next generation tissue investigation tools widely available to the research community, enabling and facilitating discovery of clinically relevant disease mechanisms in a digital 3D tissue space.

-- Retinal pigment epithelium (RPE) aging is an important cause of vision loss. As RPE aging is accompanied by changes in cell morphological features, an accurate segmentation of RPE cells is a prerequisite to such morphology analyses. Due the overwhelmingly large cell number, manual annotations of RPE cell borders are time-consuming. Computer based methods do not work well on cells with weak or missing borders in the impaired RPE sheet regions. To address such a challenge, we develop a semi-supervised deep learning approach, namely MultiHeadGAN, to segment low contrast cells from impaired regions in RPE flatmount images. The developed deep learning model has a multi-head structure that allows model training with only a small scale of human annotated data. To strengthen model learning effect, we further train our model with RPE cells without ground truth cell borders by generative adversarial networks. Additionally, we develop a new shape loss to guide the network to produce closed cell borders in the segmentation results. Compared with other state-of-the-art deep learning approaches, our method demonstrates superior qualitative and quantitative performance.Suggested by our extensive experiments, our developed deep learning method can accurately segment cells from RPE flatmount microscopy images and is promising to support large scale cell morphological analyses for RPE aging investigations.

-- Liver fibrosis staging is clinically important for liver disease progression prediction. As the portal tract fibrotic quantity and size in a liver biopsy correlate with the fibrosis stage, an accurate analysis of portal tract regions is clinically critical. Manual annotations of portal tract regions, however, are time-consuming and subject to large inter- and intra-observer variability. To address such a challenge, we develop a Multi-up-sampling and Spatial Attention guided UNet model (MUSA-UNet) to segment liver portal tract regions in whole-slide images of liver tissue slides. To enhance the segmentation performance, we propose to use depth-wise separable convolution, the spatial attention mechanism, the residual connection, and multiple up-sampling paths in the developed model. Compared with other state-of-the-art deep learning methods, our model demonstrates both superior qualitative and quantitative performance. The clinical Scheuer fibrosis stage presents a strong correlation with the resulting average portal tract fibrotic area and portal tract percentage computed from the MUSA-UNet segmentation results. In conclusion, our developed deep learning model MUSA-UNet can accurately segment portal tract regions from whole-slide images of liver tissue biopsies and present a promising potential to assist liver disease diagnosis in a computational manner.

-- Pathology image analysis is a crucial step to support accurate cancer mechanism research, disease grading, diagnosis, and treatment planning. Although there is a high demand for precise nuclei morphology analyses, nuclei segmentation, the prerequisite step, mostly is completed manually by pathologists in practice. While the newly emerging deep learning algorithms can help automate the process, the lack of sufficient pathology training images with well-annotated ground truth in most tissue-based biomedical research inevitably limits the performance of deep learning systems. In this study, we propose to address this problem by a convolutional neural network with foveal blur, a blurring algorithm mimicking human vision, that enriches datasets with multiple local nuclei regions of interest derived from large pathology images for enhanced nuclei segmentation performance. As the convolutional neural network is not able to explicitly learn the nuclei shape, we propose a human knowledge boosted deep learning system by inclusion to the convolutional neural network new loss function terms capturing shape prior knowledge and imposing smoothness constraints. The high segmentation accuracy and competitive processing speed of our proposed method from three independent datasets suggest its promising potential for automating histopathology nuclei segmentation in biomedical research and clinical settings where efficient high-quality nuclei segmentation are demanded.

-- In this study, we present a multi-resolution deep learning model HistoCAE for viable tumor segmentation in whole-slide liver histopathology images. We propose convolutional autoencoder (CAE) based framework with a customized reconstruction loss function for image reconstruction, followed by a classification module to classify each image patch as tumor vs non-tumor. The resulting patch-based prediction results are spatially combined to generate the final segmentation result for each WSI. Our proposed model presents superior performance to other benchmark models with extensive experiments, suggesting its efficacy for viable tumor area segmentation with liver whole-slide images.

-- Liver steatosis is known as the abnormal accumulation of lipids within cells. An accurate quantification of steatosis area within the liver histopathological microscopy images plays an important role in liver disease diagnosis and trans- plantation assessment. Such a quantification analysis often requires a precise steatosis segmentation that is challenging due to abundant presence of highly overlapped steatosis droplets. In this paper, a deep learning model Mask-RCNN is used to segment the steatosis droplets in clumps. Extended from Faster R-CNN, Mask-RCNN can predict object masks in addition to bounding box detection. With transfer learning, the resulting model is promising to support liver disease diagnosis and allograft rejection prediction in future clinical practice.

-- In triple negative breast cancer (TNBC) treatment, early prediction of pathological complete response (PCR) from chemotherapy before surgical operations is crucial for optimal treatment planning. We propose a novel deep learning-based system to predict PCR to neoadjuvant chemotherapy for TNBC patients with multi-stained histopathology images of serial tissue sections. By first performing tumor cell detection and recognition in a cell detection module, we produce a set of feature maps that capture cell type, shape, and location information. Next, a newly designed spatial attention module integrates such feature maps with original pathology images in multiple stains for enhanced PCR prediction in a dedicated prediction module. We compare it with baseline models that either use a single-stained slide or have no spatial attention module in place. Additionally, the heatmaps generated from the spatial attention module can help pathologists in targeting tissue regions important for disease assessment. Our system presents high efficiency and effectiveness and improves interpretability, making it highly promising for immediate clinical and translational impact.

-- The pathological complete response to neoadjuvant chemotherapy is one good assessment measure in breast cancer treatment that is proven to be highly associated with patients' overall survival. While the accurate early prediction of PCR before neoadjuvant chemotherapy treatment is in high demand, the existing PCR prediction systems are too limited in accuracy to be deployed in real clinical settings. In this work, we proposed one convolutional neural network based system for predicting PCR from three different modalities of pathology images (hematoxylin and eosin (H&E), KI-67, and PHH3 stained) captured from adjacent tissue slices. One spatial attention mechanism is deployed to the system to guide it to pay attention to specific areas where pathologists consider important and informative. With integrated usage of different image modalities, the system can avoid the limitation of spatial homogeneity from original CNN. Our proposed model achieved a high patient-level accuracy, outperforming baselines and other state-of-the-art systems. Its inspiring performance suggests its great potential in real clinical usage.

-- Oncotype DX Recurrence Score (RS) has been widely used to predict chemotherapy benefits in patients with estrogen receptor-positive breast cancer. Studies showed that the features used in Magee equations correlate with RS. We aimed to examine whether deep learning (DL)-based histology image analyses can enhance such correlations. We retrieved 382 cases with RS diagnosed between 2011 and 2015 from the Emory University and the Ohio State University. All patients received surgery. DL models were developed to detect nuclei of tumor cells and tumor-infiltrating lymphocytes (TILs) and segment tumor cell nuclei in hematoxylin and eosin (H&E) stained histopathology whole slide images (WSIs). Based on the DL-based analysis, we derived image features from WSIs, such as tumor cell number, TIL number variance, and nuclear grades. The entire patient cohorts were divided into one training set (125 cases) and two validation sets (82 and 175 cases) based on the data sources and WSI resolutions. The training set was used to train the linear regression models to predict RS. For prediction performance comparison, we used independent variables from Magee features alone or the combination of WSI-derived image and Magee features. Our results suggest that DL-based digital pathological features can enhance Magee feature correlation with RS.

-- We have developed a complete and fully automated method for simultaneous segmentation of multiple nuclei in histology images with mutual occlusion. This method framework consists of a seed detection algorithm for nuclei contour initialization, and an integrated contour deformable model that incorporates region, shape, and boundary information. The developed nucleus seed detection algorithm utilizes joint information of spatial connectivity, distance constraint, image edge map, and a shape-based voting map derived from eigenvalue analysis of Hessian matrix across multiple scales. In the resulting nuclei segmentation step, a sparse representation-based shape term is introduced to better guide nuclei contour convergence with shape prior information from the shape library. All shape representations are clustered with an L1 graph-based manifold learning method to establish a concise and representative shape prior library. A dynamic nuclei occlusion term is proposed in the level set-based variational model to dynamically deal with occlusion events involving a variable number of nuclei. In-detail theoretical derivation for level set functional optimization and offer a solution to optimize the loss function. A numerical optimization algorithm is used to iteratively search for the desired level set functions. Experiments on histopathologic images produce better results as compared with other state of the arts, suggesting the effectiveness of our detection and segmentation approach for cell analysis with histopathologic images

-- To address biomedical object tracking problem with time-lapse fluorescence image volumes, we design an object tracking and motion pattern analysis approach that generalizes particle filtering approach to improve the tracking performance in cases where object intensity distributions are non-Gaussian. We present two specific non-Gaussian state models to extend the applicability of the particle filtering tracking algorithm to biomedical research. In addition, we propose a new tracking management strategy to accelerate the model updating. With this new mapping step, our tracking strategy can avoid object state mismatch commonly seen in particle filtering when certain objects disappear from the scope of view. The performance of our approach is demonstrated with on both artificial image sequences and real time-lapse fluorescent image datasets that capture 2D bacteria and 3D lung cancer cells in motion. The experimental results are promising, suggesting the potential of the application of our approach to diverse biological and cancer research.

-- We have designed an efficient video analysis method track tagged honey bees. The analysis pipeline identifies tags in image frames, links sequences of corresponding tags into “tracks” for each individual insect, infers the tag identifier, and provides a user-friendly graphical environment for editing tracking data by five main modules: video preprocessing, tag detection, digit recognition, track assembly and a graphical editor. The video preprocessor accepts a raw video as input and generates a background image and, optionally, a cropped video file as output. The cropped video file only includes the active regions detected within the raw video, and decreases the searching space for the follow-up tag analysis. Each frame of the cropped video is searched for tag regions, and the resulting individual tag images are extracted, saved, and logged in an annotation file. Tag images are preprocessed and provided to the Tesseract optical character recognition engine for digit recognition. The orientation with the highest average confidence is chosen as the correct orientation, and digit recognition results are appended to the annotation file. Tag data from different frames are linked as tracks based on their spatial locations and sizes. The developed software also provides a user-friendly graphical environment for editing tracking data. The developed method and software implements individual-level tracking function, presenting strong promise to help distinguish disease transmission patterns caused by the interactions of individual insects.

-- The introduction of high-throughput scanning technology has allowed for routine digital 3D reconstruction of serial whole slide histology images. However, registration of serial whole slide images is a challenge due to the overwhelmingly large image size. A single image may contain several gigabytes of data. The resulting 3D volumes can easily exceed modern computer memory limits, thus precluding the direct use of existing reconstruction methods. Based on current methods, accessing subvolumes of tissues requires the prior registration of the entire full-resolution whole slide image volume, leading to a high computational cost. To solve this problem, we develop a hierarchical image registration method that computes image deformation on-the-fly. In this way, we can minimize the computational burden and focus analysis on the tissue subvolumes of interest to domain experts. As the nonrigid registration at the low resolution is applied to the rigidly registered image volume and only alignment transformations for all adjacent image pairs are available, we are developing a novel recursive mapping method that maps each pixel in the reference coordinate system to the corresponding location in a target image with use of the mapped nonrigid and rigid propagated transformations from low to high image resolution. In addition, we are also developing a multi-resolution, block-wise registration approach that works in a coarse-to-fine manner for multi-stain image registration.

-- A study on glioblastoma (GBM) cell’s invasive property is difficult with both in-vivo human patient and animal models. Therefore, GBM tumor resections are extracted from human brains with neurosurgery and grown in-vitro as GBM neurosphere cell lines instead. We have access to a spectrum of molecular representative human GBM cell lines with distinct biological behaviors. Motility evaluation is then conducted with cells in-vitro, which is prevalent in current research practice. However, such studies are limited by the fact that cells in-vitro can only migrate in a two-dimensional space. As a further step, such in-vivo cells from GBM cell lines are explanted to biologically relevant 3D scaffolds (200 μm in thickness) where live GBM cells in ex-vivo have three degrees of freedom for migration. Such an ex-vivo setting provides an opportunity to replicate the authentic brain environment, where cells interact with extracellular matrices as they migrate. With a three-dimensional spinning disc confocal image scanner (Nikon A1R) / a multiphoton imaging microscope (Zeiss 710 MP with a Chameleon Vision S Ti:Sapphire laser) for larger 3D structures (e.g. spheroids, 1 mm thick tissues), 3D fluorescent images of cell dynamics in the ex-vivo setting are acquired. We are developing 3D microscopy image analysis tools, with special focus on the development of methods to quantitatively measure the cell migration capacity deemed as a good indicator of the dramatic shift in tumor expansion property. The resulting image analysis workflow involves 3D cell segmentation, maximum cross-section detection, cell feature extraction, and 4D cell tracking. The resulting analysis workflow will be used to analyze and characterize ensemble cell invasion power in 3D space with cell lines under different molecular programs.

-- Fitting geometric models to objects of interest in images is one of the most classical problems studied in computer vision field. In addition, object representation is a common problem in diverse biomedical image processing applications in practice. As a result of its strong representation power and flexibility, conic is one of the geometric primitives widely used in a large number of image analysis applications. As opposed to most existing conic fitting methods minimizing the fitting error with the use of the second order polynomial representation, we haved contributed to this fundamental research area with a new ellipse-fitting framework from which two elegant ellipse-fitting algorithms have been derived. We propose a new method that formulates the geometric fitting problem as a process of seeking for the optimal mapping to a bivariate normal distribution model. As a result, some critical disadvantages tightly coupled with those methods following the routine polynomial representation can be well overcome. These algorithms are elegant with the mathematical formulation and novel by manifestation of the inherent connections between statistical models and ellipse fitters. With these new methods, objects of interests from numerous applications can be well characterized, ranging from cell and follicule head description in microscopy image analysis, stent profile reconstruction for cardiovascular operation assessment, to tissue biopsy representation for high throughput screening analysis.

-- Active contours, or more figuratively snakes, are a class of methods that search for and represent image features, usually object boundaries, with deformable contours driven by the net influence of both internal and external forces. The internal forces are designed in a way such that the contours are managed to maintain their tensions and rigidities, whereas the the external forces are derived from image data that encourage contours to conform to desired image features. We have developed a new pressure-like force that not only improves contour convergence rate, but also encourages contours to conform to concave regions. We introduce the steerable pressure force (SPF) for parametric active contour models by leveraging the original pressure force as its building block. Moreover, it distinguishes itself from the tradition pressure force with the great convenience of dynamically steering the direction of the pressure force to be conformed to the image content. Unlike the traditional pressure force, this new force does not require users’ input for the force direction and is steerable according to the image content. Better convergence rate as well as force normalization consistency of this new force are presented when compared with other commonly-used forces on synthetic images. Results on a MRI image smoothed at different levels demonstrate the robustness of this new force to noise.

-- As an effort to build an automated and objective system for pathologic image analysis, we have developed a self-reliant computerized image processing method for identifying nuclei, a basic biological unit of diagnostic utility, in microscopy images of glioma tissue samples. The complete analysis includes multiple processing steps, involving mode detection with color and spatial information for pixel clustering, background normalization leveraging morphological operations, boundary refinement with deformable models, and clumped nuclei separation. The developed analysis algorithm is sufficiently robust to considerable image variations inexorably coupled in microscopy images. Computerized nuclei detection results are in good concordance with human markups by both visual appraisement and quantitative measures. This suggests that the developed method is promising for generating quantitative and reliable analysis results to support further glioma analysis.

-- The diagnosis of diffuse gliomas requires the careful inspection of large amounts of visual data. Identifying tissue regions that inform diagnosis is a cumbersome task for human reviewers and is a process prone to inter-reader variability. To address this problem, we have developed an automatic method for identifying critical diagnostic regions within whole-slide microscopy images of gliomas. We frame the problem of critical region identification as a texture-based content retrieval task in the sense that each image is represented by a set of discriminating texture features. Both linear and nonlinear dimensionality reduction techniques are utilized to explore the intrinsic dimensionality of the feature space where images are classified by classification and regression trees with performances improved by a newly extended multi-class gentle boosting (MCGB) mechanism.

-- EMLDA is an iterative segmentation method utilizing the Fisher-Rao criterion as the kernel of the generic Expectation-Maximization (EM) algorithm. Linear Discriminant Analysis (LDA), a supervised classification technique, serves as the kernel of the EM-algorithm and iteratively groups data projected to a lower-dimensional feature space in such a way that the separability across all classes is maximized. In the E step, the “missing data” is estimated given the feature data and the model parameters of the current guess. In the M step, the model parameters are re-estimated in a way that maximize the criterion function using the updated “full data”.

--Hepatic steatosis droplet quantification with histology biopsies has high clinical significance for risk stratification and management of patients with fatty liver diseases and in the decision to use donor livers for transplantation. We present a steatosis segmentation model to identify individual steatosis and delineate boundaries of overlapping steatosis instances in whole slide microscopy images of liver biopsies. Specifically, a region-based module is designed to segment the foreground steatosis droplet region from background pixels, while a boundary module is introduced to learn the perceptual boundary features for each overlapped steatosis region. Next, the region and boundary information are combined to train the third deep neural network responsible for dividing steatosis droplets in clumps. The proposed network architecture is named as DeEp LearnINg stEATosis sEgmentation (DELINEATE). In addition, we use a spatial indexing based approach to identify partial steatosis droplet components from neighboring patches, and have them efficiently assembled by MaReIA, a tool we developed in our prior work.

--An accurate steatosis quantification with pathology tissue samples is of high clinical importance. However, such pathology measurement is manually made in most clinical practices, subject to severe reader variability due to large sampling bias and poor reproducibility. Although some computerized automated methods are developed to quantify the steatosis regions, they present limited analysis capacity for high resolution whole-slide microscopy images and accurate overlapped steatosis division. In this work, we propose a method that extracts an individual whole tissue piece at high resolution with minimum background area by estimating tissue bounding box and rotation angle. This is followed by the segmentation and segregation of steatosis regions with high curvature point detection and an ellipse fitting quality assessment method. We validate our method with isolated and overlapped steatosis regions in liver tissue images. The experimental results suggest that our method is promising for enhanced support of steatosis quantization during the pathology review for liver disease treatment.

--The prevalent gold standard for steatosis assessment is simply by human visual inspections of liver histopathological sections. However, this process is subject to large intra- and inter-observer variability. Hailed as a new alternative solution, digital pathology is an emerging field that uses digitized high-resolution images of histology sections for machine-based image processing. As each tissue slide is projected to a two-dimensional microscopy image space, it is not unusual to identify a large number of tissue regions with overlapped steatosis droplets in clumps. Such spatial alignment nature, combined with substantial variations of size, staining color, and structure appearance, presents a serious technical barrier for an accurate segmentation of individual steatosis droplets, leading to erroneous steatosis feature computation and size quantification. Due to its promising performance, deep learning has become a successful alternative solution to biomedical image analysis recently. In this work, we adopt the Mask-RCNN based deep learning method and successfully customize it to segment overlapped steatosis droplets in whole-slide histopathology images of live sections. To establish a large training data efficiently, we propose to transfer our prior work on nuclei segmentation for liver slides and efficiently screen results by a domain expert for a valid training data set. The proposed method can separate highly clumped steatosis droplets and recover their precise contours with high accuracy.

-- Our research work also involves microscopy image analysis of steatosis area for liver transplantation assessment, as quantitating hepatic steatosis is important in many liver diseases and liver transplantation. Liver steatosis is an abnormal accumulation of lipid (fat) in liver cells. In liver transplantation, we need to quantify the degree of steatosis associated with post-transplant dysfunction. However, steatosis estimation by pathologists has inherent intra- and inter-observer variability. To address this problem, We have developed a computerized image analysis paradigm enabling quantitative characterizations of steatosis areas in microscopy images of pediatric liver biopsies. With the same set of patients, we also acquired the lipid measurements from magnetic resonance imaging data analysis for correlation investigation. Our results suggest a high correlation between the steatosis areas quantized with microscopy images and the lipid percentages calculated from radiology imaging data.

-- Additionally, we compared and contrasted computerized techniques with magnetic resonance imaging measurements, pathologist visual scoring, and clinical parameters. Computerized methods applied to whole slide images included a commercial positive pixel count algorithm and our in-house image analysis method. For all liver samples, including pediatric, adult, frozen section, and permanent specimens, statistically significant correlations were observed between pathology, radiology, and each image analysis modality, with the strongest correlations in the pediatric cohort. Statistically significant relationships were observed between each method and with body mass index and with albumin but not with alanine aminotransferase or aspartate aminotransferase. Although pathologist assessments correlated, the absolute values of hepatic steatosis visual assessment were susceptible to intra- and inter-observer variability, particularly for microvesicular steatosis. Image analysis, pathologist assessments, radiology measurements, and several clinical parameters all showed correlations in this study, providing evidence for the utility of each method in different clinical and research settings.

-- As 2D microscopy images can only present 3D histopathological structures at discrete planes, they present significant information loss. Therefore, we are developing image analysis tools for 3D microscopy examiniations. This set of tools can derive accurate and informative 3D imaging features to help researchers and clinicians better analyze 3D biological structures. For example, we can calculate the inter-branch distance, characterize the branching pattern, and generate a histogram of branch lengths and thicknesses for the entire 3D tissue volume for vessel characterizations. We also aim to characterize the relationship between distinct types of structures in 3D space, for example, the average distance between a bile duct (or a cell) and its nearest blood vessel, to identify better clinically relevant patient stratification protocols and molecularly correlated phenotypic signature.

-- We synergized multi-resolution, multi-source data of brain tumors to better address scientific questions about disease onset and progression. We intensively made use of microscopy imaging data, MRI image features and omics data for large-scale correlation study. For imaging study particularly, we used the publicly available The Cancer Genome Atlas database for this study. Note some data can be accessed directly from the acquisition platforms, e.g. the molecular data. Others, e.g. the phenotypic information from microscopy images are only accessible after analysis. In short, we aim to extract anatomic characterization at cellular level of tissue pathology, integrate with multiple types of “omic” information, and create categories of jointly classified data to describe pathophysiology, predict prognosis and response to treatment.

-- To mitigate the low reproducibility and inter-observer agreement from human pathologic review process, we have developed a computerized image analysis to quantitatively and reproducibly measure histologic structures on a large-scale. We present an end-to-end image analysis and data integration pipeline for large-scale morphologic analysis of pathology images and demonstrate the ability to correlate phenotypic groups with molecular data and clinical outcomes. We demonstrate our method in the context of glioblastoma (GBM), with specific focus on the degree of the oligodendroglioma component. Over 200 million nuclei in digitized pathology slides from 117 GBMs in the Cancer Genome Atlas were quantitatively analyzed, followed by multiplatform correlation of nuclear features with molecular and clinical data. For each nucleus, a Nuclear Score (NS) was calculated based on the degree of oligodendroglioma appearance, using a regression model trained from the optimal feature set. Using the frequencies of neoplastic nuclei in low and high NS intervals, we were able to cluster patients into three well-separated disease groups that contained low, medium, or high Oligodendroglioma Component (OC). We showed that machine-based classification of GBMs with high oligodendroglioma component uncovered a set of tumors with strong associations with PDGFRA amplification, proneural transcriptional class, and expression of the oligodendrocyte signature genes MBP, HOXD1, PLP1, MOBP and PDGFRA. Quantitative morphologic features within the GBMs that correlated most strongly with oligodendrocyte gene expression were high nuclear circularity and low eccentricity.

-- Additionally, we have developed a methodology to subclassify disease with image analysis techniques. Morphologic signatures that represent patient-specific tumor morphology are derived from the analysis of hundreds of millions of cells in digitized whole slide images. Clustering these signatures aggregates tumors into groups with cohesive morphologic characteristics. This methodology is demonstrated with an analysis of glioblastoma, using data from The Cancer Genome Atlas to identify a prognostically significant morphology-driven subclassification, in which clusters are correlated with transcriptional, genetic, and epigenetic events. Analysis of glioblastoma identified three prognostically significant patient clusters, each characterized by molecular events in nuclear compartment signaling, developmental and cell cycle checkpoint pathways. This analysis demonstrates the potential of high-throughput morphometrics for the subclassification of disease, establishing an approach that complements genomics.

-- Neuroblastoma (NB)is one of the most frequently occurring cancerous tumors in children. Based on the International Neuroblastoma Pathology Classification defined by World Heath Organization, diagnosis of NBs involves the analysis of tumor differenitation grade and stroma degree.

-- The current pathology evaluations for patients with this disease require pathologists to identify certain morphological characteristics with microscopic examinations of tumor tissues. Thanks to the advent of modern digital scanners, it is now feasible to scan cross-section tissue specimens and acquire whole-slide digital images. As a result, computerized analysis of these image scan generate key quantifiable parameters and assist pathologists with accurate evaluations.

-- To automatically grade NB differentation, we developed a large set of image analysis techniques, applied them to histological images of haematoxylin and eosin (H&E) stained NB slides, and identified image regions associated with distinct differentiation grades, including undifferentiated, poorly-differentiated, differentiating grade.Texture features derived from segmented components of tissues were extracted and processed by anautomated classifier group trained with sampleimages with different grades of neuroblastic differentiation in a multi-resolution framework.

-- Addtionally, we proposed an image analysis system that classifies each slide as either stroma-rich or stroma-poor based on the degree of Schwannian stromal development. The resulting statistical framework performs the classification based on texture features extracted using co-occurrence statistics and local binary patterns. Due to the high resolution of digitize dwhole-slide images, we proposed a multi-resolution approach that mimics the way pathologists evaluates physical slides under microscope. The image analys starts from the lowest resolution and switches to higher resolutions when necessary. We employed an offline feature selection step, which determines the most discriminative features at each resolution level during the training step.