Cell Image Analysis

バイオイメージング技術は急速に発展してきており,様々な生命現象が新たに観察できるようになり,生物学分野に革新を起こしつつあります.このような研究において,より生体内に近い状態での解析は非常に重要になります.しかし,これまでのディッシュ上での観察と比べ,生体内では,細胞や分子等の観察対象が超高密度・大量に分布しており,画像データからの自動定量評価が難しいという課題があり,生物学分野における定量化研究の障壁となっています.静的・動的データの定量的な評価を行うためには,個々の細胞のセグメンテーション及びトラッキングをすることが求められます.しかし,バイオイメージ画像においては,人や車等の一般物体の追跡と比べ,「低コントラストで境界が非常に曖昧」「形状が様々に変化する」「対象同士の見た目が非常に類似している」といった様々な課題があります.このような課題を解決し,自動定量化を実現するため,自動細胞領域検出及びトラッキング等の細胞画像認識手法の研究開発を行っています.



Spatial-Temporal Mitosis Detection in Phase-Contrast Microscopy via Likelihood Map Estimation by 3DCNN

Automated mitotic detection in time-lapse phasecontrast microscopy provides us much information for cell behavior analysis, and thus several mitosis detection methods have been proposed. However, these methods still have two problems; 1) they cannot detect multiple mitosis events when there are closely placed. 2) they do not consider the annotation gaps, which may occur since the appearances of mitosis cells are very similar before and after the annotated frame. In this paper, we propose a novel mitosis detection method that can detect multiple mitosis events in a candidate sequence and mitigate the human annotation gap via estimating spatialtemporal likelihood map by 3DCNN. In this training, the loss gradually decreases with the gap size between groundtruth and estimation. This mitigates the annotation gaps. Our method outperformed the compared methods in terms of F1- score using challenging dataset that contains the data under four different conditions.



Cell Tracking with Deep Learning for Cell Detection and Motion Estimation in Low-Frame-Rate

Cell behavior analysis in High-throughput biological experiments is important for research and discovery in biology and medicine. To perform the high-throughput experiments, it requires to capture images in low frame rate in order to recode images on multi-points. In such low frame rate image sequence, movements of cells between successive frames are often larger than distances to nearby cells, and thus current methods based on proximity do not work properly. In this study, we propose a cell tracking method that enables to track cells in low frame rate by simultaneously estimating all of the cell motions in an image. In the experiments under dense conditions in low frame rate, our method outperformed the other methods.

  • [J. Hayashida and R. Bise, MICCAI,2019]

Weakly Supervised Cell Instance Segmentation in Dense by Propagating from Detection Map

Cell shape analysis is important in biomedical research. Deep learning methods may perform to segment individual cells if they use suffcient training data that the boundary of each cell is annotated. However, it is very time-consuming for preparing such detailed annotation for many cell culture conditions. In this paper, we propose a weakly supervised method that can segment individual cell regions who touch each other with unclear boundaries in dense conditions without the training data for cell regions. We demonstrated the effecacy of our method using several data-set including multiple cell types captured by several types of microscopy. Our method achieved the highest accuracy compared with several conventional methods. In addition, we demonstrated that our method can perform without any annotation by using fluorescence images that cell nuclear were stained as training data.

  • [K. Nishimura, E.D.F. Ker, and R. Bise, MICCAI,2019]
  • [code]


Cell Detection Method from Redundant Candidates under the Non-Overlapping Constraints



Cell detection in microscopy images is essential for automated cell behavior analysis including cell shape analysis and cell tracking. Robust cell detection in high-density and lowcontrast images is still challenging since cells often touch and partially overlap, forming a cell cluster with blurry intercellular boundaries. In such cases, current methods tend to detect multiple cells as a cluster. If the control parameters are adjusted to separate the touching cells, other problems often occur: a single cell may be segmented into several regions, and cells in lowintensity regions may not be detected. To solve these problems, we first detect redundant candidate regions, which include many false positives but in turn very few false negatives, by allowing candidate regions to overlap with each other. Next, the score for how likely the candidate region contains the main part of a single cell is computed for each cell candidate using supervised learning. Then we select an optimal set of cell regions from the redundant regions under non-overlapping constraints, where each selected region looks like a single cell and the selected regions do not overlap. We formulate this problem of optimal region selection as a binary linear programming problem under non-overlapping constraints. We demonstrated the effectiveness of our method for several types of cells in microscopy images. Our method performed better than five representative methods, achieving an F-measure of over 0.9 for all data sets. Experimental application of the proposed method to 3D images demonstrated that also works well for 3D cell detection.

Cell Segmentation in Microscopy Imagery Using a Bag of Local Bayesian Classifiers

Cell segmentation in microscopy imagery is essential for many bioimage applications such as cell tracking. To segment cells from the background accurately, we present a pixel classification approach that is independent of cell type or imaging modality. We train a set of Bayesian classifiers from clustered local training image patches. Each Bayesian classifier is an expert to make decision in its specific domain. The decision from the mixture of experts determines how likely a new pixel is a cell pixel. We demonstrate the effectiveness of this approach on four cell types with diverse morphologies under different microscopy imaging modalities.

Detection of Hematopoietic Stem Cells in Microscopy Images Using a Bank of Ring Filters

We present a method for robustly detecting hematopoietic stem cells (HSCs) in phase contrast microscopy images. HSCs appear to be easy to detect since they typically appear as round objects. However, when HSCs are touching and overlapping, showing the variations in shape and appearance, standard pattern detection methods, such as Hough transform and correlation, do not perform well. The proposed method exploits the output pattern of a ring filter bank applied to the input image, which consists of a series of matched filters with multiple-radius ring-shaped templates. By modeling the profile of each filter response as a quadratic surface, we explore the variations of peak curvatures and peak values of the filter responses when the ring radius varies. The method is validated on thousands of phase contrast microscopy images with different acquisition settings, achieving 96.5% precision and 94.4% recall.



Mitosis Detection

We propose an effective approach for automated mitosis detection using phase-contrast time-lapse microscopy, which is a non-destructive imaging modality, thereby allowing continuous monitoring of cells in culture. In our approach, we present a probabilistic model for event detection, which can simultaneously 1) identify spatio-temporal patch sequences that contain a mitotic event and 2) localize a birth event, defined as the time and location at which cell division is completed and two daughter cells are born. Our approach significantly outperforms previous approaches in terms of both detection accuracy and computational efficiency, when applied to multipotent C3H10T1/2 mesenchymal and C2C12 myoblastic stem cell populations.



Cell Tracking based on Frame by Frame Data Association

The image segmentation algorithm segments blobs from input images that can be individual cells or cell clusters (overlapped cells), and the mitosis detection algorithm locates birth events where and when one cell divides into two cells. Based on the outputs of these two algorithms, we developed a cell-blob correspondence algorithm performing data association between the cells in the previous frame and the blobs segmented in the current frame. In detail, the association algorithm makes the following hypotheses with corresponding likelihood for all possible cell actions.



Cell Tracking based on Global Data Association

Automated cell tracking in populations is important for research and discovery in biology and medicine. We propose a cell tracking method based on global spatiotemporal data association which considers hypotheses of initialization, termination, translation, division and false positive in an integrated formulation. Firstly, reliable tracklets (i.e., short trajectories) are generated by linking detection responses based on frame-by-frame association. Next, these tracklets are globally associated over time to obtain final cell trajectories and lineage trees. During global association, tracklets form tree structures where a mother cell divides into two daughter cells. We formulate the global association for tree structures as a maximum-a-posteriori (MAP) problem and solve it by linear programming. This approach is quantitatively evaluated on sequences with thousands of cells captured over several days.






Application



Wound Healing Assay

The wound healing assay in vitro is widely used for research and discovery in biology and medicine. This assay allows for observing the healing process in vitro in which the cells on the edges of the artificial wound migrate toward the wound area. The influence of different culture conditions can be measured by observing the change in the size of the wound area. For further investigation, more detailed measurements of the cell behaviors are required. We present an application of automatic cell tracking in phase-contrast microscopy images to wound healing assay. The cell behaviors under three different culture conditions have been analyzed. Our cell tracking system can track individual cells during the healing process and provide detailed spatio-temporal measurements of cell behaviors including cell density, cell migration speed and direction, and the statistics of cell mitosis events. The application demonstrates the effectiveness of automatic cell tracking for quantitative and detailed analysis of the cell behaviors in wound healing assay in vitro.

Tracing behavior of endothelial cells promotes vascular network formation

The in vitro formation of network structures derived from endothelial cells in grafts before transplantation contributes to earlier engraftment. In a previous study, endothelial cells migrated to form a net-shaped structure in co-culture. However, the specific network formation behavior of endothelial cells during migration remains unclear. In this study, we demonstrated the tracing behavior and cell cycle of endothelial cells using Fucci-labeled (Fluorescent Ubiquitination-based Cell Cycle Indicator) endothelial cells. Here, we observed the co-culture of Fucci-labeled human umbilical vein endothelial cells (HUVECs) together with normal human dermal fibroblasts (NHDFs) using time-lapse imaging and analyzed by multicellular concurrent tracking. In the G0/G1 period, HUVECs migrate faster than in the S/G2/M period, because G0/G1 is the mobile phase and S/G2/M is the proliferation phase in the cell cycle. When HUVECs are co-cultured, they tend to move randomly until they find existing tracks that they then follow to form clusters. Extracellular matrix (ECM) staining showed that collagen IV, laminin and thrombospondin deposited in accordance with endothelial cell networks. Therefore the HUVECs may migrate on the secreted ECM and exhibit tracing behavior, where the HUVECs migrate toward each other. These results suggested that ECM and a cell phase contributed to form a network by accelerating cell migration.

An engineered approach to stem cell culture: automating the decision process for real-time adaptive subculture of stem cells

Current cell culture practices are dependent upon human operators and remain laborious and highly subjective, resulting in large variations and inconsistent outcomes, especially when using visual assessments of cell confluency to determine the appropriate time to subculture cells. Although efforts to automate cell culture with robotic systems are underway, the majority of such systems still require human intervention to determine when to subculture. Thus, it is necessary to accurately and objectively determine the appropriate time for cell passaging. Optimal stem cell culturing that maintains cell pluripotency while maximizing cell yields will be especially important for efficient, cost-effective stem cell-based therapies. Toward this goal we developed a real-time computer vision-based system that monitors the degree of cell confluency with a precision of 0.791±0.031 and recall of 0.559±0.043. The system consists of an automated phase-contrast time-lapse microscope and a server. Multiple dishes are sequentially imaged and the data is uploaded to the server that performs computer vision processing, predicts when cells will exceed a pre-defined threshold for optimal cell confluency, and provides a Web-based interface for remote cell culture monitoring. Human operators are also notified via text messaging and e-mail 4 hours prior to reaching this threshold and immediately upon reaching this threshold. This system was successfully used to direct the expansion of a paradigm stem cell population, C2C12 cells. Computer-directed and human-directed control subcultures required 3 serial cultures to achieve the theoretical target cell yield of 50 million C2C12 cells and showed no difference for myogenic and osteogenic differentiation. This automated vision-based system has potential as a tool toward adaptive real-time control of subculturing, cell culture optimization and quality assurance/quality control, and it could be integrated with current and developing robotic cell cultures systems to achieve complete automation.

Mechanical characterization of adult stem cells from bone marrow and perivascular niches

Therapies using adult stem cells often require mechanical manipulation such as injection or incorporation into scaffolds. However, force-induced rupture and mechanosensitivity of cells during manipulation is largely ignored. Here, we image cell mechanical structures and perform a biophysical characterization of three different types of human adult stem cells: bone marrow CD34+ hematopoietic, bone marrow mesenchymal and perivascular mesenchymal stem cells. We use micropipette aspiration to characterize cell mechanics and quantify deformation of subcellular structures under force and its contribution to global cell deformation. Our results suggest that CD34+ cells are mechanically suitable for injection systems since cells transition from solid- to fluid-like at constant aspiration pressure, probably due to a poorly developed actin cytoskeleton. Conversely, mesenchymal stem cells from the bone marrow and perivascular niches are more suitable for seeding into biomaterial scaffolds since they are mechanically robust and have developed cytoskeletal structures that may allow cellular stable attachment and motility through solid porous environments. Among these, perivascular stem cells cultured in 6% oxygen show a developed cytoskeleton but a more compliant nucleus, which can facilitate the penetration into pores of tissues or scaffolds. We confirm the relevance of our measurements using cell motility and migration assays and measure survival of injected cells. Since different types of adult stem cells can be used for similar applications, we suggest considering mechanical properties of stem cells to match optimal mechanical characteristics of therapies.




Data Set



Phase Contrast Time-Lapse Microscopy Datasets with Automated and Manual Cell Tracking Annotations

Phase contrast time-lapse microscopy is a non-destructive technique that generates large volumes of image-based information to quantify the behaviour of individual cells or cell populations. To guide the development of algorithms for computer-aided cell tracking and analysis, 48 time-lapse image sequences, each spanning approximately 3.5 days, were generated with accompanying ground truths for C2C12 myoblast cells cultured under 4 different media conditions, including with fibroblast growth factor 2 (FGF2), bone morphogenetic protein 2 (BMP2), FGF2?+?BMP2, and control (no growth factor). The ground truths generated contain information for tracking at least 3 parent cells and their descendants within these datasets and were validated using a two-tier system of manual curation. This comprehensive, validated dataset will be useful in advancing the development of computer-aided cell tracking algorithms and function as a benchmark, providing an invaluable opportunity to deepen our understanding of individual and population-based cell dynamics for biomedical research.