ImPACT「イノベーティブな可視化技術による新成長産業の創出」



高齢化社会の到来に伴い、健康長寿で豊かな生活を実現し、病気や介護への不安を解消させる技術サポートが求められています。NIIは、病気の早期診断や超精密検査の実現を目指すImPACTに参加し、生体や物体内部を非侵襲・非破壊でリアルタイム三次元可視化する光超音波イメージングの高度化を行っています。光超音波システムは、レーザー照射により発生する超音波を検出し可視化する最先端計測技術です。この技術は、非侵襲・非破壊である上に、透過して深部まで照射できる光と超音波の両方の特性を活かし、肉眼では見えない様々な対象の可視化を可能にします。本研究では、コンピュータビジョン技術により、鮮明な画像を得るイメージング技術の高度化や、様々な情報を用いた画像解析による診断支援を行っています。例えば、撮影中の患者の体動による画質劣化に対して、画像の位置合わせにより患者の動きを補正し、画質改善した診断しやすい画像を提供できるようになります。また、疾病に関係が深い血管状態を把握するため、血管構造の自動抽出技術の開発を進めています。


Virtual Blood Vessels using Stereo X-ray Images

We propose a fully automatic system to reconstruct and visualize 3D blood vessels in Augmented Reality (AR) system from stereo X-ray images with bones and body fat. Currently, typical 3D imaging technologies are expensive and carrying the risk of irradiation exposure. To reduce the potential harm, we only need to take two X-ray images before visualizing the vessels. Our system can effectively reconstruct and visualize vessels in following steps. We first conduct initial segmentation using Markov Random Field and then refine segmentation in an entropy based post-process. We parse the segmented vessels by extracting their centerlines and generating trees. We propose a coarse-to-fine scheme for stereo matching, including initial matching using affine transform and dense matching using Hungarian algorithm guided by Gaussian regression. Finally, we render and visualize the reconstructed model in a HoloLens based AR system, which can essentially change the way of visualizing medical data. We have evaluated its performance by using synthetic and real stereo X-ray images, and achieved satisfactory quantitative and qualitative results.


Blood Vessel Extraction by Semi-Supervised Learning

In this paper, we focus on semi-supervised learning for biomedical image segmentation, so as to take advantage of huge unlabelled data. We observe that there usually exist some homogeneous connected areas of low confidence in biomedical images, which tend to confuse the classifier trained with limited labelled samples. To cope with this difficulty, we propose to construct forest oriented super pixels(voxels) to augment the standard random forest classifier, in which super pixels(voxels) are built upon the forest based code. Compared to the state-of-the-art, our proposed method shows superior segmentation performance on challenging 2D/3D biomedical images. The full implementation (based on Matlab) is available at https://github.com/lingucv/ssl_superpixels.

Separation of Transmitted Light and Scattering Components in Transmitted Microscopy



In transmitted light microscopy, a specimen tends to be observed as unclear. This is caused by a phenomenon that an image sensor captures the sum of these scattered light rays traveled from different paths due to scattering. To cope with this problem, we propose a novel computational photography approach for separating directly transmitted light from the scattering light in a transmitted light microscope by using high-frequency lighting. We first investigated light paths and clarified what types of light overlap in transmitted light microscopy. The scattered light can be simply represented and removed by using the difference in observations between focused and unfocused conditions, where the high-frequency illumination becomes homogeneous. Our method makes a novel spatial multiple-spectral absorption analysis possible, which requires absorption coefficients to be measured in each spectrum at each position. Experiments on real biological tissues demonstrated the effectiveness of our method.

Vascular Registration in Photoacosutic Imaging by Low-Rank Alignment



Photoacoustic (PA) imaging has been gaining attention as a new imaging modality that can non-invasively visualize blood vessels inside biological tissues. In the process of imaging large body parts through multi-scan fusion, alignment turns out to be an important issue, since body motion degrades image quality. In this paper, we carefully examine the characteristics of PA images and propose a novel registration method that achieves better alignment while effectively decomposing the shot volumes into low-rank foreground (blood vessels), dense background (noise), and sparse complement (corruption) components on the basis of the PA characteristics. The results of experiments using a challenging real data-set demonstrate the efficacy of the proposed method, which significantly improved image quality, and had the best alignment accuracy among the state-of-the-art methods tested.

3D Structure Modeling of Dense Capillaries by Tracking



A newly developed imaging technique called light-sheet laser microscopy imaging can visualize the detailed 3D structures of capillaries. Capillaries form complicated network structures in the obtained data, and this makes it difficult to model vessel structures by existing methods that implicitly assume simple tree structures for blood vessels. To cope with such dense capillaries with network structures, we propose to track the flow of blood vessels along a base-axis using a multiple-object tracking framework. We first track multiple blood vessels in cross-sectional images along a single axis to make the trajectories of blood vessels, and then connect these blood vessels to reveal their entire structures. This framework is efficient to track densely distributed vessels since it uses only a single cross-sectional plane. The network structure is then generated in the post-processing by connecting blood vessels on the basis of orientations of the trajectories. The results of experiments using a challenging real data-set demonstrate the efficacy of the proposed method, which are capable of modeling dense capillaries.


Cell Image Analysis

バイオイメージング技術は急速に発展してきており,様々な生命現象が新たに観察できるようになり,生物学分野に革新を起こしつつあります.このような研究において,より生体内に近い状態での解析は非常に重要になります.しかし,これまでのディッシュ上での観察と比べ,生体内では,細胞や分子等の観察対象が超高密度・大量に分布しており,画像データからの自動定量評価が難しいという課題があり,生物学分野における定量化研究の障壁となっています.静的・動的データの定量的な評価を行うためには,個々の細胞のセグメンテーション及びトラッキングをすることが求められます.しかし,バイオイメージ画像においては,人や車等の一般物体の追跡と比べ,「低コントラストで境界が非常に曖昧」「形状が様々に変化する」「対象同士の見た目が非常に類似している」といった様々な課題があります.このような課題を解決し,自動定量化を実現するため,自動細胞領域検出及びトラッキング等の細胞画像認識手法の研究開発を行っています.


Cell Detection



Cell detection in microscopy images is essential for automated cell behavior analysis including cell shape analysis and cell tracking. Robust cell detection in high-density and lowcontrast images is still challenging since cells often touch and partially overlap, forming a cell cluster with blurry intercellular boundaries. In such cases, current methods tend to detect multiple cells as a cluster. If the control parameters are adjusted to separate the touching cells, other problems often occur: a single cell may be segmented into several regions, and cells in lowintensity regions may not be detected. To solve these problems, we first detect redundant candidate regions, which include many false positives but in turn very few false negatives, by allowing candidate regions to overlap with each other. Next, the score for how likely the candidate region contains the main part of a single cell is computed for each cell candidate using supervised learning. Then we select an optimal set of cell regions from the redundant regions under non-overlapping constraints, where each selected region looks like a single cell and the selected regions do not overlap. We formulate this problem of optimal region selection as a binary linear programming problem under non-overlapping constraints. We demonstrated the effectiveness of our method for several types of cells in microscopy images. Our method performed better than five representative methods, achieving an F-measure of over 0.9 for all data sets. Experimental application of the proposed method to 3D images demonstrated that also works well for 3D cell detection.



Mitosis Detection

We propose an effective approach for automated mitosis detection using phase-contrast time-lapse microscopy, which is a non-destructive imaging modality, thereby allowing continuous monitoring of cells in culture. In our approach, we present a probabilistic model for event detection, which can simultaneously 1) identify spatio-temporal patch sequences that contain a mitotic event and 2) localize a birth event, defined as the time and location at which cell division is completed and two daughter cells are born. Our approach significantly outperforms previous approaches in terms of both detection accuracy and computational efficiency, when applied to multipotent C3H10T1/2 mesenchymal and C2C12 myoblastic stem cell populations.



Cell Tracking based on Frame by Frame Data Association

The image segmentation algorithm segments blobs from input images that can be individual cells or cell clusters (overlapped cells), and the mitosis detection algorithm locates birth events where and when one cell divides into two cells. Based on the outputs of these two algorithms, we developed a cell-blob correspondence algorithm performing data association between the cells in the previous frame and the blobs segmented in the current frame. In detail, the association algorithm makes the following hypotheses with corresponding likelihood for all possible cell actions.



Cell Tracking based on Global Data Association

Automated cell tracking in populations is important for research and discovery in biology and medicine. We propose a cell tracking method based on global spatiotemporal data association which considers hypotheses of initialization, termination, translation, division and false positive in an integrated formulation. Firstly, reliable tracklets (i.e., short trajectories) are generated by linking detection responses based on frame-by-frame association. Next, these tracklets are globally associated over time to obtain final cell trajectories and lineage trees. During global association, tracklets form tree structures where a mother cell divides into two daughter cells. We formulate the global association for tree structures as a maximum-a-posteriori (MAP) problem and solve it by linear programming. This approach is quantitatively evaluated on sequences with thousands of cells captured over several days.

Wound Healing Assay

The wound healing assay in vitro is widely used for research and discovery in biology and medicine. This assay allows for observing the healing process in vitro in which the cells on the edges of the artificial wound migrate toward the wound area. The influence of different culture conditions can be measured by observing the change in the size of the wound area. For further investigation, more detailed measurements of the cell behaviors are required. We present an application of automatic cell tracking in phase-contrast microscopy images to wound healing assay. The cell behaviors under three different culture conditions have been analyzed. Our cell tracking system can track individual cells during the healing process and provide detailed spatio-temporal measurements of cell behaviors including cell density, cell migration speed and direction, and the statistics of cell mitosis events. The application demonstrates the effectiveness of automatic cell tracking for quantitative and detailed analysis of the cell behaviors in wound healing assay in vitro.