研究者詳細 - 小田　昌宏

Depth estimation from single-shot monocular endoscope image using image domain adaptation and edge-aware depth estimation 査読有り

Oda Masahiro, Itoh Hayato, Tanaka Kiyohito, Takabatake Hirotsugu, Mori Masaki, Natori Hiroshi, Mori Kensaku

Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 2021年12月

担当区分：筆頭著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization

We propose a depth estimation method from a single-shot monocular endoscopic image using Lambertian surface translation by domain adaptation and depth estimation using multi-scale edge loss. We employ a two-step estimation process including Lambertian surface translation from unpaired data and depth estimation. The texture and specular reflection on the surface of an organ reduce the accuracy of depth estimations. We apply Lambertian surface translation to an endoscopic image to remove these texture and reflections. Then, we estimate the depth by using a fully convolutional network (FCN). During the training of the FCN, improvement of the object edge similarity between an estimated image and a ground truth depth image is important for getting better results. We introduced a muti-scale edge loss function to improve the accuracy of depth estimation. We quantitatively evaluated the proposed method using real colonoscopic images. The estimated depth values were proportional to the real depth values. Furthermore, we applied the estimated depth images to automated anatomical location identification of colonoscopic images using a convolutional neural network. The identification accuracy of the network improved from 69.2% to 74.1% by using the estimated depth images.

DOI： 10.1080/21681163.2021.2012835

Attention-Guided Pancreatic Duct Segmentation from Abdominal CT Volumes 査読有り国際共著

Shen C., Roth H.R., Yuichiro H., Oda M., Miyamoto T., Sato G., Mori K.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12969 LNCS 巻頁： 46 - 55 2021年11月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Pancreatic duct dilation indicates a high risk of pancreatic ductal adenocarcinoma (PDAC), the deadliest cancer with a poor prognosis. Segmentation of dilated pancreatic duct from CT taken from patients without PDAC shows the potential to assist the early detection of PDAC. Most current researches include pancreatic duct segmentation as one additional class for patients who have already detected PDAC. However, the dilated pancreatic duct for people who have not yet developed PDAC is typically much smaller, making the segmentation difficult. Deep learning-based segmentation on tiny components is challenging because of the large imbalance between the target object and irrelevant regions. In this work, we explore an attention-guided approach for dilated pancreatic duct segmentation as a screening tool for pre-PDAC patients, enhancing the pancreas regions’ concentration and ignoring the unnecessary features. We employ a multi-scale aggregation to combine the information at different scales to improve the segmentation performance further. Our proposed multi-scale pancreatic attention-guided approach achieved a Dice score of 54.16% on dilated pancreatic duct dataset, which shows a significant improvement over prior techniques.

DOI： 10.1007/978-3-030-90874-4_5

Multi-task Federated Learning for Heterogeneous Pancreas Segmentation 査読有り国際共著

Shen C., Wang P., Roth H.R., Yang D., Xu D., Oda M., Wang W., Fuh C.S., Chen P.T., Liu K.L., Liao W.C., Mori K.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12969 LNCS 巻頁： 101 - 110 2021年11月

Federated learning (FL) for medical image segmentation becomes more challenging in multi-task settings where clients might have different categories of labels represented in their data. For example, one client might have patient data with “healthy” pancreases only while datasets from other clients may contain cases with pancreatic tumors. The vanilla federated averaging algorithm makes it possible to obtain more generalizable deep learning-based segmentation models representing the training data from multiple institutions without centralizing datasets. However, it might be sub-optimal for the aforementioned multi-task scenarios. In this paper, we investigate heterogeneous optimization methods that show improvements for the automated segmentation of pancreas and pancreatic tumors in abdominal CT images with FL settings.

DOI： 10.1007/978-3-030-90874-4_10

Intestine Segmentation with Small Computational Cost for Diagnosis Assistance of Ileus and Intestinal Obstruction 査読有り

Oda H., Hayashi Y., Kitasaka T., Takimoto A., Hinoki A., Uchida H., Suzuki K., Oda M., Mori K.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12969 LNCS 巻頁： 3 - 12 2021年11月

This paper proposes an intestine segmentation method from CT volumes for the intestinal obstruction and the ileus diagnosis assistance. The previous method was built based on the 3D U-Net, whose computational cost was high. Nevertheless, there was no confirmation that the 3-dimensional network contributed to the segmentation performance. In this paper, we propose a method utilizing the 2D U-Net on behalf of the previous methods’ 3D U-Net. Experimental results using 110 CT volumes showed that both the proposed (2D U-Net) and previous (3D U-Net) methods achieved similar scores for the segmentation accuracy and system usefulness. In addition, the proposed method’s inference was 0.5 min on average, around 8-times faster than the 3D U-Net’s. Although subsequent processes still require more than 20 min for each case, utilizing lightweight networks is essential for practical use in the future, especially for emergency diagnosis.

DOI： 10.1007/978-3-030-90874-4_1

COVID-19 Infection Segmentation from Chest CT Images Based on Scale Uncertainty 査読有り

Oda M., Zheng T., Hayashi Y., Otake Y., Hashimoto M., Akashi T., Aoki S., Mori K.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12969 LNCS 巻頁： 88 - 97 2021年11月

担当区分：筆頭著者,　責任著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

This paper proposes a segmentation method of infection regions in the lung from CT volumes of COVID-19 patients. COVID-19 spread worldwide, causing many infected patients and deaths. CT image-based diagnosis of COVID-19 can provide quick and accurate diagnosis results. An automated segmentation method of infection regions in the lung provides a quantitative criterion for diagnosis. Previous methods employ whole 2D image or 3D volume-based processes. Infection regions have a considerable variation in their sizes. Such processes easily miss small infection regions. Patch-based process is effective for segmenting small targets. However, selecting the appropriate patch size is difficult in infection region segmentation. We utilize the scale uncertainty among various receptive field sizes of a segmentation FCN to obtain infection regions. The receptive field sizes can be defined as the patch size and the resolution of volumes where patches are clipped from. This paper proposes an infection segmentation network (ISNet) that performs patch-based segmentation and a scale uncertainty-aware prediction aggregation method that refines the segmentation result. We design ISNet to segment infection regions that have various intensity values. ISNet has multiple encoding paths to process patch volumes normalized by multiple intensity ranges. We collect prediction results generated by ISNets having various receptive field sizes. Scale uncertainty among the prediction results is extracted by the prediction aggregation method. We use an aggregation FCN to generate a refined segmentation result considering scale uncertainty among the predictions. In our experiments using 199 chest CT volumes of COVID-19 cases, the prediction aggregation method improved the dice similarity score from 47.6% to 62.1%.

DOI： 10.1007/978-3-030-90874-4_9

Aorta-aware GAN for non-contrast to artery contrasted CT translation and its application to abdominal aortic aneurysm detection. 査読有り

Hu T, Oda M, Hayashi Y, Lu Z, Kumamaru KK, Akashi T, Aoki S, Mori K

International Journal of Computer Assisted Radiology and Surgery 17 巻 ( 1 ) 頁： 97 - 105 2021年10月

Graph Cuts Loss to Boost Model Accuracy and Generalizability for Medical Image Segmentation 査読有り

Zheng Zhou, Oda Masahiro, Mori Kensaku

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021) 2021-October 巻頁： 3297 - 3306 2021年10月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Proceedings of the IEEE International Conference on Computer Vision

Segmentation accuracy and generalization ability are essential for deep learning models, especially in medical image segmentation. We present a novel, robust yet straightforward loss function to boost model accuracy and generalizability for medical image segmentation. We reformulate the graph cuts cost function to a loss function for supervised learning. The graph cuts loss innately focuses on a dual penalty to optimize the regional properties and boundary regularization. We benchmark the proposed loss on six public retinal vessel segmentation datasets with a comprehensive intra-dataset and cross-dataset evaluation. Results reveal that the proposed loss is more generalizable, narrowing the performance gap between different architectures. Besides, models trained with our loss show higher segmentation accuracy and better generalization ability than those trained with other mainstream losses. Moreover, we extend our loss to other segmentation tasks, e.g., left atrium and liver tumor segmentation. The proposed loss still achieves comparable performance to the state-of-the-art, demonstrating its potential for any N-D segmentation problem. The code is available at https://github.com/zzh_enggit/graphcutsloss.

DOI： 10.1109/ICCVW54120.2021.00369

Super-Resolution by Latent Space Exploration: Training with Poorly-Aligned Clinical and Micro CT Image Dataset 査読有り

Zheng T., Oda H., Hayashi Y., Nakamura S., Oda M., Mori K.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12965 LNCS 巻頁： 24 - 33 2021年9月

This paper proposes a super-resolution (SR) method, for performing SR on a poorly-aligned dataset. Super-resolution methods commonly needs aligned low-resolution (LR) and high-resolution (HR) images for training. For obtaining paired LR and HR images in medical imaging, we need to align low and high-resolution data using image registration technology. However, since the hardness of aligning LR and HR images, the aligned LR-HR dataset is always low quality. Conventional SR methods always fail to train using poorly-aligned datasets since these methods need high-quality LR-HR datasets. To tackle this problem, we propose a two-step framework for SR using poorly-aligned datasets. In the first step, we decompose image representation into two parts: one is a content code that captures the image content; the other is a style code that captures the image style and anatomy difference between LR / HR images. To perform SR of a given LR image, we input the content code and a latent variable simultaneously into the SR network to obtain an SR result. In the second step, using the trained SR network and an LR image, we search for a content code, and a style code for generating the most proper SR image. This is conducted by searching for the best content code and the best style code by latent space exploration. We conducted experiments using a poorly-aligned clinical-micro CT lung specimen dataset. Experimental results illustrated the proposed method outperformed conventional SR methods by increasing SSIM from 0.309 to 0.312, and have much more convincing perceptual quality than conventional SR methods.

DOI： 10.1007/978-3-030-87592-3_3

Performance improvement of weakly supervised fully convolutional networks by skip connections for brain structure segmentation 査読有り

Sugino Takaaki, Roth Holger R., Oda Masahiro, Kin Taichi, Saito Nobuhito, Nakajima Yoshikazu, Mori Kensaku

Medical Physics 48 巻 ( 11 ) 頁： 7215 - 7227 2021年9月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Medical Physics

Purpose: For the planning and navigation of neurosurgery, we have developed a fully convolutional network (FCN)-based method for brain structure segmentation on magnetic resonance (MR) images. The capability of an FCN depends on the quality of the training data (i.e., raw data and annotation data) and network architectures. The improvement of annotation quality is a significant concern because it requires much labor for labeling organ regions. To address this problem, we focus on skip connection architectures and reveal which skip connections are effective for training FCNs using sparsely annotated brain images. Methods: We tested 2D FCN architectures with four different types of skip connections. The first was a U-Net architecture with horizontal skip connections that transfer feature maps at the same scale from the encoder to the decoder. The second was a U-Net++ architecture with dense convolution layers and dense horizontal skip connections. The third was a full-resolution residual network (FRRN) architecture with vertical skip connections that pass feature maps between each downsampled scale path and the full-resolution scale path. The last one was a hybrid architecture with a combination of horizontal and vertical skip connections. We validated the effect of skip connections on medical image segmentation from sparse annotation based on these four FCN architectures, which were trained under the same conditions. Results: For multiclass segmentation of the cerebrum, cerebellum, brainstem, and blood vessels from sparsely annotated MR images, we performed a comparative evaluation of segmentation performance among the above four FCN approaches: U-Net, U-Net++, FRRN, and hybrid architectures. The experimental results show that the horizontal skip connections in the U-Net architectures were effective for the segmentation of larger sized objects, whereas the vertical skip connections in the FRRN architecture improved the segmentation of smaller sized objects. The hybrid architecture with both horizontal and vertical skip connections achieved the best results of the four FCN architectures. We then performed an ablation study to explore which skip connections in the FRRN architecture contributed to the improved segmentation of blood vessels. In the ablation study, we compared the segmentation performance between architectures with a horizontal path (HP), an HP and vertical up paths (HP+VUPs), an HP and vertical down paths (HP+VDPs), and an HP and vertical up and down paths (FRRN). We found that the vertical up paths were effective in improving the segmentation of smaller sized objects. Conclusions: This paper investigated which skip connection architectures were effective for multiclass brain segmentation from sparse annotation. Consequently, using vertical skip connections with horizontal skip connections allowed FCNs to improve segmentation performance.

DOI： 10.1002/mp.15192

Binary polyp-size classification based on deep-learned spatial information 査読有り

Itoh Hayato, Oda Masahiro, Jiang Kai, Mori Yuichi, Misawa Masashi, Kudo Shin-Ei, Imai Kenichiro, Ito Sayo, Hotta Kinichi, Mori Kensaku

INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERYInternational Journal of Computer Assisted Radiology and Surgery 16 巻頁： 1817 - 1828 2021年9月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：International Journal of Computer Assisted Radiology and Surgery

Purpose: The size information of detected polyps is an essential factor for diagnosis in colon cancer screening. For example, adenomas and sessile serrated polyps that are ≥ 10 mm are considered advanced, and shorter surveillance intervals are recommended for smaller polyps. However, sometimes the subjective estimations of endoscopists are incorrect and overestimate the sizes. To circumvent these difficulties, we developed a method for automatic binary polyp-size classification between two polyp sizes: from 1 to 9 mm and ≥ 10 mm. Method: We introduce a binary polyp-size classification method that estimates a polyp’s three-dimensional spatial information. This estimation is comprised of polyp localisation and depth estimation. The combination of location and depth information expresses a polyp’s three-dimensional shape. In experiments, we quantitatively and qualitatively evaluate the proposed method using 787 polyps of both protruded and flat types. Results: The proposed method’s best classification accuracy outperformed the fine-tuned state-of-the-art image classification methods. Post-processing of sequential voting increased the classification accuracy and achieved classification accuracy of 0.81 and 0.88 for polyps ranging from 1 to 9 mm and others that are ≥ 10 mm. Qualitative analysis revealed the importance of polyp localisation even in polyp-size classification. Conclusions: We developed a binary polyp-size classification method by utilising the estimated three-dimensional shape of a polyp. Experiments demonstrated accurate classification for both protruded- and flat-type polyps, even though the flat type have ambiguous boundary between a polyp and colon wall.

DOI： 10.1007/s11548-021-02477-z

Depth-based branching level estimation for bronchoscopic navigation 査読有り

Wang Cheng, Hayashi Yuichiro, Oda Masahiro, Kitasaka Takayuki, Takabatake Hirotsugu, Mori Masaki, Honma Hirotoshi, Natori Hiroshi, Mori Kensaku

International Journal of Computer Assisted Radiology and Surgery 16 巻 ( 10 ) 頁： 1795 - 1804 2021年8月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：International Journal of Computer Assisted Radiology and Surgery

Purpose: Bronchoscopists rely on navigation systems during bronchoscopy to reduce the risk of getting lost in the complex bronchial tree-like structure and the homogeneous bronchus lumens. We propose a patient-specific branching level estimation method for bronchoscopic navigation because it is vital to identify the branches being examined in the bronchus tree during examination. Methods: We estimate the branching level by integrating the changes in the number of bronchial orifices and the camera motions among the frames. We extract the bronchial orifice regions from a depth image, which is generated using a cycle generative adversarial network (CycleGAN) from real bronchoscopic images. We calculate the number of orifice regions using the vertical and horizontal projection profiles of the depth images and obtain the camera-moving direction using the feature point-based camera motion estimation. The changes in the number of bronchial orifices are combined with the camera-moving direction to estimate the branching level. Results: We used three in vivo and one phantom case to train the CycleGAN model and four in vivo cases to validate the proposed method. We manually created the ground truth of the branching level. The experimental results showed that the proposed method can estimate the branching level with an average accuracy of 87.6%. The processing time per frame was about 61 ms. Conclusion: Experimental results show that it is feasible to estimate the branching level using the number of bronchial orifices and camera-motion estimation from real bronchoscopic images.

DOI： 10.1007/s11548-021-02460-8

Unsupervised colonoscopic depth estimation by domain translations with a Lambertian-reflection keeping auxiliary task 査読有り

深層学習を用いた医用画像処理と画像を超えた応用招待有り

小田昌宏

生体医工学 59 巻 ( 0 ) 頁： 256 - 256 2021年6月

担当区分：筆頭著者記述言語：日本語掲載種別：研究論文（研究会，シンポジウム資料等）出版者・発行元：公益社団法人日本生体医工学会

<p>深層学習は医用画像処理において重要な手法の一つとなった．医用画像処理に含まれる画像分類，検出，セグメンテーションにおいては深層学習が多くの問題を解決し，高い成果を挙げている．画像モダリティに着目すると，眼底画像や放射線画像を対象とした深層学習の適用例が多く，実用化レベルに達したものもある．本発表では，手法や画像モダリティの面から深層学習を用いた医用画像処理を概観し，応用例を紹介する．また，深層学習の新たな応用についても紹介する．深層学習は画像に限らず様々なデータを説明変数と目的変数とすることが可能である．この性質を利用し，画像，テキスト，センサ計測データ等を用いる新たな手法を構築できる．応用例と共に深層学習の新たな可能性について考える．</p>

DOI： 10.11239/jsmbe.Annual59.256

Artificial intelligence-assisted colonic endocytoscopy for cancer recognition: a multicenter study 査読有り

Yuichi Mori , Shin-ei Kudo , Masashi Misawa , Kinichi Hotta , Ohtsuka Kazuo , Shoichi Saito , Hiroaki Ikematsu , Yutaka Saito , Takahisa Matsuda , Takeda Kenichi , Toyoki Kudo , Tetsuo Nemoto , Hayato Itoh , Kensaku Mori

Endoscopy International Open 頁： E1004 - E1011 2021年6月

Itoh Hayato, Oda Masahiro, Mori Yuichi, Misawa Masashi, Kudo Shin-Ei, Imai Kenichiro, Ito Sayo, Hotta Kinichi, Takabatake Hirotsugu, Mori Masaki, Natori Hiroshi, Mori Kensaku

INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERYInternational Journal of Computer Assisted Radiology and Surgery 16 巻 ( 6 ) 頁： 989 - 1001 2021年6月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：International Journal of Computer Assisted Radiology and Surgery

Purpose: A three-dimensional (3D) structure extraction technique viewed from a two-dimensional image is essential for the development of a computer-aided diagnosis (CAD) system for colonoscopy. However, a straightforward application of existing depth-estimation methods to colonoscopic images is impossible or inappropriate due to several limitations of colonoscopes. In particular, the absence of ground-truth depth for colonoscopic images hinders the application of supervised machine learning methods. To circumvent these difficulties, we developed an unsupervised and accurate depth-estimation method. Method: We propose a novel unsupervised depth-estimation method by introducing a Lambertian-reflection model as an auxiliary task to domain translation between real and virtual colonoscopic images. This auxiliary task contributes to accurate depth estimation by maintaining the Lambertian-reflection assumption. In our experiments, we qualitatively evaluate the proposed method by comparing it with state-of-the-art unsupervised methods. Furthermore, we present two quantitative evaluations of the proposed method using a measuring device, as well as a new 3D reconstruction technique and measured polyp sizes. Results: Our proposed method achieved accurate depth estimation with an average estimation error of less than 1 mm for regions close to the colonoscope in both of two types of quantitative evaluations. Qualitative evaluation showed that the introduced auxiliary task reduces the effects of specular reflections and colon wall textures on depth estimation and our proposed method achieved smooth depth estimation without noise, thus validating the proposed method. Conclusions: We developed an accurate depth-estimation method with a new type of unsupervised domain translation with the auxiliary task. This method is useful for analysis of colonoscopic images and for the development of a CAD system since it can extract accurate 3D information.

DOI： 10.1007/s11548-021-02398-x

Extremely imbalanced Subarachnoid Hemorrhage detection based on DenseNet-LSTM network with Class-Balanced Loss and Transfer Learning 査読有り

Deep learning system for automatic detection of bladder tumors in cystoscopic images 査読有り

Mutaguchi J., Oda M., Ueda S., Kinoshita F., Naganuma H., Matsumoto T., Lee K., Monji K., Kashiwagi K., Takeuchi A., Shiota M., Inokuchi J., Mori K., Eto M.

EUROPEAN UROLOGY 79 巻頁： S1022 - S1023 2021年6月

COVID-19診断支援AI開発における名古屋大学の取り組み招待有り

小田昌宏, 鄭通, 林雄一郎, 森健策

Medical Imaging Technology 39 巻 ( 1 ) 頁： 13 - 19 2021年2月

Lu Zhongyang, Oda Masahiro, Hayashi Yuichiro, Hu Tao, Itoh Hayato, Watadani Takeyuki, Abe Osamu, Hashimoto Masahiro, Jinzaki Masahiro, Mori Kensaku

MEDICAL IMAGING 2021: COMPUTER-AIDED DIAGNOSIS 11597 巻 2021年2月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Progress in Biomedical Optics and Imaging - Proceedings of SPIE

Subarachnoid Hemorrhage (SAH) detection is a critical, severe problem that confused clinical residents for a long time. With the rise of deep learning technologies, SAH detection made a significant breakthrough in recent ten years. Whereas, the performances are significantly degraded on imbalanced data, makes deep learning models have always suffered criticism. In this study, we present a DenseNet-LSTM network with Class-Balanced Loss and the transfer learning strategy to solve the SAH detection problem on an extremely imbalanced dataset. Compared to the previous works, the proposed framework not merely effectively integrate greyscale features the and spatial information from the consecutive CT scans, but also employ Class-Balanced loss and transfer learning to alleviate the adverse effects and broaden feature diversity respectively on an extreme SAH cases scarcity dataset, mimicking the actual situation of emergency departments. Comprehensive experiments are conducted on a dataset, consisted of 2,519 cases without hemorrhage cases and only 33 cases with SAH. Experimental results demonstrate the F-measure score of SAH detection achieved a remarkable improvement, the backbone DenseNet121 gained around 33% promotion after transfer learning, and on this basis, importing the Class-Balanced Loss and the LSTM structure, the F-measure score further increased 6.1% and 2.7% sequentially.

DOI： 10.1117/12.2582088