Faculty Profiles - Deguchi Daisuke

Papers - Deguchi Daisuke

Division display >> ／ All the affair displays 1 - 259 of about 259

Semantic-Centric Alignment for Zero-shot Panoptic Segmentation with Limited Data Reviewed Open Access

Jialei Chen, Daisuke Deguchi, Dongyue Li, Xu Zheng, Seigo Ito, Hiroshi Murase, Qi Fan

International Journal of Computer Vision 2026.2

Pedestrians' Gaze Object Estimation in Traffic Scene Considering Relationships Among Multiple Pedestrians Reviewed

Hiroyuki Tatsumi, Daisuke Deguchi, Seigo Ito, Hiroshi Murase

Proceedings of International Workshop on Advanced Image Technology (IWAIT) 2026 2026.1

Split Matching for Inductive Zero-shot Semantic Segmentation Reviewed

Jialei Chen, Xu Zheng, Dongyue Li, CHONG YI, Seigo Ito, Danda Pani Paudel, Luc Van Gool, Hiroshi Murase, Daisuke Deguchi

Proceedings of British Machine Vision Conference (BMVC 2025) 2025.12

Semantic-driven Distillation for Semantic Segmentation with Unknown Classes

Jialei Chen, Dongyue Li, CHONG YI, Xu Zheng, Seigo Ito, Hiroshi Murase, Daisuke Deguchi

Proceedings of 2025 IEEE Conference on Intelligent Transportation Systems (ITSC2025) 2025.11

CQVPR: Landmark-aware Contextual Queries for Visual Place Recognition Reviewed

Dongyue Li, Jialei Chen, Seigo Ito, Hiroshi Murase, Daisuke Deguchi

Proceedings of 2025 IEEE Conference on Intelligent Transportation Systems (ITSC2025) 2025.10

CLIP-to-Seg Distillation for Zero-shot Semantic Segmentation Reviewed Open Access

Jialei Chen, Zhenzhen Quan, Chenkai Zhang, Xu Zheng, Daisuke Deguchi, Hiroshi Murase

IEEE Transactions on Circuits and Systems for Video Technology (TCSVT) 2025.9

Language：English Publishing type：Research paper (scientific journal)

CLIP has greatly advanced zero-shot segmentation by leveraging its strong visual-language association and generalization capability.
However, directly adapting CLIP for segmentation often yields suboptimal results due to inconsistencies between image and pixel-level prediction objectives. Additionally, merely combining segmentation and CLIP models often leads to disjoint optimization, introducing significant computational overhead and additional parameters. To address these issues, we propose a novel CLIP-to-Seg Distillation approach, incorporating global and local distillation to flexibly transfer CLIP’s powerful zero-shot generalization capability to existing closed-set segmentation models. Global distillation leverages CLS tokens to condense segmentation features and distills high-level concepts to the segmentation model via image-level features. Local distillation adapts CLIP's local semantic transferability to dense prediction tasks using object-level features, aided by pseudo-mask generation for latent class mining. To further generalize the CLIP-distilled segmentation model, we generate latent text embeddings for the mined latent classes by coordinating their text embeddings and dense features.
Our method equips existing closed-set segmentation models with strong generalization capabilities for open concepts through effective and flexible CLIP-to-Seg distillation. Without relying on the CLIP model or introducing extra inference overhead, our method seamlessly integrates into existing closed-set segmentation models and enables zero-shot capability, achieving state-of-the-art performance on multiple benchmarks.

DOI： 10.1109/TCSVT.2025.3616588

Open Access

Semantic matters: A constrained approach for zero-shot video action recognition Reviewed

Zhenzhen Quan, Jialei Chen, Daisuke Deguchi, Jie Sun, Chenkai Zhang, Yujun Li, Hiroshi Murase

Pattern Recognition Vol. 162 page： 111402 2025.6

A Cross-Modal Knowledge Distillation Approach for RGB-to-Infrared Video Action Recognition Reviewed

Zhenzhen Quan, Daisuke Deguchi, Jialei Chen, Chenkai Zhang, Yujun Li, Seigo Ito, Hiroshi Murase

Proceedings of Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV2025) 2025.2

Multi-group Vision Semantic Centroid for Semantic Segmentation Reviewed

Jialei Chen, Daisuke Deguchi, Chenkai Zhang, Zhenzhen Quan, Seigo Ito, Hiroshi Murase

Proceedings of Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV2025) 2025.2

Language：English Publishing type：Research paper (international conference proceedings)

Multi-media processing has achieved great success based on
semantic segmentation. Semantic segmentation can be viewed as pixel-
clustering based on semantic prototypes. However, existing methods fo-
cus more on consistent semantics while ignoring the consistency in vi-
sion, making this task challenging. Motivated by the success of discrete
visual representation learning, we propose Multi-group Visual Semantic
Centroid (MVSC) to better cluster the pixels while maintaining consis-
tent semantics of the dense features for any image encoder. Specifically,
we randomly initialize multiple groups of prototypes as multi-groups in
visual space. The visual features are also randomly split into the same
groups and forced to be aligned with the corresponding prototypes. Then
these visual prototypes are projected into the semantic space and super-
vised by the same classifier as the dense features. Compared with existing
methods, MVSC further considers the visual space and thus facilitates
the task. Experimental results on COCO-Stu! show great improvements
compared with previous methods.

CROCODILE: Crop-based Contrastive Discriminative Learning for Enhancing Explainability of End-to-End Driving Models Reviewed

Chenkai Zhang, Daisuke Deguchi, Jialei Chen, Zhenzhen Quan, Hiroshi Murase

Proceedings of ACCV Workshop on Machine Learning and Computing for Visual Semantic Analysis (MLCSA2024) page： 378 - 393 2024.12

Early prediction of At-risk students through Learning-Activity Forecasting Reviewed

Yuya Ozaki, Daisuke Deguchi, Haruya Kyutoku, Hiroshi Murase

Proceedings of 32nd International Conference on Computers in Education (ICCE2024) 2024.11

Toward Explainable End-to-End Driving Models via Simplified Objectification Constraints Reviewed Open Access

Chenkai Zhang, Daisuke Deguchi, Jialei Chen, Hiroshi Murase

IEEE Transactions on Intelligent Transportation Systems Vol. 25 ( 10 ) page： 14521 - 14534 2024.10

Language：English Publishing type：Research paper (scientific journal)

The end-to-end driving models (E2EDMs) convert environmental information into driving actions using a complex transformation which makes E2EDMs have high prediction accuracy. Due to the black-box nature of transformation, the E2EDMs have low explainability. To solve this problem, explanation methods are used to generate explanations for observation. Based on current explanation methods, previous studies tried to further improve the explainability of E2EDMs by integrating an object detection module, however, these methods have many problems: Firstly, due to the requirement of the object detection module, they lack flexibility. Secondly, they neglect an essential property, i.e. , simplicity, to improve explainability. In this paper, since humans prefer object-level and simple explanations in driving tasks, we argue that explainability is decided by two properties which are the objectification degree (the extent to which driving related-object features are utilized) and simplification degree (the simplicity of the explanation), thus we propose Simplified Objectification Branches (SOB) to improve the explainability of E2EDMs. Firstly, this structure could be integrated into any existing E2EDMs and thus have high flexibility. Secondly, the SOB explicitly improves the simplification degree without sacrificing the objectification degree of the explanations. By designing several indicators, i.e. , heatmap satisfaction, driving action reproduction score, deception level, etc. , we proved that SOB could help E2EDMs generate better explanations. Notably, the SOB could also further enhance E2EDMs’ prediction accuracy.

DOI： 10.1109/TITS.2024.3385754

Open Access

Frozen is better than learning: A new design of prototype-based classifier for semantic segmentation Reviewed Open Access

Jialei Chen, Daisuke Deguchi, Chenkai Zhang, Xu Zheng, Hiroshi Murase

Pattern Recognition Vol. 152 2024.8

Language：English Publishing type：Research paper (scientific journal)

Semantic segmentation models comprise an encoder to extract features and a classifier for prediction. However, the learning of the classifier suffers from the ambiguity which is caused by two factors: (1) the weights of a classifier for similar categories may have positive similarities lowing the performance for similar categories, named correlation ambiguity, and (2) the classifier is prone to predict the category with a larger L2 norm and vice versa, termed prior ambiguity. To comedy the issues, we propose Category-Basis Prototype (CBP), frozen and mutually orthogonalized prototypes with equal L2 norm. Orthogonalization prevents the prototypes from being similar to each other and the equality decouples the prediction from the L2 norm. To better shape the feature space, we propose Online Centroid Contrastive Loss (OCCL) equipped with centroid and category- level losses. Experiments show that our method yields compelling results over two widely applied benchmarks indicating the effectiveness of our methods.

DOI： 10.1016/j.patcog.2024.110431

Open Access

Eye-Contact Transformer: Eye-Contact Detection of Distant Pedestrians Considering Scene Context

Ryusei Hata, Daisuke Deguchi, Takatsugu Hirayama, Yasutomo Kawanishi, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J107-D ( 4 ) page： 231 - 242 2024.4

Centroid Module for Shaping Feature Space in Semantic Segmentation

Jialei Chen, Daisuke Deguchi, Chenkai Zhang, Hiroshi Murase

Proceedings of 3rd Asia Conference on Cloud Computing, Computer Vision and Image Processing (3CVIP 2024) 2024.3

Subjective Baggage-Weight Estimation Based on Human Walking Behavior Reviewed Open Access

Masaya Mizuno, Tomohiro Fujita, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase

IEEE Access Vol. 12 page： 39390 - 39398 2024.3

Category-Level Object Pose Estimation in Heavily Cluttered Scenes by Generalized Two-Stage Shape Reconstructor Reviewed Open Access

Hiroki Tatemichi, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEEE Access Vol. 12 page： 33440 - 33448 2024.3

Comprehensive Evaluation of End-to-End Driving Model Explanations for Autonomous Vehicles Reviewed

Chenkai Zhang, Daisuke Deguchi, Jialei Chen, Hiroshi Murase

Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP) 2024 Vol. 2 ( 2184-4321 ) page： 509 - 518 2024.2

Language：English Publishing type：Research paper (international conference proceedings)

Deep learning technology has rapidly advanced, leading to the development of End-to-End driving models (E2EDMs) for autonomous vehicles with high prediction accuracy. To comprehend the prediction results of these E2EDMs, one of the most representative explanation methods is attribution-based. There are two kinds of attribution-based explanation methods: pixel-level and object-level. Usually, the heatmaps illustrate the importance of pixels and objects in the prediction results, serving as explanations for E2EDMs. Since there are many attribution-based explanation methods, evaluation methods are proposed to determine which one is better at improving the explainability of E2EDMs.
Fidelity measures the explanation's faithfulness to the model's prediction method, which is a bottommost property. However, no evaluation method could measure the fidelity difference between object-level and pixel-level explanations, making the current evaluation incomplete. In addition, without considering fidelity, previous evaluation methods may advertise manipulative explanations that solely seek human satisfaction (persuasibility). Therefore, we propose an evaluation method that further considers fidelity, our method enables a comprehensive evaluation that proves the object-level explanations genuinely outperform pixel-level explanations in fidelity and persuasibility, thus could better improve the explainability of the E2EDMs.

Pedestrian's Gaze Object Detection in Traffic Scene Reviewed

Hiroto Murakami, Jialei Chen, Daisuke Deguchi, Takatsugu Hirayama, Yasutomo Kawanishi, Hiroshi Murase

Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP) 2024 Vol. 2 page： 333 - 340 2024.2

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we present a new task of detecting an object that a target pedestrian is gazing at in a traffic scene called PEdestrian's Gaze Object (PEGO). We argue that the detection of gaze object can provide important information for pedestrian's behavior prediction and can contribute to the realization of automated vehicles. For this task, we construct a dataset of in-vehicle camera images with annotations of the objects that pedestrians are gazing at. Also, we propose a Transformer-based method called PEGO Transformer to solve the PEGO detection task. The PEGO Transformer directly performs gaze object detection with the utilization of whole-body features without a high-resolution head image and a gaze heatmap which the traditional methods rely on. Experimental results showed that the proposed method could estimate pedestrian's gaze object accurately even if various objects exist in the scene.

Texture-guided transfer learning for low-quality face recognition Reviewed Open Access

Meng Zhang, Rujie Liu, Daisuke Deguchi, Hiroshi Murase

IEEE Transactions on Image Processing 2023.11

Language：English Publishing type：Research paper (scientific journal)

Although many advanced works have achieved significant progress for face recognition with deep learning and large-scale face datasets, low-quality face recognition remains a challenging problem in real-word applications, especially for unconstrained surveillance scenes. We propose a texture-guided (TG) transfer learning approach under the knowledge distillation scheme to improve low-quality face recognition performance. Unlike existing methods in which distillation loss is built on forward propagation; e.g., the output logits and intermediate features, in this study, the backward propagation gradient texture is used. More specifically, the gradient texture of low-quality images is forced to be aligned to that of its high-quality counterpart to reduce the feature discrepancy between the high-and low-quality images. Moreover, attention is introduced to derive a soft-attention (SA) version of transfer learning, termed as SA-TG, to focus on informative regions. Experiments on the benchmark low-quality face DB’s TinyFace and QMUL-SurFace confirmed the superiority of the proposed method, especially more than 6.3% Rank1 accuracy improvement is achieved on TinyFace.

DOI： 10.1109/TIP.2023.3335830

Open Access

Computational measurement of perceived pointiness from pronunciation Reviewed Open Access

Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide, Takatsugu Hirayama, Yasutomo Kawanishi, Keisuke Doman, Daisuke Deguchi

Multimedia Tools and Applications 2023.8

Refined Objectification for Improving End-to-End Driving Model Explanation Persuasibility Reviewed

Chenkai Zhang, Daisuke Deguchi, Hiroshi Murase

Proceedings of 2023 IEEE Intelligent Vehicles Symposium (IV2023) 2023.6

Estimation of targets' locations and attention degrees by spatio-temporal integration of audiences' facial orientations Reviewed

Kazuma Takeda, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Kunio Kashino

IEICE Transactions on Information and Systems Vol. J106-A ( 3 ) page： 58 - 69 2023.3

End-to-End gaze grounding from a person pictured from behind Reviewed

Hayato Yumiya, Daisuke Deguchi, Yasutomo Kawanishi, Hiroshi Murase

Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications Vol. 5 page： 898 - 905 2023.2

Subjective Baggage-Weight Estimation from Gait: Can You Estimate How Heavy the Person Feels? Reviewed

Masaya Mizuno, Tomohiro Fujita, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase

Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP) 2023 Vol. 5 page： 567 - 574 2023.2

A Preliminary Study on View Independent Panoptic Scene Change Detection Reviewed

Jiaxin Li, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase

Proceedings of International Workshop on Advanced Image Technology (IWAIT) 2023 2023.1

More Persuasive Explanation Method for End-to-End Driving Models Reviewed Open Access

Chenkai Zhang, Daisuke Deguchi, Yuki Okafuji, Hiroshi Murase

IEEE Access Vol. 11 page： 4270 - 4282 2023.1

Language：English Publishing type：Research paper (scientific journal)

With the rapid development of autonomous driving technology, a variety of high-performance end-to-end driving models (E2EDMs) are being proposed. In order to understand the computational methods of E2EDMs, pixel-level explanations methods are used to obtain the explanations of the E2EDMs. However, little attention has been paid to the excellence of the explanations of E2EDMs. Therefore, in order to build trustworthy E2EDMs, we focus on improving the persuasibility of the explanations of E2EDMs. We propose an object-level explanation method (main approach) for E2EDMs, which masks the objects in the image and then treats the change in the prediction result as the importance of the objects, then we explain the E2EDM by the importance of each object. To further validate the effectiveness of object-level explanations, we propose another approach (validation approach), which trains E2EDMs with object information as input and generates the importance of objects using general explanation methods. Both approaches generate object-level explanations, in order to compare these object-level explanations with traditional pixel-level explanations, we propose experimental methods to measure the persuasibility of explanations of E2EDMs through a subjective and objective method. The subjective method evaluates persuasibility based on the extent to which participants think the importance of features indicated by the explanations is correct. The objective method evaluates the persuasibility based on the human annotation similarity between provided with only the important part of images and provided with the complete images. The experimental results show that the object-level explanations are more persuasive than the traditional pixel-level explanations.

DOI： 10.1109/ACCESS.2023.3235739

Open Access

Analysis of Gaze while Viewing Videos at Various Playback Speeds Reviewed

Hideaki Takahira, Yasutomo Kawanishi, Takatsugu Hirayama, Hiroshi Murase, Daisuke Deguchi

Proceedings of the International Display Workshops Vol. 29 page： 702 - 705 2022.12

Detection of Birds in a 3D Environment Referring to Audio-Visual Information Reviewed

Yasutomo Kawanishi, Ichiro Ide, Baidong Chu, Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Daisuke Deguchi

Proceedings of the 18th IEEE International Conference on Advanced Video and Signal-Based Surveillance page： 1 - 7 2022.11

Language：English Publishing type：Research paper (international conference proceedings)

We propose a method to detect birds in a 3D environment referring to both audio information observed from a microphone array and visual information observed from a panorama camera. In general, in panorama images, birds appear relatively too small to be detected accurately even with the state-of-the-art deep learning models. Thus, the proposed method takes a two step approach where the birds are first roughly located referring to audio information by Sound Source Localization (SSL), and then image detection is applied within its vicinity. Through evaluation on a dataset annotated with bounding boxes surrounding the birds, we show that the proposed method improves detection performance of birds that appear in relatively small sizes in the image, in both accuracy and processing speed.

DOI： 10.1109/AVSS56176.2022.9959510

Detection of distant eye-contact using spatio-temporal pedestrian skeletons Reviewed

Ryusei Hata, Daisuke Deguchi, Takatsugu Hirayama, Yasutomo Kawanishi, Hiroshi Murase

2022.9

Analysis of Gaze Point Distribution while Viewing Surround-Sound Videos Reviewed Open Access

Hideaki Takahira, Miho Shinohara, Yusuke Nosaka, Masaya Yokouchi, Mitsuho Yamada, Takatsugu Hirayama, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

The Transactions of Human Interface Society Vol. 24 ( 3 ) page： 195 - 204 2022.8

Two-Stage Input-Output Hidden Markov Model for Car-Pedestrian Interaction Behavior Modeling Reviewed Open Access

Fumito Shinmura, Naoki Akai, Takatsugu Hirayama, HaiLong Liu, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase

Journal of Information Processing Vol. 63 ( 8 ) page： 1371 - 1382 2022.8

Intuitive Gait Modeling using Mimetic-Words for Gait Description and Generation Reviewed

Hirotaka Kato, Takatsugu Hirayama, Keisuke Doman, Ichiro Ide, Yasutomo Kawanishi, Takahiro Komamizu, Daisuke Deguchi, Hiroshi Murase

page： 240 - 245 2022.8

Context-Aware Contribution Estimation for Feature Aggregation in Video Face Recognition Reviewed Open Access

Meng Zhang, Rujie Liu, Daisuke Deguchi, Hiroshi Murase

IEEE Access Vol. 10 page： 79301 - 79310 2022.7

Language：English Publishing type：Research paper (scientific journal)

The difficulties in video-based face recognition, such as dramatic pose variations and low quality, can be alleviated by leveraging the rich complementary information between the frames. However, limited by the mini-batch training strategy, the current deep learning methods only utilizes the frames in each batch during training, which ignore the context of the entire video. In this paper, we propose a context-aware feature aggregation scheme to aggregate complementary information between different frames. Firstly, a two-branch structure is designed as the C ontext-aware feature A ggregation N etwork ( CAN ). Secondly, a context-aware training strategy using a context bank is proposed, which alleviates the limitation of mini-batch samples by using the context of the entire video or several images belonging to the same ID and thus achieves global contribution estimation result. Comparative studies on benchmark datasets, such as IJB-C, YouTube Face (YTF), PaSC and COX, confirm that the proposed approach can achieve state-of-the-art level. Meanwhile, qualitative analysis on Multi-PIE dataset indicates that the contribution learned by the CAN is reasonable and beneficial to video face recognition.

DOI： 10.1109/ACCESS.2022.3193787

Open Access

SfM-Student: Semantic Segmentation of Train Front-View Images with SfM Data Augmentation Reviewed

Yuki Furitsu, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase, Hiroki Mukoujima, Nozomi Nagamine

IEICE Transactions on Information and Systems Vol. J105-A ( 4 ) page： 48 - 57 2022.4

Safety Margin Prediction for Pedestrians Using Risk Potential Function Considering Pedestrian Attributes and Driver Skill Reviewed

Yamato Maekawa, Takatsugu Hirayama, Yasutomo Kawanishi, Daisuke Deguchi, HaiLong Liu, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J105-A ( 3 ) page： 38 - 42 2022.3

Masked face recognition with mask transfer and self-attention under the COVID-19 pandemic Reviewed Open Access

Zhang Meng, Rujie Liu, Daisuke Deguchi, Hiroshi Murase

IEEE Access Vol. 10 page： 20527 - 20538 2022.2

Language：English Publishing type：Research paper (scientific journal)

Face masks bring a new challenge to face recognition systems especially against the background of the COVID-19 pandemic. In this paper, a method mitigating the negative effects of mask defects on face recognition is proposed. Firstly, a low-cost, accurate method of masked face synthesis, i.e. mask transfer, is proposed for data augmentation. Secondly, an attention-aware masked face recognition (AMaskNet) is proposed to improve the performance of masked face recognition, which includes two modules: a feature extractor and a contribution estimator. Therein, the contribution estimator is employed to learn the contribution of the feature elements, thus achieving refined feature representation by simple matrix multiplications. Meanwhile, the end-to-end training strategy is utilized to optimize the entire model. Finally, a mask-aware similarity matching strategy(MS) is taken to improve the performance in the inference stage. The experiments show that the proposed method consistently outperforms on three masked face recognition datasets: RMFRD [1], COX [2] and Public-IvS [3]. Meanwhile, qualitative analysis experiments using CAM [4] indicate that the contribution learned by AMaskNet is more conducive to masked face recognition.

DOI： 10.1109/ACCESS.2022.3150345

Open Access

Active Learning for Human Pose Estimation based on Temporal Pose Continuity Reviewed

Taro Mori, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase, Tetsuo Inoshita

Proceedings of International Workshop on Advanced Image Technology (IWAIT) 2022 2022.1

Soft-boundary label relaxation with class placement constraints for semantic segmentation of the railway environment Reviewed

Yuki Furitsu, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase, Hiroki Mukojima, Nozomi Nagamine

Pattern Recognition Letters Vol. 150 page： 258 - 264 2021.10

Pointedness of an Image: Measuring How Pointy an Image is Perceived Reviewed

Chihaya Matsuhira, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase

Proceedings of HCII 2021 Vol. 37 page： 137 - 144 2021.7

Language：English Publishing type：Research paper (international conference proceedings)

For computers to understand human perception, metrics that can capture human perception well are important. However, there are few metrics that characterize the visual perception of humans towards images. Therefore, in this paper, we propose a novel concept and a metric of pointedness of an image, which describes how pointy an image is perceived. The algorithm is inspired by the Features from Accelerated Segment Test (FAST) algorithm for corner detection which looks on the number of continuous neighboring darker pixels surrounding each pixel. We assume that this number would be proportional to the perceived pointedness in the region around the pixel. We evaluated our method towards how well it could capture the human perception of images. To compare the method with similar metrics that describe shapes, we prepared silhouette images of both artificial shapes and natural objects. The results showed that the proposed method gave nearly equivalent perceptual performance to other metrics and also worked in a larger variety of images.

DOI： https://doi.org/10.1007/978-3-030-78635-9_20

Best next-viewpoint recommendation by selecting minimum pose ambiguity for category-level object pose estimation Reviewed Open Access

Nik Mohd Zarifie Hashim, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Journal of the Japan Society for Precision Engineering Vol. 87 ( 5 ) page： 440 - 446 2021.5

Odometry estimation from sparse LiDAR point cloud constrained by image feature correspondence Reviewed Open Access

Masayuki Shimizu, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Journal of the Japan Society for Precision Engineering Vol. 87 ( 5 ) page： 447 - 454 2021.5

Proposal of a Light-Field Descriptor Considering Light-Ray Direction Reviewed Open Access

Masayuki Shimizu, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Journal of the Japan Society for Precision Engineering Vol. 87 ( 2 ) page： 197 - 205 2021.2

ColAtt-Net: In Reducing the Ambiguity of Pedestrian Orientations on Attribute-Aware Semantic Segmentation Task Reviewed

Mahmud Dwi Sulistiyo, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Takatsugu Hirayama, Hiroshi Murase

IEEJ Transactions on Electrical and Electronic Engineering Vol. 16 ( 2 ) page： 295 - 306 2021.2

Language：English Publishing type：Research paper (scientific journal)

Semantic segmentation has become one of the trending topics in the world of computer vision and deep learning. Recently, due to an increasing demand to solve a semantic segmentation task simultaneously with attribute recognition of objects, a new task named attribute-aware semantic segmentation has been introduced. Since the task requires to handle pixel-wise object class estimation with its attributes such as a pedestrian’s body orientation, previous works had difficulties to handle ambiguous attributes such as body orientations in object-level, especially when segmenting the pedestrians with their attributes correctly. This paper proposes the ColAtt-Net that is an attribute-aware semantic segmentation model augmented by a column-wise mask branch to predict the pedestrians’ orientations in the horizontal perspective of the input image. We firmly assume that the pedestrians captured by a car-mounted camera are distributed horizontally so that for each column of the input image, the pedestrian pixels can be labeled with one orientation uniformly. In the proposed method, we split the output of the base semantic segmentation model into two branches; one branch for segmenting the object categories, while the other one, as the novel column-wise attribute branch, is to map the recognition of pedestrian’s orientations that are distributed horizontally. This method successfully enhances the performance of attribute-aware semantic segmentation by reducing the ambiguity on segmenting the pedestrian’s orientation. Improvements on the pedestrian orientation segmentation are confidently shown by the proposed method in the experimental results, both in quantitative and qualitative views. This paper also discusses how the improved performance becomes an advantage in the autonomous driving system.

DOI： https://doi.org/10.1002/tee.23296

LFIR2Pose: Pose Estimation from an Extremely Low-Resolution FIR Image Sequence Reviewed

Saki Iwata, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Tomoyoshi Aizawa

Proceedings of the 25th International Conference on Pattern Recognition 2021.1

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we propose a method for human pose estimation from a Low-resolution Far-InfraRed (LFIR) image sequence captured by a 16 × 16 FIR sensor array. Human body estimation from such a single LFIR image is a hard task. For training the estimation model, annotation of the human pose to the images is also a difficult task for human. Thus, we propose the LFIR2Pose model which accepts a sequence of LFIR images and outputs the human pose of the last frame, and also propose an automatic annotation system for the model training. Additionally, considering that the scale of human body motion is largely different among body parts, we also propose a loss function focusing on the difference. Through an experiment, we evaluated the human pose estimation accuracy using an original data set, and confirmed that human pose can be estimated accurately from an LFIR image sequence.

OMEga-GAN: Object Manifold Embedding GAN for Image Generation by Disentangling Parameters into Pose and Shape Manifolds Reviewed

Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of the 25th International Conference on Pattern Recognition page： 7945 - 7952 2021.1

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we propose Object Manifold Embedding GAN (Ω-GAN) to generate images of variously shaped and arbitrarily posed objects from a noise variable sampled from a distribution defined over the pose and the shape manifolds in a vector space. We introduce Parametric Manifold Sampling to sample noise variables from a distribution over the pose manifold to conditionally generate object images in arbitrary poses by tuning the pose parameter. We also introduce Object Identity Loss for clearly disentangling the pose and shape parameters, which allows us to maintain the shape of the object instance when only the pose parameter is changed. Through evaluation, we confirmed that the proposed Ω-GAN could generate variously shaped object images in arbitrary poses by changing the pose and shape parameters independently. We also introduce an application of the proposed method for object pose estimation, through which we confirmed that the object poses in the generated images are accurate.

Median-Shape Representation Learning for Category-Level Object Pose Estimation in Cluttered Environments Reviewed

Hiroki Tatemichi, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Ayako Amma, Hiroshi Murase

Proceedings of the 25th International Conference on Pattern Recognition page： 4473 - 4480 2021.1

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we propose an occlusion-robust pose
estimation method of an unknown object instance in an
object category from a depth image. In a cluttered
environment, objects are often occluded mutually. For
estimating the pose of an object in such a situation, a
method that de-occludes the unobservable area of the object
would be effective. However, there are two difficulties;
occlusion causes offset between the actual object center
and the center of its observable area, and different
instances in a category may have different shapes. To cope
with these difficulties, we propose a two-stage
Encoder-Decoder model to extract features with objects
whose centers are aligned to the image center. In the
model, we also propose the Median-shape Reconstructor as
the second stage to absorb shape variations in a category.
By evaluating the method with both a large-scale virtual
dataset and a real dataset, we confirmed the proposed
method achieves good performance on pose estimation of an
occluded object from a depth image.

Human Skeleton Estimation Method Focusing on the Difference of Human Position and Action from an Extremely Low-Resolution FIR Image Sequence Reviewed Open Access

Saki Iwata, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Tomoyoshi Aizawa

Journal of the Japan Society for Precision Engineering Vol. 87 ( 1 ) page： 99 - 106 2021.1

Imageability Estimation using Visual and Language Features Reviewed

Chihaya Matsuhira, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase

Proceedings of the 2020 International Conference on Multimedia Retrieval page： 306–310 2020.10

Language：English Publishing type：Research paper (international conference proceedings)

Imageability is a concept from Psycholinguistics quantizing the human perception of words. However, existing datasets are created through subjective experiments and are thus very small. Therefore, methods to automatically estimate the imageability can be helpful. For an accurate automatic imageability estimation, we extend the idea of a psychological hypothesis called Dual-Coding Theory, that discusses the connection of our perception towards visual information
and language information, and also focus on the relationship between the pronunciation of a word and its imageability. In this research, we propose a method to estimate imageability of words
using both visual and language features extracted from corresponding data. For the estimation, we use visual features extracted from low- and high-level image features, and language features extracted from textual features and phonetic features of words. Evaluations how that our proposed method can estimate imageability more accurately than comparative methods, implying the contribution of each feature to the imageability.

DOI： 10.1145/3372278.3390731

Modeling Eye-Gaze Behavior of Electric Wheelchair Drivers via Inverse Reinforcement Learning Reviewed

Yamato Maekawa, Naoki Akai, Takatsugu Hirayama, Luis Yoichi Morales, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Proceedings of 2020 IEEE International Conference on Intelligent Transportation Systems page： 158 - 164 2020.9

Language：English Publishing type：Research paper (international conference proceedings)

It is intuitively obvious that eye-gaze behaviors of experienced drivers are different from those of novice drivers. However, it is not easy to understand the difference in their behavior quantitatively. In this work, we present an explainable eye-gaze behavior modeling method for electric wheelchair drivers based on Inverse Reinforcement Learning (IRL). We first create feature maps that represent risk factors during driving. These feature maps are able to represent not only to what but also from where drivers pay attention. IRL uses the feature maps to learn the reward representing the eye-gaze behaviors and allows us to see important features via the automatic acquisition of the reward. Through analysis of the learned model, we show quantitative evidence that eye-gaze behaviors of experienced drivers are better-balanced by paying attention to multiple risks.

DOI： 10.1109/ITSC45102.2020.9294255

Estimating the imageability of words by mining visual characteristics from crawled image data Reviewed Open Access

Marc Aurel Kastner, Ichiro Ide, Frank Nack, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase

Multimedia Tools and Applications Vol. 79 ( 25 ) page： 18167 - 18199 2020.7

Performance Boost of Attribute-aware Semantic Segmentation via Data Augmentation for Driver Assistance Reviewed

Mahmud Dwi Sulistiyo, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Takatsugu Hirayama, Hiroshi Murase

page： 293 - 298 2020.6

Language：English Publishing type：Research paper (international conference proceedings)

This paper is an extension of our work in developing an attribute-aware semantic segmentation method which focuses on pedestrian understanding in a traffic scene. Recently, the trending topic of semantic segmentation has been expanded to be able to collaborate with the object’s attributes recognition task; Here, it refers to recognizing a pedestrian’s body orientation. The attribute-aware semantic segmentation can be more beneficial for driver assistance compared to the conventional semantic segmentation because it can provide a more informative output to the system. In this paper, we conduct a study of the data augmentation usage as an effort to enhance the performance of the attribute-aware semantic segmentation task. The experiments show that the proposed method in augmenting the training data is able to improve the model’s performance. We also demonstrate some of qualitative results and discuss the benefits to a driver assistance system.

DOI： 10.1109/ICoICT49345.2020.9166219

Estimation of the Number of Vehicles by Regression Based on Parts Detection Reviewed Open Access

Fumito Shinmura, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Vol. 32 ( 3 ) page： 705 - 712 2020.6

Simultaneous Image Matching for Person Re-identification via the Stable Marriage Algorithm Reviewed

Nik Mohd Zarifie Hashim, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEEJ Transactions ON Electrical AND Electronic Engineering Vol. 15 ( 6 ) page： 909 - 917 2020.4

SOANets: Encoder-Decoder based Skeleton Orientation Alignment Network for White Cane User Recognition from 2D Human Skeleton Sequence Reviewed

Naoki Nishida, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Jun Piao

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2020 Vol. 5 page： 435 - 443 2020.2

Language：English Publishing type：Research paper (international conference proceedings)

In recent years, various facilities have been deployed to support visually impaired people. However, accidents caused by visual disabilities still occur. In this paper, to support the visually impaired people in public areas, we aim to identify whether a pedestrian image sequence obtained by a surveillance camera indicates the presence of a white cane user by analyzing the temporal transition of a human skeleton represented as 2D coordinates. Our previously proposed method aligns the orientation of the human skeletons to various orientations and identifies a white cane user from the corresponding sequences, relying on multiple classifiers related to each orientation. The method employs an exemplar-based approach to perform the alignment. However, it heavily depends on the number of exemplars and consumes excessive memory. In this paper, we propose a method to align 2D human skeleton representation sequence to various orientations using the proposed Skeleton Orientation Alignment Networks (SOANets) based on an encoder-decoder model. Using SOANets, we can obtain 2D skeleton representation sequences aligned to various orientations, extract richer skeleton features, and recognize white cane users accurately. We conducted an experiment to confirm that the
proposed method improves the recognition rate by 16%, compared to the method that does not use the skeleton
orientation alignment.

Development and Evaluation of "KamiRepo" Web Service with Return of Handwritten Assignments via LMS Reviewed Open Access

Shigeki Ohira, Shunya Seiya, Ryuya Ito, Kosuke Okamoto, Ukyo Tanikawa, Daisuke Deguchi, Tomoki Toda

Vol. 6 ( 1 ) page： 52 - 68 2020.2

Occlusion-Aware Skeleton Trajectory Representation for Abnormal Behavior Detection Reviewed

Onur Temuroglu, Yasutomo Kawanishi, Daisuke Deguchi, Takatsugu Hirayama, Ichiro Ide, Hiroshi Murase, Mayuu Iwasaki, Atsushi Tsukada

2020.2

Language：English Publishing type：Research paper (international conference proceedings)

Surveillance cameras are expected to play a large role in the development of ITS technologies. They can be used to detect abnormally behaving individuals which can then be reported to drivers nearby. There are multiple works that tackle the problem of abnormal behavior detection. However, most of these works make use of appearance features which have redundant information and are susceptible to noise. While there are also works that make use of pose skeleton representation, they do not consider well how to handle cases with occlusions, which can occur due to the simple reason of pedestrian orientation preventing some joints from appearing in the frame clearly. In this paper, we propose a skeleton trajectory representation that enables handling of occlusions. We also propose a framework for pedestrian abnormal behavior detection that uses the proposed representation and detect relatively hard-to-notice anomalies such as drunk walking. The experiments we conducted show that our method outperforms other representation methods.

DOI： 10.1007/978-981-15-4818-5_9

Browsing Visual Sentiment Datasets Using Psycholinguistic Groundings Open Access

Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase

Lecture Note in Computer Science Vol. 11962 page： 697 - 702 2020.1

Language：English Publishing type：Research paper (international conference proceedings)

Recent multimedia applications commonly use text and imagery from Social Media for tasks related to sentiment research. As such, there are various image datasets for sentiment research for popular classification tasks. However, there has been little research regarding the relationship between the sentiment of images and its annotations from a multi-modal standpoint. In this demonstration, we built a tool to visualize psycholinguistic groundings for a sentiment dataset. For each image, individual psycholinguistic ratings are computed from the image's metadata. A sentiment-psycholinguistic spatial embedding is computed to show a clustering of images across different classes close to human perception. Our interactive browsing tool can visualize the data in various ways, highlighting different psycholinguistic groundings with heatmaps.

DOI： 10.3758/s13428-018-1099-3

Open Access

More-Natural Mimetic Words Generation for Fine-Grained Gait Description Reviewed

Hirotaka Kato, Takatsugu Hirayama, Ichiro Ide, Keisuke Doman, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase

Lecture Note in Computer Science Vol. 11962 page： 214 - 225 2020.1

Language：English Publishing type：Research paper (international conference proceedings)

A mimetic word is used to verbally express the manner of a phenomenon intuitively. The Japanese language is known to have a greater number of mimetic words in its vocabulary than most other languages. Especially, since human gaits are one of the most commonly represented behavior by mimetic words in the language, we consider that it should be suitable for labels of fine-grained gait recognition. In addition, Japanese mimetic words have a more decomposable structure than these in other languages such as English. So it is said that they have sound-symbolism and their phonemes are strongly related to the impressions of various phenomena. Thanks to this, native Japanese speakers can express their impressions on them briefly and intuitively using various mimetic words. Our previous work proposed a framework to convert the body-parts movements to an arbitrary mimetic word by a regression model. The framework introduced a phonetic space" based on sound-symbolism, and it enabled fine-grained gait description using the generated mimetic words consisting of an arbitrary combination of phonemes. However, this method did not consider the "naturalness" of the description. Thus, in this paper, we propose an improved mimetic word generation module considering its naturalness, and update the description framework. Here, we define the co-occurrence frequency of phonemes composing a mimetic word as the naturalness. To investigate the co-occurrence frequency, we collected many mimetic words through a subjective experiment. As a result of evaluation experiments, we confirmed that the proposed module could describe gaits with more natural mimetic words while maintaining the description accuracy."

DOI： 10.1007/978-3-030-37734-2_18

Attribute-Aware Loss Function for Accurate Semantic Segmentation Considering the Pedestrian Orientations Reviewed Open Access

Mahmud Dwi Sulistiyo, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Takatsugu Hirayama, Jiang-Yu Zheng, Hiroshi Murase

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences Vol. E103-A ( 1 ) page： 231 - 242 2020.1

Language：English Publishing type：Research paper (scientific journal)

Numerous applications such as autonomous driving, satellite imagery sensing, and biomedical imaging use computer vision as an important tool for perception tasks. For Intelligent Transportation Systems (ITS), it is required to precisely recognize and locate scenes in sensor data. Semantic segmentation is one of computer vision methods intended to perform such tasks. However, the existing semantic segmentation tasks label each pixel with a single object’s class. Recognizing object attributes, e.g., pedestrian orientation, will be more informative and help for a better scene understanding. Thus, we propose a method to perform semantic segmentation with pedestrian attribute recognition simultaneously. We introduce an attribute-aware loss function that can be applied to an arbitrary base model. Furthermore, a re-annotation to the existing Cityscapes dataset enriches the ground-truth labels by annotating the attributes of pedestrian orientation. We implement the proposed method and compare the experimental results with others. The attribute-aware semantic segmentation shows the ability to outperform baseline methods both in the traditional object segmentation task and the expanded attribute detection task.

DOI： 10.1587/transfun.2019TSP0001

Analysis of Effective Flicker Light Patterns to Improve the Pedestrian Detectability from a Driver for an Intelligent Headlight System Reviewed Open Access

Takashi Maeda, Takatsugu Hirayama, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Journal of the Japan Society for Precision Engineering Vol. 85 ( 12 ) page： 1157 - 1162 2019.12

Cyclist Recognition via Size-Adaptable PointNet Reviewed Open Access

Taiki Yamamoto, Fumito Shinmura, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Kazuki Kato, Hiroshi Murase

Journal of the Japan Society for Precision Engineering Vol. 85 ( 12 ) page： 1117 - 1126 2019.12

Scene-Adaptive Driving Area Prediction based on Automatic Label Acquisition from Driving Information Reviewed

Takuya Migishima, Haruya Kyutoku, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Proceedings of the 5th IAPR Asian Conference on Pattern Recognition (ACPR2019) 2019.11

Semantic Segmentation of Railway Images Considering Temporal Continuity Reviewed

Yuki Furitsu, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase, Hiroki Mukoujima, Nozomi Nagamine

Proceedings of the 5th IAPR Asian Conference on Pattern Recognition (ACPR2019) 2019.11

An Analysis of How Driver Experience Affects Eye-Gaze Behavior for Robotic Wheelchair Operation Reviewed

Yamato Maekawa, Naoki Akai, Takatsugu Hirayama, Luis Yoichi Morales, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Proceedings of 2019 IEEE International Conference on Computer Vision (ICCV) Workshops page： 4443 - 4451 2019.11

Language：English Publishing type：Research paper (international conference proceedings)

Drivers obtain information on surrounding environment using their eyesights. Experienced eye-gaze behavior is needed when driving at places where multiple risks exist to prepare for and avoid them. In this work, we analyze the change in eye-gaze behavior in such situations while a driver gains experience on the operation of a robotic wheelchair. Accurate distance information in the traffic environment is important to analyze the eye-gaze behavior. However, almost all previous works analyze eye-gaze behavior in a 2D environment, so they could not obtain accurate distance information. For this reason, we analyze eye-gaze behavior in 3D space. Concretely, we developed a novel eye-gaze behavior analysis platform based on a robotic wheelchair and estimated the driver's attention in 3D space. We try to analyze the eye-gaze behavior considering a useful field-of-view in 3D space based on the distance information instead of only the fixation point to investigate the objects that a driver implicitly pays attention to and from where s/he focuses on them. Results show that novice drivers pay attention to a single risk at a time. In contrast, they pay more attention to multiple risks simultaneously as they gain experience. Additionally, we discuss what features are effective to model the eye-gaze behavior based on the results.

DOI： 10.1109/ICCVW.2019.00545

Exemplar-based Pseudo-Viewpoint Rotation for White-Cane User Recognition from a 2D Human Pose Sequence Reviewed

Naoki Nishida, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Jun Piao

2019.9

Similar Seasonal-Geo-Region Mining based on Visual Concepts in Social Media Photos Reviewed

Yasutomo Kawanishi, Ichiro Ide, Chen Lu, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase

Proceedings of the 5th IEEE Conference on Multimedia BigData page： 86 - 93 2019.9

Estimation of the attractiveness of food photography based on image features Reviewed Open Access

Kazuma Takahashi, Tatsumi Hattori, Keisuke Doman, Yasutomo Kawanishi, Takatsugu Hirayama, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

IEICE Trans. on Information and Systems Vol. E102-D ( 8 ) page： 1590 - 1593 2019.8

Hand Orientation Estimation in Probability Density Form

Kazuaki Kondo, Daisuke Deguchi, Atsushi Shimada

2019.6

Estimating the Imageability of a sentence for image caption evaluation

Kazuki Umemura, Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase

2019.4

Estimating the visual variety of concepts by referring to Web popularity Reviewed

Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase

Multimedia Tools and Applications Vol. 78 ( 7 ) page： 9463 - 9488 2019.4

Language：English Publishing type：Research paper (scientific journal)

Increasingly sophisticated methods for data processing demand knowledge on the semantic relationship between language and vision. New fields of research like Explainable AI demand to step away from black-boxed approaches and understanding how the underlying semantics of data sets and AI models work. Advancements in Psycholinguistics suggest, that there is a relationship from language perception to how language production and sentence creation work. In this paper, a method to measure the visual variety of concepts is proposed to quantify the semantic gap between vision and language. For this, an image corpus is recomposed using ImageNet and Web data. Web-based metrics for measuring the popularity of sub-concepts are used as a weighting to ensure that the image composition in a dataset is as natural as possible. Using clustering methods, a score describing the visual variety of each concept is determined. A crowd-sourced survey is conducted to create ground-truth values applicable for this research. The evaluations show that the recomposed image corpus largely improves the measured variety compared to previous datasets. The results are promising and give additional knowledge about the relationship of language and vision.

DOI： 10.1007/s11042-018-6528-x

Person Re-Detection from an In-Vehicle Camera Video Referring to Observations from Other Vehicles Open Access

Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

The Journal of the Institute of Image Electronics Engineers of Japan Vol. 48 ( 2 ) page： 273 - 277 2019.4

Summarization of multiple news videos considering the consistency of audio-visual contents Reviewed

Ye Zhang, Ryunosuke Tanishige, Ichiro Ide, Keisuke Doman, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase

International Journal of Semantic Computing Vol. 13 ( 1 ) page： 135 - 155 2019.3

Hard Negative Mining from in-Vehicle Camera Images based on Multiple Observations of Background Patterns Reviewed Open Access

Masashi Hontani, Haruya Kyutoku, David Robert Wong, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2019 Vol. 5 page： 435 - 442 2019.2

Language：English Publishing type：Research paper (international conference proceedings)

In recent years, the demand for highly accurate pedestrian detectors has increased due to the development of advanced driving support systems.
For the training of an accurate pedestrian detector, it is important to collect a large number of training samples.
To support this, this paper proposes a ``hard negative'' mining method to automatically extract background images which tend to be erroneously detected as pedestrians.
Negative samples are selected based on the assumption that frequent patterns observed multiple times in the same location are most likely parts of the background scene.
As a result of an evaluation using in-vehicle camera images captured along the same route, we confirmed that the proposed method can automatically collect false positive samples accurately.
We also confirmed that a highly accurate detector can be constructed using the additional negative samples.

DOI： 10.5220/0007376804350442

Open Access

Next Viewpoint Recommendation by Pose Ambiguity Minimization for Accurate Object Pose Estimation Reviewed Open Access

Nik Mohd Zarifie Hashim, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Ayako Amma, Norimasa Kobori

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2019 Vol. 5 page： 60 - 67 2019.2

Language：English Publishing type：Research paper (international conference proceedings)

3D object pose estimation by using a depth sensor is one of the important tasks in activities by robots. To reduce the pose ambiguity of an estimated object pose, several methods for multiple viewpoint pose estimation have been proposed. However, these methods need to select the viewpoints carefully to obtain better results. If the pose of the target object is ambiguous from the current observation, we could not decide where we should move the sensor to set as the next viewpoint. In this paper, we propose a best next viewpoint recommendation method by minimizing the pose ambiguity of the object by making use of the current pose estimation result as a latent variable. We evaluated viewpoints recommended by the proposed method and confirmed that it helps us to gain better pose estimation results than several comparative methods on a synthetic dataset.

DOI： 10.5220/0007366700600067

Open Access

Pedestrian Intensive Scanning for Active-scan LIDAR Reviewed

Taiki Yamamoto, Fumito Shinmura, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2019 Vol. 5 page： 313 - 320 2019.2

Language：English Publishing type：Research paper (international conference proceedings)

In recent years, LIDAR is playing an important role as a sensor for understanding environments of a vehicle’s surroundings. Active-scan LIDAR is being actively developed as a LIDAR that can control the laser irradiation direction arbitrary and rapidly. In comparison with conventional uniform-scan LIDAR (e.g. Velodyne HDL-64e), Active-scan LIDAR enables us to densely scan even distant pedestrians. In addition, if appropriately controlled, this sensor has the potential to reduce unnecessary laser irradiations towards non-target objects. Although there are some preliminary studies on pedestrian scanning strategy for Active-scan LIDARs, in the best of our knowledge, an efficient method has not been realized yet. Therefore, this paper proposes a novel pedestrian scanning method based on orientation aware pedestrian likelihood estimation using the orientation-wise pedestrian’s shape models with local distribution of measured points. To evaluate the effectiveness of the proposed method, we conducted experiments by simulating Active-scan LIDAR using point-clouds from the KITTI dataset. Experimental results showed that the proposed method outperforms conventional methods.

DOI： 10.5220/0007359903130320

Voting-based Hand-Waving Gesture Spotting from a Low-Resolution Far-Infrared Image Sequence Reviewed

Yasutomo Kawanishi, Chisato Toriyama, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Tomoyoshi Aizawa, Masato Kawade

Proceedings of 2018 IEEE International Conference on Visual Communications and Image Processing (VCIP2018) 2018.12

Language：English Publishing type：Research paper (international conference proceedings)

We propose a temporal spotting method of a hand gesture from a low-resolution far-infrared image sequence captured by a far-infrared sensor array. The sensor array captures the spatial distribution of far-infrared intensity as a thermal image by detecting far-infrared waves emitted from heat sources. It is difficult to spot a hand gesture from a sequence of thermal images captured by the sensor due to its low-resolution, heavy noise, and varying duration of the gesture. Therefore, we introduce a voting-based approach to spot the gesture with template matching-based gesture recognition. We confirm the effectiveness of the proposed temporal spotting method in several settings.

DOI： 10.1109/VCIP.2018.8698650

Gaze-inspired Learning for Estimating the Attractiveness of a Food Photo Reviewed

Akinori Sato, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of 20th IEEE Int. Symposium on Multimedia (ISM2018) page： 36 - 43 2018.12

Language：English Publishing type：Research paper (international conference proceedings)

The number of food photos posted to the Web has been increasing. Most of the users prefer to post delicious-looking food photos. They, however, do not always look delicious. A previous work proposed a method for estimating the attractiveness of food photos, that is, the degree of how much a food photo looks delicious, as an assistive technology for taking a delicious-looking food photo. This method extracted image features from the entire food photo to evaluate the impression. In our work, we conduct a preference experiment where subjects are asked to compare a pair of food photos and measure their gaze. The proposed method extracts image features from local regions selected based on the gaze information and estimates the attractiveness of a food photo by learning regression parameters. Experimental results showed the effectiveness of extracting image features from outside the gaze regions rather than inside them.

DOI： 10.1109/ISM.2018.00015

Action Recognition Using a Far-Infrared Sensor Array by Convolutional RNN Reviewed Open Access

Takayuki Kawashima, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Tomoyoshi Aizawa, Masato Kawade

Journal of the Japan Society for Precision Engineering Vol. 84 ( 12 ) page： 1025 - 1032 2018.12

Pedestrian Detection from a Sparse LIDAR Point-Cloud Reviewed Open Access

Yoshiki Tatebe, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Journal of the Japan Society for Precision Engineering Vol. 84 ( 12 ) page： 1017 - 1024 2018.12

Localizing the Gaze Target of a Crowd of People Reviewed

Yuki Kodama, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Hidehisa Nagano, Kunio Kashino

2018.12

Language：English Publishing type：Research paper (international conference proceedings)

What target is focused on by many people?
Analysis of the target is a crucial task, especially in a cinema, a stadium, and so on.
However, it is very difficult to estimate the gaze of each person in a crowd accurately and simultaneously with existing image-based eye tracking methods, since the image resolution of each person becomes low when we capture the whole crowd with a distant camera.
Therefore, we introduce a new approach for localizing the gaze target focused on by a crowd of people.
The proposed framework aggregates the individually estimated results of each person's gaze.
It enables us to localize the target being focused on by them even though each person's gaze localization from a low-resolution image is inaccurate.
We analyze the effects of an aggregation method on the localization accuracy using images capturing a crowd of people in a tennis stadium under the assumption that all of the people are focusing on the same target, and also investigate the effect of the number of people involved in the aggregation on the localization accuracy.
As a result, the proposed method showed the ability to improve the localization accuracy as it is applied to a larger crowd of people.

Analyzing Headlight Flicker Patterns for Improving the Pedestrian Detectability from a Driver Reviewed

Takashi Maeda, Takatsugu Hirayama, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of 2018 IEEE 21st International Conference on Intelligent Transportation Systems (ITSC2018) page： 3113 - 3118 2018.11

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we analyze headlight flicker patterns which improve the pedestrian detectability from a driver. Recently, headlights are becoming capable of selectively projecting light on a pedestrian in addition to the normal forward projection. However, it is still not clear how the light should be
projected to effectively improve the visibility of the pedestrian. We actually analyze nine flicker patterns by controlling duty ratios and durations of lighting time, and conduct experiments in field and laboratory settings. As a result, we reveal that a specific fundamental frequency is effective for improving the pedestrian detectability from a driver. We also conclude that the difference between the two settings are not significant.

Estimating the Scene-wise Reliability of LiDAR Pedestrian Detectors Reviewed

Haruya Kyutoku, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Kazuki Kato, Hiroshi Murase

Proceedings of 2018 IEEE 21th International Conference on Intelligent Transportation Systems (ITSC2018) page： 3511 - 3516 2018.11

Language：English Publishing type：Research paper (international conference proceedings)

Nowadays, development of driving support systems and autonomous driving systems have become active.
Pedestrian detection from in-vehicle sensors is one of the most important technologies for these systems.
However, outputs of pedestrian detectors can not be fully trusted in real environments.
Therefore, we propose an estimation system of pedestrian detector's reliabilities for a given scene.
This paper proposes a scene-wise reliability calculation method for LiDAR-based detectors, and a construction method for their estimators.
Here, the problem is how we can define the reliability.
The proposed method defines the reliability considering oversights as the strictest threshold without oversights.
Meanwhile, it defines the reliability considering false detections as the loosest threshold without false detections.
Experimental results showed that the proposed method could properly represent the reliability of a given scene, and estimate their reliability.

Attribute-aware Semantic Segmentation of Road Scenes for Understanding Pedestrian Orientations Reviewed

Mahmud Dwi Sulistiyo, Yasutomo Kawanishi, Daisuke Deguchi, Takatsugu Hirayama, Ichiro Ide, Jiang-Yu Zheng, Hiroshi Murase

Proceedings of 2018 IEEE 21th International Conference on Intelligent Transportation Systems (ITSC2018) page： 2698 - 2703 2018.11

Language：English Publishing type：Research paper (international conference proceedings)

Semantic segmentation is an interesting task for many deep learning researchers for scene understanding. However, recognizing details about objects’ attributes can be more informative and also helpful for a better scene understanding in intelligent vehicle use cases. This paper introduces a method for simultaneous semantic segmentation and pedestrian attributes recognition. A modified dataset built on top of the Cityscapes dataset is created by adding attribute classes corresponding to pedestrian orientation attributes. The proposed method extends the SegNet model and is trained by using both the original and the attribute-enriched datasets. Based on an experiment, the proposed attribute-aware semantic segmentation approach shows the ability to slightly improve the performance on the Cityscapes dataset, which is capable of expanding its classes in this case through additional data training.

Estimation of Driver's Insight for Safe Passing based on Pedestrian Attributes Reviewed

Fumito Shinmura, Yasutomo Kawanishi, Daisuke Deguchi, Takatsugu Hirayama, Ichiro Ide, Hiroshi Murase, Hironobu Fujiyoshi

Proceedings of 2018 IEEE 21th International Conference on Intelligent Transportation Systems (ITSC2018) page： 1041 - 1046 2018.11

Language：English Publishing type：Research paper (international conference proceedings)

In order to reduce traffic accidents between a vehicle and a pedestrian, recognition of a pedestrian who has a possibility of collision with a vehicle should be helpful. However, since a pedestrian may suddenly change his/her direction and cross the road, it is difficult to predict his/her behavior directly. Here, we focus on the fact that experienced drivers usually pass by a pedestrian while preparing to step on the brake at any moment when they feel danger. If driver assistant systems can estimate such experienced driver’s decisions, they could early detect the pedestrian in danger of collision. Therefore, we classify the driver’s decisions into three types by referring to the accelerator operation of drivers, and propose a method to estimate the type of the driver’s decision. The drivers are considered to decide their actions focusing on various behaviors and states of a pedestrian, namely pedestrian’s attributes. Since the driver’s decisions change along the timeline, the use of a temporal context is considered to be effective. Thus, in this paper, we propose an estimation method using a recurrent neural network architecture with the pedestrian’s attributes as input. We constructed a dataset collected by experienced drivers in control of the vehicle and evaluated the performance, and then confirmed the effectiveness of the use of pedestrian’s attributes.

Image Synthesis of Eye Areas for Perceptually Establishing Eye-contacts between Video Conference Participants Reviewed Open Access

Takuya Inoue, Takatsugu Hirayama, Tomokazu Takahashi, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Takayuki Kurozumi, Kunio Kashino

IEEJ Transactions on Electronics, Information and Systems Vol. 138 ( 11 ) page： 1399 - 1409 2018.11

Epipolar geometry-based ego-localization using an in-vehicle monocular camera

Haruya Kyutoku, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

2018.10

Language：English Publishing type：Research paper (international conference proceedings)

Nowadays, development of driving support systems and autonomous driving systems have become active.
Vehicle ego-localization using in-vehicle sensors is one of the most important technologies for these systems.
Accordingly, various attempts to localize own vehicle from in-vehicle sensors have been made.
In general, the estimation accuracy of the traveling direction is lower than in the lateral direction.
Therefore, we propose a highly accurate method for ego-localization of the traveling direction based on epipolar geometry using an in-vehicle monocular camera.
The proposed method makes correspondences between in-vehicle camera images and database images with location information, and calculates the location using locations annotated to the corresponding database images.
However, there are many gaps due to the difference of speed and trajectory of vehicles even if the images are obtained along the same road.
To overcome this problem, the distance between the input image and the database image is calculated by the distance metric based on the epipolar geometry and the local feature method.
An experiment was conducted using actual images with correct locations, and we confirmed the effectiveness of the proposed method from its results.

Vehicle counting via car parts detection from an in-vehicle camera image

Fumito Shinmura, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

2018.10

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a method to count vehicles from an in-vehicle camera image by regression based on car parts detection. In the case of an in-vehicle camera image, since vehicles are frequently occluded by other vehicles in traffic congestion, it is difficult to accurately count vehicles. Therefore, we propose a method to count vehicles by regression based on the number of visible car parts. For this, we make an estimator by learning the relation between the number of visible car parts and that of vehicles by Support Vector Regression. We evaluated our method using in-vehicle camera images recorded in an actual environment, where the proposed method performed better than counting detected vehicles.

People Tracking across Non-overlapping Camera Views via Camera Dropout and Trajectory Ensemble Reviewed

Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J101-D ( 8 ) page： 1079 - 1088 2018.8

Visualization of Real World Activity on Group Work

Daisuke Deguchi, Kazuaki Kondo, Atsushi Shimada

Proceedings of 20th International Conference on HCI International 2018 (HCII2018) Vol. LNCS 10902 page： 23 - 37 2018.7

Authorship：Lead author Language：English Publishing type：Research paper (international conference proceedings)

Group work is widely introduced and practiced as a method to achieve the learning goal efficiently by collaborating group members. However, since most types of group works are carried out in the real environment, it is very difficult to perform formative assessment and real time evaluation without students' feedbacks. Therefore, there is a strong demand to develop a method that supports evaluation of group work. To support evaluation of group work, this paper proposes a method to visualize the real world activity during group work by using first person view cameras and wearable sensors. Here, the proposed method visualizes three scores: (1) individual attention, (2) hand visibility, (3) individual activity. To evaluate the performance and analyze the relationships between scores, we conducted experiments of Marshmallow challenge" that is a collaborative work to construct a tower using marshmallow and spaghetti within a limit of time. Through the experiments, we confirmed that the proposed method has potential to become a evaluation tool for visualizing the activity of the group work."

DOI： 10.1007/978-3-319-91131-1_2

Describing Gaits by Onomatopoeias with Sound Symbolism Reviewed Open Access

Hirotaka Kato, Takatsugu Hirayama, Keisuke Doman, Ichiro Ide, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase

Transactions of the Japanese Society for Artificial Intelligence Vol. 33 ( 4 ) page： 1 - 9 2018.7

Pedestrian Detectability Estimation Considering Visual Adaptation to Drastic Illumination Change Reviewed Open Access

Yuki Imaeda, Takatsugu Hirayama, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. E101-D ( 5 ) page： 1457 - 1462 2018.5

Development of 'KamiRepo' System with Automatic Student Identification to Handle Handwritten Assignments on LMS Reviewed

Shunya Seiya, Ryuya Ito, Kosuke Okamoto, Ukyo Tanikawa, Shigeki Ohira, Daisuke Deguchi, Tomoki Toda

Proceedings of 2018 the IEEE Global Engineering Education Conference (EDUCON2018) page： 841 - 848 2018.4

Language：English Publishing type：Research paper (international conference proceedings)

A Learning Management System (LMS) has become a fundamental tool for higher education, and a framework to leverage digital education data in the LMS has attracted attention. On the other hand, there is strong demand to deal with various education data provided not only from electronic media but also non-electronic media, such as a handwritten assignment. To solve this problem, this paper describes the development of 'KamiRepo' system to make it possible to automatically upload handwritten assignments to the LMS. In this system, optical character recognition (OCR) is performed to identify scanned handwritten assignments of individual students and read their scores. Then, their scanned files automatically separated from the entire file of the scanned handwritten assignments are returned to the individual students through LMS together with their corresponding scores. Compared with a conventional system using the dedicated multifunction printer, our developed system is capable of 1) using general-purpose scanners, 2) using a user interface on Web browser, and 3) achieving accurate student identification. We have launched this system in our university in April 2017 and have evaluated its effectiveness. The experimental results using real data collected for 6 months showed that our system achieved 99.7% of success rate in the automatic upload process.

A Preliminary Study on Optimizing Person Re-identification using Stable Marriage Algorithm Reviewed

Nik Mohd Zarifie Hashim, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

page： 6 2018.2

Language：English Publishing type：Research paper (international conference proceedings)

Person re-identification gains an increasing interest
in the surveillance image processing field due to its’ importance
for security. Most approaches to solve the person re-identification
problem match persons one-by-one. However, redundant match-
ing where one of the person is selected for the matching pair
several times often occurs. It also degrades the overall image
matching performance. To overcome the issue, in this paper,
we propose a method which solves the person re-identification
problem for multiple persons simultaneously. Instead of one-
by-one matching, we consider person re-identification as an
instance of the Stable Marriage Problem (SMP). The result of an
experiment showed that the proposed method outperforms some
of the existing state-of-the-art methods applied to the VIPeR
dataset.

Efficient Pedestrian Scanning by Active Scan LIDAR Reviewed

Taiki Yamamoto, Fumito Shinmura, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Proceedings of International Workshop on Advanced Image Technology (IWAIT) 2018 page： 1 - 4 2018.1

Pedestrian Detection from Sparse Point-Cloud using 3DCNN Reviewed

Yoshiki Tatebe, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase, Utsushi Sakai

Proceedings of International Workshop on Advanced Image Technology (IWAIT) 2018 page： 1 - 4 2018.1

Automatic selection of Web contents towards automatic authoring of a video biography Reviewed Open Access

Ichiro Ide, Yasutomo Kawanishi, Kyoka Kunishiro, Frank Nack, Daisuke Deguchi, Hiroshi Murase

Proceedings of 19th IEEE Int. Symposium on Multimedia (ISM2017) page： 304 - 307 2017.12

Summarization of news videos considering the consistency of auditory and visual contents Reviewed

Ichiro Ide, Ye Zhang, Ryunosuke Tanishige, Keisuke Doman, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase

Proceedings of 19th IEEE Int. Symposium on Multimedia (ISM2017) page： 193 - 199 2017.12

Toward Describing Human Gaits by Onomatopoeias Reviewed

Hirotaka Kato, Takatsugu Hirayama, Yasutomo Kawanishi, Keisuke Doman, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV2017) Workshops page： 1573 - 1580 2017.10

Driver's Decision Analysis in Terms of Pedestrian Attributes -A Case Study in Passing by a Pedestrian-

Fumito Shinmura, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Hironobu Fujiyoshi

Proceedings of Workshop on Human Factors in Intelligent Vehicles page： 32 - 36 2017.10

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we report a case study on driver's decision in terms of pedestrian attributes. Among various traffic situations, the situation that a vehicle passes by a pedestrian is one of the major situations. To build a safety driving system that supports a non-experienced driver in such a situation, we analyzed how experienced drivers decide to handle the vehicle in such a situation. Since pedestrian’s behavior can be considered as a key factor for the decision, and also the behavior is different depending on their attributes," such as walking or stopping, noticing the vehicle or not, using a smartphone, etc., we analyzed what pedestrian's attributes affect the driver's decisions. For the analysis, we first built a large-scale dataset of driving data. Using the dataset, we clarified what attributes are dominant for the driver's decision."

Detection of Similar Geo-Regions based on Visual Concepts in Social Photos Reviewed

Hiroki Takimoto, Magali Philippe, Yasutomo Kawanishi, Ichiro Ide, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase

2017.9

Language：English Publishing type：Research paper (international conference proceedings)

Travel destination recommendation is useful to support travel.Considering the recommendation of regions within the destination area to visit, it could be difficult for the users to explicitly indicate their preference.Therefore, we considered that it would be more intuitive to recommend regions in the destination area that are similar to a region already well-known to the user.Thus, in this paper, we propose a method for the detection of similar geo-regions based on Visual Concepts in social photos.We report experimental results and analyses by applying the proposed method to the YFCC100M dataset.

Proposal of an encoded marker for working robots: An encoded marker easy to detect in various positions and under blur Reviewed

Norimasa Kobori, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Electronics and Communications in Japan Vol. 100 ( 10 ) page： 59 - 69 2017.9

Action Recognition from Extremely Low-Resolution Thermal Image Sequence Reviewed

Takayuki Kawashima, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Tomoyoshi Aizawa, Masato Kawade

Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS2017) 2017.9

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a Deep Learning-based action recognition method from an extremely low-resolution thermal image sequence. The method recognizes daily actions by humans (e.g. walking, sitting down, standing up, etc.) and abnormal actions (e.g. falling down) without privacy concerns. While privacy concerns can be ignored, it is difficult to compute feature points and to obtain a clear edge of the human body from an extremely low-resolution thermal image. To address these problems, this paper proposes a Deep Learning-based action recognition method that combines convolution layers and an LSTM layer for learning spatio-temporal representation, whose inputs are the thermal images and their frame differences cropped by the gravity center of human regions. The effectiveness of the proposed method was confirmed through experiments.

DOI： 10.1109/AVSS.2017.8078497

Estimation of the attractiveness of food photography focusing on main ingredients Reviewed

Kazuma Takahashi, Keisuke Doman, Yasutomo Kawanishi, Takatsugu Hirayama, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of 9th Workshop on Cooking and Eating Activities (CEA2017) in conjunction with IJCAI2017 page： 1 - 6 2017.8

Regression of Feature Scale Tracklets for Decimeter Visual Localization Reviewed

David Robert Wong, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Image and Vision Computing Vol. 68 page： 53 - 63 2017.8

Language：English Publishing type：Research paper (scientific journal)

Localization along a route is an everyday necessity for in-vehicle navigation systems, and a vital task for automated driving technologies. Visual ego-localization promises reliable accuracy even in challenging urban environments where Global Positioning Systems (GPSs) can fail. Using cameras for localization against a pre-constructed database requires either the creation of a dense three-dimensional feature point map and pose estimation of a query camera relative to this map, or image matching along a database route to determine the capture position of the query camera based on the most similar database image. While the latter method is potentially less computationally intensive and can provide a more compact database, localization accuracy is limited by the discrete positioning information at database frame capture locations. In this paper we propose an image matching method that makes use of image features which are pre-matched during database construction, allowing linear regression coefficients for the relationship between capture position and feature scale to be calculated. The capture position of matched query features can then be estimated to sub-database spacing resolution. By incorporating the visual localization system into a Bayes estimator, we demonstrate an average monocular vision localization accuracy of 0.33 m in tests on actual vehicle image streams.

DOI： 10.1016/j.imavis.2017.07.004

Headgear Recognition by Decomposing Human Images in the Thermal Infrared Spectrum Reviewed

Brahmastro Kresnaraman, Yasutomo Kawanishi, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

Proceedings of the 15th International Conference on Quality in Research (QiR2017) page： 164 - 168 2017.7

Language：English Publishing type：Research paper (international conference proceedings)

Surveillance systems play a critical role in security and surveillance. A surveillance system with cameras that work in the visible spectrum is sufficient for most cases. However, problems may arise during the night, or in areas with less than ideal illumination conditions. Cameras with thermal infrared technology can be a better option in these situations since they do not rely on illumination to observe the environment. Furthermore, in our daily lives, it is common for humans to wear headgears such as glasses, masks, and hats. In surveillance, such headgears can be a hindrance to the identification of a person, and hence pose a certain degree of risk. This is not ideal in areas where the identity of a person is important, for example, in a bank. Therefore, in this paper we propose a headgear recognition method using an innovative decomposition approach on thermal infrared images. The decomposition method is based on Robust Principal Component Analysis, a modification of the popular Principal Component Analysis. The proposed method performs decomposition on a human image and isolates headgears in the image for recognition purposes. Experiments were conducted to evaluate the capability of the proposed method. The results show a positive outcome when compared with other methods.

DOI： 10.1109/QIR.2017.8168475

Trajectory Ensemble: Multiple Persons Consensus Tracking across Non-overlapping Multiple Cameras over Randomly Dropped Camera Networks Reviewed

Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

page： 1471 - 1477 2017.7

Language：English Publishing type：Research paper (international conference proceedings)

Multiple person tracking over a camera network is usually performed by matching person images between adjacent cameras. It easily fails by a temporal appearance change of the persons caused by environmental illumination and observation orientation of a camera. To solve this problem, matching person images across not only adjacent cameras but also cameras multiple hops away in the camera network is effective, but such relaxation of spatiotemporal cues also cause tracking failure due to the increase of matching candidates. To avoid the failure, we introduce Random Camera Drop" to generate different camera networks which relax the spatio-temporal cues partially and randomly. We then, integrate tracking results over the networks to a consensus tracking result by a novel concept "Trajectory Ensemble", an extension of unsupervised ensemble learning for the multiple person tracking over a camera network problem. We evaluated the framework on several virtual datasets generated from a public dataset, "Shinpuhkan 2014 dataset" and confirmed that the proposed method achieves the highest tracking results among several comparative methods."

DOI： 10.1109/CVPRW.2017.190

Monocular Localization within Sparse Voxel Maps Reviewed

David Robert Wong, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV2017) page： 493 - 498 2017.6

Language：English Publishing type：Research paper (international conference proceedings)

We introduce a method that uses a single camera to localize a vehicle within a pre-constructed map consisting of a voxel occupancy grid and road-line marker positions. Sophisticated mapping hardware is capable of creating high accuracy 3D maps of road environments, but localizing a vehicle within such maps is one of the challenges at the forefront of automated driving. A solution which is robust to dynamic environments, while using only inexpensive sensors, is a difficult problem. In addition, maps that enable precise localization consume a lot of data which is impractical for the expansive environments encountered in real-world road networks. We show how using the area of edge regions shared between rendered views of a compact voxel map and in-vehicle camera images can be coupled with non-linear optimization methods to determine the camera position and pose.

Proposal of a spectral random dots marker using local feature for posture estimation Reviewed

Norimasa Kobori, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of IEEE Virtual Reality 2017 page： 223 - 224 2017.3

Deep Manifold Embedding for 3D Object Pose Estimation Reviewed Open Access

Hiroshi Ninomiya, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Norimasa Kobori, Yusuke Nakano

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2017 Vol. 5 page： 173 - 178 2017.3

Human Wearable Attribute Recognition Using Probability-Map-Based Decomposition of Thermal Infrared Images Reviewed Open Access

Brahmastro Kresnaraman, Yasutomo Kawanishi, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences Vol. E100-A ( 3 ) page： 854 - 864 2017.3

Language：English Publishing type：Research paper (scientific journal)

This paper addresses the attribute recognition problem, a
field of research that is dominated by studies in the visible spectrum. Only
a few works are available in the thermal spectrum, which is fundamentally
different from the visible one. This research performs recognition specif-
ically on wearable attributes, such as glasses and masks. Usually these
attributes are relatively small in size when compared with the human body,
on top of a large intra-class variation of the human body itself, therefore
recognizing them is not an easy task. Our method utilizes a decomposi-
tion framework based on Robust Principal Component Analysis (RPCA) to
extract the attribute information for recognition. However, because it is dif-
ficult to separate the body and the attributes without any prior knowledge,
noise is also extracted along with attributes, hampering the recognition
capability. We made use of prior knowledge; namely the location where
the attribute is likely to be present. The knowledge is referred to as the
Probability Map, incorporated as a weight in the decomposition by RPCA.
Using the Probability Map, we achieve an attribute-wise decomposition.
The results show a significant improvement with this approach compared
to the baseline, and the proposed method achieved the highest performance
in average with a 0:83 F-score.

DOI： 10.1587/transfun.E100.A.854

Open Access

Can We Detect Pedestrians using Low-resolution LIDAR? Reviewed Open Access

Yoshiki Tatebe, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase, Utsushi Sakai

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2017 Vol. 5 page： 157 - 164 2017.2

Wheelchair-user Detection Combined with Parts-based Tracking Reviewed

Ukyo Tanikawa, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Ryo Kawai

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2017 Vol. 5 page： 165 - 172 2017.2

Can a Driver Assistance System Determine if a Driver is Perceiving a Pedestrian? Reviewed Open Access

Yuki Imaeda, Takatsugu Hirayama, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2017 Vol. 4 page： 611 - 616 2017.2

Single Camera Vehicle Localization Using Feature Scale Tracklets Reviewed

David Robert Wong, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences Vol. E100-A ( 2 ) page： 702 - 713 2017.2

Language：English Publishing type：Research paper (scientific journal)

Advances in intelligent vehicle systems have led to modern automobiles being able to aid drivers with tasks such as lane following and automatic braking. Such automated driving tasks increasingly require reliable ego-localization. Although there is a large number of sensors that can be employed for this purpose, the use of a single camera still remains one of the most appealing, but also one of the most challenging. GPS localization in urban environments may not be reliable enough for automated driving systems, and various combinations of range sensors and inertial navigation systems are often too complex and expensive for a consumer setup. Therefore accurate localization with a single camera is a desirable goal.
In this paper we propose a method for vehicle localization using images captured from a single vehicle-mounted camera and a pre-constructed database.
Image feature points are extracted, but the calculation of camera poses is not required---instead we make use of the feature points' scale.
For image feature-based localization methods, matching of many features against candidate database images is time consuming, and database sizes can become large. Therefore, here we propose a method that constructs a database with pre-matched features of known good scale stability. This limits the number of unused and incorrectly matched features, and allows recording of the database scales into ``tracklets''. These ``Feature scale tracklets'' are used for fast image match voting based on scale comparison with corresponding query image features. This process reduces the number of image-to-image matching iterations that need to be performed while improving the localization stability. We also present an analysis of the system performance using a dataset with high accuracy ground truth. We demonstrate robust vehicle positioning even in challenging lane change and real traffic situations.

DOI： 10.1587/transfun.E100.A.702

A Classification Method of Cooking Operations Based on Temporal Patterns of Gaze Transitions and Blinks Reviewed

Hiroya Inoue, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J100-A ( 1 ) page： 12 - 23 2017.1

Misclassification Tolerable Learning for Robust Pedestrian Orientation Classification Reviewed

Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Hironobu Fujiyoshi

page： 481 - 486 2016.12

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we propose a multiclass classifier training method which reduces fatal" misclassifications by cost-relaxation of "tolerable" misclassifications in one-against-all classifiers training, named misclassification tolerable learning. In a binary classifier in the one-against-all classifiers, we introduce a new class group "conceptually similar classes," whose class labels are similar to the positive class. In the case of pedestrian orientation classification, the conceptually similar classes are defined as neighboring orientations to the positive orientation. We consider the misclassification of the conceptually similar classes to the positive class as tolerable misclassification. By relaxing the cost of the tolerable misclassifications, our proposed classification method reduces fatal misclassifications of non-similar classes. We evaluated the cost-relaxation effectiveness on several public datasets and confirmed that the proposed method outperforms the normal SVM on all of the datasets in the soft criterion by achieving 78.63% recognition rate on PDC Dataset."

Pedestrian Detection by Scene Adaptation Based on False Positive Mining Reviewed Open Access

Yuki Suzuki, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Journal of the Japan Society for Precision Engineering Vol. 82 ( 12 ) page： 1085 - 1091 2016.12

Recognition of Texting-While-Walking by Joint Features based on Arm and Head Poses Reviewed

Fumito Shinmura, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Hironobu Fujiyoshi

2016.11

Language：English Publishing type：Research paper (international conference proceedings)

Pedestrians texting-while-walking" increase the risk of traffic accidents, since they are often not paying attention to their surrounding environments and fails to notice approaching vehicles. Thus, the recognition of texting-while-walking from an in-vehicle camera should be helpful for safety driving assistance. In this paper, we propose a method to recognize a pedestrian texting-while-walking from in-vehicle camera images. The proposed approach focuses on the characteristic relationship between the arm and the head poses observed during a texting-while-walking behavior. In this paper, Pose-Dependent Joint HOG feature is proposed as a novel feature, which uses parts locations as prior knowledge and describes the cooccurrence of the arm and the head poses. To show the effectiveness of the proposed method, we constructed a dataset and evaluated it."

Moving camera background-subtraction for obstacle detection on railway tracks Reviewed

Hiroki Mukoujima, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase, Masato Ukai, Nozomi Nagamine, Ryuta Nakasone

Proceedings of 2016 IEEE International Conference on Image Processing (ICIP2016) page： 3967 - 3971 2016.9

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a method for detecting obstacles using train frontal view videos.
In recent years, obstacles detection using in-vehicle camera is actively developed for satisfying demands from various applications.
In the field of obstacle detection, most methods employ machine learning approach, and they can detect only trained obstacles, such as pedestrian, bicycle, etc.
Therefore, they cannot detect untrained general obstacles.
To overcome this problem, this paper propose a background subtraction method using moving camera.
The proposed method first computes frame-by-frame correspondences between the present and the database train frontal view image sequences, and detects obstacles by applying image subtraction to corresponding frames.
To confirm the effectiveness of the proposed method, experiments were conducted by using several image sequences captured at the experimental railroad track.
The experimental result showed that the proposed method could detect various obstacles accurately and effectively.

DOI： 10.1109/ICIP.2016.7533104

Subjective Sensing of Real World Activity on Group Study Reviewed

Daisuke Deguchi, Kazuaki Kondo, Atsushi Shimada

Proceedings of the 18th International Conference on Collaboration Technologies (CollabTech2016) page： 5 - 8 2016.9

Authorship：Lead author Language：English Publishing type：Research paper (international conference proceedings)

Collaborative learning is efficient teaching/learning method, and it is widely introduced and practiced in various situations. However, it has a difficulty to perform formative assessment and real time evaluation without students' feedbacks. Therefore, demand for technologies to support formative assessment in collaborative learning is increasing. To tackle this problem, we have started the research project for automatic sensing and visualization of real world activities in collaborative learning. In this paper, we will report details about preliminary group work experiments and its results with visualization tool.

Proposal of an Encoded Marker for Working Robots Reviewed

Norimasa Kobori, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEEJ Transactions on Electronics, Information and Systems Vol. 136 ( 9 ) page： 1367 - 1375 2016.9

Parts Selective DPM for detection of pedestrians possessing an umbrella Reviewed

Yuto Shimbo, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of 2016 IEEE Intelligent Vehicles Symposium (IV2016) page： 1053 - 1058 2016.6

Language：English Publishing type：Research paper (international conference proceedings)

In recent years, pedestrian detection from an invehicle camera has been attracting attention.However, in the case of a raining situation, the detection accuracy decreases because the head of a pedestrian tends to be occluded by an umbrella. In oder to handle such cases, in this paper, as a variation of the Deformable Part Model (DPM) which is widely used in the field of object recognition, we propose Parts Selective DPM (PS-DPM)" which selectively chooses the original part filters and additional part filters trained independently. In the detection of pedestrians possessing an umbrella, the selection of head and umbrella parts will make pedestrian detection more robust to the occlusion. We conducted experiments to evaluate the performance of the proposed method. As a result, pedestrian detection with the proposed PS-DPM achieved high detection accuracy in rainy weather, compared with the detection by the conventional DPM. Moreover, we confirmed that it did not decrease the pedestrian detection accuracy in fine weather."

Using Super-Pixels and Human Probability Map for Automatic Human Subject Segmentation Reviewed

Esmaeil Pourjam, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences Vol. E99-A ( 5 ) page： 943 - 953 2016.5

Language：English Publishing type：Research paper (scientific journal)

Human body segmentation has many applications in a wide
variety of image processing tasks, from intelligent vehicles to entertainment.
A substantial amount of research has been done in the field of segmentation
and it is still one of the active research areas, resulting in introduction of
many innovative methods in literature. Still, until today, a method that can
overcome the human segmentation problems and adapt itself to different
kinds of situations, has not been introduced. Many of methods today try to
use the graph-cut framework to solve the segmentation problem. Although
powerful, these methods rely on a distance penalty term (intensity difference
or RGB color distance). This term does not always lead to a good separation
between two regions. For example, if two regions are close in color, even
if they belong to two different objects, they will be grouped together, which
is not acceptable. Also, if one object has multiple parts with different
colors, e.g. humans wear various clothes with different colors and patterns,
each part will be segmented separately. Although this can be overcome
by multiple inputs from user, the inherent problem would not be solved.
In this paper, we have considered solving the problem by making use of a
human probability map, super-pixels and Grab-cut framework. Using this
map relives us from the need for matching the model to the actual body,
thus helps to improve the segmentation accuracy. As a result, not only the
accuracy has improved, it also became comparable to the state-of-the-art
interactive methods.

DOI： 10.1587/transfun.E99.A.943

A study on estimating the attractiveness of food photography Reviewed

Kazuma Takahashi, Keisuke Doman, Yasutomo Kawanishi, Takatsugu Hirayama, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of 2016 IEEE Second International Conference on Multimedia Big Data page： 444 - 449 2016.4

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a method for estimating the attractiveness of food photos in order to assist a user to shoot them attractively.
The proposed method extracts both color and shape features from input food images, and then integrates them according to a regression scheme.
By this way, the proposed method estimates the attractiveness of an unknown food photo.
We also created a food image dataset taken from various 3D-angles for each food category, and set target values of their attractiveness through subjective experiments.
Then, we evaluated the performance of the proposed method in two different ways of constructing the attractiveness estimator: One that constructs it for each food category, and the other that constructs a common attractiveness estimator for all food categories.
Experimental results showed the effectiveness of the proposed method in addition to the necessity for adaptively selecting the estimator depending on the appearance of foods for further performance improvement.

DOI： 10.1109/BigMM.2016.41

Reconstructing Face Image from the Thermal Infrared Spectrum to the Visible Spetrum Reviewed Open Access

Brahmastro Kresnaraman, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

Sensors, special issue on "Infrared and THz Sensing and Imaging" Vol. 16 ( 4 ) page： 1 - 16 2016.4

Language：English Publishing type：Research paper (scientific journal)

During the night or in poorly lit areas, thermal cameras are a better choice instead of normal cameras for security surveillance because they do not rely on illumination. A thermal camera is able to detect a person within its view, but identification from only thermal information is not an easy task. The purpose of this paper is to reconstruct the face image of a person from the thermal spectrum to the visible spectrum. After the reconstruction, further image processing can be employed, including identification/recognition. Concretely, we propose a two-step thermal-to-visible-spectrum reconstruction method based on Canonical Correlation Analysis (CCA). The reconstruction is done by utilizing the relationship between images in both thermal infrared and visible spectra obtained by CCA. The whole image is processed in the first step while the second step processes patches in an image. Results show that the proposed method gives satisfying results with the two-step approach and outperforms comparative methods in both quality and recognition evaluations.

DOI： 10.3390/s16040568

Open Access

Subtraction-Based General Forward Obstacle Detection from In-Vehicle Camera Images using Extended Census Transform Reviewed Open Access

Haruya Kyutoku, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

IEEJ Transactions on Electronics, Information and Systems Vol. 136 ( 4 ) page： 588 - 589 2016.4

A classification method of cooking operations based on eye movement patterns Reviewed

Hiroya Inoue, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of Ninth Biennial ACM Symposium on Eye Tracking Research and Applications (ETRA2016) page： 205 - 208 2016.3

HandWaving Gesture Detection Using a Far-Infrared Sensor Array with Thermo-Spatial Region of Interest Reviewed

Chisato Toriyama, Yasutomo Kawanishi, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Tomoyoshi Aizawa, Masato Kawade

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2016 page： 545 - 551 2016.2

Image transformation of eye areas for synthesizing eye-contacts in video conferencing Reviewed

Takuya Inoue, Tomokazu Takahashi, Takatsugu Hirayama, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Takayuki Kurozumi, Kunio Kashino

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2016 Vol. 3 page： 273 - 279 2016.2

Human Wearable Attribute Recognition using Decomposition of Thermal Infrared Images Reviewed

Brahmastro Kresnaraman, Yasutomo Kawanishi, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

Proceedings of the 22th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV2016) page： 123 - 127 2016.2

Language：English Publishing type：Research paper (international conference proceedings)

This paper addresses an attribute recognition problem in thermal images, specifically on worn objects such as hat and glasses. Although attribute recognition is a growing research field, there are not much work done in thermal infrared spectrum. In this spectrum, since illumination is not a problem, it could be a better option to be used in nighttime or poorly lit areas. The proposed method uses only the attribute information and excludes the unnecessary information for the recognition. To achieve this, we propose attribute recognition based on feature decomposition using Robust Principal Component Analysis (RPCA). An experiment to evaluate the capability of the proposed method was conducted on the dataset created for this research. The results show that the proposed method outperformed the method without decomposition by 14% in average with a maximum of 27% increase in a specific attribute.

Human Body Segmentation Using Texture Aware Grab-cut and Statistical Shape Models Reviewed

Esmaeil Pourjam, Yasutomo Kawanishi, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of the 22th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV2016) page： 28 - 33 2016.2

Language：English Publishing type：Research paper (international conference proceedings)

Segmentation is one of active areas in computer vision field with application in many areas from entertainment to intelligent vehicles (IVs). Among the objects, humans themselves have always been among the most interested subjects because of
their special features. Since human body has an articulated structure, modeling and recognizing different variations in the body has proved to be very difficult. Wearing various kinds of clothes in different situations which can have a completely dissimilar appearance based on the clothing type, makes the modeling much more difficult. Add to this, the common problems of vision like illumination changes, blurring due to camera movements, etc. make the problem even more difficult. Thus having a system that can segment human subjects accurately can be useful in many applications. In this paper, we propose a system for segmenting human subjects using Statistical Shapes Models (SSM) feedback and a texture aware version of Grab-cut which incorporates texture feature for improving the segmentation accuracy. Our experiments show that the proposed system has an acceptable accuracy compared to the state-of-the art interactive methods and much better than the conventional ones.

Human tracking using far-infrared sensor array (in Japanese) Reviewed

Takashi Hosono, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Tomoyoshi Aizawa, Masato Kawade

IEICE Transactions on Information and Systems Vol. J99-D ( 1 ) page： 119 - 129 2016.1

Prediction of driver's pedestrian detectability considering characteristics of human fields-of-view while driving (in Japanese) Reviewed

Ryunosuke Tanishige, Keisuke Doman, Daisuke Deguchi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J99-D ( 1 ) page： 56 - 66 2016.1

Position Interpolation using Feature Point Scale for Decimeter Visual Localization Reviewed

David Robert Wong, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV2015) Workshops page： 1 - 8 2015.12

Language：English Publishing type：Research paper (international conference proceedings)

Vehicle ego-localization is a critical task not only for in-car navigation systems, but also for emerging intelligent and autonomous vehicle technologies. Visual localization methods that determine current location by performing image matching against a pre-constructed database have an accuracy limited by the spatial distance between database images. In this paper we propose a method that uses the scale of feature points to interpolate the position of the query image between two database images. We show how this simple contribution offers an appreciable improvement in localization accuracy with an extremely minimal increase in processing time, especially when used in conjunction with image matching methods that already monitor feature scale. Our experiments showed an increase of up to 33\% in average localization accuracy when compared to a method without any interpolation.

DOI： 10.1109/ICCVW.2015.22

Detector Ensemble based on False Positive Mining for Pedestrian Detection Reviewed

Yuki Suzuki, Daisuke Deguchi, Yasutomo Kawanishi, Ichiro Ide, Hiroshi Murase

Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition (ACPR2015) 2015.11

Generation of a video summary on a news topic based on SNS responses to news stories Reviewed

Kosuke Kato, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proc. 4th Int Workshop on Crowdsourcing for Multimedia (CrowdMM'15) in conjunction with ACMMM2015 page： 21 - 26 2015.10

Distant Pedestrian Re-Detection from an in-Vehicle Camera based on Detections by Other Vehicles Reviewed

Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Hironobu Fujiyoshi

Proceedings of 2015 IEEE Conference on Intelligent Transportation Systems (ITSC2015) page： 1215 - 1220 2015.9

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we propose the re-detection paradigm, which is a detection with prior knowledge of the detection targets, and we introduce an implementation of the re-detection for distant pedestrian detection from an in-vehicle camera. We focus on the fact that other vehicles including forward vehicles can observe and detect pedestrians before the own vehicle observes them. Since appearances of pedestrians do not significantly change even though their locations are different, sharing images of the detected pedestrians among the vehicles, the own vehicle can use them as prior knowledge for detecting them again. Results of applying the proposed method to a dataset obtained by an in-vehicle camera demonstrate that the accuracy of pedestrian detection results can be significantly increased if prior knowledge of the pedestrians could be obtained.

DOI： 10.1109/ITSC.2015.200

Tastes and textures estimation of foods based on the analysis of its ingredients list and image Reviewed

Hiroki Matsunaga, Keisuke Doman, Takatsugu Hirayama, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Vol. 9281 page： 326 - 333 2015.9

Language：English Publishing type：Research paper (international conference proceedings)

Recently, the number of cooking recipes on the Web is increasing.
However, it is difficult to search them by tastes or textures although
they are actually important considering the nature of the contents.
Therefore, we propose a method for estimating the tastes and the
textures of a cooking recipe by analyzing them.
Concretely, the proposed method refers to an ingredients feature from
the ``ingredients list'' and image features from the ``food image'' in
a cooking recipe.
We confirmed the effectiveness of the proposed method through an
experiment.

Statistical Shape Feedback for Human Subject Segmentation Reviewed Open Access

Esmaeil Pourjam, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEEJ Transactions on Electronics, Information and Systems Vol. 135-C ( 8 ) page： 1000 - 1008 2015.8

Language：English Publishing type：Research paper (scientific journal)

Human segmentation is one of the most interesting yet most challenging subjects in the field of object segmentation and image processing. It can be used in various types of applications from image retrieval to robotics and human machine interfaces, including even entertainment. Many researches have been done on this subject and it is still one of active research areas. But until now, a method for accurate segmentation in different conditions has not been introduced. In this paper, we present Statistical Shape Feedback Segmentation” (SSFSeg) method, which is a way to automatically segment human subjects (pedestrians) from single images. Our main contributions in this paper are: 1) Using human shape model as priors for Grab-cut segmentation. 2) Implementation of a feedback system which provides a coarse-to-fine way of generating more accurate shapes. For this task, we try to use masks generated by the Statistical Shape Model (SSM) algorithm as a prior input for the Grab-cut technique to segment the desired human subject in the image without user interaction. To achieve this, we propose a feedback framework for the SSM sample generation. Our experiments confirmed that the segmentation error of our proposed method is less than half of the Grab-cut method."

DOI： 10.1541/ieejeiss.135.1000

Open Access

Typicality analysis of the combination of ingredients in a cooking recipe for assisting the arrangement of ingredients Reviewed

Satoshi Yokoi, Keisuke Doman, Takatsugu Hirayama, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of 2015 IEEE International Conference on Multimedia and Expo (ICME2012) page： 1 - 6 2015.7

Language：English Publishing type：Research paper (international conference proceedings)

As the number of cooking recipes posted on the Web increases, it becomes difficult to find a cooking recipe that a user needs. Moreover, even if it can be done, it is still difficult for users to arrange the cooking recipe, for example, by replacing ingredients with different ones. To deal with such problems, we propose a framework for typicality analysis of the combination of ngredients. The framework calculates a typicality value for each combination of ingredients. The list of ingredients can be arranged by adjusting the typicality value by adding or removing ingredients iteratively. The effectiveness of the proposed framework was confirmed through subjective experiments.

Pedestrian Orientation Classification Utilizing Single-Chip Coaxial RGB-ToF Camera Reviewed

Fumito Shinmura, Yasutomo Kawanishi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Hironobu Fujiyoshi

Proceedings of Workshop on Environment Perception for Automated On-road Vehicles in conjunction with 2015 IEEE Intelligent Vehicles Symposium (IV2015) page： 7 - 11 2015.6

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a method for pedestrian orientation classification. In image recognition, the accuracy is often degraded by the influence of background. In addition, it is also difficult to remove the background and extract only the human body from an image. To overcome these problems, we utilize a single-chip RGB-ToF camera. This camera can acquire RGB and depth images along the same optical axis at the same moment, and thus segmentation of the RGB image becomes easier by using the coaxial depth image. Our proposed method segmented a human body from its background accurately, which lead to the improvement of the accuracy of pedestrian orientation classification.

Translating Apereo Software: A Case Study using Sakai and Transifex

Yuji Tokiwa, Daisuke Deguchi, Juan Jose Merono Sanchez, Jose Mariano Lujan Gonzalez, Diego del Blanco Orobitg

2015.6

Language：English Publishing type：Research paper (international conference proceedings)

Web based translation support systems such as Crowdin and Transifex make it easy for regional Sakai communities to collaborate globally in translation. In the fall and winter of 2014, Spanish Sakai community and Japanese Sakai community are collaborating in translation of Sakai 10 using Transifex as a common translation platform. This collaboration brought a lot of things to two communities. For instance, to develop a tool to import resource bundle files to Transifex in a specific manner, we can have an ease of use platform to translate modular designed software such as Sakai. And this platform will be extended for every regional community and for every project in Apereo community.
During this session we will talk about followings;
(1) Overview
(2) Benefits for Apereo community
(3) Context dependent translation by gettext Portable Object
(4) Community translation strategy in Sakai Spanish Users (S2U)
(5) Automatized process by Jenkins

Multi-modal scene duplicate detection from news videos focusing on human faces Reviewed

Haruka Kumagai, Keisuke Doman, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

International Journal of Semantic Computing Vol. 9 ( 2 ) page： 215 - 237 2015.6

Language：English Publishing type：Research paper (scientific journal)

In this paper, as a tool for structuring a large volume of archived news videos according to their semantic contents, we propose a method that effectively detects scene duplicates, assuming the presence of a person speaking in the videos. A scene duplicate is defined here as a pair of video segments taken at the same event from different viewpoints. When considering scenes where a subject is speaking in news videos, referring to the audio channel could be effective to detect scene duplicates regardless of viewpoints. However, it cannot be relied on when external audio sources overlap the original one or when the subject is actually not speaking. In contrast, the image channel can be useful in most cases. However, significant difference in viewpoints could prevent accurate detection. Therefore, we propose a method that combines the results obtained from both audio and image channels in order to improve the accuracy of scene duplicate detection from news videos. The proposed method was evaluated through an experiment with actual broadcast news videos by comparing it with a conventional method. As a result, we confirmed that the detection accuracy significantly improved by the proposed method in both recall and precision.

DOI： 10.1142/S1793351X15400048

Localization of Open Source Softoware for Higher EducationCommon translation memory Reviewed

Yuji Tokiwa, Daisuke Deguchi, Makoto Miyazaki, Naoshi Hiraoka, Toshihiro Kita, Shoji Kajita

IPSJ Journal of Digital Practices Vol. 6 ( 2 ) page： 79 - 88 2015.4

Adaptive Reference Image Selection for Temporal Object Removal from Frontal In-vehicle Camera Image Sequences Reviewed

Toru Kotsuka, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2015 page： 233 - 239 2015.3

Language：English Publishing type：Research paper (international conference proceedings)

In recent years, image inpainting is widely used to remove undesired objects from an image. Especially, the removal of temporal objects, such as pedestrians and vehicles, in street-view databases such as Google Street View has many applications in Intelligent Transportation Systems (ITS). To remove temporal objects, Uchiyama et al. proposed a method that combined multiple image sequences captured along the same route. However, when spatial alignment inside an image group does not work well, the quality of the output image of this method is often affected. For example, large temporal objects existing in only one image create regions that do not correspond to other images in the group, and the image created from aligned images becomes distorted. One solution to this problem is to select adaptively the reference image containing only small temporal objects for spatial alignment. Therefore, this paper proposes a method to remove temporal objects by integration of multiple image sequences with an adaptive reference image selection mechanism.

DOI： 10.5220/0005357102330239

Estimation of Human Orientation using Coaxial RGB-Depth Images Reviewed

Fumito Shinmura, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Hironobu Fujiyoshi

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2015 page： 113 - 120 2015.3

Language：English Publishing type：Research paper (international conference proceedings)

Estimation of human orientation contributes to improving the accuracy of human behavior recognition. However,
estimation of human orientation is a challenging task because of the variable appearance of the human
body. The wide variety of poses, sizes and clothes combined with a complicated background degrades the
estimation accuracy. Therefore, we propose a method for estimating human orientation using coaxial RGBDepth
images. This paper proposes Depth Weighted Histogram of Oriented Gradients (DWHOG) feature
calculated from RGB and depth images. By using a depth image, the outline of a human body and the texture
of a background can be easily distinguished. In the proposed method, a region having a large depth gradient
is given a large weight. Therefore, features at the outline of the human body are enhanced, allowing robust
estimation even with complex backgrounds. In order to combine RGB and depth images, we utilize a newly
available single-chip RGB-ToF camera, which can capture both RGB and depth images taken along the same
optical axis. We experimentally confirmed that the proposed method can estimate human orientation robustly
to complex backgrounds, compared to a method using conventional HOG features.

DOI： 10.5220/0005305301130120

Scene duplicate detection from news videos using image-audio matching focusing on human faces Reviewed

Haruka Kumagai, Keisuke Doman, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of the 16th IEEE International Symposium on Multimedia (ISM2014) page： 71 - 77 2014.12

Event detection based on twitter enthusiasm degree for generating a sports highlight video Reviewed

Keisuke Doman, Taishi Tomita, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

page： 949 - 952 2014.11

Language：English Publishing type：Research paper (international conference proceedings)

This paper presents a Twitter-based event detection method based on Twitter Enthusiasm Degrees (TED)" toward generating a highlight video of a sports game. Existing methods not only depend on both languages and sports types but also often falsely detect non-target events. In contrast, the proposed method detects sports events using TEDs calculated from several kinds of string features independent of languages and sports. We applied the proposed method to actual sports games, and compared the detected events with the events present in broadcasted highlight videos, and confirmed the effectiveness and the language and sports type independencies of the proposed method."

DOI： 10.1145/2647868.2654973

Human Tracking using a Far-Infrared Sensor Array and a Thermo-Spatial Sensitive Histogram

細野峻司, 高橋友和, 出口大輔, 井手一郎, 村瀬洋, 相澤知禎, 川出雅人

Proceedings of 2nd Workshop on User-Centred Computer Vision 2014.11

Prediction of Driver's Pedestrian Detectability by Image Processing Adaptive to Visual Fields of View Reviewed

Ryunosuke Tanishige, Daisuke Deguchi, Keisuke Doman, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

Proceedings of 2014 IEEE Conference on Intelligent Transportation Systems (ITSC2014) page： 1388 - 1393 2014.10

Vision-based Vehicle Localization using a Visual Street Map with Embedded SURF Scale Reviewed

David Robert Wong, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of Computer Vision in Vehicle Technology with Special Session on Micro Aerial Vehicles (CVVT2014) 2014.9

Language：English Publishing type：Research paper (international conference proceedings)

Accurate vehicle positioning is important not only for in-car navigation systems but is also a requirement for emerging autonomous driving methods. Consumer level GPS are inaccurate in a number of driving environments such as in tunnels or areas where tall buildings cause satellite shadowing. Current vision-based methods typically rely on the integration of multiple sensors or fundamental matrix calculation which can be unstable when the baseline is small.

In this paper we present a novel visual localization method which uses a visual street map and extracted SURF image features. By monitoring the difference in scale of features matched between input images and the visual street map within a Dynamic Time Warping framework, stable localization in the direction of motion is achieved without calculation of the fundamental or essential matrices.

We present the system performance in real traffic environments. By comparing localization results with a high accuracy GPS ground truth, we demonstrate that accurate vehicle positioning is achieved.

Spatial People Density Estimation from Multiple Viewpoints by Memory Based Regression Reviewed

Yoshimune Tabuchi, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Takayuki Kurozumi, Kunio Kashino

Proceedings of the 22nd IAPR International Conference on Pattern Recognition (ICPR2014) page： 2209 - 2214 2014.8

Language：English Publishing type：Research paper (international conference proceedings)

Crowd analysis using cameras has attracted much attention for public safety and marketing. Among techniques of the crowd analysis, we focus on spatial people density estimation which estimates the number of people for each small area in a floor region. However, spatial people density cannot be estimated accurately for an area far from the camera because of the occlusion by people in a closer area. Therefore, we propose a method using a memory based regression method with images captured from cameras from multiple viewpoints. This method is realized by looking up a table that consists of correspondences between people density maps and crowd appearances. Since the crowd appearances include situations where various occlusions occur, an estimation robust to occlusion should be realized. In an experiment, we examined the effectiveness of the proposed method.

DOI： 10.1109/ICPR.2014.384

Single Camera Vehicle Localization Using SURF Scale and Dynamic Time Warping Reviewed

David Robert Wong, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of 2014 the IEEE Intelligent Vehicles Symposium (IV2014) page： 681 - 686 2014.6

Language：English Publishing type：Research paper (international conference proceedings)

Vehicle ego-localization is an essential process for many driver assistance and autonomous driving systems. The traditional solution of GPS localization is often unreliable in urban environments where tall buildings can cause shadowing of the satellite signal and multipath propagation. Typical visual feature based localization methods rely on calculation of the fundamental matrix which can be unstable when the baseline is small.
In this paper we propose a novel method which uses the scale of matched SURF image features and Dynamic Time Warping to perform stable localization. By comparing SURF feature scales between input images and a pre-constructed database, stable localization is achieved without the need to calculate the fundamental matrix. In addition, 3D information is added to the database feature points in order to perform lateral localization, and therefore lane recognition.
From experimental data captured from real traffic environments, we show how the proposed system can provide high localization accuracy relative to an image database, and can also perform lateral localization to recognize the vehicle's current lane.

Estimation of Traffic Sign Visibility Considering Local and Global Features in a Driving Environment Reviewed

Keisuke Doman, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase, Utsushi Sakai

Proceedings of 2014 IEEE Intelligent Vehicles Symposium (IV2014) page： 202 - 207 2014.6

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a camera-based visibility estimation method for a traffic sign.
The visibility here indicates how a visual target is easy to be detected and recognized by a human driver (not a machine).
This research aims at realizing a nuisance-free driver assistance system which sorts out information depending on the visibility of a visual target, in order to prevent driver distraction.
Our previous study on estimating the visibility of a traffic sign considered only the effect of the local region around a target, assuming the situation that a driver's gaze is around it.
The proposed method integrates both the local features and global features in a driving environment without such an assumption.
The global features evaluate the positional relationships between traffic signs and the appearance around the fixation point of a driver's gaze, which considers the effect of the driver's entire field of view.
Experimental results showed the effectiveness of incorporating the global features for estimating the visibility of a traffic sign.

Ego-localization by Searching in a Streetview Database using a Sequence of Image Distance based on the Positional Relation between In-vehicle Cameras Reviewed Open Access

Haruya Kyutoku, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

IEEJ Transactions on Electronics, Information and Systems Vol. 134 ( 5 ) page： 702 - 710 2014.5

Estimation of the Representative Story Transition in a Chronological Semantic Structure of News Topics Reviewed

Kosuke Kato, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of the 4th International Conference on Multimedia Retrieval (ICMR2014) page： 487 - 491 2014.4

Language：English Publishing type：Research paper (international conference proceedings)

It is important to track the flow of topics to thoroughly understand the contents. Accordingly, a method that structures the chronological semantic relations between news stories, namely a 'topic thread structure' has been proposed.It allows the comprehensive understanding of a topic by chronologically tracking stories one by one from the initial story. However, this task imposes a user to watch many stories when it contains various sub-topics. Thus, we propose method that estimates the representative story transition in a topic thread structure. In the proposed method, features obtained from a story and those from the topic thread structure are used for the estimation. We confirmed the effectiveness of the proposed method by comparing the results obtained from the proposed method to the ground truth obtained from votes in a subjective experiment. However, this task imposes a user to watch many stories when it contains various sub-topics. Thus, we propose method that estimates the representative story transition in a topic thread structure. In the proposed method, features obtained from a story and those from the topic thread structure are used for the estimation. We confirmed the effectiveness of the proposed method by comparing the results obtained from the proposed method to the ground truth obtained from votes in a subjective experiment

DOI： 10.1145/2578726.2578800

Can a Human be a Sensor? Reviewed

Atsushi Shimada, Daisuke Deguchi, Kazuaki Kondo, Takuya Funatomi

Proceedings of the 20th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV2014) page： 242 - 245 2014.2

Language：English Publishing type：Research paper (international conference proceedings)

This paper discusses a novel sensing strategy to retrieve real-world information. Instead of using sensor devices such as cameras, microphones and so on, the proposed approach involves people in real-world sensing to acquire requested information as accurately and quickly as possible. We call the proposed sensing strategy as Human Cloud Sensing (HCS)". In this paper, we introduce the concept of HCS and report some experimental results which were conducted to investigate the feasibility of "Can a human be a sensor?"."

Environment adaptive pedestrian detection using in-vehicle camera and GPS Reviewed

Daichi Suzuo, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Hiroyuki Ishida, Yoshiko Kojima

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2014 page： 354 - 361 2014.1

Language：English Publishing type：Research paper (international conference proceedings)

In recent years, accurate pedestrian detection from in-vehicle camera images is focused to develop a safety driving assistance system. Currently, successful methods are based on statistical learning. However, in such methods, it is necessary to prepare a large amount of training images. Thus, the decrease in the number of training images degrades the detection accuracy. That is, in driving environments with few or no training images, it is difficult to detect pedestrians accurately. Therefore, we propose an approach that collects training images automatically to build classifiers for various driving environments. This is expected to realize highly accurate pedestrian detection by using an appropriate classifier corresponding to the current location. The proposed method consists of three steps; Classification of driving scenes, collection of non-pedestrian images and training of classifiers for each scene class, and associating a scene-class-specific classifier with GPS location information. Through experiments, we confirmed the effectiveness of the method compared to baseline methods.

DOI： 10.5220/0004677003540361

Exemplar-Based Human Body Super-Resolution for Surveillance Camera Systems Reviewed

Kento Nishibori, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2014 page： 115 - 121 2014.1

Language：English Publishing type：Research paper (international conference proceedings)

In this paper, we propose an exemplar-based super-resolution method applied to a human body in a surveillance video. Since persons are usually captured as low-resolution images by a video surveillance system, it is sometimes necessary to perform detection and identification of persons from not only a human face but also from the human body appearance. The super-resolution for a human body image is difficult because the appearances of person images vary according to the color of clothing and the posture of persons. Thus, we focus on the high-frequency components that could restore the lost high-frequency components of the low-resolution image regardless to the variation of the clothing. Therefore, the purpose of the work presented in this paper is to apply the exemplar-based super-resolution using high-frequency components for a low-resolution human body image to generate a high-resolution human body image so that both computer systems and humans can identify persons more accurately. As a result of experiments, we confirmed the effectiveness of the proposed super-resolution method.

DOI： 10.5220/0004686101150121

Scene dependent classifiers for pedestrian detection

Hidefumi Yoshida, Daichi Suzuo, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

2013.11

Automatic authoring of a domestic cooking video based on the description of cooking instructions Reviewed

Yasuhiro Hayashi, Keisuke Doman, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of 5th International Workshop on Multimedia for Cooking and Eating Activities (CEA'13) page： 21 - 26 2013.10

Blur-invariant Traffic Sign Recognition Using Compact Local Phase Quantization Reviewed

Saleh Aly, Daisuke Deguchi, Hiroshi Murase

Proceedings of 2013 IEEE Conference on Intelligent Transportation Systems (ITSC2013) page： 821 - 827 2013.10

Construction of a traffic sign detector based on voting type co-training Reviewed

Yuji Kojima, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of 2013 IEEE Conference on Intelligent Transportation Systems (ITSC2013) page： 1137 - 1142 2013.10

Prediction of Pedestrian Detectability for Drivers by Image Processing and its Driver Adaptation Reviewed

Ryunosuke Tanishige, Daisuke Deguchi, Keisuke Doman, Yoshito Mekada, Ichiro Ide, Hiroshi Murase, Naoki Nitanda

Proceedings of the 6th Biennial Workshop on Digital Signal Processing for In-Vehicle Systems (DSP2013) page： 1 - 4 2013.9

Language：English Publishing type：Research paper (international conference proceedings)

In recent years, advances in pedestrian detection technology have resulted in the development of driving assistance systems that notify the drivers of the presence of pedestrians. However, warning of all of the presence of pedestrians would confuse the driver. Therefore, the driver should only be notified of the less detectable pedestrians to avoid confusion. To achieve this, it is necessary to develop a method to predict the driver’s perception performance of pedestrian detectability. This paper proposes a method that predicts the pedestrian detectability considering the difference between individual drivers. The proposed method constructs a predictor specific to each driver, in order to predict the pedestrian detectability precisely. To obtain the ground truth of the pedestrian detectability, we conducted an experiment by human subjects using images from an in-vehicle camera including pedestrians. From the comparison between the output of the proposed method and the actual detectability, we confirmed that the proposed method significantly reduces the prediction error in comparison with the existing methods.

Pedestrian Detection by Scene Dependent Classifiers with Generative Learning Reviewed

Hidefumi Yoshida, Daichi Suzuo, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Takashi Machida, Yoshiko Kojima

Proceedings of 2013 IEEE Intelligent Vehicles Symposium (IV2013) page： 654 - 659 2013.6

Language：English Publishing type：Research paper (international conference proceedings)

Recently, pedestrian detection from in-vehicle camera images is becoming an crucial technology for Intelligent Transportation Systems (ITS). However, it is difficult to detect pedestrians accurately in various scenes by obtaining training samples. To tackle this problem, we propose a method to construct scene dependent classifiers to improve the accuracy of pedestrian detection. The proposed method selects an appropriate classifier based on the scene information that is a category of appearance associated with location information. To construct scene dependent classifiers, the proposed method introduces generative learning for synthesizing scene dependent training samples. Experimental results showed that the detection accuracy of the proposed method outperformed the comparative method, and we confirmed that scene dependent classifiers improved the accuracy of pedestrian detection.

Segmentation of Human Instances Using Grab-cut and Active Shape Model Feedback Reviewed

Esmaeil Pourjam, Ichiro Ide, Daisuke Deguchi, Hiroshi Murase

Proceedings of IAPR Conference on Machine Vision Applications (MVA) 2013 page： 77 - 80 2013.5

Cross-pose face recognition Reviewed

Xi Li, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEICE Transaction on Information and Systems, Special Section on Face Perception and Recognition Vol. E96-D ( 3 ) page： 531 - 537 2013.3

Language：English Publishing type：Research paper (scientific journal)

This paper presents an approach for cross-pose face recognition by virtual view generation using an appearance clustering based local view transition model. Previously, the traditional global pattern based view transition model (VTM) method was extended to its local version called LVTM, which learns the linear transformation of pixel values between frontal and non-frontal image pairs from training images using partial image in a small region for each location, instead of transforming the entire image pattern. In this paper, we show that the accuracy of the appearance transition model and the recognition rate can be further improved by better exploiting the inherent linear relationship between frontal-nonfrontal face image patch pairs. This is achieved based on the observation that variations in appearance caused by pose are closely related to the corresponding 3D structure and intuitively frontal-nonfrontal patch pairs from more similar local 3D face structures should have a stronger linear relationship. Thus for each specific location, instead of learning a common transformation as in the LVTM, the corresponding local patches are first clustered based on an appearance similarity distance metric and then the transition models are leaned separately for each cluster. In the testing stage, each local patch for the input non-frontal probe image is transformed using the learned local view transition model corresponding to the most visually similar cluster. The experimental results on a real-world face dataset demonstrated the superiority of the proposed method in terms of recognition rate.

Two-step super-resolution with binary and grid constraints for low resolution QR-code recognition Reviewed Open Access

Yuji Kato, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J96-D ( 2 ) page： 328 - 337 2013.2

Detection of Biased Broadcast Sports Video Highlights by Attribute-Based Tweets Analysis Reviewed

Takashi Kobayashi, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Lecture Note in Computer Science Vol. 7733 page： 364 - 373 2013.1

Language：English Publishing type：Research paper (international conference proceedings)

We propose a method for detecting biased-highlights in a broadcast sports video according to viewers’ attributes obtained from a large number of tweets. Recently, Twitter is widely used to make real-time play-by-play comments on TV programs, especially on sports games. This trend enables us to effectively acquire the viewers’ interests in a large mass. In order to make use of such tweets for highlight detection in broadcast sports video, the proposed method first performs an attribute analysis on the set of tweets issued by each user to classify which team he/she supports. It then detects biased-highlights by referring to the number of tweets made by viewers with a specific attribute.

DOI： 10.1007/978-3-642-35728-2_35

A Study of Gaze Estimation Using Head and Body Pose Information Reviewed

Nobuhiro Funatsu, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of International Workshop on Advanced Image Technology (IWAIT) 2013 page： 231 - 235 2013.1

Language：English Publishing type：Research paper (international conference proceedings)

Gaze estimation from an image is an important technique for tasks such as driver monitoring and measuring of advertising effectiveness. For this, most existing methods require a high quality image of eyes. However, It is difficult to obtain the eye images when eye occlusion occurs due to sunglasses or face rotation. Another approach approximates head directions to gaze directions. However, the gaze direction is affected not only by the head direction but also by the body direction. Thus, we are studying a method that estimates the gaze direction accurately using information on both head and body pose directions. Experimental result showed that the proposed method could estimate gaze directions more accurately than by using the information on only head directions.

Estimation of the human performance for pedestrian detectability based on visual search and motion features Reviewed

Masashi Wakayama, Daisuke Deguchi, Keisuke Doman, Ichiro Ide, Hiroshi Murase, Yukimasa Tamatsu

Proceedings of the 21st IAPR International Conference on Pattern Recognition (ICPR2012) page： 1940 - 1943 2012.11

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a method for estimating the human performance of pedestrian detectability from in-vehicle camera images in order to warn a driver of the positions of pedestrians in an appropriate timing. By introducing features related to visual search and motion of the target, the proposed method estimates the detectability of pedestrians accurately. Support Vector Regression (SVR) is used to estimate the detectability. Here, SVR is trained using features calculated by the proposed method with the ground truth obtained through experiments with human subjects. From experiments using in-vehicle camera images, we confirmed that the proposed features were effective to estimate the detectability of pedestrians.

Virtual View Generation using Clustering based Local View Transition Model Reviewed

Xi Li, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of ACCV Workshop on Face Analysis: The Intersection of Computer Vision and Human Perception 2012.11

Language：English Publishing type：Research paper (international conference proceedings)

This paper presents an approach for realistic virtual view generation using appearance clustering based local view transition model, with its target application on cross-pose face recognition. Previously, the traditional global pattern based view transition model (VTM) method was extended to its local version called LVTM, which learns the linear transformation of pixel values between frontal and non-frontal image pairs using partial image in a small region for each location, rather than transforming the entire image pattern. In this paper, we show that the accuracy of the appearance transition model and the recognition rate can be further improved by better exploiting the inherent linear relationship between frontal-nonfrontal face image patch pairs. For each specific location, instead of learning a common transformation as in the LVTM, the corresponding local patches are first clustered based on appearance similarity distance metric and then the transition models are learned separately for each cluster. In the testing stage, each local patch for the input non-frontal probe image is transformed using the learned local view transition model corresponding to the most visually similar cluster. The experimental results on a real-world face dataset demonstrated the superiority of the proposed method in terms of recognition rate.

Ego-Localization Based on Frame Correspondences between In-Vehicle Camera Images and Streetview Database Reviewed

Haruya Kyutoku, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J95-D ( 11 ) page： 1973 - 1982 2012.11

Subtraction-Based Forward Obstacle Detection Using Illumination Insensitive Feature for Driving-Support Reviewed

Haruya Kyutoku, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

Proceedings of Computer Vision in Vehicle Technology: From Earth to Mars (CVVT2012) Vol. Part II ( LNCS 7584 ) page： 515 - 525 2012.10

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a method for detecting general obstacles on a road by subtracting present and past in-vehicle camera images.
The image-subtraction-based object detection approach can be applied to detect any kind of obstacles although the existing learning-based methods detect only specific obstacles.
To detect general obstacles, the proposed method first computes a frame-by-frame correspondence between the present and the past in-vehicle camera image sequences, and then registrates road surfaces between the frames.
Finally, obstacles are detected by applying image subtraction to the registrated road surface regions with an illumination insensitive feature for robust detection.
Experiments were conducted by using several image sequences captured by an actual in-vehicle camera to confirm the effectiveness of the proposed method.
The experimental results shows that the proposed method can detect general obstacles accurately at a distance enough to avoid them safely even in situations with different illuminations.

DOI： 10.1007/978-3-642-33868-7_51

Visibility Estimation of Traffic Signals under Rainy Weather Conditions for Smart Driving Support Reviewed

Ryuhei Sato, Keisuke Doman, Daisuke Deguchi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase, Yukimasa Tamatsu

Proceedings of 2012 IEEE Conference on Intelligent Transportation Systems (ITSC2012) page： 1321 - 1326 2012.9

Language：English Publishing type：Research paper (international conference proceedings)

The aim of this work is to support a driver by
notifying the information of traffic signals in accordance with
their visibility. To avoid traffic accidents, the driver should
detect and recognize surrounding objects, especially traffic
signals. However, when driving a vehicle under rainy weather
conditions, it is difficult for drivers to detect or to recognize
objects existing in the road environment in comparison with fine
weather conditions. Therefore, this paper proposes a method
for estimating the visibility of traffic signals for drivers under
rainy weather conditions by image processing. The proposed
method is based on the concept of visual noise known in the
field of cognitive science, and extracts two types of visual noise
features which ware considered that they affect the visibility of
traffic signals. We expect to improve the accuracy of visibility
estimation by combining the visual noise features with the
texture feature introduced in a previous work. Experimental
results showed that the proposed method could estimate the
visibility of traffic signals more accurately under rainy weather
conditions.

DOI： 10.1109/ITSC.2012.6338838

Detection and classification of repetitious human motions combining shift variant and invariant features Reviewed

Ichiro Ide, Taku Kuhara, Daisuke Deguchi, Tomokazu Takahashi, Hiroshi Murase

Proceedings of the 3rd International Conference on Emerging Security Technologies (EST2012) page： 86 - 89 2012.9

Robust Face Super-Resolution Using Free-Form Deformations For Low-Quality Surveillance Video Reviewed

Tomonari Yoshida, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of 2012 IEEE International Conference on Multimedia and Expo (ICME2012) page： 368 - 373 2012.7

Speech shot extraction from broadcast news videos Reviewed

Shogo Kumagai, Keisuke Doman, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

International Journal of Semantic Computing Vol. 6 ( 2 ) page： 179 - 204 2012.6

Language：English Publishing type：Research paper (scientific journal)

We propose a method for discriminating between a speech shot and a narrated shot to extract genuine speech shots from a broadcast news video. Speech shots in news videos contain a wealth of multimedia information of the speaker, and could thus be considered valuable as archived material. In order to extract speech shots from news videos, there is an approach that uses the position and size of a face region. However, it is difficult to extract them with only such an approach, since news videos contain non-speech shots where the speaker is not the subject that appears in the screen, namely, narrated shots. To solve this problem, we propose a method to discriminate between a speech shot and a narrated shot in two stages. The first stage of the proposed method directly evaluates the inconsistency between a subject and a speaker based on the co-occurrence between lip motion and voice. The second stage of the proposed method evaluates based on the intra- and inter-shot features that focus on the tendency of speech shots. With the combination of both stages, the proposed method accurately discriminates between a speech shot and a narrated shot. In the experiments, the overall accuracy of speech shots extraction by the proposed method was 0.871. Therefore, we confirmed the effectiveness of the proposed method.

DOI： 10.1142/S1793351X12400077

Real-time marker-free patient registration for electromagnetic navigated bronchoscopy: a phantom study Reviewed

Daisuke Deguchi, Marco Feuerstein, Takayuki Kitasaka, Yasuhito Suenaga, Ichiro Ide, Hiroshi Murase, Kazuyoshi Imaizumi, Yoshinori Hasegawa, Kensaku Mori

International Journal of Computer Assisted Radiology and Surgery Vol. 7 ( 3 ) page： 359 - 369 2012.5

Authorship：Lead author Language：English Publishing type：Research paper (scientific journal)

Purpose
To perform bronchoscopy safely and smoothly, it is very important to develop a bronchoscopic guidance system. Transbronchial lung biopsy (TBLB) with a bronchoscopic guidance system especially should permit safe image-guided procedures. Recently, electromagnetic tracking (EMT) is utilized to track the tip of the bronchoscope camera in real time. For most tracking methods using position sensors, registration between tracking data and previously acquired reference image data, such as CT image, is performed using natural landmarks of the patient or fiducial markers attached to the patient, whose positions need to be measured manually by the physician before the actual bronchoscopy. Therefore, this paper proposes a marker-free CT-to-patient registration method utilizing bronchoscope's position and orientation obtained by the EMT.

Methods
We developed a guidance system that is able to track the tip of the bronchoscope camera in real time. In the case of a guidance system that uses position sensors, natural landmarks of the patient or fiducial markers attached to the patient are needed to obtain the correspondence between EMT outputs and previously acquired reference image data, such as CT image. This paper proposes a registration method without landmarks or fiducials by estimating the transformation matrix between the patient and the CT image taken prior to the bronchoscopic examination. This estimation is performed by computing correspondences between the outputs of the EMT sensor and airways extracted from the CT image. As ambiguities between EMT measurements and their corresponding airway branches may arise at airway bifurcations, we introduce a stable airway branch selection mechanism for improving the robustness of the estimation of the transformation matrix. To evaluate the performance of the proposed method, we applied the method to a rubber bronchial phantom and added virtual breathing motion to the sensor output.

Results
Experimental results show that the accuracy of our proposed method is within 2.0-3.0 mm (without breathing motion) and 2.5-3.5 mm (with breathing motion). The proposed method could also track a bronchoscope camera in real time.

Conclusions
We developed a method for CT-to-patient registration using a position sensor without fiducial markers and natural landmarks. Endoscopic guided biopsy of lung lesions is feasible using a marker-free CT-to-patient registration method.

DOI： 10.1007/s11548-011-0626-9

Construction of a local attraction map according to social visual attention Reviewed

Ichiro Ide, Jiani Wang, Masafumi Noda, Tomokazu Takahashi, Daisuke Deguchi, Hiroshi Murase

Proceedings of the 5th International Conference on Intelligent Interactive Multimedia Systems and Services (IIMSS2012) page： 153 - 162 2012.5

Development and comparison of new hybrid motion tracking for bronchoscopic navigation Reviewed

Xiongbiao Luo, Marco Feuerstein, Daisuke Deguchi, Takayuki Kitasaka, Hirotsugu Takabatake, Kensaku Mori

Medical Image Analysis Vol. 16 ( 3 ) page： 577 - 596 2012.4

Language：English Publishing type：Research paper (scientific journal)

This paper presents a new hybrid camera motion tracking method for bronchoscopic navigation combining SIFT, epipolar geometry analysis, Kalman filtering, and image registration. In a thorough evaluation, we compare it to state-of-the-art tracking methods. Our hybrid algorithm for predicting bronchoscope motion uses SIFT features and epipolar constraints to obtain an estimate for inter-frame pose displacements and Kalman filtering to find an estimate for the magnitude of the motion. We then execute bronchoscope tracking by performing image registration initialized by these estimates. This procedure registers the actual bronchoscopic video and the virtual camera images generated from 3D chest CT data taken prior to bronchoscopic examination for continuous bronchoscopic navigation. A comparative assessment of our new method and the state-of-the-art methods is performed on actual patient data and phantom data. Experimental results from both datasets demonstrate a significant performance boost of navigation using our new method. Our hybrid method is a promising means for bronchoscope tracking, and outperforms other methods based solely on Kalman filtering or image features and image registration.

DOI： 10.1016/j.media.2010.11.001

Integration of generative learning and multiple pose classifiers for pedestrian detection Reviewed

Hidefumi Yoshida, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Kunihiro Goto, Yoshikatsu Kimura, Takashi Naito

Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP) 2012 Vol. 1 page： 567 - 572 2012.2

Language：English Publishing type：Research paper (international conference proceedings)

Recently, pedestrian detection from in-vehicle camera images is becoming an important technology in ITS (Intelligent Transportation System).
However, it is difficult to detect pedestrians stably due to the variety of their poses and their backgrounds.
To tackle this problem, we propose a method to detect various pedestrians from in-vehicle camera images by using multiple classifiers corresponding to various pedestrian pose classes.
Since pedestrians’ pose varies widely, it is difficult to construct a single classifier that can detect pedestrians with various poses stably.
Therefore, this paper constructs multiple classifiers optimized for variously posed pedestrians by classifying pedestrian images into multiple pose classes.
Also, to reduce the bias and the cost for preparing numerous pedestrian images for each pose class for learning, the proposed method employs a generative learning method.
Finally, the proposed method constructs multiple classifiers by using the synthesized pedestrian images.
Experimental results showed that the detection accuracy of the proposed method outperformed comparative methods, and we confirmed that the proposed method could detect variously posed pedestrians stably.

DOI： 10.5220/0003817305670572

Construction of an accurate traffic sign detector by automatic gathering of traffic sign images based on retrospective tracking Reviewed Open Access

Daisuke Deguchi, Keisuke Doman, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J95-D ( 1 ) page： 76 - 84 2012.1

Improvement of Vehicle Ego-localization by Sequential Matching of Feature-points from an Aerial Image and an In-vehicle Camera Image Reviewed Open Access

Masafumi Noda, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Yoshiko Kojima, Takashi Naito

IEICE Transactions on Information and Systems Vol. J95-D ( 1 ) page： 111 - 121 2012.1

Estimation of traffic sign visibility based on the integration of contrast features and appearance features Reviewed Open Access

Keisuke Doman, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase, Yukimasa Tamatsu

IEICE Transactions on Information and Systems Vol. J95-D ( 1 ) page： 122 - 130 2012.1

Classification of near-duplicate video clips in a large-scale broadcast video archive based on their appearance patterns Reviewed Open Access

Yuji Shamoto, Ichiro Ide, Daisuke Deguchi, Tomokazu Takahashi, Hiroshi Murase

Journal of Information Processing Vol. 52 ( 12 ) page： 3593 - 3598 2011.12

Creation of a Sight-seeing Map with Visual Classification of Photos on the Web Reviewed Open Access

Jiani Wang, Masafumi Noda, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Journal of Information Processing Vol. 52 ( 12 ) page： 3588 - 3592 2011.12

Detection of Inconsistency between Subject and Speaker based on the Co-occurrence of Lip Motion and Voice Towards Speech Scene Extraction from News Videos Reviewed

Shogo Kumagai, Keisuke Doman, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of 2011 IEEE International Symposium on Multimedia (ISM2011) page： 311 - 318 2011.12

Language：English Publishing type：Research paper (international conference proceedings)

We propose a method to detect the inconsistency between a subject and the speaker for extracting speech scenes from news videos.
Speech scenes in news videos contain a wealth of multimedia information, and are valuable as archived material.
In order to extract speech scenes from news videos, there is an approach that uses the position and size of a face region.
However, it is difficult to extract them with only such approach, since news videos contain non-speech scenes where the speaker is not the subject, such as narrated scenes.
To solve this problem, we propose a method to discriminate between speech scenes and narrated scenes based on the co-occurrence between a subject's lip motion and the speaker's voice.
The proposed method uses lip shape and degree of lip opening as visual features representing a subject's lip motion, and uses voice volume and phoneme as audio feature representing a speaker's voice.
Then, the proposed method discriminates between speech scenes and narrated scenes based on the correlations of these features.
We report the results of experiments on videos captured in a laboratory condition and also on actual broadcast news videos.
Their results showed the effectiveness of our method and the feasibility of our research goal.

Detection of Repetitive Cooking Motions in a Cooking Video Reviewed Open Access

Taku Kuhara, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J94-D ( 12 ) page： 1983 - 1985 2011.12

Generation of an In-Vehicle Camera Image Sequence without Temporary Objects by Selection of Partial Images from Multiple Image Sequences Reviewed Open Access

Hiroyuki Uchiyama, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J94-D ( 12 ) page： 2093 - 2104 2011.12

Scene Segmentation of Wedding Party Videos by Scenario-based Matching with Example Videos Reviewed

Kazuki Sawai, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of the 19th ACM International Multimedia Conference (ACM-MM2011) page： 1545 - 1548 2011.12

Accurate and Robust Attitude Estimation using MEMS Gyroscopes and a Monocular Camera Reviewed Open Access

Norimasa Kobori, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase

Transactions of the Society of Instrument and Control Engineers Vol. 47 ( 10 ) page： 442 - 449 2011.10

Low resolution QR-code recognition by applying super-resolution using the property of QR-codes Reviewed

Yuji Kato, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase

Proceedings of the 11th International Conference on Document Analysis and Recognition (ICDAR2011) page： 992 - 996 2011.9

Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes a method for low resolution QR-code recognition. A QR-code is a two-dimensional binary symbol that can embed various information such as characters and numbers. To recognize a QR-code correctly and stably,the resolution of an input image should be high. In practice,however, recognition of a QR-code is usually difficult due to
low resolution when it is captured from a distance. In this paper, we propose a method to improve the performance of low resolution QR-code recognition by using the super-resolution technique that generates a high resolution image from multiple low-resolution images. Although a QR-code is a binary pattern, it is observed as a grayscale image due to the degradation through the capturing process. Especially the pixels around the borders between white and black regions become ambiguous. To overcome this problem, the proposed method introduces a binary pattern constraint to generate super-resolved images appropriate for recognition. Experimental results showed that a recognition rate of 98% an be achieved by the proposed method, which is a 15.7% improvement in comparison with a method using a conventional super-resolution method.

Power-efficient hardware architecture of K-Means clustering with Bayesian-Information-Criterion processor for multimedia processing applications Reviewed

Tse-Wei Chen, Chih-Hao Sun, Hsiao-Hang Su, Shao-Yi Chien, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

IEEE Journal on Emerging and Selected Topics in Circuits and Systems Vol. 1 ( 3 ) page： 357 - 368 2011.9

Language：English Publishing type：Research paper (scientific journal)

A power-efficient K-Means hardware architecture that can automatically estimate the number of clusters in the clustering process is proposed. The contributions of this work include two main aspects. The first is the integration of the hierarchical data sampling in the hardware to accelerate the clustering speed. The second is the development of the Bayesian-Information-Criterion (BIC) Processor" to estimate the number of clusters of K-Means. The architecture of the "BIC Processor" is designed based on the simplification of the BIC computations, and the precision of the logarithm function is also analyzed. The experiments show that the proposed architecture can be employed in different multimedia applications, such as motion segmentation and edge-adaptive noise reduction. Besides, the gate count of the hardware is 51 K with the 90-nm complimentary metal-oxide-semiconductor technology. It is also shown that this work can achieve high efficiency compared with a GPU, and the power consumption scales well with the number of clusters and the number of dimensions. The power consumption ranges between 10.72 and 12.95 mW in different modes when the operating frequency is 233 MHz."

DOI： 10.1109/JETCAS.2011.2165231

Towards the Creation of a Cooking Recipe for Inexperienced Users by Supplementing Multimedia Information Reviewed Open Access

Yuka Shidochi, Ichiro Ide, Yuichi Nakamura, Daisuke Deguchi, Tomokazu Takahashi, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J94-A ( 7 ) page： 544 - 547 2011.7

On-road Obstacle Detection by Comparing Present and Past In-vehicle Camera Images Reviewed

Haruya Kyutoku, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

Proceedings of IAPR Conference on Machine Vision Applications (MVA) 2011 page： 357 - 360 2011.6

Estimation of Traffic Sign Visibility Considering Temporal Environmental Changes for Smart Driver Assistance Reviewed

Keisuke Doman, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase, Yukimasa Tamatsu

Proceedings of 2011 IEEE Intelligent Vehicles Symposium (IV2011) page： 667 - 672 2011.6

Language：English Publishing type：Research paper (international conference proceedings)

We propose a visibility estimation method for traffic signs considering temporal environmental changes, as a part of work for the realization of nuisance-free driver assistance systems. Recently, the number of driver assistance systems in a vehicle is increasing. Accordingly, it is becoming important to sort out appropriate information provided from them, because providing too much information may cause driver distraction. To solve such a problem, we focus on a visibility estimation method for controlling the information according to the visibility of a traffic sign. The proposed method sequentially captures a traffic sign by an in-vehicle camera, and estimates its accumulative visibility by integrating a series of instantaneous visibility. By this way, even if the environmental conditions may change temporally and complicatedly, we can still accurately estimate the visibility that the driver perceives in an actual traffic scene. We also investigate the performance of the proposed method and show its effectiveness.

Road Image Update using In-vehicle Camera Images and Aerial Image Reviewed

Masafumi Noda, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Yoshiko Kojima, Takashi Naito

Proceedings of 2011 IEEE Intelligent Vehicles Symposium (IV2011) page： 460 - 465 2011.6

Language：English Publishing type：Research paper (international conference proceedings)

Road image is becoming important for several
applications such as car navigation systems, traffic environment
research, city modeling. Usually, a road image can be obtained
from an aerial image but the resolution of the aerial image is
often low, or it contains occlusions by obstacles. Therefore, the
update of road image is required. In this paper, we propose a
road image mosaicing method using in-vehicle camera images
and an aerial image. We first perform image registration of road
regions between these images, and then, we generate a large road
image by performing image mosaicing of road regions in invehicle
camera images. In an experiment, we achieved resolution
improvement and occlusions removal, and also succeeded in
update of a large road image.

3-D Line Segment Reconstruction Using an In-Vehicle Camera for Free Space Detection Reviewed

Hiroyuki Uchiyama, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase

Proceedings of 2011 IEEE Intelligent Vehicles Symposium (IV2011) page： 290 - 295 2011.6

Language：English Publishing type：Research paper (international conference proceedings)

Free space detection is very important for vehicle navigation and safe driving. 3-D line segment reconstruction
of a street is important for the free space detection because a street-view includes many line segments. For the free space
detection, we propose a method for reconstructing 3-D line segments in a streetscape using a monocular in-vehicle camera.
The 3-D reconstruction of the line segments is achieved by using each three images from an image sequence. Once accurate
camera poses of these images are obtained, one of the remaining crucial problems is to match the line segments between the
images correctly. A strategy for finding correspondence of the line segments is as follows: First, the correspondences of line
segment candidates are searched by using a two-view constraint. However, the two-view constraint has difficulty on determining
an unique correspondence geometrically. Therefore, the candidates of the line segment correspondences are reduced using
a three-view constraint. In order to improve the accuracy, the proposed method exploits a color feature of the line segment
and a preliminary knowledge of the vehicle motion. Finally, the line segments are reconstructed using the correspondences.
From an experimental result, we confirmed the effectiveness of the proposed method. Application to the free space detection
demonstrated the usefulness of the reconstructed line segments.

Intelligent Traffic Sign Detector: Adaptive Learning Based on Online Gathering of Training Samples Reviewed

Daisuke Deguchi, Mitsunori Shirasuna, Keisuke Doman, Ichiro Ide, Hiroshi Murase

Proceedings of 2011 IEEE Intelligent Vehicles Symposium (IV2011) page： 72 - 77 2011.6

Authorship：Lead author Language：English Publishing type：Research paper (international conference proceedings)

This paper proposes an intelligent traffic sign detector using adaptive learning based on online gathering of training samples from in-vehicle camera image sequences. To detect traffic signs accurately from in-vehicle camera images, various training samples of traffic signs are needed. In addition, to reduce false alarms, various background images should also be prepared before constructing the detector. However, since their appearances vary widely, it is difficult to obtain them exhaustively by manual intervention. Therefore, the proposed method simultaneously obtains both traffic sign images and background images from in-vehicle camera images. Especially, to reduce false alarms, the proposed method gathers background images that were easily mis-detected by a previously constructed traffic sign detector, and re-trains the detector by using them as negative samples. By using retrospectively tracked traffic sign images and background images as positive and negative training samples, respectively, the proposed method constructs a highly accurate traffic sign detector automatically. Experimental results showed the effectiveness of the proposed method.

Detection of Road Markings Recorded in In-Vehicle Camera Images by Using Position-Dependent Classifiers Reviewed Open Access

Masafumi Noda, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Yoshiko Kojima, Takashi Naito

IEEJ Transactions on Industry Applications Vol. 131 ( 4 ) page： 466 - 474 2011.4

Accurate and Robust Attitude Estimation of a Moving Vehicle Using MEMS Gyroscopes and a Monocular Camera Reviewed

Norimasa Kobori, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase

Proceedings of 2010 Workshop on Picture Coding and Image Processing (WPCIP 2010) 2010.12

Language：English Publishing type：Research paper (international conference proceedings)

In order to estimate accurate attitude of mobile robots and　vehicles, we propose a hybrid system which combines a lowcost　monocular camera with a gyro sensor. A gyro sensor has　drift errors that accumulate over time. On the other hand, a　camera cannot obtain the rotation continuously in the case　where feature points cannot be extracted from images, although　the accuracy is better than a gyro sensor. To solve　these problems, we propose a method for combining these　sensors based on an Extended Kalman Filter.

Ego-Localization by Spatio-Temporal Matching between Videos Obtained from Cameras with Different Viewing Angles Using an Extended DTW Method Reviewed Open Access

Hiroyuki Uchiyama, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J93-D ( 12 ) page： 2659 - 2665 2010.12

Multimedia supplementation to a cooking recipe text for facilitating its understanding to inexperienced users Reviewed

Ichiro Ide, Yuka Shidochi, Yuichi Nakamura, Daisuke Deguchi, Tomokazu Takahashi, Hiroshi Murase

Proceedings of the 2nd Workshop on Cooking and Eating Activities (CEA2010) page： 242 - 247 2010.12

Frontal Face Generation from Multiple Low-Resolution Non-Frontal Faces for Face Recognition Reviewed

Yuki Kono, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of the 10th International Workshop on Visual Surveillance 2010 2010.11

Improvement of a traffic sign detector by retrospective gathering of training samples from in-vehicle camera image sequences Reviewed

Daisuke Deguchi, Keisuke Doman, Ichiro Ide, Hiroshi Murase

Proceedings of Computer Vision in Vehicle Technology: From Earth to Mars (CVVT2010) 2010.11

Vehicle Ego-localization by Matching In-vehicle Camera Images to an Aerial Image Reviewed

Masafumi Noda, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Yoshiko Kojima, Takashi Naito

Proceedings of Computer Vision in Vehicle Technology: From Earth to Mars (CVVT2010) 2010.11

Marker-Free Registration for Electromagnetic Navigation Bronchoscopy under Respiratory Motion

Marco Feuerstein, Takamasa Sugiura, Daisuke Deguchi, Tobias Reichl, Takayuki Kitasaka, Kensaku Mori

Proceedings of the 5th International Workshop on Medical Imaging and Augmented Reality (MIAR2010) Vol. LNCS 6326 page： 237 - 246 2010.9

Construction of a traffic sign detector using generative learning method considering color variation Reviewed Open Access

Keisuke Doman, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

IEICE Transactions on Information and Systems Vol. J93-D ( 8 ) page： 1375 - 1385 2010.8

Removal of Moving Objects from a Street-View Image by Fusing Multiple Image Sequences Reviewed Open Access

Hiroyuki Uchiyama, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase

Proceedings of the 20th IAPR International Conference on Pattern Recognition (ICPR2010) page： 3456 - 3459 2010.8

Classification of near-duplicate video segments based on their appearance patterns Reviewed Open Access

Ichiro Ide, Yuji Shamoto, Daisuke Deguchi, Tomokazu Takahashi, Hiroshi Murase

Proceedings of the 20th IAPR International Conference on Pattern Recognition (ICPR2010) page： 3129 - 3133 2010.8

Estimation of Traffic Sign Visibility Toward Smart Driver Assistance Reviewed Open Access

Keisuke Doman, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase, Yukimasa Tamatsu

Proceedings of 2010 IEEE Intelligent Vehicles Symposium (IV2010) page： 45 - 50 2010.6

Construction of cascaded traffic sign detector using generative learning Reviewed

Keisuke Doman, Daisuke Deguchi, Tomokazu Takahashi, Yoshito Mekada, Ichiro Ide, Hiroshi Murase

Proceedings of the 4th International Conference on Innovative Computing, Information and Control (ICICIC2009) 2009.12

Labeling News Topic Threads with Wikipedia Entries Reviewed

Tomoki Okuoka, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase

Proceedings of the 1st International Workshop on Content-Based Audio/Video Analysis for Novel TV Services (CBTV2009) page： 501 - 504 2009.12

Automated Anatomical Labeling of Bronchial Branches Extracted from CT Datasets Based on Machine Learning and Combination Optimization and Its Application to Bronchoscope Guidance Reviewed

Kensaku Mori, Shunsuke Ota, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Shingo Iwano, Yoshinori Hasegawa, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of the 12th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI2009) Vol. LNCS 5762, Part II page： 707 - 714 2009.9

Language：English Publishing type：Research paper (international conference proceedings)

This paper presents a method for the automated anatomical labeling of bronchial branches extracted from 3D CT images based on machine learning and combination optimization. We also show applications of anatomical labeling on a bronchoscopy guidance system. This paper performs automated labeling by using machine learning and combination optimization. The actual procedure consists of four steps: (a) extraction of tree structures of the bronchus regions extracted from CT images, (b) construction of AdaBoost classifiers, (c) computation of candidate names for all branches by using the classifiers, (d) selection of best combination of anatomical names. We applied the proposed method to 90 cases of 3D CT datasets. The experimental results showed that the proposed method can assign correct anatomical names to 86.9% of the bronchial branches up to the sub-segmental lobe branches. Also, we overlaid the anatomical names of bronchial branches on real bronchoscopic views to guide real bronchoscopy.

DOI： 10.1007/978-3-642-04271-3_86

Low-resolution Character Recognition by Video-based Super-resolution Reviewed

大倉直, 出口大輔, 高橋友和, 井手一郎, 村瀬洋

Proceedings of the 10th International Conference on Document Analysis and Recognition (ICDAR2009) page： 191 - 195 2009.7

Selective image similarity measure for bronchoscope tracking based on image registration Reviewed

Daisuke Deguchi, Kensaku Mori, Marco Feuerstein, Takayuki Kitasaka, Calvin R. Maurer Jr., Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Medical Image Analysis Vol. 13 ( 4 ) page： 621 - 633 2009.6

Authorship：Lead author Language：English Publishing type：Research paper (scientific journal)

We propose a selective method of measurement for computing image similarities based on characteristic structure extraction and demonstrate its application to flexible endoscope navigation, in particular to a bronchoscope navigation system. Camera motion tracking is a fundamental function required for image-guided treatment or therapy systems. In recent years, an ultra-tiny electromagnetic sensor commercially became available, and many image-guided treatment or therapy systems use this sensor for tracking the camera position and orientation. However, due to space limitations, it is difficult to equip the tip of a bronchoscope with such a position sensor, especially in the case of ultra-thin bronchoscopes. Therefore, continuous image registration between real and virtual bronchoscopic images becomes an efficient tool for tracking the bronchoscope. Usually, image registration is done by calculating the image similarity between real and virtual bronchoscopic images. Since global schemes to measure image similarity, such as mutual information, squared gray-level difference, or cross correlation, average differences in intensity values over an entire region, they fail at tracking of scenes where less characteristic structures can be observed. The proposed method divides an entire image into a set of small subblocks and only selects those in which characteristic shapes are observed. Then image similarity is calculated within the selected subblocks. Selection is done by calculating feature values within each subblock. We applied our proposed method to eight pairs of chest X-ray CT images and bronchoscopic video images. The experimental results revealed that bronchoscope tracking using the proposed method could track up to 1600 consecutive bronchoscopic images (about 50s) without external position sensors. Tracking performance was greatly improved in comparison with a standard method utilizing squared gray-level differences of the entire images.

DOI： 10.1016/j.media.2009.06.001

An improved method for automated recognition of bronchial tree structure by optimizing combinations of candidate anatomical names Reviewed

Takayuki Kitasaka, Shunsuke Ota, Daisuke Deguchi, Kensaku Mori, Yasuhito Suenaga, Yoshinori Hasegawa, Kazuyoshi Imaizumi, Shingo Iwano, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

International Journal of Computer Assisted Radiology and Surgery Vol. 4 page： 327 - 328 2009.6

Ego-localization using Streetscape Image Sequences from In-vehicle Cameras Reviewed Open Access

Hiroyuki Uchiyama, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase

Proceedings of 2009 IEEE Intelligent Vehicles Symposium (IV2009) page： 185 - 190 2009.6

Automatic Calibration of an In-vehicle Gaze Tracking System Using Driver's Typical Gaze Behavior Reviewed Open Access

Kenji Yamashiro, Daisuke Deguchi, Tomokazu Takahashi, Ichiro Ide, Hiroshi Murase, Kazunori Higuchi, Takashi Naito

Proceedings of 2009 IEEE Intelligent Vehicles Symposium (IV2009) page： 998 - 1003 2009.6

Recognition of Road Markings from In-Vehicle Camera Images by a Generative Learning Method Reviewed

Masafumi Noda, Tomokazu Takahashi, Daisuke Deguchi, Ichiro Ide, Hiroshi Murase, Yoshiko Kojima, Takashi Naito

Proceedings of IAPR Conference on Machine Vision Applications (MVA) 2009 page： 514 - 517 2009.5

A method for accelerating bronchoscope tracking based on image registration by using GPU Reviewed

Takamasa Sugiura, Daisuke Deguchi, Marco Feuerstein, Takayuki Kitasaka, Yasuhito Suenaga, Kensaku Mori

Proceedings of SPIE Medical Imaging 2009 Vol. 7261 page： 726108 - 1 2009.2

Automatic mediastinal lymph node detection in chest CT Reviewed

Marco Feuerstein, Daisuke Deguchi, Takayuki Kitasaka, Shingo Iwano, Kazuyoshi Imaizumi, Yoshinori Hasegawa, Yasuhito Suenaga, Kensaku Mori

Proceedings of SPIE Medical Imaging 2009 Vol. 7260 page： 1 - 11 2009.2

Development of a Common Platform for Computer-Aided Diagnosis of Medical Images Reviewed Open Access

Yukitaka Nimura, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Kensaku Mori

Medical Imaging Technology Vol. 26 ( 5 ) page： 327 - 337 2008.11

PLUTO: A Common CAD Platform for Multiorgan Multidisease CAD (MoMd-CAD) Reviewed

Yukitaka Nimura, Daisuke Deguchi, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga

RSNA (Radiological Society of North America) Scientific Assembly and Annual Meeting Program 2008 page： 904 2008.11

A Method for Accelerating Bronchoscope Tracking Based on Image Registration by GPGPU Reviewed

Takamasa Sugiura, Daisuke Deguchi, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga

Proceedings of International Workshop on Augmented environments for Medical Imaging and Computer-aided Surgery (AMI-ARCS) 2008 page： 130 - 139 2008.9

Improvement of Accuracy of Marker-Free Bronchoscope Tracking Using Electromagnetic Tracker Based on Bronchial Branch Information Reviewed

Kensaku Mori, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Yoshinori Hasegawa, Kazuyoshi Imaizumi, Hirotsugu Takabatake

Proceedings of the 11th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI2008) Vol. LNCS 5242, Part II page： 535 - 542 2008.9

Augmented display of anatomical names of bronchial branches for bronchoscopy assistance Reviewed

Shunsuke Ota, Daisuke Deguchi, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Yoshinori Hasegawa, Kazuyoshi Imaizumi, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of the 4th International Workshop on Medical Imaging and Augmented Reality (MIAR2008) Vol. LNCS 5128 page： 377 - 384 2008.8

An improved method for bronchoscope tracking using an electromagnetic tracker without fiducial markers Reviewed

Daisuke Deguchi, Kazuyoshi Ishitani, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Yoshinori Hasegawa, Kazuyoshi Imaizumi

Proceedings of the 22st International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2008) Vol. 3 page： S301 2008.6

Automated anatomical labeling of bronchaial branches using multiple classifiers and its application to bronchoscopy guidance Reviewed

Shunsuke Ota, Daisuke Deguchi, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Yoshinori Hasegawa, Kazuyoshi Imaizumi, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of the 22st International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2008) Vol. 3 page： S269 2008.6

Fiducial-free bronchoscope tracking using ultra-tiny electromagnetic tracker based on non-rigid transformation technique Reviewed

Kensaku Mori, Kazuyoshi Ishitani, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Yoshinori Hasegawa, Kazuyoshi Imaizumi

Proceedings of the 22st International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2008) Vol. 3 page： S105 2008.6

Automated anatomical labeling of bronchial branches using multiple classifiers and its application to bronchoscopy guidance based on fusion of virtual and real bronchoscopy Reviewed

Shunsuke Ota, Daisuke Deguchi, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Yoshinori Hasegawa, Kazuyoshi Imaizumi, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of SPIE Medical Imaging 2008 Vol. 6916 page： 1 - 12 2008.2

Bronchoscope Tracking Without Fiducial Markers Using Ultra-tiny Electromagnetic Tracking System and Its Evaluation in Different Environments Reviewed

Kensaku Mori, Daisuke Deguchi, Kazuyoshi Ishitani, Takayuki Kitasaka, Yasuhito Suenaga, Yoshinori Hasegawa, Kazuyoshi Imaizumi, Hirotsugu Takabatake

Proceedings of the 10th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI2007) Vol. LNCS 4792, Part II page： 644 - 651 2007.10

A method for bronchoscope tracking using position sensor without fiducial markers Reviewed

Daisuke Deguchi, Kazuyoshi Ishitani, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Yoshinori Hasegawa, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of the 21st International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2007) Vol. 2 page： S11-S13 2007.6

A method for bronchoscope tracking using position sensor without fiducial markers Reviewed

Daisuke Deguchi, Kazuyoshi Ishitani, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of SPIE Medical Imaging 2007 Vol. 6511 page： 1 - 0 2007.2

Easy and stable bronchoscope camera calibration technique for bronchoscope navigation system Reviewed

Kazuyoshi Ishitani, Daisuke Deguchi, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of SPIE Medical Imaging 2007 Vol. 6509 page： 1 - 1 2007.2

Compensation of Electromagnetic trackintg system using an optical tracker and its application to bronchoscopy navigation system Reviewed

Kensaku Mori, Kazuyoshi Ishitani, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of SPIE Medical Imaging 2007 Vol. 6509 page： 1 - 0 2007.2

Evaluation of a prostate biopsy strategy for cancer detection using a computer simulation system with virtual needle biopsy for three-dimensional prostate models Reviewed

Masanori Noguchi, Daisuke Deguchi, Junichiro Toriwaki, Kensaku Mori, Yoshito Mekada, Kei Matsuoka

International Journal of Urology Vol. 13 ( 10 ) page： 1296 - 1303 2006.10

Language：English Publishing type：Research paper (scientific journal)

AIM:We evaluated a prostate biopsy strategy for cancer detection using a computer simulation system with virtual needle biopsy for three-dimensional (3D) prostate models.
METHODS:Two 3D prostate models with a volume of 25 cc or 50 cc were constructed from computed tomographic images of radical prostatectomy specimens. The peripheral zone (PZ) and transition zone (TZ) were arranged in the prostate models according to the anatomical information. Four thousand patterns of cancer lesions were automatically inserted into each prostate model with a proportion of 75% in PZ and 25% in TZ. Average hit rates (AHR) in prostate models were evaluated both with an increased number of biopsy cores and various angles of virtual needle biopsy. The influence of adding secondary tumors for hit rates was also evaluated.
RESULTS:For both sizes, the laterally angled biopsy in 4-8 core biopsy schemes showed higher AHR than the vertical plane biopsy, while the vertical plane biopsy in 10-18 core biopsy schemes showed higher AHR than the laterally angled biopsy. A higher number of biopsy cores increased the AHR of secondary tumors.
CONCLUSIONS:Our results suggest that it is important in prostate cancer detection to change the needle placement according to the number of biopsy cores and the size of the prostate.

DOI： 10.1111/j.1442-2042.2006.01561.x

A preliminary study on introduction of a magnetic tracking sensor for bronchoscope navigation system Reviewed

Kazuyoshi Ishitani, Daisuke Deguchi, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of International Workshop on Augmented environments for Medical Imaging and Computer-aided Surgery (AMI-ARCS) 2006 2006.10

Bronchoscope Tracking Based on Image Registration Using Multiple Initial Starting Points Estimated by Motion Prediction Reviewed

Kensaku Mori, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori, Calvin R. Maurer Jr.

Proceedings of the 9th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI2006) Vol. LNCS 4191, Part II page： 645 - 652 2006.10

Detection of liver cancer regions from dynamic CT images Reviewed

Yuichiro Hayashi, Daisuke Deguchi, Takayuki Kitasaka, Kensaku Mori, Yoshito Mekada, Yasuhito Suenaga

Proceedings of the 20th International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2006) Vol. 1 page： 524 - 525 2006.6

Image-based bronchoscope tracking using motion predication and multiple searches Reviewed

Kensaku Mori, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of the 20th International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2006) Vol. 1 page： 178 - 181 2006.6

A method for bronchoscope tracking by combining a position sensor and image registration Reviewed

Daisuke Deguchi, Kenta Akiyama, Kensaku Mori, Takayuki Kitasaka, Yasuhito Suenaga, Calvin R. Maurer Jr., Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Computer Aided Surgery Vol. 11 ( 3 ) page： 109 - 117 2006.5

Authorship：Lead author Language：English Publishing type：Research paper (scientific journal)

This paper describes a method for tracking a bronchoscope by combining a position sensor and image registration. A bronchoscopy guidance system is a tool for providing real-time navigation information acquired from pre-operative CT images to a physician during a bronchoscopic examination. In this system, one of the fundamental functions is tracking a bronchoscope's camera motion. Recently, a very small electromagnetic position sensor has become available. It is possible to insert this sensor into a bronchoscope's working channel to obtain the bronchoscope's camera motion. However, the accuracy of its output is inadequate for bronchoscope tracking. The proposed combination of the sensor and image registration between real and virtual bronchoscopic images derived from CT images is quite useful for improving tracking accuracy. Furthermore, this combination has enabled us to achieve a real-time bronchoscope guidance system. We performed evaluation experiments for the proposed method using a rubber phantom model. The experimental results showed that the proposed system allowed the bronchoscope's camera motion to be tracked at 2.5 frames per second.

Branch identification method for CT-guided bronchoscopy based on eigenspace image matching between real and virtual bronchoscopic images Reviewed

Riyoko Shinohara, Kensaku Mori, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of SPIE Medical Imaging 2006 Vol. 6143 page： 14 - 1 2006.2

A method for automated liver region extraction basing upon estimation of CT value distributions from multi-phase CT images Reviewed Open Access

Daisuke Deguchi, Yuichiro Hayashi, Takayuki Kitasaka, Kensaku Mori, Yoshito Mekada, Yasuhito Suenaga, Jun-ichi Hasegawa, Junichiro Toriwaki

Journal of Computer Aided Diagnosis of Medical Images Vol. 9 ( 4 ) page： 36 - 48 2006.1

Development of a Virtual Needle Biopsy Simulation System for the Virtual Prostate Reviewed

Daisuke Deguchi, Kensaku Mori, Yoshito Mekada, Jun-ichi Hasegawa, Junichiro Toriwaki, Masanori Noguchi

Systems and Computers in Japan Vol. 37 ( 1 ) page： 93 - 104 2006.1

Authorship：Lead author Language：English Publishing type：Research paper (scientific journal)

This paper discusses the virtual prostate needle biopsy system and the construction of the virtual prostate model, considering the actual distribution of prostate abnormalities. A needle biopsy simulation is performed for the constructed virtual prostate, and the actual biopsy procedure in the clinical situation is evaluated. The prostate needle biopsy is a histologic diagnosis procedure in which a sample is acquired from the prostate tissue by needle biopsy and is inspected under a microscope. In order to achieve reliable prostate needle biopsy, it is necessary to consider systematically and numerically the number of needles required, and their locations and insertion angles.
For such a purpose, a system is developed in which a virtual prostate is constructed on a computer and the biopsy procedure is evaluated quantitatively by performing virtually the prostate needle biopsy. Furthermore, two different models are constructed, for the prostate with and without hypertrophy, based on the actual statistical distribution data for
prostate abnormalities. Each model is partitioned into the peripheral zone and the transition zone. The constructed
virtual prostate is input into the virtual needle biopsy simulation system, and three different systematic biopsy procedures actually used at clinical sites, and four different needle biopsy procedures, are experimentally evaluated.
The experiments show that the insertion angle that maximizes the hit probability is not always the same as the
insertion angle that maximizes cancer sample acquisition. It is evident that the proposed method can indicate a biopsy
procedure which realizes a high hit probability with a small number of needles.

DOI： 10.1002/scj.20181

Hybrid Bronchoscope Tracking Using a Magnetic Tracking Sensor and Image Registration Reviewed

Kensaku Mori, Daisuke Deguchi, Kenta Akiyama, Takayuki Kitasaka, Calvin R. Maurer Jr., Yasuhito Suenaga, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of the 8th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI2005) Vol. LNCS 3750, Part II page： 543 - 550 2005.10

A method for bronchoscope tracking by combining a position sensor and image registration Reviewed

Daisuke Deguchi, Kenta Akiyama, Takayuki Kitasaka, Kensaku Mori, Yasuhito Suenaga, Jun-ichi Hasegawa, Junichiro Toriwaki, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori

Proceedings of the 19th International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2005) Vol. 1281 page： 630 - 635 2005.6

Development of a method for automated liver region extraction from contrasted 3D abdominal X-ray CT images Reviewed Open Access

Yuichiro Hayashi, Daisuke Deguchi, Kensaku Mori, Yoshito Mekada, Yasuhito Suenaga, Junichiro Toriwaki

Journal of Computer Aided Diagnosis of Medical Images Vol. 8 ( 1_3 ) page： 18 - 30 2004.11

Fast and Accurate Bronchoscope Tracking using Image Registration and Motion Prediction Reviewed

Jiro Nagao, Kensaku Mori, Tsutomu Enjoji, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Jun-ichi Hasegawa, Junichiro Toriwaki, Hirotsugu Takabatake, Hiroshi Natori

Proceedings of the 7th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI2004) Vol. LNCS 3217, Part II page： 551 - 558 2004.9

Improved camera-tracking method by combining motion prediction and image registration for bronchoscope navigation system Reviewed

Kensaku Mori, Tsutomu Enjoji, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Junichiro Toriwaki, Hiroshi Natori, Hirotsugu Takabatake

Proceedings of the 18th International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2004) Vol. 1268 page： 1261 2004.6

New image similarity measures for bronchoscope tracking based on image registration between virtual and real bronchoscopic images Reviewed

Kensaku Mori, Tsutomu Enjoji, Daisuke Deguchi, Takayuki Kitasaka, Yasuhito Suenaga, Junichiro Toriwaki, Hirotsugu Takabatake, Hiroshi Natori

Proceedings of SPIE Medical Imaging 2004 Vol. 5639 page： 165 - 176 2004.2

Development of a simulation system of the virtual needle biopsy for the virtual prostate Reviewed

Daisuke Deguchi, Kensaku Mori, Yoshito Mekada, Jun-ichi Hasegawa, Junichiro Toriwaki, Masanori Noguchi

IEICE Transactions on Information and Systems Vol. J87-D-II ( 1 ) page： 281 - 289 2004.1

New Image Similarity Measure for Bronchoscope Tracking Based on Image Registration Reviewed

Daisuke Deguchi, Kensaku Mori, Yasuhito Suenaga, Jun-ichi Hasegawa, Junichiro Toriwaki, Hirotsugu Takabatake, Hiroshi Natori

Proceedings of the 6th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI2003) Vol. LNCS 2878 page： 399 - 406 2003.11

New calculation method of image similarity for endoscope tracking based on image registration in endoscope navigation Reviewed

Daisuke Deguchi, Kensaku Mori, Yasuhito Suenaga, Jun-ichi Hasegawa, Junichiro Toriwaki, Hiroshi Natori, Hirotsugu Takabatake

Proceedings of the 17th International Congress and Exhibition on Computer Assisted Radiology and Surgery (CARS2003) Vol. 1256 page： 460 - 466 2003.6

Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images Reviewed

Kensaku Mori, Daisuke Deguchi, Jun Sugiyama, Yasuhito Suenaga, Junichiro Toriwaki, Calvin R. Maurer Jr., Hirotsugu Takabatake, Hiroshi Natori

Medical Image Analysis Vol. 6 page： 321 - 336 2002.9

Language：English Publishing type：Research paper (scientific journal)

This paper describes a method for tracking the camera motion of a flexible endoscope, in particular a bronchoscope, using epipolar geometry analysis and intensity-based image registration. The method proposed here does not use a positional sensor attached to the endoscope. Instead, it tracks camera motion using real endoscopic (RE) video images obtained at the time of the procedure and X-ray CT images acquired before the endoscopic examination. A virtual endoscope system (VES) is used for generating virtual endoscopic (VE) images. The basic idea of this tracking method is to find the viewpoint and view direction of the VES that maximizes a similarity measure between the VE and RE images. To assist the parameter search process, camera motion is also computed directly from epipolar geometry analysis of the RE video images. The complete method consists of two steps: (a) rough estimation using epipolar geometry analysis and (b) precise estimation using intensity-based image registration. In the rough registration process, the method computes camera motion from optical flow patterns between two consecutive RE video image frames using epipolar geometry analysis. In the image registration stage, we search for the VES viewing parameters that generate the VE image that is most similar to the current RE image. The correlation coefficient and the mean square intensity difference are used for measuring image similarity. The result obtained in the rough estimation process is used for restricting the parameter search area. We applied the method to bronchoscopic video image data from three patients who had chest CT images. The method successfully tracked camera motion for about 600 consecutive frames in the best case. Visual inspection suggests that the tracking is sufficiently accurate for clinical use. Tracking results obtained by performing the method without the epipolar geometry analysis step were substantially worse. Although the method required about 20 s to process one frame, the results demonstrate the potential of image-based tracking for use in an endoscope navigation system.

DOI： 10.1016/S1361-8415(02)00089-0

Camera motion tracking of real bronchoscope using epipolar geometry analysis and CT derived bronchoscopic images Reviewed

Daisuke Deguchi, Kensaku Mori, Jun-ichi Hasegawa, Junichiro Toriwaki, Hirotsugu Takabatake, Hiroshi Natori

Proceedings of SPIE Medical Imaging 2002 Vol. 4683 page： 30 - 41 2002.2

A method for tracking the camera motion of real endoscope by epipolar geometry analysis and virtual endoscopy system Reviewed

Kensaku Mori, Daisuke Deguchi, Jun-ichi Hasegawa, Yasuhito Suenaga, Junichiro Toriwaki, Hirotsugu Takabatake, Hiroshi Natori

Proceedings of the 4th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2001 Vol. LNCS 2208 page： 1 - 8 2001.10

　 PREV - NEXT 　

<Deguchi Daisuke>

Papers - Deguchi Daisuke