Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

Award type：Award from international society, conference, symposium, etc.  Country：Japan

Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

2件受賞

Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

Award type：Honored in official journal of a scientific society, scientific journal  Country：Japan

Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

Award type：Award from Japanese society, conference, symposium, etc.  Country：Japan

Award type：Award from international society, conference, symposium, etc. 

Language：English   Publishing type：Research paper (scientific journal)  

The end-to-end driving models (E2EDMs) convert environmental information into driving actions using a complex transformation which makes E2EDMs have high prediction accuracy. Due to the black-box nature of transformation, the E2EDMs have low explainability. To solve this problem, explanation methods are used to generate explanations for observation. Based on current explanation methods, previous studies tried to further improve the explainability of E2EDMs by integrating an object detection module, however, these methods have many problems: Firstly, due to the requirement of the object detection module, they lack flexibility. Secondly, they neglect an essential property, i.e. , simplicity, to improve explainability. In this paper, since humans prefer object-level and simple explanations in driving tasks, we argue that explainability is decided by two properties which are the objectification degree (the extent to which driving related-object features are utilized) and simplification degree (the simplicity of the explanation), thus we propose Simplified Objectification Branches (SOB) to improve the explainability of E2EDMs. Firstly, this structure could be integrated into any existing E2EDMs and thus have high flexibility. Secondly, the SOB explicitly improves the simplification degree without sacrificing the objectification degree of the explanations. By designing several indicators, i.e. , heatmap satisfaction, driving action reproduction score, deception level, etc. , we proved that SOB could help E2EDMs generate better explanations. Notably, the SOB could also further enhance E2EDMs’ prediction accuracy.

DOI： 10.1109/TITS.2024.3385754

Language：English   Publishing type：Research paper (scientific journal)  

Semantic segmentation models comprise an encoder to extract features and a classifier for prediction. However, the learning of the classifier suffers from the ambiguity which is caused by two factors: (1) the weights of a classifier for similar categories may have positive similarities lowing the performance for similar categories, named correlation ambiguity, and (2) the classifier is prone to predict the category with a larger L2 norm and vice versa, termed prior ambiguity. To comedy the issues, we propose Category-Basis Prototype (CBP), frozen and mutually orthogonalized prototypes with equal L2 norm. Orthogonalization prevents the prototypes from being similar to each other and the equality decouples the prediction from the L2 norm. To better shape the feature space, we propose Online Centroid Contrastive Loss (OCCL) equipped with centroid and category- level losses. Experiments show that our method yields compelling results over two widely applied benchmarks indicating the effectiveness of our methods.

DOI： 10.1016/j.patcog.2024.110431

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.14923/transinfj.2023PDP0024

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/ACCESS.2024.3376656

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/ACCESS.2024.3372658

Language：English   Publishing type：Research paper (international conference proceedings)  

Deep learning technology has rapidly advanced, leading to the development of End-to-End driving models (E2EDMs) for autonomous vehicles with high prediction accuracy. To comprehend the prediction results of these E2EDMs, one of the most representative explanation methods is attribution-based. There are two kinds of attribution-based explanation methods: pixel-level and object-level. Usually, the heatmaps illustrate the importance of pixels and objects in the prediction results, serving as explanations for E2EDMs. Since there are many attribution-based explanation methods, evaluation methods are proposed to determine which one is better at improving the explainability of E2EDMs.
Fidelity measures the explanation's faithfulness to the model's prediction method, which is a bottommost property. However, no evaluation method could measure the fidelity difference between object-level and pixel-level explanations, making the current evaluation incomplete. In addition, without considering fidelity, previous evaluation methods may advertise manipulative explanations that solely seek human satisfaction (persuasibility). Therefore, we propose an evaluation method that further considers fidelity, our method enables a comprehensive evaluation that proves the object-level explanations genuinely outperform pixel-level explanations in fidelity and persuasibility, thus could better improve the explainability of the E2EDMs.

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we present a new task of detecting an object that a target pedestrian is gazing at in a traffic scene called PEdestrian's Gaze Object (PEGO). We argue that the detection of gaze object can provide important information for pedestrian's behavior prediction and can contribute to the realization of automated vehicles. For this task, we construct a dataset of in-vehicle camera images with annotations of the objects that pedestrians are gazing at. Also, we propose a Transformer-based method called PEGO Transformer to solve the PEGO detection task. The PEGO Transformer directly performs gaze object detection with the utilization of whole-body features without a high-resolution head image and a gaze heatmap which the traditional methods rely on. Experimental results showed that the proposed method could estimate pedestrian's gaze object accurately even if various objects exist in the scene.

Language：English   Publishing type：Research paper (scientific journal)  

Although many advanced works have achieved significant progress for face recognition with deep learning and large-scale face datasets, low-quality face recognition remains a challenging problem in real-word applications, especially for unconstrained surveillance scenes. We propose a texture-guided (TG) transfer learning approach under the knowledge distillation scheme to improve low-quality face recognition performance. Unlike existing methods in which distillation loss is built on forward propagation; e.g., the output logits and intermediate features, in this study, the backward propagation gradient texture is used. More specifically, the gradient texture of low-quality images is forced to be aligned to that of its high-quality counterpart to reduce the feature discrepancy between the high-and low-quality images. Moreover, attention is introduced to derive a soft-attention (SA) version of transfer learning, termed as SA-TG, to focus on informative regions. Experiments on the benchmark low-quality face DB’s TinyFace and QMUL-SurFace confirmed the superiority of the proposed method, especially more than 6.3% Rank1 accuracy improvement is achieved on TinyFace.

DOI： 10.1109/TIP.2023.3335830

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1007/s11042-023-15732-z

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/IV55152.2023.10186742

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.14923/transfunj.2022HAP0006

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

With the rapid development of autonomous driving technology, a variety of high-performance end-to-end driving models (E2EDMs) are being proposed. In order to understand the computational methods of E2EDMs, pixel-level explanations methods are used to obtain the explanations of the E2EDMs. However, little attention has been paid to the excellence of the explanations of E2EDMs. Therefore, in order to build trustworthy E2EDMs, we focus on improving the persuasibility of the explanations of E2EDMs. We propose an object-level explanation method (main approach) for E2EDMs, which masks the objects in the image and then treats the change in the prediction result as the importance of the objects, then we explain the E2EDM by the importance of each object. To further validate the effectiveness of object-level explanations, we propose another approach (validation approach), which trains E2EDMs with object information as input and generates the importance of objects using general explanation methods. Both approaches generate object-level explanations, in order to compare these object-level explanations with traditional pixel-level explanations, we propose experimental methods to measure the persuasibility of explanations of E2EDMs through a subjective and objective method. The subjective method evaluates persuasibility based on the extent to which participants think the importance of features indicated by the explanations is correct. The objective method evaluates the persuasibility based on the human annotation similarity between provided with only the important part of images and provided with the complete images. The experimental results show that the object-level explanations are more persuasive than the traditional pixel-level explanations.

DOI： 10.1109/ACCESS.2023.3235739

Language：English   Publishing type：Research paper (international conference proceedings)  

We analyze gaze while viewing videos at various playback speeds.
The aims of this paper are to evaluate the comprehension and viewability of videos.
In our experiment, we used 0.75x to 3.0x speed, and measured gaze.
We report differences in gaze behavior among different playback speeds in viewing same video.

Language：English   Publishing type：Research paper (international conference proceedings)  

We propose a method to detect birds in a 3D environment referring to both audio information observed from a microphone array and visual information observed from a panorama camera. In general, in panorama images, birds appear relatively too small to be detected accurately even with the state-of-the-art deep learning models. Thus, the proposed method takes a two step approach where the birds are first roughly located referring to audio information by Sound Source Localization (SSL), and then image detection is applied within its vicinity. Through evaluation on a dataset annotated with bounding boxes surrounding the birds, we show that the proposed method improves detection performance of birds that appear in relatively small sizes in the image, in both accuracy and processing speed.

DOI： 10.1109/AVSS56176.2022.9959510

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.11184/his.24.3_195

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.20729/00218987

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/MIPR54900.2022.00050

Language：English   Publishing type：Research paper (scientific journal)  

The difficulties in video-based face recognition, such as dramatic pose variations and low quality, can be alleviated by leveraging the rich complementary information between the frames. However, limited by the mini-batch training strategy, the current deep learning methods only utilizes the frames in each batch during training, which ignore the context of the entire video. In this paper, we propose a context-aware feature aggregation scheme to aggregate complementary information between different frames. Firstly, a two-branch structure is designed as the C ontext-aware feature A ggregation N etwork ( CAN ). Secondly, a context-aware training strategy using a context bank is proposed, which alleviates the limitation of mini-batch samples by using the context of the entire video or several images belonging to the same ID and thus achieves global contribution estimation result. Comparative studies on benchmark datasets, such as IJB-C, YouTube Face (YTF), PaSC and COX, confirm that the proposed approach can achieve state-of-the-art level. Meanwhile, qualitative analysis on Multi-PIE dataset indicates that the contribution learned by the CAN is reasonable and beneficial to video face recognition.

DOI： 10.1109/ACCESS.2022.3193787

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.14923/transfunj.2021JAP1021

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.14923/transfunj.2021JAL2005

Language：English   Publishing type：Research paper (scientific journal)  

Face masks bring a new challenge to face recognition systems especially against the background of the COVID-19 pandemic. In this paper, a method mitigating the negative effects of mask defects on face recognition is proposed. Firstly, a low-cost, accurate method of masked face synthesis, i.e. mask transfer, is proposed for data augmentation. Secondly, an attention-aware masked face recognition (AMaskNet) is proposed to improve the performance of masked face recognition, which includes two modules: a feature extractor and a contribution estimator. Therein, the contribution estimator is employed to learn the contribution of the feature elements, thus achieving refined feature representation by simple matrix multiplications. Meanwhile, the end-to-end training strategy is utilized to optimize the entire model. Finally, a mask-aware similarity matching strategy(MS) is taken to improve the performance in the inference stage. The experiments show that the proposed method consistently outperforms on three masked face recognition datasets: RMFRD [1], COX [2] and Public-IvS [3]. Meanwhile, qualitative analysis experiments using CAM [4] indicate that the contribution learned by AMaskNet is more conducive to masked face recognition.

DOI： 10.1109/ACCESS.2022.3150345

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.patrec.2021.07.014

Language：English   Publishing type：Research paper (international conference proceedings)  

For computers to understand human perception, metrics that can capture human perception well are important. However, there are few metrics that characterize the visual perception of humans towards images. Therefore, in this paper, we propose a novel concept and a metric of pointedness of an image, which describes how pointy an image is perceived. The algorithm is inspired by the Features from Accelerated Segment Test (FAST) algorithm for corner detection which looks on the number of continuous neighboring darker pixels surrounding each pixel. We assume that this number would be proportional to the perceived pointedness in the region around the pixel. We evaluated our method towards how well it could capture the human perception of images. To compare the method with similar metrics that describe shapes, we prepared silhouette images of both artificial shapes and natural objects. The results showed that the proposed method gave nearly equivalent perceptual performance to other metrics and also worked in a larger variety of images.

DOI： https://doi.org/10.1007/978-3-030-78635-9_20

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.2493/jjspe.87.440

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.2493/jjspe.87.447

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.2493/jjspe.87.197

Language：English   Publishing type：Research paper (scientific journal)  

Semantic segmentation has become one of the trending topics in the world of computer vision and deep learning. Recently, due to an increasing demand to solve a semantic segmentation task simultaneously with attribute recognition of objects, a new task named attribute-aware semantic segmentation has been introduced. Since the task requires to handle pixel-wise object class estimation with its attributes such as a pedestrian’s body orientation, previous works had difficulties to handle ambiguous attributes such as body orientations in object-level, especially when segmenting the pedestrians with their attributes correctly. This paper proposes the ColAtt-Net that is an attribute-aware semantic segmentation model augmented by a column-wise mask branch to predict the pedestrians’ orientations in the horizontal perspective of the input image. We firmly assume that the pedestrians captured by a car-mounted camera are distributed horizontally so that for each column of the input image, the pedestrian pixels can be labeled with one orientation uniformly. In the proposed method, we split the output of the base semantic segmentation model into two branches; one branch for segmenting the object categories, while the other one, as the novel column-wise attribute branch, is to map the recognition of pedestrian’s orientations that are distributed horizontally. This method successfully enhances the performance of attribute-aware semantic segmentation by reducing the ambiguity on segmenting the pedestrian’s orientation. Improvements on the pedestrian orientation segmentation are confidently shown by the proposed method in the experimental results, both in quantitative and qualitative views. This paper also discusses how the improved performance becomes an advantage in the autonomous driving system.

DOI： https://doi.org/10.1002/tee.23296

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose a method for human pose estimation from a Low-resolution Far-InfraRed (LFIR) image sequence captured by a 16 × 16 FIR sensor array. Human body estimation from such a single LFIR image is a hard task. For training the estimation model, annotation of the human pose to the images is also a difficult task for human. Thus, we propose the LFIR2Pose model which accepts a sequence of LFIR images and outputs the human pose of the last frame, and also propose an automatic annotation system for the model training. Additionally, considering that the scale of human body motion is largely different among body parts, we also propose a loss function focusing on the difference. Through an experiment, we evaluated the human pose estimation accuracy using an original data set, and confirmed that human pose can be estimated accurately from an LFIR image sequence.

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose Object Manifold Embedding GAN (Ω-GAN) to generate images of variously shaped and arbitrarily posed objects from a noise variable sampled from a distribution defined over the pose and the shape manifolds in a vector space. We introduce Parametric Manifold Sampling to sample noise variables from a distribution over the pose manifold to conditionally generate object images in arbitrary poses by tuning the pose parameter. We also introduce Object Identity Loss for clearly disentangling the pose and shape parameters, which allows us to maintain the shape of the object instance when only the pose parameter is changed. Through evaluation, we confirmed that the proposed Ω-GAN could generate variously shaped object images in arbitrary poses by changing the pose and shape parameters independently. We also introduce an application of the proposed method for object pose estimation, through which we confirmed that the object poses in the generated images are accurate.

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose an occlusion-robust pose
estimation method of an unknown object instance in an
object category from a depth image. In a cluttered
environment, objects are often occluded mutually. For
estimating the pose of an object in such a situation, a
method that de-occludes the unobservable area of the object
would be effective. However, there are two difficulties;
occlusion causes offset between the actual object center
and the center of its observable area, and different
instances in a category may have different shapes. To cope
with these difficulties, we propose a two-stage
Encoder-Decoder model to extract features with objects
whose centers are aligned to the image center. In the
model, we also propose the Median-shape Reconstructor as
the second stage to absorb shape variations in a category.
By evaluating the method with both a large-scale virtual
dataset and a real dataset, we confirmed the proposed
method achieves good performance on pose estimation of an
occluded object from a depth image.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Imageability is a concept from Psycholinguistics quantizing the human perception of words. However, existing datasets are created through subjective experiments and are thus very small. Therefore, methods to automatically estimate the imageability can be helpful. For an accurate automatic imageability estimation, we extend the idea of a psychological hypothesis called Dual-Coding Theory, that discusses the connection of our perception towards visual information
and language information, and also focus on the relationship between the pronunciation of a word and its imageability. In this research, we propose a method to estimate imageability of words
using both visual and language features extracted from corresponding data. For the estimation, we use visual features extracted from low- and high-level image features, and language features extracted from textual features and phonetic features of words. Evaluations how that our proposed method can estimate imageability more accurately than comparative methods, implying the contribution of each feature to the imageability.

DOI： 10.1145/3372278.3390731

Language：English   Publishing type：Research paper (international conference proceedings)  

It is intuitively obvious that eye-gaze behaviors of experienced drivers are different from those of novice drivers. However, it is not easy to understand the difference in their behavior quantitatively. In this work, we present an explainable eye-gaze behavior modeling method for electric wheelchair drivers based on Inverse Reinforcement Learning (IRL). We first create feature maps that represent risk factors during driving. These feature maps are able to represent not only to what but also from where drivers pay attention. IRL uses the feature maps to learn the reward representing the eye-gaze behaviors and allows us to see important features via the automatic acquisition of the reward. Through analysis of the learned model, we show quantitative evidence that eye-gaze behaviors of experienced drivers are better-balanced by paying attention to multiple risks.

DOI： 10.1109/ITSC45102.2020.9294255

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1007/s11042-019-08571-4

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper is an extension of our work in developing an attribute-aware semantic segmentation method which focuses on pedestrian understanding in a traffic scene. Recently, the trending topic of semantic segmentation has been expanded to be able to collaborate with the object’s attributes recognition task; Here, it refers to recognizing a pedestrian’s body orientation. The attribute-aware semantic segmentation can be more beneficial for driver assistance compared to the conventional semantic segmentation because it can provide a more informative output to the system. In this paper, we conduct a study of the data augmentation usage as an effort to enhance the performance of the attribute-aware semantic segmentation task. The experiments show that the proposed method in augmenting the training data is able to improve the model’s performance. We also demonstrate some of qualitative results and discuss the benefits to a driver assistance system.

DOI： 10.1109/ICoICT49345.2020.9166219

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

In recent years, various facilities have been deployed to support visually impaired people. However, accidents caused by visual disabilities still occur. In this paper, to support the visually impaired people in public areas, we aim to identify whether a pedestrian image sequence obtained by a surveillance camera indicates the presence of a white cane user by analyzing the temporal transition of a human skeleton represented as 2D coordinates. Our previously proposed method aligns the orientation of the human skeletons to various orientations and identifies a white cane user from the corresponding sequences, relying on multiple classifiers related to each orientation. The method employs an exemplar-based approach to perform the alignment. However, it heavily depends on the number of exemplars and consumes excessive memory. In this paper, we propose a method to align 2D human skeleton representation sequence to various orientations using the proposed Skeleton Orientation Alignment Networks (SOANets) based on an encoder-decoder model. Using SOANets, we can obtain 2D skeleton representation sequences aligned to various orientations, extract richer skeleton features, and recognize white cane users accurately. We conducted an experiment to confirm that the
proposed method improves the recognition rate by 16%, compared to the method that does not use the skeleton
orientation alignment.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Surveillance cameras are expected to play a large role in the development of ITS technologies. They can be used to detect abnormally behaving individuals which can then be reported to drivers nearby. There are multiple works that tackle the problem of abnormal behavior detection. However, most of these works make use of appearance features which have redundant information and are susceptible to noise. While there are also works that make use of pose skeleton representation, they do not consider well how to handle cases with occlusions, which can occur due to the simple reason of pedestrian orientation preventing some joints from appearing in the frame clearly. In this paper, we propose a skeleton trajectory representation that enables handling of occlusions. We also propose a framework for pedestrian abnormal behavior detection that uses the proposed representation and detect relatively hard-to-notice anomalies such as drunk walking. The experiments we conducted show that our method outperforms other representation methods.

Language：English   Publishing type：Research paper (international conference proceedings)  

Recent multimedia applications commonly use text and imagery from Social Media for tasks related to sentiment research. As such, there are various image datasets for sentiment research for popular classification tasks. However, there has been little research regarding the relationship between the sentiment of images and its annotations from a multi-modal standpoint. In this demonstration, we built a tool to visualize psycholinguistic groundings for a sentiment dataset. For each image, individual psycholinguistic ratings are computed from the image's metadata. A sentiment-psycholinguistic spatial embedding is computed to show a clustering of images across different classes close to human perception. Our interactive browsing tool can visualize the data in various ways, highlighting different psycholinguistic groundings with heatmaps.

DOI： 10.3758/s13428-018-1099-3

Language：English   Publishing type：Research paper (international conference proceedings)  

A mimetic word is used to verbally express the manner of a phenomenon intuitively. The Japanese language is known to have a greater number of mimetic words in its vocabulary than most other languages. Especially, since human gaits are one of the most commonly represented behavior by mimetic words in the language, we consider that it should be suitable for labels of fine-grained gait recognition. In addition, Japanese mimetic words have a more decomposable structure than these in other languages such as English. So it is said that they have sound-symbolism and their phonemes are strongly related to the impressions of various phenomena. Thanks to this, native Japanese speakers can express their impressions on them briefly and intuitively using various mimetic words. Our previous work proposed a framework to convert the body-parts movements to an arbitrary mimetic word by a regression model. The framework introduced a phonetic space" based on sound-symbolism, and it enabled fine-grained gait description using the generated mimetic words consisting of an arbitrary combination of phonemes. However, this method did not consider the "naturalness" of the description. Thus, in this paper, we propose an improved mimetic word generation module considering its naturalness, and update the description framework. Here, we define the co-occurrence frequency of phonemes composing a mimetic word as the naturalness. To investigate the co-occurrence frequency, we collected many mimetic words through a subjective experiment. As a result of evaluation experiments, we confirmed that the proposed module could describe gaits with more natural mimetic words while maintaining the description accuracy."

DOI： 10.1007/978-3-030-37734-2_18

Language：English   Publishing type：Research paper (scientific journal)  

Numerous applications such as autonomous driving, satellite imagery sensing, and biomedical imaging use computer vision as an important tool for perception tasks. For Intelligent Transportation Systems (ITS), it is required to precisely recognize and locate scenes in sensor data. Semantic segmentation is one of computer vision methods intended to perform such tasks. However, the existing semantic segmentation tasks label each pixel with a single object’s class. Recognizing object attributes, e.g., pedestrian orientation, will be more informative and help for a better scene understanding. Thus, we propose a method to perform semantic segmentation with pedestrian attribute recognition simultaneously. We introduce an attribute-aware loss function that can be applied to an arbitrary base model. Furthermore, a re-annotation to the existing Cityscapes dataset enriches the ground-truth labels by annotating the attributes of pedestrian orientation. We implement the proposed method and compare the experimental results with others. The attribute-aware semantic segmentation shows the ability to outperform baseline methods both in the traditional object segmentation task and the expanded attribute detection task.

DOI： 10.1587/transfun.2019TSP0001

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： https://doi.org/10.2493/jjspe.85.1157

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.2493/jjspe.85.1117

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Drivers obtain information on surrounding environment using their eyesights. Experienced eye-gaze behavior is needed when driving at places where multiple risks exist to prepare for and avoid them. In this work, we analyze the change in eye-gaze behavior in such situations while a driver gains experience on the operation of a robotic wheelchair. Accurate distance information in the traffic environment is important to analyze the eye-gaze behavior. However, almost all previous works analyze eye-gaze behavior in a 2D environment, so they could not obtain accurate distance information. For this reason, we analyze eye-gaze behavior in 3D space. Concretely, we developed a novel eye-gaze behavior analysis platform based on a robotic wheelchair and estimated the driver's attention in 3D space. We try to analyze the eye-gaze behavior considering a useful field-of-view in 3D space based on the distance information instead of only the fixation point to investigate the objects that a driver implicitly pays attention to and from where s/he focuses on them. Results show that novice drivers pay attention to a single risk at a time. In contrast, they pay more attention to multiple risks simultaneously as they gain experience. Additionally, we discuss what features are effective to model the eye-gaze behavior based on the results.

DOI： 10.1109/ICCVW.2019.00545

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1587/transinf.2018EDL8219

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

Increasingly sophisticated methods for data processing demand knowledge on the semantic relationship between language and vision. New fields of research like Explainable AI demand to step away from black-boxed approaches and understanding how the underlying semantics of data sets and AI models work. Advancements in Psycholinguistics suggest, that there is a relationship from language perception to how language production and sentence creation work. In this paper, a method to measure the visual variety of concepts is proposed to quantify the semantic gap between vision and language. For this, an image corpus is recomposed using ImageNet and Web data. Web-based metrics for measuring the popularity of sub-concepts are used as a weighting to ensure that the image composition in a dataset is as natural as possible. Using clustering methods, a score describing the visual variety of each concept is determined. A crowd-sourced survey is conducted to create ground-truth values applicable for this research. The evaluations show that the recomposed image corpus largely improves the measured variety compared to previous datasets. The results are promising and give additional knowledge about the relationship of language and vision.

DOI： 10.1007/s11042-018-6528-x

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1142/S1793351X19500016

Language：English   Publishing type：Research paper (international conference proceedings)  

In recent years, the demand for highly accurate pedestrian detectors has increased due to the development of advanced driving support systems.
For the training of an accurate pedestrian detector, it is important to collect a large number of training samples.
To support this, this paper proposes a ``hard negative'' mining method to automatically extract background images which tend to be erroneously detected as pedestrians.
Negative samples are selected based on the assumption that frequent patterns observed multiple times in the same location are most likely parts of the background scene.
As a result of an evaluation using in-vehicle camera images captured along the same route, we confirmed that the proposed method can automatically collect false positive samples accurately.
We also confirmed that a highly accurate detector can be constructed using the additional negative samples.

Language：English   Publishing type：Research paper (international conference proceedings)  

3D object pose estimation by using a depth sensor is one of the important tasks in activities by robots. To reduce the pose ambiguity of an estimated object pose, several methods for multiple viewpoint pose estimation have been proposed. However, these methods need to select the viewpoints carefully to obtain better results. If the pose of the target object is ambiguous from the current observation, we could not decide where we should move the sensor to set as the next viewpoint. In this paper, we propose a best next viewpoint recommendation method by minimizing the pose ambiguity of the object by making use of the current pose estimation result as a latent variable. We evaluated viewpoints recommended by the proposed method and confirmed that it helps us to gain better pose estimation results than several comparative methods on a synthetic dataset.

Language：English   Publishing type：Research paper (international conference proceedings)  

In recent years, LIDAR is playing an important role as a sensor for understanding environments of a vehicle’s surroundings. Active-scan LIDAR is being actively developed as a LIDAR that can control the laser irradiation direction arbitrary and rapidly. In comparison with conventional uniform-scan LIDAR (e.g. Velodyne HDL-64e), Active-scan LIDAR enables us to densely scan even distant pedestrians. In addition, if appropriately controlled, this sensor has the potential to reduce unnecessary laser irradiations towards non-target objects. Although there are some preliminary studies on pedestrian scanning strategy for Active-scan LIDARs, in the best of our knowledge, an efficient method has not been realized yet. Therefore, this paper proposes a novel pedestrian scanning method based on orientation aware pedestrian likelihood estimation using the orientation-wise pedestrian’s shape models with local distribution of measured points. To evaluate the effectiveness of the proposed method, we conducted experiments by simulating Active-scan LIDAR using point-clouds from the KITTI dataset. Experimental results showed that the proposed method outperforms conventional methods.

Language：English   Publishing type：Research paper (international conference proceedings)  

We propose a temporal spotting method of a hand gesture from a low-resolution far-infrared image sequence captured by a far-infrared sensor array. The sensor array captures the spatial distribution of far-infrared intensity as a thermal image by detecting far-infrared waves emitted from heat sources. It is difficult to spot a hand gesture from a sequence of thermal images captured by the sensor due to its low-resolution, heavy noise, and varying duration of the gesture. Therefore, we introduce a voting-based approach to spot the gesture with template matching-based gesture recognition. We confirm the effectiveness of the proposed temporal spotting method in several settings.

DOI： 10.1109/VCIP.2018.8698650

Language：English   Publishing type：Research paper (international conference proceedings)  

The number of food photos posted to the Web has been increasing. Most of the users prefer to post delicious-looking food photos. They, however, do not always look delicious. A previous work proposed a method for estimating the attractiveness of food photos, that is, the degree of how much a food photo looks delicious, as an assistive technology for taking a delicious-looking food photo. This method extracted image features from the entire food photo to evaluate the impression. In our work, we conduct a preference experiment where subjects are asked to compare a pair of food photos and measure their gaze. The proposed method extracts image features from local regions selected based on the gaze information and estimates the attractiveness of a food photo by learning regression parameters. Experimental results showed the effectiveness of extracting image features from outside the gaze regions rather than inside them.

DOI： 10.1109/ISM.2018.00015

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.2493/jjspe.84.1017

Language：English   Publishing type：Research paper (international conference proceedings)  

What target is focused on by many people?
Analysis of the target is a crucial task, especially in a cinema, a stadium, and so on.
However, it is very difficult to estimate the gaze of each person in a crowd accurately and simultaneously with existing image-based eye tracking methods, since the image resolution of each person becomes low when we capture the whole crowd with a distant camera.
Therefore, we introduce a new approach for localizing the gaze target focused on by a crowd of people.
The proposed framework aggregates the individually estimated results of each person's gaze.
It enables us to localize the target being focused on by them even though each person's gaze localization from a low-resolution image is inaccurate.
We analyze the effects of an aggregation method on the localization accuracy using images capturing a crowd of people in a tennis stadium under the assumption that all of the people are focusing on the same target, and also investigate the effect of the number of people involved in the aggregation on the localization accuracy.
As a result, the proposed method showed the ability to improve the localization accuracy as it is applied to a larger crowd of people.

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we analyze headlight flicker patterns which improve the pedestrian detectability from a driver. Recently, headlights are becoming capable of selectively projecting light on a pedestrian in addition to the normal forward projection. However, it is still not clear how the light should be
projected to effectively improve the visibility of the pedestrian. We actually analyze nine flicker patterns by controlling duty ratios and durations of lighting time, and conduct experiments in field and laboratory settings. As a result, we reveal that a specific fundamental frequency is effective for improving the pedestrian detectability from a driver. We also conclude that the difference between the two settings are not significant.

Language：English   Publishing type：Research paper (international conference proceedings)  

Nowadays, development of driving support systems and autonomous driving systems have become active.
Pedestrian detection from in-vehicle sensors is one of the most important technologies for these systems.
However, outputs of pedestrian detectors can not be fully trusted in real environments.
Therefore, we propose an estimation system of pedestrian detector's reliabilities for a given scene.
This paper proposes a scene-wise reliability calculation method for LiDAR-based detectors, and a construction method for their estimators.
Here, the problem is how we can define the reliability.
The proposed method defines the reliability considering oversights as the strictest threshold without oversights.
Meanwhile, it defines the reliability considering false detections as the loosest threshold without false detections.
Experimental results showed that the proposed method could properly represent the reliability of a given scene, and estimate their reliability.

Language：English   Publishing type：Research paper (international conference proceedings)  

Semantic segmentation is an interesting task for many deep learning researchers for scene understanding. However, recognizing details about objects’ attributes can be more informative and also helpful for a better scene understanding in intelligent vehicle use cases. This paper introduces a method for simultaneous semantic segmentation and pedestrian attributes recognition. A modified dataset built on top of the Cityscapes dataset is created by adding attribute classes corresponding to pedestrian orientation attributes. The proposed method extends the SegNet model and is trained by using both the original and the attribute-enriched datasets. Based on an experiment, the proposed attribute-aware semantic segmentation approach shows the ability to slightly improve the performance on the Cityscapes dataset, which is capable of expanding its classes in this case through additional data training.

Language：English   Publishing type：Research paper (international conference proceedings)  

In order to reduce traffic accidents between a vehicle and a pedestrian, recognition of a pedestrian who has a possibility of collision with a vehicle should be helpful. However, since a pedestrian may suddenly change his/her direction and cross the road, it is difficult to predict his/her behavior directly. Here, we focus on the fact that experienced drivers usually pass by a pedestrian while preparing to step on the brake at any moment when they feel danger. If driver assistant systems can estimate such experienced driver’s decisions, they could early detect the pedestrian in danger of collision. Therefore, we classify the driver’s decisions into three types by referring to the accelerator operation of drivers, and propose a method to estimate the type of the driver’s decision. The drivers are considered to decide their actions focusing on various behaviors and states of a pedestrian, namely pedestrian’s attributes. Since the driver’s decisions change along the timeline, the use of a temporal context is considered to be effective. Thus, in this paper, we propose an estimation method using a recurrent neural network architecture with the pedestrian’s attributes as input. We constructed a dataset collected by experienced drivers in control of the vehicle and evaluated the performance, and then confirmed the effectiveness of the use of pedestrian’s attributes.

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： https://doi.org/10.1541/ieejeiss.138.1399

Language：English   Publishing type：Research paper (international conference proceedings)  

Nowadays, development of driving support systems and autonomous driving systems have become active.
Vehicle ego-localization using in-vehicle sensors is one of the most important technologies for these systems.
Accordingly, various attempts to localize own vehicle from in-vehicle sensors have been made.
In general, the estimation accuracy of the traveling direction is lower than in the lateral direction.
Therefore, we propose a highly accurate method for ego-localization of the traveling direction based on epipolar geometry using an in-vehicle monocular camera.
The proposed method makes correspondences between in-vehicle camera images and database images with location information, and calculates the location using locations annotated to the corresponding database images.
However, there are many gaps due to the difference of speed and trajectory of vehicles even if the images are obtained along the same road.
To overcome this problem, the distance between the input image and the database image is calculated by the distance metric based on the epipolar geometry and the local feature method.
An experiment was conducted using actual images with correct locations, and we confirmed the effectiveness of the proposed method from its results.

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a method to count vehicles from an in-vehicle camera image by regression based on car parts detection. In the case of an in-vehicle camera image, since vehicles are frequently occluded by other vehicles in traffic congestion, it is difficult to accurately count vehicles. Therefore, we propose a method to count vehicles by regression based on the number of visible car parts. For this, we make an estimator by learning the relation between the number of visible car parts and that of vehicles by Support Vector Regression. We evaluated our method using in-vehicle camera images recorded in an actual environment, where the proposed method performed better than counting detected vehicles.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

Group work is widely introduced and practiced as a method to achieve the learning goal efficiently by collaborating group members. However, since most types of group works are carried out in the real environment, it is very difficult to perform formative assessment and real time evaluation without students' feedbacks. Therefore, there is a strong demand to develop a method that supports evaluation of group work. To support evaluation of group work, this paper proposes a method to visualize the real world activity during group work by using first person view cameras and wearable sensors. Here, the proposed method visualizes three scores: (1) individual attention, (2) hand visibility, (3) individual activity. To evaluate the performance and analyze the relationships between scores, we conducted experiments of Marshmallow challenge" that is a collaborative work to construct a tower using marshmallow and spaghetti within a limit of time. Through the experiments, we confirmed that the proposed method has potential to become a evaluation tool for visualizing the activity of the group work."

DOI： 10.1007/978-3-319-91131-1_2

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1587/transinf.2017EDL8215

Language：English   Publishing type：Research paper (international conference proceedings)  

A Learning Management System (LMS) has become a fundamental tool for higher education, and a framework to leverage digital education data in the LMS has attracted attention. On the other hand, there is strong demand to deal with various education data provided not only from electronic media but also non-electronic media, such as a handwritten assignment. To solve this problem, this paper describes the development of 'KamiRepo' system to make it possible to automatically upload handwritten assignments to the LMS. In this system, optical character recognition (OCR) is performed to identify scanned handwritten assignments of individual students and read their scores. Then, their scanned files automatically separated from the entire file of the scanned handwritten assignments are returned to the individual students through LMS together with their corresponding scores. Compared with a conventional system using the dedicated multifunction printer, our developed system is capable of 1) using general-purpose scanners, 2) using a user interface on Web browser, and 3) achieving accurate student identification. We have launched this system in our university in April 2017 and have evaluated its effectiveness. The experimental results using real data collected for 6 months showed that our system achieved 99.7% of success rate in the automatic upload process.

Language：English   Publishing type：Research paper (international conference proceedings)  

Person re-identification gains an increasing interest
in the surveillance image processing field due to its’ importance
for security. Most approaches to solve the person re-identification
problem match persons one-by-one. However, redundant match-
ing where one of the person is selected for the matching pair
several times often occurs. It also degrades the overall image
matching performance. To overcome the issue, in this paper,
we propose a method which solves the person re-identification
problem for multiple persons simultaneously. Instead of one-
by-one matching, we consider person re-identification as an
instance of the Stable Marriage Problem (SMP). The result of an
experiment showed that the proposed method outperforms some
of the existing state-of-the-art methods applied to the VIPeR
dataset.

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ISM.2017.54

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ISM.2017.33

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we report a case study on driver's decision in terms of pedestrian attributes. Among various traffic situations, the situation that a vehicle passes by a pedestrian is one of the major situations. To build a safety driving system that supports a non-experienced driver in such a situation, we analyzed how experienced drivers decide to handle the vehicle in such a situation. Since pedestrian’s behavior can be considered as a key factor for the decision, and also the behavior is different depending on their attributes," such as walking or stopping, noticing the vehicle or not, using a smartphone, etc., we analyzed what pedestrian's attributes affect the driver's decisions. For the analysis, we first built a large-scale dataset of driving data. Using the dataset, we clarified what attributes are dominant for the driver's decision."

Language：English   Publishing type：Research paper (international conference proceedings)  

Travel destination recommendation is useful to support travel.Considering the recommendation of regions within the destination area to visit, it could be difficult for the users to explicitly indicate their preference.Therefore, we considered that it would be more intuitive to recommend regions in the destination area that are similar to a region already well-known to the user.Thus, in this paper, we propose a method for the detection of similar geo-regions based on Visual Concepts in social photos.We report experimental results and analyses by applying the proposed method to the YFCC100M dataset.

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1002/ecj.11987

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a Deep Learning-based action recognition method from an extremely low-resolution thermal image sequence. The method recognizes daily actions by humans (e.g. walking, sitting down, standing up, etc.) and abnormal actions (e.g. falling down) without privacy concerns. While privacy concerns can be ignored, it is difficult to compute feature points and to obtain a clear edge of the human body from an extremely low-resolution thermal image. To address these problems, this paper proposes a Deep Learning-based action recognition method that combines convolution layers and an LSTM layer for learning spatio-temporal representation, whose inputs are the thermal images and their frame differences cropped by the gravity center of human regions. The effectiveness of the proposed method was confirmed through experiments.

DOI： 10.1109/AVSS.2017.8078497

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1145/3106668.3106670

Language：English   Publishing type：Research paper (scientific journal)  

Localization along a route is an everyday necessity for in-vehicle navigation systems, and a vital task for automated driving technologies. Visual ego-localization promises reliable accuracy even in challenging urban environments where Global Positioning Systems (GPSs) can fail. Using cameras for localization against a pre-constructed database requires either the creation of a dense three-dimensional feature point map and pose estimation of a query camera relative to this map, or image matching along a database route to determine the capture position of the query camera based on the most similar database image. While the latter method is potentially less computationally intensive and can provide a more compact database, localization accuracy is limited by the discrete positioning information at database frame capture locations. In this paper we propose an image matching method that makes use of image features which are pre-matched during database construction, allowing linear regression coefficients for the relationship between capture position and feature scale to be calculated. The capture position of matched query features can then be estimated to sub-database spacing resolution. By incorporating the visual localization system into a Bayes estimator, we demonstrate an average monocular vision localization accuracy of 0.33 m in tests on actual vehicle image streams.

DOI： 10.1016/j.imavis.2017.07.004

Language：English   Publishing type：Research paper (international conference proceedings)  

Surveillance systems play a critical role in security and surveillance. A surveillance system with cameras that work in the visible spectrum is sufficient for most cases. However, problems may arise during the night, or in areas with less than ideal illumination conditions. Cameras with thermal infrared technology can be a better option in these situations since they do not rely on illumination to observe the environment. Furthermore, in our daily lives, it is common for humans to wear headgears such as glasses, masks, and hats. In surveillance, such headgears can be a hindrance to the identification of a person, and hence pose a certain degree of risk. This is not ideal in areas where the identity of a person is important, for example, in a bank. Therefore, in this paper we propose a headgear recognition method using an innovative decomposition approach on thermal infrared images. The decomposition method is based on Robust Principal Component Analysis, a modification of the popular Principal Component Analysis. The proposed method performs decomposition on a human image and isolates headgears in the image for recognition purposes. Experiments were conducted to evaluate the capability of the proposed method. The results show a positive outcome when compared with other methods.

DOI： 10.1109/QIR.2017.8168475

Language：English   Publishing type：Research paper (international conference proceedings)  

Multiple person tracking over a camera network is usually performed by matching person images between adjacent cameras. It easily fails by a temporal appearance change of the persons caused by environmental illumination and observation orientation of a camera. To solve this problem, matching person images across not only adjacent cameras but also cameras multiple hops away in the camera network is effective, but such relaxation of spatiotemporal cues also cause tracking failure due to the increase of matching candidates. To avoid the failure, we introduce Random Camera Drop" to generate different camera networks which relax the spatio-temporal cues partially and randomly. We then, integrate tracking results over the networks to a consensus tracking result by a novel concept "Trajectory Ensemble", an extension of unsupervised ensemble learning for the multiple person tracking over a camera network problem. We evaluated the framework on several virtual datasets generated from a public dataset, "Shinpuhkan 2014 dataset" and confirmed that the proposed method achieves the highest tracking results among several comparative methods."

Language：English   Publishing type：Research paper (international conference proceedings)  

We introduce a method that uses a single camera to localize a vehicle within a pre-constructed map consisting of a voxel occupancy grid and road-line marker positions. Sophisticated mapping hardware is capable of creating high accuracy 3D maps of road environments, but localizing a vehicle within such maps is one of the challenges at the forefront of automated driving. A solution which is robust to dynamic environments, while using only inexpensive sensors, is a difficult problem. In addition, maps that enable precise localization consume a lot of data which is impractical for the expansive environments encountered in real-world road networks. We show how using the area of edge regions shared between rendered views of a compact voxel map and in-vehicle camera images can be coupled with non-linear optimization methods to determine the camera position and pose.

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/VR.2017.7892257

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.5220/0006101201730178

Language：English   Publishing type：Research paper (scientific journal)  

This paper addresses the attribute recognition problem, a
field of research that is dominated by studies in the visible spectrum. Only
a few works are available in the thermal spectrum, which is fundamentally
different from the visible one. This research performs recognition specif-
ically on wearable attributes, such as glasses and masks. Usually these
attributes are relatively small in size when compared with the human body,
on top of a large intra-class variation of the human body itself, therefore
recognizing them is not an easy task. Our method utilizes a decomposi-
tion framework based on Robust Principal Component Analysis (RPCA) to
extract the attribute information for recognition. However, because it is dif-
ficult to separate the body and the attributes without any prior knowledge,
noise is also extracted along with attributes, hampering the recognition
capability. We made use of prior knowledge; namely the location where
the attribute is likely to be present. The knowledge is referred to as the
Probability Map, incorporated as a weight in the decomposition by RPCA.
Using the Probability Map, we achieve an attribute-wise decomposition.
The results show a significant improvement with this approach compared
to the baseline, and the proposed method achieved the highest performance
in average with a 0:83 F-score.

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.5220/0006100901570164

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.5220/0006101101650172

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.5220/0006229306110616

Language：English   Publishing type：Research paper (scientific journal)  

Advances in intelligent vehicle systems have led to modern automobiles being able to aid drivers with tasks such as lane following and automatic braking. Such automated driving tasks increasingly require reliable ego-localization. Although there is a large number of sensors that can be employed for this purpose, the use of a single camera still remains one of the most appealing, but also one of the most challenging. GPS localization in urban environments may not be reliable enough for automated driving systems, and various combinations of range sensors and inertial navigation systems are often too complex and expensive for a consumer setup. Therefore accurate localization with a single camera is a desirable goal.
In this paper we propose a method for vehicle localization using images captured from a single vehicle-mounted camera and a pre-constructed database.
Image feature points are extracted, but the calculation of camera poses is not required---instead we make use of the feature points' scale.
For image feature-based localization methods, matching of many features against candidate database images is time consuming, and database sizes can become large. Therefore, here we propose a method that constructs a database with pre-matched features of known good scale stability. This limits the number of unused and incorrectly matched features, and allows recording of the database scales into ``tracklets''. These ``Feature scale tracklets'' are used for fast image match voting based on scale comparison with corresponding query image features. This process reduces the number of image-to-image matching iterations that need to be performed while improving the localization stability. We also present an analysis of the system performance using a dataset with high accuracy ground truth. We demonstrate robust vehicle positioning even in challenging lane change and real traffic situations.

DOI： 10.1587/transfun.E100.A.702

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose a multiclass classifier training method which reduces fatal" misclassifications by cost-relaxation of "tolerable" misclassifications in one-against-all classifiers training, named misclassification tolerable learning. In a binary classifier in the one-against-all classifiers, we introduce a new class group "conceptually similar classes," whose class labels are similar to the positive class. In the case of pedestrian orientation classification, the conceptually similar classes are defined as neighboring orientations to the positive orientation. We consider the misclassification of the conceptually similar classes to the positive class as tolerable misclassification. By relaxing the cost of the tolerable misclassifications, our proposed classification method reduces fatal misclassifications of non-similar classes. We evaluated the cost-relaxation effectiveness on several public datasets and confirmed that the proposed method outperforms the normal SVM on all of the datasets in the soft criterion by achieving 78.63% recognition rate on PDC Dataset."

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.2493/jjspe.82.1085

Language：English   Publishing type：Research paper (international conference proceedings)  

Pedestrians texting-while-walking" increase the risk of traffic accidents, since they are often not paying attention to their surrounding environments and fails to notice approaching vehicles. Thus, the recognition of texting-while-walking from an in-vehicle camera should be helpful for safety driving assistance. In this paper, we propose a method to recognize a pedestrian texting-while-walking from in-vehicle camera images. The proposed approach focuses on the characteristic relationship between the arm and the head poses observed during a texting-while-walking behavior. In this paper, Pose-Dependent Joint HOG feature is proposed as a novel feature, which uses parts locations as prior knowledge and describes the cooccurrence of the arm and the head poses. To show the effectiveness of the proposed method, we constructed a dataset and evaluated it."

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a method for detecting obstacles using train frontal view videos.
In recent years, obstacles detection using in-vehicle camera is actively developed for satisfying demands from various applications.
In the field of obstacle detection, most methods employ machine learning approach, and they can detect only trained obstacles, such as pedestrian, bicycle, etc.
Therefore, they cannot detect untrained general obstacles.
To overcome this problem, this paper propose a background subtraction method using moving camera.
The proposed method first computes frame-by-frame correspondences between the present and the database train frontal view image sequences, and detects obstacles by applying image subtraction to corresponding frames.
To confirm the effectiveness of the proposed method, experiments were conducted by using several image sequences captured at the experimental railroad track.
The experimental result showed that the proposed method could detect various obstacles accurately and effectively.

DOI： 10.1109/ICIP.2016.7533104

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

Collaborative learning is efficient teaching/learning method, and it is widely introduced and practiced in various situations. However, it has a difficulty to perform formative assessment and real time evaluation without students' feedbacks. Therefore, demand for technologies to support formative assessment in collaborative learning is increasing. To tackle this problem, we have started the research project for automatic sensing and visualization of real world activities in collaborative learning. In this paper, we will report details about preliminary group work experiments and its results with visualization tool.

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.1541/ieejeiss.136.1367

Language：English   Publishing type：Research paper (international conference proceedings)  

In recent years, pedestrian detection from an invehicle camera has been attracting attention.However, in the case of a raining situation, the detection accuracy decreases because the head of a pedestrian tends to be occluded by an umbrella. In oder to handle such cases, in this paper, as a variation of the Deformable Part Model (DPM) which is widely used in the field of object recognition, we propose Parts Selective DPM (PS-DPM)" which selectively chooses the original part filters and additional part filters trained independently. In the detection of pedestrians possessing an umbrella, the selection of head and umbrella parts will make pedestrian detection more robust to the occlusion. We conducted experiments to evaluate the performance of the proposed method. As a result, pedestrian detection with the proposed PS-DPM achieved high detection accuracy in rainy weather, compared with the detection by the conventional DPM. Moreover, we confirmed that it did not decrease the pedestrian detection accuracy in fine weather."

Language：English   Publishing type：Research paper (scientific journal)  

Human body segmentation has many applications in a wide
variety of image processing tasks, from intelligent vehicles to entertainment.
A substantial amount of research has been done in the field of segmentation
and it is still one of the active research areas, resulting in introduction of
many innovative methods in literature. Still, until today, a method that can
overcome the human segmentation problems and adapt itself to different
kinds of situations, has not been introduced. Many of methods today try to
use the graph-cut framework to solve the segmentation problem. Although
powerful, these methods rely on a distance penalty term (intensity difference
or RGB color distance). This term does not always lead to a good separation
between two regions. For example, if two regions are close in color, even
if they belong to two different objects, they will be grouped together, which
is not acceptable. Also, if one object has multiple parts with different
colors, e.g. humans wear various clothes with different colors and patterns,
each part will be segmented separately. Although this can be overcome
by multiple inputs from user, the inherent problem would not be solved.
In this paper, we have considered solving the problem by making use of a
human probability map, super-pixels and Grab-cut framework. Using this
map relives us from the need for matching the model to the actual body,
thus helps to improve the segmentation accuracy. As a result, not only the
accuracy has improved, it also became comparable to the state-of-the-art
interactive methods.

DOI： 10.1587/transfun.E99.A.943

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a method for estimating the attractiveness of food photos in order to assist a user to shoot them attractively.
The proposed method extracts both color and shape features from input food images, and then integrates them according to a regression scheme.
By this way, the proposed method estimates the attractiveness of an unknown food photo.
We also created a food image dataset taken from various 3D-angles for each food category, and set target values of their attractiveness through subjective experiments.
Then, we evaluated the performance of the proposed method in two different ways of constructing the attractiveness estimator: One that constructs it for each food category, and the other that constructs a common attractiveness estimator for all food categories.
Experimental results showed the effectiveness of the proposed method in addition to the necessity for adaptively selecting the estimator depending on the appearance of foods for further performance improvement.

Language：English   Publishing type：Research paper (scientific journal)  

During the night or in poorly lit areas, thermal cameras are a better choice instead of normal cameras for security surveillance because they do not rely on illumination. A thermal camera is able to detect a person within its view, but identification from only thermal information is not an easy task. The purpose of this paper is to reconstruct the face image of a person from the thermal spectrum to the visible spectrum. After the reconstruction, further image processing can be employed, including identification/recognition. Concretely, we propose a two-step thermal-to-visible-spectrum reconstruction method based on Canonical Correlation Analysis (CCA). The reconstruction is done by utilizing the relationship between images in both thermal infrared and visible spectra obtained by CCA. The whole image is processed in the first step while the second step processes patches in an image. Results show that the proposed method gives satisfying results with the two-step approach and outperforms comparative methods in both quality and recognition evaluations.

DOI： 10.3390/s16040568

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.1541/ieejeiss.136.588

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1145/2857491.2857500

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.5220/0005718105450551

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.5220/0005668702730279

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper addresses an attribute recognition problem in thermal images, specifically on worn objects such as hat and glasses. Although attribute recognition is a growing research field, there are not much work done in thermal infrared spectrum. In this spectrum, since illumination is not a problem, it could be a better option to be used in nighttime or poorly lit areas. The proposed method uses only the attribute information and excludes the unnecessary information for the recognition. To achieve this, we propose attribute recognition based on feature decomposition using Robust Principal Component Analysis (RPCA). An experiment to evaluate the capability of the proposed method was conducted on the dataset created for this research. The results show that the proposed method outperformed the method without decomposition by 14% in average with a maximum of 27% increase in a specific attribute.

Language：English   Publishing type：Research paper (international conference proceedings)  

Segmentation is one of active areas in computer vision field with application in many areas from entertainment to intelligent vehicles (IVs). Among the objects, humans themselves have always been among the most interested subjects because of
their special features. Since human body has an articulated structure, modeling and recognizing different variations in the body has proved to be very difficult. Wearing various kinds of clothes in different situations which can have a completely dissimilar appearance based on the clothing type, makes the modeling much more difficult. Add to this, the common problems of vision like illumination changes, blurring due to camera movements, etc. make the problem even more difficult. Thus having a system that can segment human subjects accurately can be useful in many applications. In this paper, we propose a system for segmenting human subjects using Statistical Shapes Models (SSM) feedback and a texture aware version of Grab-cut which incorporates texture feature for improving the segmentation accuracy. Our experiments show that the proposed system has an acceptable accuracy compared to the state-of-the art interactive methods and much better than the conventional ones.

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.14923/transinfj.2015JDP7031

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.14923/transinfj.2015HAP0015

Language：English   Publishing type：Research paper (international conference proceedings)  

Vehicle ego-localization is a critical task not only for in-car navigation systems, but also for emerging intelligent and autonomous vehicle technologies. Visual localization methods that determine current location by performing image matching against a pre-constructed database have an accuracy limited by the spatial distance between database images. In this paper we propose a method that uses the scale of feature points to interpolate the position of the query image between two database images. We show how this simple contribution offers an appreciable improvement in localization accuracy with an extremely minimal increase in processing time, especially when used in conjunction with image matching methods that already monitor feature scale. Our experiments showed an increase of up to 33\% in average localization accuracy when compared to a method without any interpolation.

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1145/2810188.2810189

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose the re-detection paradigm, which is a detection with prior knowledge of the detection targets, and we introduce an implementation of the re-detection for distant pedestrian detection from an in-vehicle camera. We focus on the fact that other vehicles including forward vehicles can observe and detect pedestrians before the own vehicle observes them. Since appearances of pedestrians do not significantly change even though their locations are different, sharing images of the detected pedestrians among the vehicles, the own vehicle can use them as prior knowledge for detecting them again. Results of applying the proposed method to a dataset obtained by an in-vehicle camera demonstrate that the accuracy of pedestrian detection results can be significantly increased if prior knowledge of the pedestrians could be obtained.

DOI： 10.1109/ITSC.2015.200

Language：English   Publishing type：Research paper (international conference proceedings)  

Recently, the number of cooking recipes on the Web is increasing.
However, it is difficult to search them by tastes or textures although
they are actually important considering the nature of the contents.
Therefore, we propose a method for estimating the tastes and the
textures of a cooking recipe by analyzing them.
Concretely, the proposed method refers to an ingredients feature from
the ``ingredients list'' and image features from the ``food image'' in
a cooking recipe.
We confirmed the effectiveness of the proposed method through an
experiment.

Language：English   Publishing type：Research paper (scientific journal)  

Human segmentation is one of the most interesting yet most challenging subjects in the field of object segmentation and image processing. It can be used in various types of applications from image retrieval to robotics and human machine interfaces, including even entertainment. Many researches have been done on this subject and it is still one of active research areas. But until now, a method for accurate segmentation in different conditions has not been introduced. In this paper, we present Statistical Shape Feedback Segmentation” (SSFSeg) method, which is a way to automatically segment human subjects (pedestrians) from single images. Our main contributions in this paper are: 1) Using human shape model as priors for Grab-cut segmentation. 2) Implementation of a feedback system which provides a coarse-to-fine way of generating more accurate shapes. For this task, we try to use masks generated by the Statistical Shape Model (SSM) algorithm as a prior input for the Grab-cut technique to segment the desired human subject in the image without user interaction. To achieve this, we propose a feedback framework for the SSM sample generation. Our experiments confirmed that the segmentation error of our proposed method is less than half of the Grab-cut method."

DOI： 10.1541/ieejeiss.135.1000

Language：English   Publishing type：Research paper (international conference proceedings)  

As the number of cooking recipes posted on the Web increases, it becomes difficult to find a cooking recipe that a user needs. Moreover, even if it can be done, it is still difficult for users to arrange the cooking recipe, for example, by replacing ingredients with different ones. To deal with such problems, we propose a framework for typicality analysis of the combination of ngredients. The framework calculates a typicality value for each combination of ingredients. The list of ingredients can be arranged by adjusting the typicality value by adding or removing ingredients iteratively. The effectiveness of the proposed framework was confirmed through subjective experiments.

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a method for pedestrian orientation classification. In image recognition, the accuracy is often degraded by the influence of background. In addition, it is also difficult to remove the background and extract only the human body from an image. To overcome these problems, we utilize a single-chip RGB-ToF camera. This camera can acquire RGB and depth images along the same optical axis at the same moment, and thus segmentation of the RGB image becomes easier by using the coaxial depth image. Our proposed method segmented a human body from its background accurately, which lead to the improvement of the accuracy of pedestrian orientation classification.

Language：English   Publishing type：Research paper (international conference proceedings)  

Web based translation support systems such as Crowdin and Transifex make it easy for regional Sakai communities to collaborate globally in translation. In the fall and winter of 2014, Spanish Sakai community and Japanese Sakai community are collaborating in translation of Sakai 10 using Transifex as a common translation platform. This collaboration brought a lot of things to two communities. For instance, to develop a tool to import resource bundle files to Transifex in a specific manner, we can have an ease of use platform to translate modular designed software such as Sakai. And this platform will be extended for every regional community and for every project in Apereo community.
During this session we will talk about followings;
(1) Overview
(2) Benefits for Apereo community
(3) Context dependent translation by gettext Portable Object
(4) Community translation strategy in Sakai Spanish Users (S2U)
(5) Automatized process by Jenkins

Language：English   Publishing type：Research paper (scientific journal)  

In this paper, as a tool for structuring a large volume of archived news videos according to their semantic contents, we propose a method that effectively detects scene duplicates, assuming the presence of a person speaking in the videos. A scene duplicate is defined here as a pair of video segments taken at the same event from different viewpoints. When considering scenes where a subject is speaking in news videos, referring to the audio channel could be effective to detect scene duplicates regardless of viewpoints. However, it cannot be relied on when external audio sources overlap the original one or when the subject is actually not speaking. In contrast, the image channel can be useful in most cases. However, significant difference in viewpoints could prevent accurate detection. Therefore, we propose a method that combines the results obtained from both audio and image channels in order to improve the accuracy of scene duplicate detection from news videos. The proposed method was evaluated through an experiment with actual broadcast news videos by comparing it with a conventional method. As a result, we confirmed that the detection accuracy significantly improved by the proposed method in both recall and precision.

DOI： 10.1142/S1793351X15400048

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

In recent years, image inpainting is widely used to remove undesired objects from an image. Especially, the removal of temporal objects, such as pedestrians and vehicles, in street-view databases such as Google Street View has many applications in Intelligent Transportation Systems (ITS). To remove temporal objects, Uchiyama et al. proposed a method that combined multiple image sequences captured along the same route. However, when spatial alignment inside an image group does not work well, the quality of the output image of this method is often affected. For example, large temporal objects existing in only one image create regions that do not correspond to other images in the group, and the image created from aligned images becomes distorted. One solution to this problem is to select adaptively the reference image containing only small temporal objects for spatial alignment. Therefore, this paper proposes a method to remove temporal objects by integration of multiple image sequences with an adaptive reference image selection mechanism.

DOI： 10.5220/0005357102330239

Language：English   Publishing type：Research paper (international conference proceedings)  

Estimation of human orientation contributes to improving the accuracy of human behavior recognition. However,
estimation of human orientation is a challenging task because of the variable appearance of the human
body. The wide variety of poses, sizes and clothes combined with a complicated background degrades the
estimation accuracy. Therefore, we propose a method for estimating human orientation using coaxial RGBDepth
images. This paper proposes Depth Weighted Histogram of Oriented Gradients (DWHOG) feature
calculated from RGB and depth images. By using a depth image, the outline of a human body and the texture
of a background can be easily distinguished. In the proposed method, a region having a large depth gradient
is given a large weight. Therefore, features at the outline of the human body are enhanced, allowing robust
estimation even with complex backgrounds. In order to combine RGB and depth images, we utilize a newly
available single-chip RGB-ToF camera, which can capture both RGB and depth images taken along the same
optical axis. We experimentally confirmed that the proposed method can estimate human orientation robustly
to complex backgrounds, compared to a method using conventional HOG features.

DOI： 10.5220/0005305301130120

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ISM.2014.43

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper presents a Twitter-based event detection method based on Twitter Enthusiasm Degrees (TED)" toward generating a highlight video of a sports game. Existing methods not only depend on both languages and sports types but also often falsely detect non-target events. In contrast, the proposed method detects sports events using TEDs calculated from several kinds of string features independent of languages and sports. We applied the proposed method to actual sports games, and compared the detected events with the events present in broadcasted highlight videos, and confirmed the effectiveness and the language and sports type independencies of the proposed method."

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Accurate vehicle positioning is important not only for in-car navigation systems but is also a requirement for emerging autonomous driving methods. Consumer level GPS are inaccurate in a number of driving environments such as in tunnels or areas where tall buildings cause satellite shadowing. Current vision-based methods typically rely on the integration of multiple sensors or fundamental matrix calculation which can be unstable when the baseline is small.

In this paper we present a novel visual localization method which uses a visual street map and extracted SURF image features. By monitoring the difference in scale of features matched between input images and the visual street map within a Dynamic Time Warping framework, stable localization in the direction of motion is achieved without calculation of the fundamental or essential matrices.

We present the system performance in real traffic environments. By comparing localization results with a high accuracy GPS ground truth, we demonstrate that accurate vehicle positioning is achieved.

Language：English   Publishing type：Research paper (international conference proceedings)  

Crowd analysis using cameras has attracted much attention for public safety and marketing. Among techniques of the crowd analysis, we focus on spatial people density estimation which estimates the number of people for each small area in a floor region. However, spatial people density cannot be estimated accurately for an area far from the camera because of the occlusion by people in a closer area. Therefore, we propose a method using a memory based regression method with images captured from cameras from multiple viewpoints. This method is realized by looking up a table that consists of correspondences between people density maps and crowd appearances. Since the crowd appearances include situations where various occlusions occur, an estimation robust to occlusion should be realized. In an experiment, we examined the effectiveness of the proposed method.

Language：English   Publishing type：Research paper (international conference proceedings)  

Vehicle ego-localization is an essential process for many driver assistance and autonomous driving systems. The traditional solution of GPS localization is often unreliable in urban environments where tall buildings can cause shadowing of the satellite signal and multipath propagation. Typical visual feature based localization methods rely on calculation of the fundamental matrix which can be unstable when the baseline is small.
In this paper we propose a novel method which uses the scale of matched SURF image features and Dynamic Time Warping to perform stable localization. By comparing SURF feature scales between input images and a pre-constructed database, stable localization is achieved without the need to calculate the fundamental matrix. In addition, 3D information is added to the database feature points in order to perform lateral localization, and therefore lane recognition.
From experimental data captured from real traffic environments, we show how the proposed system can provide high localization accuracy relative to an image database, and can also perform lateral localization to recognize the vehicle's current lane.

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a camera-based visibility estimation method for a traffic sign.
The visibility here indicates how a visual target is easy to be detected and recognized by a human driver (not a machine).
This research aims at realizing a nuisance-free driver assistance system which sorts out information depending on the visibility of a visual target, in order to prevent driver distraction.
Our previous study on estimating the visibility of a traffic sign considered only the effect of the local region around a target, assuming the situation that a driver's gaze is around it.
The proposed method integrates both the local features and global features in a driving environment without such an assumption.
The global features evaluate the positional relationships between traffic signs and the appearance around the fixation point of a driver's gaze, which considers the effect of the driver's entire field of view.
Experimental results showed the effectiveness of incorporating the global features for estimating the visibility of a traffic sign.

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.1541/ieejeiss.134.702

Language：English   Publishing type：Research paper (international conference proceedings)  

It is important to track the flow of topics to thoroughly understand the contents. Accordingly, a method that structures the chronological semantic relations between news stories, namely a 'topic thread structure' has been proposed.It allows the comprehensive understanding of a topic by chronologically tracking stories one by one from the initial story. However, this task imposes a user to watch many stories when it contains various sub-topics. Thus, we propose method that estimates the representative story transition in a topic thread structure. In the proposed method, features obtained from a story and those from the topic thread structure are used for the estimation. We confirmed the effectiveness of the proposed method by comparing the results obtained from the proposed method to the ground truth obtained from votes in a subjective experiment. However, this task imposes a user to watch many stories when it contains various sub-topics. Thus, we propose method that estimates the representative story transition in a topic thread structure. In the proposed method, features obtained from a story and those from the topic thread structure are used for the estimation. We confirmed the effectiveness of the proposed method by comparing the results obtained from the proposed method to the ground truth obtained from votes in a subjective experiment

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper discusses a novel sensing strategy to retrieve real-world information. Instead of using sensor devices such as cameras, microphones and so on, the proposed approach involves people in real-world sensing to acquire requested information as accurately and quickly as possible. We call the proposed sensing strategy as Human Cloud Sensing (HCS)". In this paper, we introduce the concept of HCS and report some experimental results which were conducted to investigate the feasibility of "Can a human be a sensor?"."

Language：English   Publishing type：Research paper (international conference proceedings)  

In recent years, accurate pedestrian detection from in-vehicle camera images is focused to develop a safety driving assistance system. Currently, successful methods are based on statistical learning. However, in such methods, it is necessary to prepare a large amount of training images. Thus, the decrease in the number of training images degrades the detection accuracy. That is, in driving environments with few or no training images, it is difficult to detect pedestrians accurately. Therefore, we propose an approach that collects training images automatically to build classifiers for various driving environments. This is expected to realize highly accurate pedestrian detection by using an appropriate classifier corresponding to the current location. The proposed method consists of three steps; Classification of driving scenes, collection of non-pedestrian images and training of classifiers for each scene class, and associating a scene-class-specific classifier with GPS location information. Through experiments, we confirmed the effectiveness of the method compared to baseline methods.

DOI： 10.5220/0004677003540361

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose an exemplar-based super-resolution method applied to a human body in a surveillance video. Since persons are usually captured as low-resolution images by a video surveillance system, it is sometimes necessary to perform detection and identification of persons from not only a human face but also from the human body appearance. The super-resolution for a human body image is difficult because the appearances of person images vary according to the color of clothing and the posture of persons. Thus, we focus on the high-frequency components that could restore the lost high-frequency components of the low-resolution image regardless to the variation of the clothing. Therefore, the purpose of the work presented in this paper is to apply the exemplar-based super-resolution using high-frequency components for a low-resolution human body image to generate a high-resolution human body image so that both computer systems and humans can identify persons more accurately. As a result of experiments, we confirmed the effectiveness of the proposed super-resolution method.

DOI： 10.5220/0004686101150121

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1145/2506023.2506028

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

In recent years, advances in pedestrian detection technology have resulted in the development of driving assistance systems that notify the drivers of the presence of pedestrians. However, warning of all of the presence of pedestrians would confuse the driver. Therefore, the driver should only be notified of the less detectable pedestrians to avoid confusion. To achieve this, it is necessary to develop a method to predict the driver’s perception performance of pedestrian detectability. This paper proposes a method that predicts the pedestrian detectability considering the difference between individual drivers. The proposed method constructs a predictor specific to each driver, in order to predict the pedestrian detectability precisely. To obtain the ground truth of the pedestrian detectability, we conducted an experiment by human subjects using images from an in-vehicle camera including pedestrians. From the comparison between the output of the proposed method and the actual detectability, we confirmed that the proposed method significantly reduces the prediction error in comparison with the existing methods.

Language：English   Publishing type：Research paper (international conference proceedings)  

Recently, pedestrian detection from in-vehicle camera images is becoming an crucial technology for Intelligent Transportation Systems (ITS). However, it is difficult to detect pedestrians accurately in various scenes by obtaining training samples. To tackle this problem, we propose a method to construct scene dependent classifiers to improve the accuracy of pedestrian detection. The proposed method selects an appropriate classifier based on the scene information that is a category of appearance associated with location information. To construct scene dependent classifiers, the proposed method introduces generative learning for synthesizing scene dependent training samples. Experimental results showed that the detection accuracy of the proposed method outperformed the comparative method, and we confirmed that scene dependent classifiers improved the accuracy of pedestrian detection.

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

This paper presents an approach for cross-pose face recognition by virtual view generation using an appearance clustering based local view transition model. Previously, the traditional global pattern based view transition model (VTM) method was extended to its local version called LVTM, which learns the linear transformation of pixel values between frontal and non-frontal image pairs from training images using partial image in a small region for each location, instead of transforming the entire image pattern. In this paper, we show that the accuracy of the appearance transition model and the recognition rate can be further improved by better exploiting the inherent linear relationship between frontal-nonfrontal face image patch pairs. This is achieved based on the observation that variations in appearance caused by pose are closely related to the corresponding 3D structure and intuitively frontal-nonfrontal patch pairs from more similar local 3D face structures should have a stronger linear relationship. Thus for each specific location, instead of learning a common transformation as in the LVTM, the corresponding local patches are first clustered based on an appearance similarity distance metric and then the transition models are leaned separately for each cluster. In the testing stage, each local patch for the input non-frontal probe image is transformed using the learned local view transition model corresponding to the most visually similar cluster. The experimental results on a real-world face dataset demonstrated the superiority of the proposed method in terms of recognition rate.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

We propose a method for detecting biased-highlights in a broadcast sports video according to viewers’ attributes obtained from a large number of tweets. Recently, Twitter is widely used to make real-time play-by-play comments on TV programs, especially on sports games. This trend enables us to effectively acquire the viewers’ interests in a large mass. In order to make use of such tweets for highlight detection in broadcast sports video, the proposed method first performs an attribute analysis on the set of tweets issued by each user to classify which team he/she supports. It then detects biased-highlights by referring to the number of tweets made by viewers with a specific attribute.

DOI： 10.1007/978-3-642-35728-2_35

Language：English   Publishing type：Research paper (international conference proceedings)  

Gaze estimation from an image is an important technique for tasks such as driver monitoring and measuring of advertising effectiveness. For this, most existing methods require a high quality image of eyes. However, It is difficult to obtain the eye images when eye occlusion occurs due to sunglasses or face rotation. Another approach approximates head directions to gaze directions. However, the gaze direction is affected not only by the head direction but also by the body direction. Thus, we are studying a method that estimates the gaze direction accurately using information on both head and body pose directions. Experimental result showed that the proposed method could estimate gaze directions more accurately than by using the information on only head directions.

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a method for estimating the human performance of pedestrian detectability from in-vehicle camera images in order to warn a driver of the positions of pedestrians in an appropriate timing. By introducing features related to visual search and motion of the target, the proposed method estimates the detectability of pedestrians accurately. Support Vector Regression (SVR) is used to estimate the detectability. Here, SVR is trained using features calculated by the proposed method with the ground truth obtained through experiments with human subjects. From experiments using in-vehicle camera images, we confirmed that the proposed features were effective to estimate the detectability of pedestrians.

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper presents an approach for realistic virtual view generation using appearance clustering based local view transition model, with its target application on cross-pose face recognition. Previously, the traditional global pattern based view transition model (VTM) method was extended to its local version called LVTM, which learns the linear transformation of pixel values between frontal and non-frontal image pairs using partial image in a small region for each location, rather than transforming the entire image pattern. In this paper, we show that the accuracy of the appearance transition model and the recognition rate can be further improved by better exploiting the inherent linear relationship between frontal-nonfrontal face image patch pairs. For each specific location, instead of learning a common transformation as in the LVTM, the corresponding local patches are first clustered based on appearance similarity distance metric and then the transition models are learned separately for each cluster. In the testing stage, each local patch for the input non-frontal probe image is transformed using the learned local view transition model corresponding to the most visually similar cluster. The experimental results on a real-world face dataset demonstrated the superiority of the proposed method in terms of recognition rate.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a method for detecting general obstacles on a road by subtracting present and past in-vehicle camera images.
The image-subtraction-based object detection approach can be applied to detect any kind of obstacles although the existing learning-based methods detect only specific obstacles.
To detect general obstacles, the proposed method first computes a frame-by-frame correspondence between the present and the past in-vehicle camera image sequences, and then registrates road surfaces between the frames.
Finally, obstacles are detected by applying image subtraction to the registrated road surface regions with an illumination insensitive feature for robust detection.
Experiments were conducted by using several image sequences captured by an actual in-vehicle camera to confirm the effectiveness of the proposed method.
The experimental results shows that the proposed method can detect general obstacles accurately at a distance enough to avoid them safely even in situations with different illuminations.

DOI： 10.1007/978-3-642-33868-7_51

Language：English   Publishing type：Research paper (international conference proceedings)  

The aim of this work is to support a driver by
notifying the information of traffic signals in accordance with
their visibility. To avoid traffic accidents, the driver should
detect and recognize surrounding objects, especially traffic
signals. However, when driving a vehicle under rainy weather
conditions, it is difficult for drivers to detect or to recognize
objects existing in the road environment in comparison with fine
weather conditions. Therefore, this paper proposes a method
for estimating the visibility of traffic signals for drivers under
rainy weather conditions by image processing. The proposed
method is based on the concept of visual noise known in the
field of cognitive science, and extracts two types of visual noise
features which ware considered that they affect the visibility of
traffic signals. We expect to improve the accuracy of visibility
estimation by combining the visual noise features with the
texture feature introduced in a previous work. Experimental
results showed that the proposed method could estimate the
visibility of traffic signals more accurately under rainy weather
conditions.

DOI： 10.1109/ITSC.2012.6338838

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/EST.2012.7

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ICME.2012.162

Language：English   Publishing type：Research paper (scientific journal)  

We propose a method for discriminating between a speech shot and a narrated shot to extract genuine speech shots from a broadcast news video. Speech shots in news videos contain a wealth of multimedia information of the speaker, and could thus be considered valuable as archived material. In order to extract speech shots from news videos, there is an approach that uses the position and size of a face region. However, it is difficult to extract them with only such an approach, since news videos contain non-speech shots where the speaker is not the subject that appears in the screen, namely, narrated shots. To solve this problem, we propose a method to discriminate between a speech shot and a narrated shot in two stages. The first stage of the proposed method directly evaluates the inconsistency between a subject and a speaker based on the co-occurrence between lip motion and voice. The second stage of the proposed method evaluates based on the intra- and inter-shot features that focus on the tendency of speech shots. With the combination of both stages, the proposed method accurately discriminates between a speech shot and a narrated shot. In the experiments, the overall accuracy of speech shots extraction by the proposed method was 0.871. Therefore, we confirmed the effectiveness of the proposed method.

DOI： 10.1142/S1793351X12400077

Authorship：Lead author   Language：English   Publishing type：Research paper (scientific journal)  

Purpose
To perform bronchoscopy safely and smoothly, it is very important to develop a bronchoscopic guidance system. Transbronchial lung biopsy (TBLB) with a bronchoscopic guidance system especially should permit safe image-guided procedures. Recently, electromagnetic tracking (EMT) is utilized to track the tip of the bronchoscope camera in real time. For most tracking methods using position sensors, registration between tracking data and previously acquired reference image data, such as CT image, is performed using natural landmarks of the patient or fiducial markers attached to the patient, whose positions need to be measured manually by the physician before the actual bronchoscopy. Therefore, this paper proposes a marker-free CT-to-patient registration method utilizing bronchoscope's position and orientation obtained by the EMT.

Methods
We developed a guidance system that is able to track the tip of the bronchoscope camera in real time. In the case of a guidance system that uses position sensors, natural landmarks of the patient or fiducial markers attached to the patient are needed to obtain the correspondence between EMT outputs and previously acquired reference image data, such as CT image. This paper proposes a registration method without landmarks or fiducials by estimating the transformation matrix between the patient and the CT image taken prior to the bronchoscopic examination. This estimation is performed by computing correspondences between the outputs of the EMT sensor and airways extracted from the CT image. As ambiguities between EMT measurements and their corresponding airway branches may arise at airway bifurcations, we introduce a stable airway branch selection mechanism for improving the robustness of the estimation of the transformation matrix. To evaluate the performance of the proposed method, we applied the method to a rubber bronchial phantom and added virtual breathing motion to the sensor output.

Results
Experimental results show that the accuracy of our proposed method is within 2.0-3.0 mm (without breathing motion) and 2.5-3.5 mm (with breathing motion). The proposed method could also track a bronchoscope camera in real time.

Conclusions
We developed a method for CT-to-patient registration using a position sensor without fiducial markers and natural landmarks. Endoscopic guided biopsy of lung lesions is feasible using a marker-free CT-to-patient registration method.

DOI： 10.1007/s11548-011-0626-9

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1007/978-3-642-29934-6_15

Language：English   Publishing type：Research paper (scientific journal)  

This paper presents a new hybrid camera motion tracking method for bronchoscopic navigation combining SIFT, epipolar geometry analysis, Kalman filtering, and image registration. In a thorough evaluation, we compare it to state-of-the-art tracking methods. Our hybrid algorithm for predicting bronchoscope motion uses SIFT features and epipolar constraints to obtain an estimate for inter-frame pose displacements and Kalman filtering to find an estimate for the magnitude of the motion. We then execute bronchoscope tracking by performing image registration initialized by these estimates. This procedure registers the actual bronchoscopic video and the virtual camera images generated from 3D chest CT data taken prior to bronchoscopic examination for continuous bronchoscopic navigation. A comparative assessment of our new method and the state-of-the-art methods is performed on actual patient data and phantom data. Experimental results from both datasets demonstrate a significant performance boost of navigation using our new method. Our hybrid method is a promising means for bronchoscope tracking, and outperforms other methods based solely on Kalman filtering or image features and image registration.

DOI： 10.1016/j.media.2010.11.001

Language：English   Publishing type：Research paper (international conference proceedings)  

Recently, pedestrian detection from in-vehicle camera images is becoming an important technology in ITS (Intelligent Transportation System).
However, it is difficult to detect pedestrians stably due to the variety of their poses and their backgrounds.
To tackle this problem, we propose a method to detect various pedestrians from in-vehicle camera images by using multiple classifiers corresponding to various pedestrian pose classes.
Since pedestrians’ pose varies widely, it is difficult to construct a single classifier that can detect pedestrians with various poses stably.
Therefore, this paper constructs multiple classifiers optimized for variously posed pedestrians by classifying pedestrian images into multiple pose classes.
Also, to reduce the bias and the cost for preparing numerous pedestrian images for each pose class for learning, the proposed method employs a generative learning method.
Finally, the proposed method constructs multiple classifiers by using the synthesized pedestrian images.
Experimental results showed that the detection accuracy of the proposed method outperformed comparative methods, and we confirmed that the proposed method could detect variously posed pedestrians stably.

DOI： 10.5220/0003817305670572

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

We propose a method to detect the inconsistency between a subject and the speaker for extracting speech scenes from news videos.
Speech scenes in news videos contain a wealth of multimedia information, and are valuable as archived material.
In order to extract speech scenes from news videos, there is an approach that uses the position and size of a face region.
However, it is difficult to extract them with only such approach, since news videos contain non-speech scenes where the speaker is not the subject, such as narrated scenes.
To solve this problem, we propose a method to discriminate between speech scenes and narrated scenes based on the co-occurrence between a subject's lip motion and the speaker's voice.
The proposed method uses lip shape and degree of lip opening as visual features representing a subject's lip motion, and uses voice volume and phoneme as audio feature representing a speaker's voice.
Then, the proposed method discriminates between speech scenes and narrated scenes based on the correlations of these features.
We report the results of experiments on videos captured in a laboratory condition and also on actual broadcast news videos.
Their results showed the effectiveness of our method and the feasibility of our research goal.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a method for low resolution QR-code recognition. A QR-code is a two-dimensional binary symbol that can embed various information such as characters and numbers. To recognize a QR-code correctly and stably,the resolution of an input image should be high. In practice,however, recognition of a QR-code is usually difficult due to
low resolution when it is captured from a distance. In this paper, we propose a method to improve the performance of low resolution QR-code recognition by using the super-resolution technique that generates a high resolution image from multiple low-resolution images. Although a QR-code is a binary pattern, it is observed as a grayscale image due to the degradation through the capturing process. Especially the pixels around the borders between white and black regions become ambiguous. To overcome this problem, the proposed method introduces a binary pattern constraint to generate super-resolved images appropriate for recognition. Experimental results showed that a recognition rate of 98% an be achieved by the proposed method, which is a 15.7% improvement in comparison with a method using a conventional super-resolution method.

Language：English   Publishing type：Research paper (scientific journal)  

A power-efficient K-Means hardware architecture that can automatically estimate the number of clusters in the clustering process is proposed. The contributions of this work include two main aspects. The first is the integration of the hierarchical data sampling in the hardware to accelerate the clustering speed. The second is the development of the Bayesian-Information-Criterion (BIC) Processor" to estimate the number of clusters of K-Means. The architecture of the "BIC Processor" is designed based on the simplification of the BIC computations, and the precision of the logarithm function is also analyzed. The experiments show that the proposed architecture can be employed in different multimedia applications, such as motion segmentation and edge-adaptive noise reduction. Besides, the gate count of the hardware is 51 K with the 90-nm complimentary metal-oxide-semiconductor technology. It is also shown that this work can achieve high efficiency compared with a GPU, and the power consumption scales well with the number of clusters and the number of dimensions. The power consumption ranges between 10.72 and 12.95 mW in different modes when the operating frequency is 233 MHz."

DOI： 10.1109/JETCAS.2011.2165231

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

We propose a visibility estimation method for traffic signs considering temporal environmental changes, as a part of work for the realization of nuisance-free driver assistance systems. Recently, the number of driver assistance systems in a vehicle is increasing. Accordingly, it is becoming important to sort out appropriate information provided from them, because providing too much information may cause driver distraction. To solve such a problem, we focus on a visibility estimation method for controlling the information according to the visibility of a traffic sign. The proposed method sequentially captures a traffic sign by an in-vehicle camera, and estimates its accumulative visibility by integrating a series of instantaneous visibility. By this way, even if the environmental conditions may change temporally and complicatedly, we can still accurately estimate the visibility that the driver perceives in an actual traffic scene. We also investigate the performance of the proposed method and show its effectiveness.

Language：English   Publishing type：Research paper (international conference proceedings)  

Road image is becoming important for several
applications such as car navigation systems, traffic environment
research, city modeling. Usually, a road image can be obtained
from an aerial image but the resolution of the aerial image is
often low, or it contains occlusions by obstacles. Therefore, the
update of road image is required. In this paper, we propose a
road image mosaicing method using in-vehicle camera images
and an aerial image. We first perform image registration of road
regions between these images, and then, we generate a large road
image by performing image mosaicing of road regions in invehicle
camera images. In an experiment, we achieved resolution
improvement and occlusions removal, and also succeeded in
update of a large road image.

Language：English   Publishing type：Research paper (international conference proceedings)  

Free space detection is very important for vehicle navigation and safe driving. 3-D line segment reconstruction
of a street is important for the free space detection because a street-view includes many line segments. For the free space
detection, we propose a method for reconstructing 3-D line segments in a streetscape using a monocular in-vehicle camera.
The 3-D reconstruction of the line segments is achieved by using each three images from an image sequence. Once accurate
camera poses of these images are obtained, one of the remaining crucial problems is to match the line segments between the
images correctly. A strategy for finding correspondence of the line segments is as follows: First, the correspondences of line
segment candidates are searched by using a two-view constraint. However, the two-view constraint has difficulty on determining
an unique correspondence geometrically. Therefore, the candidates of the line segment correspondences are reduced using
a three-view constraint. In order to improve the accuracy, the proposed method exploits a color feature of the line segment
and a preliminary knowledge of the vehicle motion. Finally, the line segments are reconstructed using the correspondences.
From an experimental result, we confirmed the effectiveness of the proposed method. Application to the free space detection
demonstrated the usefulness of the reconstructed line segments.

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes an intelligent traffic sign detector using adaptive learning based on online gathering of training samples from in-vehicle camera image sequences. To detect traffic signs accurately from in-vehicle camera images, various training samples of traffic signs are needed. In addition, to reduce false alarms, various background images should also be prepared before constructing the detector. However, since their appearances vary widely, it is difficult to obtain them exhaustively by manual intervention. Therefore, the proposed method simultaneously obtains both traffic sign images and background images from in-vehicle camera images. Especially, to reduce false alarms, the proposed method gathers background images that were easily mis-detected by a previously constructed traffic sign detector, and re-trains the detector by using them as negative samples. By using retrospectively tracked traffic sign images and background images as positive and negative training samples, respectively, the proposed method constructs a highly accurate traffic sign detector automatically. Experimental results showed the effectiveness of the proposed method.

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.1541/ieejias.131.466

Language：English   Publishing type：Research paper (international conference proceedings)  

In order to estimate accurate attitude of mobile robots and　vehicles, we propose a hybrid system which combines a lowcost　monocular camera with a gyro sensor. A gyro sensor has　drift errors that accumulate over time. On the other hand, a　camera cannot obtain the rotation continuously in the case　where feature points cannot be extracted from images, although　the accuracy is better than a gyro sensor. To solve　these problems, we propose a method for combining these　sensors based on an Extended Kalman Filter.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ICPR.2010.844

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ICPR.2010.766

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/IVS.2010.5548137

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ISM.2009.67

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper presents a method for the automated anatomical labeling of bronchial branches extracted from 3D CT images based on machine learning and combination optimization. We also show applications of anatomical labeling on a bronchoscopy guidance system. This paper performs automated labeling by using machine learning and combination optimization. The actual procedure consists of four steps: (a) extraction of tree structures of the bronchus regions extracted from CT images, (b) construction of AdaBoost classifiers, (c) computation of candidate names for all branches by using the classifiers, (d) selection of best combination of anatomical names. We applied the proposed method to 90 cases of 3D CT datasets. The experimental results showed that the proposed method can assign correct anatomical names to 86.9% of the bronchial branches up to the sub-segmental lobe branches. Also, we overlaid the anatomical names of bronchial branches on real bronchoscopic views to guide real bronchoscopy.

DOI： 10.1007/978-3-642-04271-3_86

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ICDAR.2009.168

Authorship：Lead author   Language：English   Publishing type：Research paper (scientific journal)  

We propose a selective method of measurement for computing image similarities based on characteristic structure extraction and demonstrate its application to flexible endoscope navigation, in particular to a bronchoscope navigation system. Camera motion tracking is a fundamental function required for image-guided treatment or therapy systems. In recent years, an ultra-tiny electromagnetic sensor commercially became available, and many image-guided treatment or therapy systems use this sensor for tracking the camera position and orientation. However, due to space limitations, it is difficult to equip the tip of a bronchoscope with such a position sensor, especially in the case of ultra-thin bronchoscopes. Therefore, continuous image registration between real and virtual bronchoscopic images becomes an efficient tool for tracking the bronchoscope. Usually, image registration is done by calculating the image similarity between real and virtual bronchoscopic images. Since global schemes to measure image similarity, such as mutual information, squared gray-level difference, or cross correlation, average differences in intensity values over an entire region, they fail at tracking of scenes where less characteristic structures can be observed. The proposed method divides an entire image into a set of small subblocks and only selects those in which characteristic shapes are observed. Then image similarity is calculated within the selected subblocks. Selection is done by calculating feature values within each subblock. We applied our proposed method to eight pairs of chest X-ray CT images and bronchoscopic video images. The experimental results revealed that bronchoscope tracking using the proposed method could track up to 1600 consecutive bronchoscopic images (about 50s) without external position sensors. Tracking performance was greatly improved in comparison with a standard method utilizing squared gray-level differences of the entire images.

DOI： 10.1016/j.media.2009.06.001

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/IVS.2009.5164275

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/IVS.2009.5164417

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

AIM:We evaluated a prostate biopsy strategy for cancer detection using a computer simulation system with virtual needle biopsy for three-dimensional (3D) prostate models.
METHODS:Two 3D prostate models with a volume of 25 cc or 50 cc were constructed from computed tomographic images of radical prostatectomy specimens. The peripheral zone (PZ) and transition zone (TZ) were arranged in the prostate models according to the anatomical information. Four thousand patterns of cancer lesions were automatically inserted into each prostate model with a proportion of 75% in PZ and 25% in TZ. Average hit rates (AHR) in prostate models were evaluated both with an increased number of biopsy cores and various angles of virtual needle biopsy. The influence of adding secondary tumors for hit rates was also evaluated.
RESULTS:For both sizes, the laterally angled biopsy in 4-8 core biopsy schemes showed higher AHR than the vertical plane biopsy, while the vertical plane biopsy in 10-18 core biopsy schemes showed higher AHR than the laterally angled biopsy. A higher number of biopsy cores increased the AHR of secondary tumors.
CONCLUSIONS:Our results suggest that it is important in prostate cancer detection to change the needle placement according to the number of biopsy cores and the size of the prostate.

DOI： 10.1111/j.1442-2042.2006.01561.x

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Authorship：Lead author   Language：English   Publishing type：Research paper (scientific journal)  

This paper describes a method for tracking a bronchoscope by combining a position sensor and image registration. A bronchoscopy guidance system is a tool for providing real-time navigation information acquired from pre-operative CT images to a physician during a bronchoscopic examination. In this system, one of the fundamental functions is tracking a bronchoscope's camera motion. Recently, a very small electromagnetic position sensor has become available. It is possible to insert this sensor into a bronchoscope's working channel to obtain the bronchoscope's camera motion. However, the accuracy of its output is inadequate for bronchoscope tracking. The proposed combination of the sensor and image registration between real and virtual bronchoscopic images derived from CT images is quite useful for improving tracking accuracy. Furthermore, this combination has enabled us to achieve a real-time bronchoscope guidance system. We performed evaluation experiments for the proposed method using a rubber phantom model. The experimental results showed that the proposed system allowed the bronchoscope's camera motion to be tracked at 2.5 frames per second.

Language：English   Publishing type：Research paper (international conference proceedings)  

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (scientific journal)  

Authorship：Lead author   Language：English   Publishing type：Research paper (scientific journal)  

This paper discusses the virtual prostate needle biopsy system and the construction of the virtual prostate model, considering the actual distribution of prostate abnormalities. A needle biopsy simulation is performed for the constructed virtual prostate, and the actual biopsy procedure in the clinical situation is evaluated. The prostate needle biopsy is a histologic diagnosis procedure in which a sample is acquired from the prostate tissue by needle biopsy and is inspected under a microscope. In order to achieve reliable prostate needle biopsy, it is necessary to consider systematically and numerically the number of needles required, and their locations and insertion angles.
For such a purpose, a system is developed in which a virtual prostate is constructed on a computer and the biopsy procedure is evaluated quantitatively by performing virtually the prostate needle biopsy. Furthermore, two different models are constructed, for the prostate with and without hypertrophy, based on the actual statistical distribution data for
prostate abnormalities. Each model is partitioned into the peripheral zone and the transition zone. The constructed
virtual prostate is input into the virtual needle biopsy simulation system, and three different systematic biopsy procedures actually used at clinical sites, and four different needle biopsy procedures, are experimentally evaluated.
The experiments show that the insertion angle that maximizes the hit probability is not always the same as the
insertion angle that maximizes cancer sample acquisition. It is evident that the proposed method can indicate a biopsy
procedure which realizes a high hit probability with a small number of needles.

DOI： 10.1002/scj.20181

Language：English   Publishing type：Research paper (international conference proceedings)  

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1016/j.ics.2005.03.282

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (scientific journal)  

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1007/978-3-540-39899-8_50

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

This paper describes a method for tracking the camera motion of a flexible endoscope, in particular a bronchoscope, using epipolar geometry analysis and intensity-based image registration. The method proposed here does not use a positional sensor attached to the endoscope. Instead, it tracks camera motion using real endoscopic (RE) video images obtained at the time of the procedure and X-ray CT images acquired before the endoscopic examination. A virtual endoscope system (VES) is used for generating virtual endoscopic (VE) images. The basic idea of this tracking method is to find the viewpoint and view direction of the VES that maximizes a similarity measure between the VE and RE images. To assist the parameter search process, camera motion is also computed directly from epipolar geometry analysis of the RE video images. The complete method consists of two steps: (a) rough estimation using epipolar geometry analysis and (b) precise estimation using intensity-based image registration. In the rough registration process, the method computes camera motion from optical flow patterns between two consecutive RE video image frames using epipolar geometry analysis. In the image registration stage, we search for the VES viewing parameters that generate the VE image that is most similar to the current RE image. The correlation coefficient and the mean square intensity difference are used for measuring image similarity. The result obtained in the rough estimation process is used for restricting the parameter search area. We applied the method to bronchoscopic video image data from three patients who had chest CT images. The method successfully tracked camera motion for about 600 consecutive frames in the best case. Visual inspection suggests that the tracking is sufficiently accurate for clinical use. Tracking results obtained by performing the method without the epipolar geometry analysis step were substantially worse. Although the method required about 20 s to process one frame, the results demonstrate the potential of image-based tracking for use in an endoscope navigation system.

DOI： 10.1016/S1361-8415(02)00089-0

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese  

Language：Japanese  

カメラを用いた群集解析は公共の安全やマーケテイングなどに需要がある。我々は群集解析の中でも、小領域毎の人数を求める空間的な人数分布推定の実現を目指している。 しかし、カメラから離れた場所では人物閣の遮蔽が発生するため、空間的な人数分布を正しく推定することは困難である。そこで本稿では、複数視点、のカメラから撮影された画像を用いて記憶型回帰により人数分布を推定する手法を紹介する。本手法は、人数分布と群集の見えの対応表を用いるものであり、予め群集の見えに遮蔽が含まれているため、遮蔽に頑健な人数分布推定手法が実現される。

Language：Japanese  

本稿では、空撮画像と車載カメラ画像の対応付けにより自車の位置を推定する手法について述べた．提案手法は、複数時刻の車載カメラ画像を利用することで、自車位置の推定性能の向上を図ったものである．実験により、推定性能の向上を確認し、提案手法の有効性を確認する．

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Event date： 2025

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：群馬大学  

Event date： 2024

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2024

Language：English   Presentation type：Oral presentation (general)  

Venue：Keio University  

Event date： 2024

Language：English   Presentation type：Oral presentation (general)  

Venue：Keio University  

In the realm of autonomous driving, end-to-end models (E2EDMs) have gained prominence due to their high predictive accuracy. Such accuracy is attributed to the utilization of a backbone pre-trained on large datasets, subsequently fine-tuned on autonomous driving datasets. However, the inherent “black box” nature of these E2EDMs poses significant challenges in terms of explainability. Current methodologies predominantly focus on generating visual explanations for the E2EDMs’ decision-making process. Numerous approaches aim to enhance the explainability of these E2EDMs by fine-tuning with complicated architectures, supplemented by additional information, e.g., object position to develop more explainable E2EDMs. In this study, we diverge from the conventional approaches where significant effort is placed during the fine-tuning phase of E2EDMs. Our method focuses on training backbones before the fine-tuning phase. This preemptive strategy enables us to fine-tune more explainable E2EDMs without the need for additional information or complex training techniques.

Event date： 2024

Language：English   Presentation type：Poster presentation  

Venue：Rome, Italy  

In this paper, we present a new task of detecting an object that a target pedestrian is gazing at in a traffic scene called PEdestrian's Gaze Object (PEGO). We argue that the detection of gaze object can provide important information for pedestrian's behavior prediction and can contribute to the realization of automated vehicles. For this task, we construct a dataset of in-vehicle camera images with annotations of the objects that pedestrians are gazing at. Also, we propose a Transformer-based method called PEGO Transformer to solve the PEGO detection task. The PEGO Transformer directly performs gaze object detection with the utilization of whole-body features without a high-resolution head image and a gaze heatmap which the traditional methods rely on. Experimental results showed that the proposed method could estimate pedestrian's gaze object accurately even if various objects exist in the scene.

Event date： 2024

Language：English   Presentation type：Oral presentation (general)  

Venue：Precise House Mantegna Roma  

Deep learning technology has rapidly advanced, leading to the development of End-to-End driving models (E2EDMs) for autonomous vehicles with high prediction accuracy. To comprehend the prediction results of these E2EDMs, one of the most representative explanation methods is attribution-based. There are two kinds of attribution-based explanation methods: pixel-level and object-level. Usually, the heatmaps illustrate the importance of pixels and objects in the prediction results, serving as explanations for E2EDMs. Since there are many attribution-based explanation methods, evaluation methods are proposed to determine which one is better at improving the explainability of E2EDMs.
Fidelity measures the explanation's faithfulness to the model's prediction method, which is a bottommost property. However, no evaluation method could measure the fidelity difference between object-level and pixel-level explanations, making the current evaluation incomplete. In addition, without considering fidelity, previous evaluation methods may advertise manipulative explanations that solely seek human satisfaction (persuasibility). Therefore, we propose an evaluation method that further considers fidelity, our method enables a comprehensive evaluation that proves the object-level explanations genuinely outperform pixel-level explanations in fidelity and persuasibility, thus could better improve the explainability of the E2EDMs.

Event date： 2024

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：別府国際コンベンションセンター ビーコンプラザ  

注視対象物推定は，自動運転車両の実現において不可欠な歩行者の行動予測のための重要な要素である．本発表では，交通シーンで歩行者が注視している物体（PEdestrians' Gaze Object: PEGO）を推定するPEGO Transformer V2を提案する．従来手法の多くは，高解像度の頭部画像と視線ヒートマップを用いて歩行者の注視対象物を推定している．しかしながら，遠方にいる歩行者に対して，高解像度の頭部画像を得られないため推定精度が低いという問題があった．そこでPEGO Transformer V2は，歩行者全身の特徴に加え，歩行者位置情報も活用することで性能改善を図る．実験の結果，提案手法であるPEGO Transformer V2は，シーン中の歩行者ごとにそれぞれ正しい注視対象物を推定できることを確認した．

Event date： 2024

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2024

Language：Japanese   Presentation type：Poster presentation  

Venue：熊本城ホール（熊本県熊本市中央区桜町3番40号）  

Event date： 2024

Language：English   Presentation type：Oral presentation (general)  

Venue：Shanghai, China  

Event date： 2024

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2024

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2024

Language：English   Presentation type：Poster presentation  

Venue：熊本城ホール（熊本県熊本市中央区桜町3番40号）  

In autonomous driving, end-to-end driving models (E2EDMs) have become increasingly popular due to their exceptional predictive performance. This performance stems from leveraging a backbone that is pre-trained on large datasets and then fine-tuned for driving tasks. However, the opaque nature of these E2EDMs presents a challenge to explainability, thus visual explanations are created to shed light on the decision-making process of E2EDMs. Currently, these E2EDMs have been made more explainable by fine-tuning the driving tasks along with a side task designed for explainability. This side task for explainability requires complex structures with auxiliary data, such as the location of objects.
In this paper, we argue that a more effective approach to enhancing the explainability of E2EDMs is to design a fine-tuning specifically focused on explainability. We propose CROp-based COntrastive DIscriminative LEarning (CROCODILE) to enhance a backbone’s capability to accurately identify object features during this additional fine-tuning for explainability. By adopting CROCODILE, we can develop more explainable E2EDMs without relying on auxiliary data or complex structures in the fine-tuning for driving tasks. Our experimental results confirm that our approach enhances the explainability of E2EDMs.

Event date： 2024

Language：English   Presentation type：Poster presentation  

Event date： 2024

Language：Japanese   Presentation type：Poster presentation  

Venue：熊本城ホール（熊本県熊本市中央区桜町3番40号）  

Event date： 2024

Language：English   Presentation type：Oral presentation (general)  

Venue：InterContinental Hanoi Landmark72  

Event date： 2024

Language：English   Presentation type：Poster presentation  

Venue：Ateneo de Manila University  

Event date： 2023

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：公立はこだて未来大学  

Event date： 2023

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：公立はこだて未来大学  

Event date： 2023

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：公立はこだて未来大学  

Event date： 2023

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2023

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2023

Language：Japanese   Presentation type：Oral presentation (invited, special)  

Event date： 2023

Language：English   Presentation type：Oral presentation (general)  

Venue：Maison Glad JEJU hotel, Jeju, Korea  

Event date： 2023

Language：English   Presentation type：Oral presentation (general)  

Venue：Lisbon, Portugal  

Event date： 2023

Language：English   Presentation type：Poster presentation  

Venue：Lisbon, Portugal  

Event date： 2023

Language：Japanese   Presentation type：Poster presentation  

Venue：アクトシティ浜松  

Event date： 2023

Language：English   Presentation type：Poster presentation  

Venue：Anchorage, AK, USA  

DOI： 10.1109/IV55152.2023.10186742

Event date： 2023

Language：English   Presentation type：Oral presentation (general)  

Venue：アクトシティ浜松  

Event date： 2023

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：沖縄国際大学  

Event date： 2023

Language：English   Presentation type：Oral presentation (general)  

Venue：アクトシティ浜松  

Event date： 2023

Language：Japanese   Presentation type：Poster presentation  

Venue：アクトシティ浜松  

Event date： 2023

Language：Japanese   Presentation type：Oral presentation (invited, special)  

Venue：中部生産性本部 セミナールーム  

Event date： 2022

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2022

Language：English   Presentation type：Oral presentation (general)  

Venue：Hybrid (The Hong Kong Polytechnic University and online)  

Event date： 2022

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：姫路市文化コンベンションセンター アクリエひめじ  

Event date： 2022

Language：English   Presentation type：Oral presentation (general)  

Venue：Online  

DOI： 10.1109/MIPR54900.2022.00050

Event date： 2022

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：姫路市文化コンベンションセンター アクリエひめじ  

Event date： 2022

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：姫路市文化コンベンションセンター アクリエひめじ  

Event date： 2022

Language：English   Presentation type：Oral presentation (general)  

Event date： 2022

Language：English   Presentation type：Poster presentation  

Venue：Fukuoka International Congress Center  

We analyze gaze while viewing videos at various playback speeds.
The aims of this paper are to evaluate the comprehension and viewability of videos.
In our experiment, we used 0.75x to 3.0x speed, and measured gaze.
We report differences in gaze behavior among different playback speeds in viewing same video.

Event date： 2022

Language：English   Presentation type：Oral presentation (general)  

Venue：Universidad Autónoma de Madrid (Hybrid)  

We propose a method to detect birds in a 3D environment referring to both audio information observed from a microphone array and visual information observed from a panorama camera. In general, in panorama images, birds appear relatively too small to be detected accurately even with the state-of-the-art deep learning models. Thus, the proposed method takes a two step approach where the birds are first roughly located referring to audio information by Sound Source Localization (SSL), and then image detection is applied within its vicinity. Through evaluation on a dataset annotated with bounding boxes surrounding the birds, we show that the proposed method improves detection performance of birds that appear in relatively small sizes in the image, in both accuracy and processing speed.

DOI： 10.1109/AVSS56176.2022.9959510

Event date： 2022

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：熊本大学工学部黒髪キャンパス  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2021

Language：English   Presentation type：Poster presentation  

Venue：Milan, Italy (Virtual)  

In this paper, we propose a method for human pose estimation from a Low-resolution Far-InfraRed (LFIR) image sequence captured by a 16 × 16 FIR sensor array. Human body estimation from such a single LFIR image is a hard task. For training the estimation model, annotation of the human pose to the images is also a difficult task for human. Thus, we propose the LFIR2Pose model which accepts a sequence of LFIR images and outputs the human pose of the last frame, and also propose an automatic annotation system for the model training. Additionally, considering that the scale of human body motion is largely different among body parts, we also propose a loss function focusing on the difference. Through an experiment, we evaluated the human pose estimation accuracy using an original data set, and confirmed that human pose can be estimated accurately from an LFIR image sequence.

Event date： 2021

Language：English   Presentation type：Poster presentation  

Venue：Milan, Italy (Virtual)  

In this paper, we propose an occlusion-robust pose
estimation method of an unknown object instance in an
object category from a depth image. In a cluttered
environment, objects are often occluded mutually. For
estimating the pose of an object in such a situation, a
method that de-occludes the unobservable area of the object
would be effective. However, there are two difficulties;
occlusion causes offset between the actual object center
and the center of its observable area, and different
instances in a category may have different shapes. To cope
with these difficulties, we propose a two-stage
Encoder-Decoder model to extract features with objects
whose centers are aligned to the image center. In the
model, we also propose the Median-shape Reconstructor as
the second stage to absorb shape variations in a category.
By evaluating the method with both a large-scale virtual
dataset and a real dataset, we confirmed the proposed
method achieves good performance on pose estimation of an
occluded object from a depth image.

Event date： 2021

Language：English   Presentation type：Poster presentation  

Venue：Milan, Italy (Virtual)  

In this paper, we propose Object Manifold Embedding GAN (Ω-GAN) to generate images of variously shaped and arbitrarily posed objects from a noise variable sampled from a distribution defined over the pose and the shape manifolds in a vector space. We introduce Parametric Manifold Sampling to sample noise variables from a distribution over the pose manifold to conditionally generate object images in arbitrary poses by tuning the pose parameter. We also introduce Object Identity Loss for clearly disentangling the pose and shape parameters, which allows us to maintain the shape of the object instance when only the pose parameter is changed. Through evaluation, we confirmed that the proposed Ω-GAN could generate variously shaped object images in arbitrary poses by changing the pose and shape parameters independently. We also introduce an application of the proposed method for object pose estimation, through which we confirmed that the object poses in the generated images are accurate.

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Online  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Online  

Event date： 2021

Language：English   Presentation type：Poster presentation  

Venue：Virtual  

For computers to understand human perception, metrics that can capture human perception well are important. However, there are few metrics that characterize the visual perception of humans towards images. Therefore, in this paper, we propose a novel concept and a metric of pointedness of an image, which describes how pointy an image is perceived. The algorithm is inspired by the Features from Accelerated Segment Test (FAST) algorithm for corner detection which looks on the number of continuous neighboring darker pixels surrounding each pixel. We assume that this number would be proportional to the perceived pointedness in the region around the pixel. We evaluated our method towards how well it could capture the human perception of images. To compare the method with similar metrics that describe shapes, we prepared silhouette images of both artificial shapes and natural objects. The results showed that the proposed method gave nearly equivalent perceptual performance to other metrics and also worked in a larger variety of images.

DOI： https://doi.org/10.1007/978-3-030-78635-9_20

Event date： 2021

Language：Japanese   Presentation type：Poster presentation  

Venue：online  

Event date： 2021

Language：English   Presentation type：Oral presentation (general)  

Venue：Online  

DOI： 10.1007/978-3-030-67835-7_6

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Online  

Event date： 2021

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Online  

Event date： 2020

Language：English   Presentation type：Poster presentation  

Venue：Dublin, Ireland (Virtual)  

Imageability is a concept from Psycholinguistics quantizing the human perception of words. However, existing datasets are created through subjective experiments and are thus very small. Therefore, methods to automatically estimate the imageability can be helpful. For an accurate automatic imageability estimation, we extend the idea of a psychological hypothesis called Dual-Coding Theory, that discusses the connection of our perception towards visual information
and language information, and also focus on the relationship between the pronunciation of a word and its imageability. In this research, we propose a method to estimate imageability of words
using both visual and language features extracted from corresponding data. For the estimation, we use visual features extracted from low- and high-level image features, and language features extracted from textual features and phonetic features of words. Evaluations how that our proposed method can estimate imageability more accurately than comparative methods, implying the contribution of each feature to the imageability.

DOI： 10.1145/3372278.3390731

Event date： 2020

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン  

Event date： 2020

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン  

Event date： 2020

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：広島大学  

Event date： 2020

Language：Japanese  

Event date： 2020

Language：Japanese  

Event date： 2020

Language：English   Presentation type：Poster presentation  

Venue：Grand Hotel Excelsior Malta  

In recent years, various facilities have been deployed to support visually impaired people. However, accidents caused by visual disabilities still occur. In this paper, to support the visually impaired people in public areas, we aim to identify whether a pedestrian image sequence obtained by a surveillance camera indicates the presence of a white cane user by analyzing the temporal transition of a human skeleton represented as 2D coordinates. Our previously proposed method aligns the orientation of the human skeletons to various orientations and identifies a white cane user from the corresponding sequences, relying on multiple classifiers related to each orientation. The method employs an exemplar-based approach to perform the alignment. However, it heavily depends on the number of exemplars and consumes excessive memory. In this paper, we propose a method to align 2D human skeleton representation sequence to various orientations using the proposed Skeleton Orientation Alignment Networks (SOANets) based on an encoder-decoder model. Using SOANets, we can obtain 2D skeleton representation sequences aligned to various orientations, extract richer skeleton features, and recognize white cane users accurately. We conducted an experiment to confirm that the
proposed method improves the recognition rate by 16%, compared to the method that does not use the skeleton
orientation alignment.

Event date： 2020

Language：English   Presentation type：Oral presentation (general)  

Venue：指宿ふれあいプラザなのはな館  

Surveillance cameras are expected to play a large role in the development of ITS technologies. They can be used to detect abnormally behaving individuals which can then be reported to drivers nearby. There are multiple works that tackle the problem of abnormal behavior detection. However, most of these works make use of appearance features which have redundant information and are susceptible to noise. While there are also works that make use of pose skeleton representation, they do not consider well how to handle cases with occlusions, which can occur due to the simple reason of pedestrian orientation preventing some joints from appearing in the frame clearly. In this paper, we propose a skeleton trajectory representation that enables handling of occlusions. We also propose a framework for pedestrian abnormal behavior detection that uses the proposed representation and detect relatively hard-to-notice anomalies such as drunk walking. The experiments we conducted show that our method outperforms other representation methods.

Event date： 2020

Language：English  

Venue：Deajeon Convention Center, Daejeon, Korea  

Recent multimedia applications commonly use text and imagery from Social Media for tasks related to sentiment research. As such, there are various image datasets for sentiment research for popular classification tasks. However, there has been little research regarding the relationship between the sentiment of images and its annotations from a multi-modal standpoint. In this demonstration, we built a tool to visualize psycholinguistic groundings for a sentiment dataset. For each image, individual psycholinguistic ratings are computed from the image's metadata. A sentiment-psycholinguistic spatial embedding is computed to show a clustering of images across different classes close to human perception. Our interactive browsing tool can visualize the data in various ways, highlighting different psycholinguistic groundings with heatmaps.

DOI： 10.3758/s13428-018-1099-3

Event date： 2020

Language：English   Presentation type：Poster presentation  

Venue：Deajeon Convention Center, Daejeon, Korea  

A mimetic word is used to verbally express the manner of a phenomenon intuitively. The Japanese language is known to have a greater number of mimetic words in its vocabulary than most other languages. Especially, since human gaits are one of the most commonly represented behavior by mimetic words in the language, we consider that it should be suitable for labels of fine-grained gait recognition. In addition, Japanese mimetic words have a more decomposable structure than these in other languages such as English. So it is said that they have sound-symbolism and their phonemes are strongly related to the impressions of various phenomena. Thanks to this, native Japanese speakers can express their impressions on them briefly and intuitively using various mimetic words. Our previous work proposed a framework to convert the body-parts movements to an arbitrary mimetic word by a regression model. The framework introduced a phonetic space" based on sound-symbolism, and it enabled fine-grained gait description using the generated mimetic words consisting of an arbitrary combination of phonemes. However, this method did not consider the "naturalness" of the description. Thus, in this paper, we propose an improved mimetic word generation module considering its naturalness, and update the description framework. Here, we define the co-occurrence frequency of phonemes composing a mimetic word as the naturalness. To investigate the co-occurrence frequency, we collected many mimetic words through a subjective experiment. As a result of evaluation experiments, we confirmed that the proposed module could describe gaits with more natural mimetic words while maintaining the description accuracy."

DOI： 10.1007/978-3-030-37734-2_18

Event date： 2020

Language：Japanese   Presentation type：Poster presentation  

Venue：オンライン  

Event date： 2020

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン  

Event date： 2020

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Online  

Event date： 2020

Language：Japanese   Presentation type：Poster presentation  

Venue：オンライン  

Event date： 2020

Language：Japanese   Presentation type：Poster presentation  

Venue：Online  

Event date： 2020

Language：Japanese   Presentation type：Poster presentation  

Venue：Online  

Event date： 2020

Language：Japanese   Presentation type：Poster presentation  

Venue：Online  

Event date： 2020

Language：English   Presentation type：Oral presentation (general)  

This paper is an extension of our work in developing an attribute-aware semantic segmentation method which focuses on pedestrian understanding in a traffic scene. Recently, the trending topic of semantic segmentation has been expanded to be able to collaborate with the object’s attributes recognition task; Here, it refers to recognizing a pedestrian’s body orientation. The attribute-aware semantic segmentation can be more beneficial for driver assistance compared to the conventional semantic segmentation because it can provide a more informative output to the system. In this paper, we conduct a study of the data augmentation usage as an effort to enhance the performance of the attribute-aware semantic segmentation task. The experiments show that the proposed method in augmenting the training data is able to improve the model’s performance. We also demonstrate some of qualitative results and discuss the benefits to a driver assistance system.

DOI： 10.1109/ICoICT49345.2020.9166219

Event date： 2020

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン  

Event date： 2020

Language：English   Presentation type：Oral presentation (general)  

Venue：Online  

Event date： 2020

Language：Japanese   Presentation type：Oral presentation (general)  

Event date： 2020

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：国立沖縄工業高等専門学校  

近年，高齢化の進展により，独居高齢者に向けた見守りシステムが注目されている．しかし，見守りシステムにはプライバシーの問題が存在する．そこで我々はプライバシーの問題を軽減でき，さらに暗闇での撮影にも強い赤外線センサアレイを用いて撮影した超低解像度FIR画像系列から人物の骨格を推定する手法を提案してきた．本発表では特に超低解像度FIR画像内での人物位置と動作の違いに着目した骨格推定法について検討する．具体的には，超低解像度FIR画像内で人物位置に依存しない特徴を抽出するため，低解像度画像において影響が大きいと考えられる中間層でのPoolingをせずに，特徴抽出の最後にGlobalMaxPoolingを導入する．さらに動作の種類に合わせて，時系列情報と空間的情報を有効活用できるネットワークを提案する．実験では，赤外線センサアレイの画角内の様々な位置で人物を撮影したデータセットを用いて，従来手法と提案手法で特定の人物位置で学習を行ない，未学習位置で骨格推定精度を評価した．その結果，人物の動作を滑らかに推定でき，定量的にも精度が向上することを確認した．

Event date： 2020

Language：Japanese   Presentation type：Poster presentation  

Venue：国立沖縄工業高等専門学校  

近年，自動運転に関する研究開発が活発になされており，限定された環境下ではあるものの公道での走行実験等も進められている．しかし，交差点は事故の危険性が高く，交差点を含む一般道を状況に応じて安全に走 行するための課題は多い．このような背景から，車載カメラを用いて詳細に周囲環境を認識し，周囲の状況を考慮 して安全に走行できる道路上の領域（走路）を推定する技術が求められている．そこで本発表では，車載カメラ画像と，目的地までの経路から得られる車両の進路情報を条件（Condition）とした，conditional GANの枠組みを利用して走行すべき走路を推定する手法を提案する．実走行データから構築したデータセットを用いた評価実験により，画像のみを入力とする比較手法に比べて，提案手法では走路推定精度が15.0%向上することを確認した．

Event date： 2020

Language：English   Presentation type：Oral presentation (general)  

Venue：Rhodes, Greece (Virtual)  

It is intuitively obvious that eye-gaze behaviors of experienced drivers are different from those of novice drivers. However, it is not easy to understand the difference in their behavior quantitatively. In this work, we present an explainable eye-gaze behavior modeling method for electric wheelchair drivers based on Inverse Reinforcement Learning (IRL). We first create feature maps that represent risk factors during driving. These feature maps are able to represent not only to what but also from where drivers pay attention. IRL uses the feature maps to learn the reward representing the eye-gaze behaviors and allows us to see important features via the automatic acquisition of the reward. Through analysis of the learned model, we show quantitative evidence that eye-gaze behaviors of experienced drivers are better-balanced by paying attention to multiple risks.

DOI： 10.1109/ITSC45102.2020.9294255

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：大分大学  

Event date： 2019

Language：English  

Venue：グランキューブ大阪（大阪府立国際会議場）  

Recent multimedia applications commonly use text and imagery from Social Media for tasks related to sentiment research. As such, there are various image datasets for sentiment research for popular classification tasks. However, there has been little research regarding the relationship between the sentiment of images and its annotations from a multi-modal standpoint. In this research, we built a tool to visualize psycholinguistic groundings for a sentiment dataset. For each image, individual psycholinguistic ratings are computed from the image's metadata. A sentiment-psycholinguistic spatial embedding is computed to show a clustering of images across different classes close to human perception. Our interactive browsing tool can visualize the data in various ways, highlighting different psycholinguistic groundings with heatmaps.

Event date： 2019

Language：English   Presentation type：Poster presentation  

Venue：Osaka International Convention Center  

Recently, helper robots become popular in our social life, especially for helping the elderly and the disabled people to perform their daily tasks at home. To handling objects, object pose estimation from a depth image is an essential task of the helper robots. However, an object’s pose is often ambiguous from an observation from only a single viewpoint. If we can observe the object from additional viewpoints, the pose estimation result will be better. Thus, we propose a next viewpoint recommendation method based on pose ambiguity minimization. We confirmed and showed the proposed method outperformed other comparative methods on synthetic object images.

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (invited, special)  

Venue：京都大学  

Event date： 2019

Language：English   Presentation type：Poster presentation  

Venue：Long Beach Convention Center, Long Beach, CA USA  

Event date： 2019

Language：Japanese   Presentation type：Poster presentation  

Venue：パシフィコ横浜アネックスホール  

近年，高齢化社会の進展が問題となっており、高齢者の健康の暮らしのためには身体機能の維持が必須である．そのためには日常的に，身体の動きを把握する必要があるが，センサ装着が必要な方法や可視光カメラを用いる方法ではセンサの装着忘れやプライバシの問題がある．
そこで，超低解像度遠赤外線画像を用いた骨格推定法を提案する．現在，高解像度な可視光画像からの骨格推定は可能であるため，本手法では可視光カメラと赤外線カメラで同時に撮影をするシステムを構築し，学習データを収集する．それを用いて，ニューラルネットワークを学習し，超低解像度遠赤外線画像から直接骨格を推定する．

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：国立オリンピック記念青少年総合センター  

Event date： 2019

Language：English   Presentation type：Oral presentation (general)  

Venue：國立成功大學（臺灣，臺南）  

Event date： 2019

Language：English   Presentation type：Poster presentation  

Venue：國立成功大學（臺灣，臺南）  

Event date： 2019

Language：English   Presentation type：Poster presentation  

Venue：國立成功大學（臺灣，臺南）  

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：法政大学 市ケ谷キャンパス  

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：早稲田大学 西早稲田キャンパス  

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：早稲田大学 西早稲田キャンパス  

近年，Semantic Segmentationは自動運転や高度運転支援の要素技術として注目を集めている．しかし，高精度なSemantic Segmentation手法の計算コストは高く，実時間処理は困難である．従って，精度を維持したまま，より高速に計算する手法が求められている．Semantic Segmentationの処理は，畳み込み演算の計算時間が大部分を占める．提案手法では，XceptionモジュールのDepthwise convolutionを垂直・水平方向に分解することで畳み込み演算のさらなる高速化を試みる．実験により，mIoUの低下を抑えつつ，パラメータ数と実行時間を削減可能なことを確認した．

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：早稲田大学 西早稲田キャンパス  

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：早稲田大学 西早稲田キャンパス  

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：福岡大学七隈キャンパス  

近年，高齢化社会の進展が問題となっており、高齢者の健康の暮らしのためには身体機能の維持が必須である．そのためには日常的に，身体の動きを把握する必要があるが，センサ装着が必要な方法や可視光カメラを用いる方法ではセンサの装着忘れやプライバシの問題がある．
そこで，超低解像度遠赤外線画像を用いた姿勢推定法を提案する．現在，高解像度な可視光画像からの骨格推定は可能であるため，本手法では可視光カメラと赤外線カメラで同時に撮影をするシステムを構築し，学習データを収集する．それを用いて，ニューラルネットワークを学習し，超低解像度遠赤外線画像から直接姿勢を推定する．

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：名古屋大学  

Event date： 2019

Language：English   Presentation type：Oral presentation (general)  

Venue：名古屋大学  

Event date： 2019

Language：Japanese   Presentation type：Poster presentation  

Venue：北九州国際会議場  

Event date： 2019

Language：Japanese   Presentation type：Poster presentation  

Venue：北九州国際会議場  

本研究では,視覚障害者を支援するために,人物の姿勢変化から歩行者系列が白杖利用者か否か分類す
ることを目的とする.しかし,人物の向きによって画像上の人物姿勢は大きく異なるため,分類しなければなら
ない姿勢パターンが多くなり,分類が困難となる.そこで本発表では,事例ベースで様々な向きの姿勢をある1つ
の向きの姿勢に正規化する手法を提案する.分類する姿勢のパターンが限定されることにより,高精度の分類が
可能となる.評価実験において,提案手法により認識率が 5% 向上することを確認した.

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：北九州国際会議場  

夜間における歩行者の交通死亡事故が多発しており，その原因として運転者による歩行者の視認が困難であることが挙げられる．近年，通常の前方照射に加えて選択的照射が可能なヘッドライトが開発されており，その技術を活かして歩行者に選択的に光を照射することで歩行者の見つけやすさを向上させることが考えられる．我々はこれまでに点滅に着目して効果的な照射法を検討し，特定の周波数において点滅光の照射が効果的であることを確認しているが， 1 つの固定された周辺光条件しか想定していなかった．そこで本研究では，周辺光条件ごとに効果的な点滅照射を分析する．具体的には，運転環境を模擬した臨場感のある仮想環境において 6 種類の周辺光条件を設定し，通常のロービーム照射，ハイビーム照射および 7 種類の点滅周波数を用いて実験を行なった．実験結果から，点滅照射が見つけやすさ向上に有効であることや，異なる周辺光条件では効果的な周波数が異なることを確認した．

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：ホテルオークラJRハウステンボス  

旅行を計画する際，満足のいく計画を立てるためには，旅先の雰囲気を予め把握することが重要である．ある地域の訪問予定時の雰囲気が，既に知っている地域のある時期の雰囲気と似ていることが分かれば，その地域の訪問時の雰囲気を直感的に把握することできる．これを実現するため，我々は旅行者の体験を反映した大量のSNS投稿写真に着目した．具体的には，写真に付随する時空間情報を用いて写真をクラスタリングし「時期情報付き地域」を獲得し，写真のVisual Conceptに基づいて時空間類似地域をマイニングすることで，時期を考慮した類似地域の組を獲得する手法を提案する．手法の有用性を示すため，Flickr に投稿された写真を用いて時空間類似地域マイニング実験を行った．また，被験者実験を通して，提案手法の妥当性を確認した．

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：ホテルオークラJRハウステンボス  

近年,投稿型料理レシピポータルサイトが普及している.ウェブ上には膨大な数の料理レシピが存在し,
掲載されている料理の種類は多岐にわたる.しかし,それに伴い,サイト利用者が検索の際に目的とする料理レシピ
件数を絞りこむことが難しくなりつつある.そこで本研究では,地域属性(スペイン料理,中華料理など)ごとに料
理の味や見た目が大きく異なることに着目し,料理レシピがもつ多様な情報から,機械学習を用いて作成される料理
の地域属性を推定する手法を提案する.学習には,料理の完成画像から得られた画像特徴,素材一覧から得られた素
材特徴,調理手順文の動詞から得られた手順特徴の 3 つを組み合わせて用いる.結果として,これら 3 つの全特徴を
用いた手法で 8 割近い推定精度が得られることを確認した.

Event date： 2019

Language：English   Presentation type：Oral presentation (general)  

Venue：Vienna House Diplomat Prague, Prague, Czech Republic  

In recent years, the demand for highly accurate pedestrian detectors has increased due to the development of advanced driving support systems.
For the training of an accurate pedestrian detector, it is important to collect a large number of training samples.
To support this, this paper proposes a ``hard negative'' mining method to automatically extract background images which tend to be erroneously detected as pedestrians.
Negative samples are selected based on the assumption that frequent patterns observed multiple times in the same location are most likely parts of the background scene.
As a result of an evaluation using in-vehicle camera images captured along the same route, we confirmed that the proposed method can automatically collect false positive samples accurately.
We also confirmed that a highly accurate detector can be constructed using the additional negative samples.

Event date： 2019

Language：English   Presentation type：Oral presentation (general)  

Venue：Vienna House Diplomat Prague, Prague, Czech Republic  

3D object pose estimation by using a depth sensor is one of the important tasks in activities by robots. To reduce the pose ambiguity of an estimated object pose, several methods for multiple viewpoint pose estimation have been proposed. However, these methods need to select the viewpoints carefully to obtain better results. If the pose of the target object is ambiguous from the current observation, we could not decide where we should move the sensor to set as the next viewpoint. In this paper, we propose a best next viewpoint recommendation method by minimizing the pose ambiguity of the object by making use of the current pose estimation result as a latent variable. We evaluated viewpoints recommended by the proposed method and confirmed that it helps us to gain better pose estimation results than several comparative methods on a synthetic dataset.

Event date： 2019

Language：English   Presentation type：Poster presentation  

Venue：Vienna House Diplomat Prague, Prague, Czech Republic  

In recent years, LIDAR is playing an important role as a sensor for understanding environments of a vehicle’s surroundings. Active-scan LIDAR is being actively developed as a LIDAR that can control the laser irradiation direction arbitrary and rapidly. In comparison with conventional uniform-scan LIDAR (e.g. Velodyne HDL-64e), Active-scan LIDAR enables us to densely scan even distant pedestrians. In addition, if appropriately controlled, this sensor has the potential to reduce unnecessary laser irradiations towards non-target objects. Although there are some preliminary studies on pedestrian scanning strategy for Active-scan LIDARs, in the best of our knowledge, an efficient method has not been realized yet. Therefore, this paper proposes a novel pedestrian scanning method based on orientation aware pedestrian likelihood estimation using the orientation-wise pedestrian’s shape models with local distribution of measured points. To evaluate the effectiveness of the proposed method, we conducted experiments by simulating Active-scan LIDAR using point-clouds from the KITTI dataset. Experimental results showed that the proposed method outperforms conventional methods.

Event date： 2019

Language：Japanese   Presentation type：Poster presentation  

Venue：グランキューブ大阪（大阪府立国際会議場）  

近年，先進国を中心に高齢化が進んでおり，電動車いすの需要が高まっている．しかし，電動車いすの利便性は高い一方で，事故が多発している．自動車の事故は視覚的な認知ミスが原因であることが多く，電動車いすも自動車と同様に運転者の視覚的な認知を支援することが事故防止に有効であると考えられる．また，熟練運転者と非熟練運転者の視行動に差異があることが知られており，熟練者の視行動に基づいて，運転者の視覚的な認知の支援をすることができると考えられる．本研究では，電動車いす運転における熟練者の視行動のモデル化を目指し，リスクが複数存在する箇所である死角あり狭路で運転する場合を想定して，熟達に伴う視行動の変化を3次元空間上で分析した．その結果，熟達するほど複数のリスクに対してバランスよく注意を払うことを確認した．

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：広島県情報プラザ  

Event date： 2019

Language：Japanese   Presentation type：Oral presentation (invited, special)  

Venue：名古屋大学  

Event date： 2019

Language：English   Presentation type：Poster presentation  

Venue：Aotea Centre (Auckland, New Zealand)  

Event date： 2019

Language：English   Presentation type：Poster presentation  

Venue：Aotea Centre (Auckland, New Zealand)