Faculty Profiles - TAKEDA, Kazuya

Personalized lane changes using subjective risk-sensitive framework

Naren Bao, Alexander Carballo, Kazuya Takeda

Towards Human-Vehicle Harmonization page： 211 - 222 2023.3

Publishing type：Part of collection (book)

Most of the current researches on autonomous vehicles' control assume that all vehicles should have the same patterns of driving implemented, resulting in conservative or average systems. However, these results may not be acceptable to drivers who prefer a more aggressive style of driving, while extremely cautious drivers may consider the standard outputs to be too aggressive. In this chapter, we introduce risk-sensitive control (RSC), an inverse optimal control algorithm that estimates risk-sensitive driving features and incorporate them into a receding-horizon controller. RSC uses a meta-learning algorithm to update the parameters of the cost function, continuously improving the controller online as more and more driving data is gathered from the user for subjective risk feedback. The estimator takes into account the individual differences in subjective risk analysis, in terms of driving features and surrounding vehicle locations, by adjusting the cost function and constraints. We test this approach using five-lane change scenarios, some safe and some risky, with 30 real drivers in a CARLA simulation environment. Based on both quantitative and qualitative evaluations, our experimental results demonstrate that the proposed framework can generate users' preferred driving commands during lane changes, that is, commands associated with lower subjective risk, outperforming conventional, model-based predictive control methods in terms of replicating the user's own driving behavior.

DOI： 10.1515/9783110981223-016

Framework for Generation and Removal of Multiple Types of Adverse Weather from Driving Scene Images Reviewed

Hanting Yang, Alexander Carballo, Yuxiao Zhang, Kazuya Takeda

SENSORS Vol. 23 ( 3 ) page： 1548 2023.2

Language：English Publishing type：Research paper (scientific journal) Publisher：MDPI

Weather variation in the distribution of image data can cause a decline in the performance of existing visual algorithms during evaluation. Adding additional samples of target domain to training data or using pre-trained image restoration methods such as de-hazing, de-raining, and de-snowing, to improve the quality of input images are two promising solutions. In this work, we propose Multiple Weather Translation GAN (MWTG), a CycleGAN-based, dual-purpose framework that simultaneously learns weather generation and its removal from image data. MWTG consists of four GANs constrained using cycle consistency that carry out domain translation tasks between hazy, rainy, snowy, and clear weather, using an asymmetric approach. To increase network capacity, we employ a spatial feature transform (SFT) layer to fuse the features extracted from the weather layer, which contains high-level domain information from the previous generators. Further, we collect an unpaired, real-world driving dataset recorded under various weather conditions called Realistic Driving Scenes under Bad Weather (RDSBW). We qualitatively and quantitatively evaluate MWTG using the RDSBW and the variation of Cityscapes that synthesize weather effects, eg., FoggyCityscape. Our experimental results suggest that MWTG can generate realistic weather in clear images and also accurately remove noise from weather images. Furthermore, the SOTA pedestrian detector ASCP is shown to achieve an impressive gain in detection precision after image restoration using the proposed MWTG method.

DOI： 10.3390/s23031548

Estimating the effect of hitting strategies in baseball using counterfactual virtual simulation with deep learning Reviewed

Hiroshi Nakahara, Kazuya Takeda, Keisuke Fujii

International Journal of Computer Science in Sport Vol. 22 ( 1 ) page： 1 - 12 2023.1

Perception and sensing for autonomous vehicles under adverse weather conditions: A survey Reviewed

Yuxiao Zhang, Alexander Carballo, Hanting Yang, Kazuya Takeda

ISPRS Journal of Photogrammetry and Remote Sensing Vol. 196 page： 146 - 177 2023.1

RSG-GCN: Predicting Semantic Relationships in Urban Traffic Scene With Map Geometric Prior

Yafu Tian, Alexander Carballo, Ruifeng Li, Kazuya Takeda

IEEE OPEN JOURNAL OF INTELLIGENT TRANSPORTATION SYSTEMS Vol. 4 page： 244 - 260 2023

Language：English Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

Automated identification of the relationships between traffic actors and surrounding objects, in order to describe their behavior and predict their intentions, has become the focus of increasing attention in the field of autonomous driving. Therefore, in this work, we propose a Road Scene Graphs-Graph Convolutional Network (RSG-GCN) as a novel, graph-based model for predicting the topological graph structure of a given traffic scene. The status of the actors and HD map information are integrated as prior knowledge, allowing the edges linking the actor nodes to capture potential semantic relationships, such as "vehicle approaching pedestrian" and "pedestrian waiting at intersection". To train this model, we created our own RSG dataset, as well as a relational dataset and benchmark derived from nuScenes. Our extensive range of experiments demonstrate that our model can more accurately predict semantic relationships and behavior in a given traffic scene than other popular traffic scene prediction models. In particular, regarding the use of HD map prior knowledge, we found that the resulting increase in accuracy significantly outweighs performance loss caused by the increase in graph size. The downstream applications of RSG include traffic scene retrieval and synthetic traffic scene generation, which are briefly described.

DOI： 10.1109/OJITS.2023.3260624

Real-Time Graph-Based Optimization for GNSS-Doppler Integrated RTK-GNSS/IμDR Positioning System in Urban Area

Aoki Takanose, Eijiro Takeuchi, Alexander Carballo, Junichi Meguro, Kazuya Takeda

IEEE Intelligent Vehicles Symposium, Proceedings Vol. 2023-June 2023

Publishing type：Research paper (international conference proceedings)

Autonomous driving of vehicles and robots requires highly accurate position information, and RTK-GNSS is expected to be utilized for this purpose. In this paper, we propose a robust and real-time operation method by introducing graph optimization into the integrated RTK-GNSS/IMU method. The proposed method is an extension of a method using vehicle trajectories that can estimate positions with lane-level accuracy even in urban areas. The position is estimated by removing GNSS multipaths from the shape of a vehicle trajectory of several hundred meters and averaging the remaining GNSS results. This method does not take into account the errors in the vehicle trajectory and cannot fully benefit from the high accuracy positioning solution of RTK-GNSS. To solve this problem, we introduce graph optimization to the base method, which treats the error state as a probabilistic model. However, general graph optimization methods have problems with processing time and outlier elimination. The proposed method solves these problems by restricting the time series data to be optimized and using a two-step optimization structure. Evaluations show that the proposed method is effective because it satisfies the requirements for real-time operation and improves accuracy compared to conventional methods.

DOI： 10.1109/IV55152.2023.10186672

Uncertainty Aware Task Allocation for Human-Automation Cooperative Recognition in Autonomous Driving Systems

Atsushi Kuribayashi, Eijiro Takeuchi, Alexander Carballo, Yoshio Ishiguro, Kazuya Takeda

IEEE Intelligent Vehicles Symposium, Proceedings Vol. 2023-June 2023

Publishing type：Research paper (international conference proceedings)

Cooperative recognition, a method to achieve human-automation cooperation in the recognition phase of the autonomous driving system, has been proposed to address the challenges in the conventional control phase cooperation, e.g., taking over vehicle control. In cooperative recognition, the operator intervenes in recognition tasks that are difficult for the automated system alone to improve driving efficiency and safety. The challenge is the integration of both human and automated systems while both participants have different characteristics, processing capabilities, and uncertainty in the decisions (recognition results). The objectives of this study are task allocation (i.e., when and for which targets the operator should intervene) taking into account the intervention efficiency and human state. And also combine the human intervention and recognition result of the automated systems to solve the uncertainties in both participants. We formulated this problem with a Partially Observable Markov Decision Process (POMDP). The simulator experiment indicated that the recognition result of the automated system and the operator's intervention were stochastically combined. The intervention requests to the operator adapted to the operator state and could be reduced while maintaining driving efficiency and minimizing risk omissions.

DOI： 10.1109/IV55152.2023.10186725

RSG-Search: Semantic Traffic Scene Retrieval Using Graph-Based Scene Representation

Yafu Tian, Alexander Carballo, Ruifeng Li, Kazuya Takeda

IEEE Intelligent Vehicles Symposium, Proceedings Vol. 2023-June 2023

Publishing type：Research paper (international conference proceedings)

Browsing specific traffic scene in large-scale dataset is an increasing demand from researchers, self-driving community and insurance companies. It is easy to search scenes with specific tags such as "rain", "snow", or "on highway". However, searching specific scene configurations, like "two vehicles waiting for a person crossing the road", is still an open problem. In this paper, we provide RSG-search, a scene-graph based traffic scene retrieval method, based on our previous research on traffic scene-graph generation. By previously translating open datasets to scene graphs, we can ignore irrelevant details, and efficiently search specific scene configuration among thousands of traffic scenes. Experiment results shows that our graph searching method is able to retrieve results for a given query with high accuracy. Our method simplifies the task of scene retrieval, opening opportunities for new applications.

DOI： 10.1109/IV55152.2023.10186641

Predictive World Models from Real-World Partial Observations.

Robin Karlsson, Alexander Carballo, Keisuke Fujii 0001, Kento Ohtani, Kazuya Takeda

MOST page： 152 - 166 2023

ViCE: Improving Dense Representation Learning by Superpixelization and Contrasting Cluster Assignment Reviewed

Robin Karlsson, Tomoki Hayashi, Keisuke Fujii, Alexander Carballo, Kento Ohtani, Kazuya Takeda

The 33rd British Machine Vision Conference (BMVC 2022) 2022.11

Methods of Gently Notifying Pedestrians of Approaching Objects when Listening to Music Reviewed

Yuki Sakashita, Yoshio Ishiguro, Kento Ohtani, Kazuya TakedaTakanori Nishino,

UIST '22: The 35th Annual ACM Symposium on User Interface Software and Technology ( 39 ) page： 1 - 4 2022.10

Improvement of anomalous sound detection method considering the distribution of embedding Reviewed

Ibuki Kuroyanagi, Tomoki Hayashi, Kazuya Takeda, Tomoki Toda

the 24th International Congress on Acoustics page： 5 2022.10

ドライブレコーダを活用した旅の要約動画自動生成システム Reviewed

尾頭花奈, 石黒祥生, 椎尾一郎, 武田一哉

コンピュータソフトウェア Vol. 39 ( 4 ) page： 4_144 - 4_157 2022.10

Automatic Fault Detection in Race Walking From a Smartphone Camera via Fine-Tuning Pose Estimation Reviewed

Tomohiro Suzuki, Kazuya Takeda, Keisuke Fujii

2022 IEEE 11th Global Conference on Consumer Electronics 2022.10

Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction Reviewed

Masakiyo Teranishi, Kazushi Tsutsui, Kazuya Takeda, Keisuke Fujii,

9th Workshop on Machine Learning and Data Mining for Sports Analytics 2022 (MLSA'22) co-located with the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery (ECML-PKDD'22) 2022.9

OPTIMIZATION OF AIRCRAFT FLIGHT PATHS CONSIDERING THE CONFLICTING PARAMETERS OF ECONOMY AND SAFETY Reviewed

Tetsuya Nagashima, Ming Ding, Keisuke FUJII & Kazuya TAKEDA

33rd Congress of the International Council of the Aeronautical Sciences 2022.9

Improvement of Serial Approach to Anomalous Sound Detection by Incorporating Two Binary Cross-Entropies for Outlier Exposure Reviewed

Ibuki Kuroyanagi, Tomoki Hayashi, Kazuya Takeda, Tomoki Toda

the 30th European Signal Processing Conference (EUSIPCO) 2022.8

Emergence of Collaborative Hunting via Multi-Agent Deep Reinforcement Learning Reviewed

Kazushi Tsutsui, Kazuya Takeda, Keisuke Fujii

International Workshop on Human Behavior Understanding (HBU'22) in conjunction with International Conference on Pattern Recognition (ICPR'22) 2022.8

Evaluating a third base coach’s decision-making via game theory and machine learning Reviewed

Hiroshi Nakahara, Kazuya Takeda, Keisuke Fujii

MathSport International, 2022 2022.7

Automatic screen-play classification in basketballvia semi-supervised learning Reviewed

Ziyi Zhang, Kazuya Takeda, Keisuke Fujii

Mathsport International 2022 2022.7

GNSS/IMU Performance Improvement Based on Acceleration Error Estimation Using Height Variation Reviewed

Aoki Takanose, Kaito Kondo, Yuta Hoda, Junichi Meguro, Kazuya Takeda

15th International Symposium on Advanced Vehicle Control (AVEC’22) Vol. Mo1B-01 2022.5

SecretSign: A Method of Finding a Specific Vehicle Privately and Quickly using Flashing Lights Reviewed

Yusuke Sakai, Hiromi Morita, Yoshio Ishiguro, Takanori Nishino, Kazuya Takeda

IEEE Intelligent Transportation Systems Magazine Vol. 14 ( 1 ) page： 216 - 227 2022.1

Data-Driven Risk-Sensitive Control for Personalized Lane Change Maneuvers

Naren Bao, Linda Capito, Dongfang Yang, Alexander Carballo, Chiyomi Miyajima, Kazuya Takeda

IEEE ACCESS Vol. 10 page： 36397 - 36415 2022

Language：English Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

Most current research in the field of autonomous vehicle control assumes that all vehicles will follow the same patterns of automated driving behavior, resulting in systems with "conservative" or "average" driving styles. These systems may not be acceptable to drivers who prefer a more aggressive style of driving, however, while extremely cautious drivers may consider the standard outputs to be too aggressive. To address this problem, in this paper, we introduce Risk Sensitive Control (RSC), an inverse optimal control algorithm that estimates risk-sensitive driving features and incorporates them into a receding-horizon controller. RSC uses a meta-learning algorithm to update the parameters of the cost function, continuously improving the controller online as more and more driving data is gathered from the user and subjective risk feedback. An estimator takes into account individual differences in subjective risk analysis, in terms of driving features and surrounding vehicle locations, by adjusting the cost function and its constraints. We test this approach using five lane change scenarios, some safe and some risky, with thirty real drivers in a CARLA simulation environment. Our quantitative and qualitative evaluations demonstrate that the proposed framework is able to generate a user's preferred driving maneuvers during lane changes, i.e., control commands the user associates with lower subjective risk, outperforming conventional, model-based predictive control methods in terms of replicating the user's own driving behavior.

DOI： 10.1109/ACCESS.2022.3163267

Real-to-Synthetic: Generating Simulator Friendly Traffic Scenes from Graph Representation

Yafu Tian, Alexander Carballo, Ruifeng Li, Kazuya Takeda

2022 IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV) page： 1615 - 1622 2022

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

Reproducing real-world traffic scenes in the simulator is fundamental to training self-driving systems. Creating a simulation scenario is a complex task, generally done manually: the ego-vehicle and other entities are placed and their trajectories dened, trying to recreate some situation found in real traffic. To reduce the manual burden, here we propose the Realto-Synthetic toolset. This toolset provides synthetic traffic scene in openDrive format, which can be directly simulated in many simulators such as SUMO or CARLA. Also, we provide a scene generator which generates near-realistic scene from minimum user effort. To maintain the similarity between real-world scene and generated one, here we introduce the concept "Road Scene Graph"(RSG). In this graph, nodes represent entities while edges stand for pairwise relationships. These relationships could be maintained in the scene generation process while the actor is generated according to the distribution sampled from realworld data. Experiments proved that by using "Road Scene Graph", our scene generator proposes a much more convenient way to configure traffic scenes rather than manually defining every actor's initial status and trajectories.

DOI： 10.1109/IV51971.2022.9827441

Deep Reinforcement Learning in a Racket Sport for Player Evaluation With Technical and Tactical Contexts.

An enhanced driver's risk perception modeling based on gate recurrent unit network

Ping Peng, Ding Weiping, Liu Yongkang, Takeda Kazuya

2022 IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV) page： 1234 - 1240 2022

Auditory and visual warning information generation of the risk object in driving scenes based on weakly supervised learning

Niu Yinjie, Ding Ming, Zhang Yuxiao, Ohtani Kento, Takeda Kazuya

2022 IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV) page： 1572 - 1577 2022

Ning Ding, Kazuya Takeda, Keisuke Fujii 0001

IEEE Access Vol. 10 page： 54764 - 54772 2022

Deepware: An Open-Source Toolkit for Developing and Evaluating Learning-Based and Model-Based Autonomous Driving Models

Shunya Seiya, Alexander Carballo, Eijiro Takeuchi, Kazuya Takeda

IEEE ACCESS Vol. 10 page： 105734 - 105743 2022

Language：English Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

In recent decades, many learning-based autonomous driving systems have been proposed, and researchers have also created toolkits for developing these systems. These toolkits allow developers to train their models easily, and then test them using simulators. Existing toolkits for learning-based autonomous driving systems have some limitations however, which include inability to reuse modules or to perform accurate comparisons with model-based systems, as well as a lack of support for middle-to-middle models. As a solution, in this paper we introduce Deepware, an end-to-end toolkit for developing and evaluating learning-based autonomous driving models. Deepware includes the tools needed for collecting and evaluating datasets, training models, and evaluating models on simulators or in real-world environments using actual vehicles. Unlike existing toolkits, we used ROS as our platform, which is a set of software frameworks for robot software development widely used in autonomous driving systems as middleware, which allows cooperation with model-based systems. This approach also allows system modules to be shared when building models. In addition, it allows the comparison of learning-based and model-based methods under the same conditions. Moreover, by extracting features from model-based systems, our toolkit can also support middle-to-middle models. The proposed Deepware toolkit and dataset are available at: https://github.com/shunchan0677/deepware.

DOI： 10.1109/ACCESS.2022.3212152

Disentangled Bad Weather Removal GAN for Pedestrian Detection

Yang Hanting, Carballo Alexander, Takeda Kazuya

2022 IEEE 95TH VEHICULAR TECHNOLOGY CONFERENCE (VTC2022-SPRING) 2022

Driving Risk and Intervention: Subjective Risk Lane Change Dataset

Naren Bao, Alexander Carballo, Kazuya Takeda

2022 IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV) page： 556 - 562 2022

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

When developing truly driverless mobility for the future, one key index used to measure the matureness of a particular self-driving technology is the driver intervention rate. One method which has proven to be effective for decreasing intervention rates is the use of personalized driving models that can mimic the driving style and preferences of a targeted user, so that autonomous driving feels safer and more natural to them. To create such models, quantitative data should be collected from users in order to determine the style of driving that a particular user, or type of user, prefers. In this paper, we introduce the Subjective Risk Lane Change (SRLC) Dataset, which includes ego vehicle driving behavior data, surrounding vehicle location information, and the subjective risk scores of users, collected during both safe and risky lane change scenarios encountered in CARLA simulators, as well as demographic information for our 30 participants. Furthermore, user intervention data for all of our participants was collected from Personalized Model Predictive Controllers during the generated lane change maneuvers. As far as the authors are able to determine, no other public dataset provides driving behavior signal and intervention timing information collected during driver interventions. Our dataset can be used to gain insights into a variety of personal driving styles, allowing the improvement of adaptive autonomous driving systems, and leading to safer and more widely accepted driverless technology.

DOI： 10.1109/IV51971.2022.9827358

Estimating counterfactual treatment outcomes over time in multi-vehicle simulation. Reviewed

Occlusion-Aware Motion Planning With Visibility Maximization via Active Lateral Position Adjustment

Narksri Patiphon, Darweesh Hatem, Takeuchi Eijiro, Ninomiya Yoshiki, Takeda Kazuya

IEEE ACCESS Vol. 10 page： 57759 - 57782 2022

Keisuke Fujii 0001, Koh Takeuchi, Atsushi Kuribayashi, Naoya Takeishi, Yoshinobu Kawahara, Kazuya Takeda

SIGSPATIAL/GIS ( 7 ) page： 7 - 4 2022

Cooperative play classification in team sports via semi-supervised learning. Reviewed

Zhang Ziyi, Kazuya Takeda, Keisuke Fujii

International Journal of Computer Science in Sport Vol. 21 ( 1 ) page： 111 - 121 2022

Improving Dense Representation Learning by Superpixelization and Contrasting Cluster Assignment.

Robin Karlsson, Tomoki Hayashi, Keisuke Fujii 0001, Alexander Carballo, Kento Ohtani, Kazuya Takeda

BMVC page： 699 - 699 2022

Autonomous Driving in Adverse Weather Conditions: A Survey Reviewed

Y. Zhang, A. Carballo, H. Yang, K. Takeda

arXiv preprint arXiv:2112.08936 2021.12

ViCE: Self-Supervised Visual Concept Embeddings as Contextual and Pixel Appearance Invariant Semantic Representations Reviewed

Karlsson, Robin; Hayashi, Tomoki; Fujii, Keisuke; Carballo, Alexander; Ohtani, Kento; Takeda, Kazuya;

arXiv preprint arXiv:2111.12460 2021.11

AN ENSEMBLE APPROACH TO ANOMALOUS SOUND DETECTION BASED ON CONFORMER-BASED AUTOENCODER AND BINARY CLASSIFIER INCORPORATED WITH METRIC LEARNING

Ibuki Kuroyanagi , Tomoki Hayashi , Yusuke Adachi , Takenori Yoshimura , Kazuya Takeda , Tomoki Toda

Proceedings of the Detection and Classification of Acoustic Scenes and Events 2021 Workshop (DCASE2021) page： 110 - 114 2021.11

End-to-End Learning-based Driving System with Branches by Emphasizing Target Direction Reviewed

Seiya, Shunya; Ohtani, Kento; Carballo, Alexander; Takeuchi, Eijiro; Takeda, Kazuya;

Vol. 52 ( 6 ) 2021.11

FollowSelect: Path-based Menu Interaction for Intuitive Navigation Reviewed

Vol. 62 ( 10 ) page： 1669 - 1680 2021.10

Flexible prediction of opponent motion with internal representation in interception behavior Reviewed

Kazushi Tsutsui, Keisuke Fujii, Kazutoshi Kudo, Kazuya Takeda

Biological Cybernetics Vol. 115 ( 5 ) page： 473 - 485 2021.10

Authorship：Last author Language：Japanese Publishing type：Research paper (scientific journal) Publisher：Springer Science and Business Media LLC

<title>Abstract</title>Skilled interception behavior often relies on accurate predictions of external objects because of a large delay in our sensorimotor systems. To deal with the sensorimotor delay, the brain predicts future states of the target based on the current state available, but it is still debated whether internal representations acquired from prior experience are used as well. Here we estimated the predictive manner by analyzing the response behavior of a pursuer to a sudden directional change of the evasive target, providing strong evidence that prediction of target motion by the pursuer was incompatible with a linear extrapolation based solely on the current state of the target. Moreover, using neural network models, we validated that nonlinear extrapolation as estimated was computationally feasible and useful even against unknown opponents. These results support the use of internal representations in predicting target motion, suggesting the usefulness and versatility of predicting external object motion through internal representations.

DOI： 10.1007/s00422-021-00891-9

Anomalous Sound Detection Using a Binary Classification Model and Class Centroids Reviewed

Other Link： https://link.springer.com/article/10.1007/s00422-021-00891-9/fulltext.html

Supervised sequential pattern mining of event sequences in sport to identify important patterns of play: an application to rugby union Reviewed

Rory Bunker, Keisuke Fujii, Hiroyuki Hanada, Ichiro Takeuchi

PLOS One 2021.9

Ibuki Kuroyanagi, Tomoki Hayashi, Kazuya Takeda , Tomoki Toda

European Signal Processing Conference2021(EUSIPCO2021) 2021.8

Prediction of Personalized Driving Behaviors via Driver-Adaptive Deep Generative Models Reviewed

Naren Bao, Alexander Carballo, Takeda Kazuya

2021 IEEE Intelligent Vehicles Symposium (IV) page： 616 - 621 2021.7

ANOMALOUS SOUND DETECTION WITH ENSEMBLE OF AUTOENCODER AND BINARY CLASSIFICATION APPROACHES

Ibuki Kuroyanagi , Tomoki Hayashi , Yusuke Adachi, , Takenori Yoshimura , Kazuya Takeda , Tomoki Toda

Proceedings of the Detection and Classification of Acoustic Scenes and Events 2021 Workshop (DCASE2021) 2021.7

Leveraging state-of-the-art ASR techniques to audio captioning Reviewed

Narisetty, Chaitanya; Hayashi, Tomoki; Ishizaki, Ryunosuke; Watanabe, Shinji; Takeda, Kazuya;

Proceedings of the Detection and Classification of Acoustic Scenes and Events 2021 Workshop (DCASE2021) page： 160 - 164 2021.7

Deadlock-free planner for occluded intersections using estimated visibility of hidden vehicles Reviewed

Patiphon Narksri, Eijiro Takeuchi, Yoshiki Ninomiya, Kazuya Takeda

Electronics (Switzerland) Vol. 10 ( 4 ) page： 1 - 28 2021.2

Language：Japanese Publishing type：Research paper (scientific journal)

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. A common approach used for planning blind intersection crossings is to assume that hypothetical vehicles are approaching the intersection at a constant speed from the occluded areas. Such an assumption can result in a deadlock problem, causing the ego vehicle to remain stopped at an intersection indefinitely due to insufficient visibility. To solve this problem and facilitate safe, deadlock-free intersection crossing, we propose a blind intersection planner that utilizes both the ego vehicle and the approaching vehicle’s visibility. The planner uses a particle filter and our proposed visibility-dependent behavior model of approaching vehicles for predicting hidden vehicles. The behavior model is designed based on an analysis of actual driving data from multiple drivers crossing blind intersections. The proposed planner was tested in a simulation and found to be effective for allowing deadlock-free crossings at intersections where a baseline planner became stuck in a deadlock. The effects of perception accuracy and sensor position on output motion were also investigated. It was found that the proposed planner delayed crossing motion when the perception was imperfect. Furthermore, our results showed that the planner decelerated less while crossing the intersection with the front-mounted sensor configuration compared to the roof-mounted configuration due to the improved visibility. The minimum speed difference between the two sensor configurations was 1.82 m/s at an intersection with relatively poor visibility and 1.50 m/s at an intersection with good visibility.

DOI： 10.3390/electronics10040411

Motion Analysis and Performance Improved Method for 3D LiDAR Sensor Data Compression Reviewed

Chenxi Tu, Eijiro Takeuchi, Alexander Carballo, Chiyomi Miyajima, Kazuya Takeda

IEEE Transactions on Intelligent Transportation Systems Vol. 22 ( 1 ) page： 243 - 256 2021.1

Language：Japanese Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

© 2000-2011 IEEE. Continuous point cloud data is being used more and more widely in practical applications such as mapping, localization and object detection in autonomous driving systems, but due to the huge volume of data involved, sharing and storing this data is currently expensive and difficult. One possible solution is the development of more efficient methods of compressing the data. Other researchers have proposed converting 3D point cloud data into 2D images, or using tree structures to store the data. In a previous study targeting streaming point cloud data, we proposed an MPEG-like compression method which utilizes simultaneous localization and mapping (SLAM) results to simulate LiDAR's operating process. In this paper, instead of imitating MPEG, we propose new strategy for more efficient reference frame distribution and more natural frame prediction, and use a different algorithm to encode the residual, greatly improving the algorithm's performance and its stability in different scenarios. We also discuss how various parameters affect compression performance. Using our proposed method, streaming point cloud data collected by LiDAR sensors can be compressed to 1/50th of its original size, with only 2 cm of Root Mean Square Error for each detected point. We evaluate our proposed method by comparing its performance with several other existing point cloud compression methods in three different driving scenarios, demonstrating that our proposed method outperforms them.

DOI： 10.1109/TITS.2019.2956066

A Comparison of Methods for Sharing Recognition Information and Interventions to Assist Recognition in Autonomous Driving System Reviewed

How to monitor multiple autonomous vehicles remotely with few observers: An active management method Reviewed

Ming Ding, Eijiro Takeuchi, Yoshio Ishiguro, Yoshiki Ninomiya, Nobuo Kawaguchi and Kazuya Takeda

2021 IEEE Intelligent Vehicles Symposium (IV) page： 1168 - 1173 2021

RSG-Net: Towards Rich Semantic Relationship Prediction for Intelligent Vehicle in Complex Environment Reviewed

Tian, Yafu; Carballo, Alexander; Li, Ruifeng; Takeda, Kazuya;

2021 IEEE Intelligent Vehicles Symposium (IV) page： 546 - 552 2021

Atsushi Kuribayashi, Eijiro Takeuchi, Alexander Carballo, Yoshio Ishiguro, Kazuya Takeda

2021 IEEE Intelligent Vehicles Symposium (IV) page： 622 - 629 2021

Automatic Generation of Road Trip Summary Video for Reminiscence and Entertainment using Dashcam Video Reviewed

Kana Bito, Itiro Siio, Yoshio Ishiguro, Kazuya Takeda

13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications page： 181 - 190 2021

Learning a Model for Inferring a Spatial Road Lane Network Graph using Self-Supervision Reviewed

Robin Karlsson, David Robert Wong, Simon Thompson, Kazuya Takeda

The 24th IEEE International Conference on Intelligent Transportation Systems(ITSC2021) page： 812 - 819 2021

A recognition phase Intervention Interface to Improve Naturalness of Autonomous Driving for Distracted Drivers Reviewed

Atsushi Kuribayashi, Eijiro Takeuchi, Alexander Carballo, Yoshio Ishiguro, Kazuya Takeda

2021 IEEE International Intelligent Transportation Systems Conference (ITSC) page： 1737 - 1744 2021

Visibility Estimation in Complex, Real-World Driving Environments Using High Definition Maps Reviewed

Patiphon Narksri, Hatem Darweesh, Eijiro Takeuchi, Yoshiki Ninomiya, Kazuya Takeda

The 24th IEEE International Conference on Intelligent Transportation Systems(ITSC2021) page： 2847 - 2854 2021

OpenPlanner 2.0: The Portable Open Source Planner for Autonomous Driving Applications Reviewed

Hatem Darweesh , Eijiro Takeuchi , and Kazuya Takeda

IEEE Intelligent Vehicles Symposium Workshops (IV Workshops) page： 313 - 318 2021

Characterization of Multiple 3D LiDARs for Localization and Mapping Performance using the NDT Algorithm Reviewed

Alexander Carballo, Abraham Monrroy, David Wong, Patiphon Narksri, Jacob Lambert, Yuki Kitsukawa, Eijiro Takeuchi, Shinpei Kato, Kazuya Takeda

2021 IEEE INTELLIGENT VEHICLES SYMPOSIUM WORKSHOPS (IV WORKSHOPS) page： 327 - 334 2021

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

In this work, we present a detailed comparison of ten different 3D LiDAR sensors for the tasks of mapping and vehicle localization, using as common reference the Normal Distributions Transform (NDT) algorithm implemented in the self-driving open source platform Autoware. LiDAR data used in this study is a subset of our LiDAR Benchmarking and Reference (LIBRE) dataset, captured independently from each sensor, from a vehicle driven on public urban roads multiple times, at different times of the day. In this study, we analyze the performance and characteristics of each LiDAR for the tasks of (1) 3D mapping including an assessment map quality based on mean map entropy, and (2) 6-DOF localization using a ground truth reference map.

DOI： 10.1109/IVWorkshops54471.2021.9669244

Cross-Lingual Voice Conversion using a Cyclic Variational Auto-encoder and a WaveNet Vocoder Reviewed

Eagleye: A Lane-Level Localization Using Low-Cost GNSS/IMU Reviewed

Aoki Takanose, Yuki Kitsukawa, Junichi Megruo, Eijiro Takeuchi, Alexander Carballo, Kazuya Takeda

2021 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops) page： 319 - 326 2021

Anomalous Sound Detection Using a Binary Classification Model and Class Centroids

Kuroyanagi Ibuki, Hayashi Tomoki, Takeda Kazuya, Toda Tomoki

29TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO 2021) page： 1995 - 1999 2021

A Method for Location Initialization of Handheld Devices using Autonomous Driving Vehicles for Interactive Systems

Ishiguro Yoshio, Takeda Kazuya

AUTOMOTIVEUI '21 ADJUNCT PROCEEDINGS: 13TH INTERNATIONAL ACM CONFERENCE ON AUTOMOTIVE USER INTERFACES AND INTERACTIVE VEHICULAR APPLICATIONS page： 102 - 106 2021

Hikaru Nakatani, Patrick Lumban Tobing, Kazuya Takeda, Tomoki Toda

2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2020 - Proceedings page： 520 - 526 2020.12

Publishing type：Research paper (international conference proceedings)

© 2020 APSIPA. We propose a novel, cross-lingual voice conversion (VC) method using a cyclic variational auto-encoder (CycleVAE). Voice conversion is the transformation of the voice of one speaker into the voice of another speaker, while cross-lingual VC performs voice conversion between speakers who speak different languages. When using VC methods based on parallel learning, it is necessary to prepare accented speech uttered by the source or target speaker, using the pronunciation system of the speaker's mother tongue. On the other hand, VC methods which use a non-parallel learning approach can utilize the natural speech data of both the source and target speakers, produced in their own native languages. It then becomes necessary, however, to deal with the issues of time-alignment and language mismatches. To address these issues, we apply CycleVAE to cross-lingual VC as a sophisticated, non-parallel method of VC. We also apply the WaveNet vocoder in the waveform generation process of CycleVAE-VC to improve overall conversion quality. Our objective and subjective experimental results when performing cross-lingual VC from a native English speaker to a native Japanese speaker confirm that the proposed method achieves a higher level of naturalness and speaker similarity than a conventional RNN-based parallel VC method using accented speech.

Generation of Origami Folding Animations from 3D Point Cloud Using Latent Space Interpolation Reviewed

Other Link： https://dblp.uni-trier.de/conf/apsipa/2020

Chiaki Nakagaito, Takanori Nishino, Kazuya Takeda

SIGGRAPH Asia 2020 Posters. SA 2020 page： 30 - 2 2020.12

Road Scene Graph: A Semantic Graph-Based Scene Representation Dataset for Intelligent Vehicles Reviewed

Yafu Tian, Alexander Carballo, Ruifeng Li, Kazuya Takeda

CoRR Vol. abs/2011.13588 2020.11

Publishing type：Research paper (scientific journal)

Rich semantic information extraction plays a vital role on next-generation
intelligent vehicles. Currently there is great amount of research focusing on
fundamental applications such as 6D pose detection, road scene semantic
segmentation, etc. And this provides us a great opportunity to think about how
shall these data be organized and exploited.
In this paper we propose road scene graph,a special scene-graph for
intelligent vehicles. Different to classical data representation, this graph
provides not only object proposals but also their pair-wise relationships. By
organizing them in a topological graph, these data are explainable,
fully-connected, and could be easily processed by GCNs (Graph Convolutional
Networks). Here we apply scene graph on roads using our Road Scene Graph
dataset, including the basic graph prediction model. This work also includes
experimental evaluations using the proposed model.

Trajectory prediction with imitation learning reflecting defensive evaluation in team sports Reviewed

Other Link： http://arxiv.org/pdf/2011.13588v1

CONFORMER-BASED SOUND EVENT DETECTION WITH SEMI-SUPERVISED LEARNING AND DATA AUGMENTATION Reviewed

Koichi Miyazaki, Tatsuya Komatsu, Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, Kazuya Takeda

Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE Workshop) page： 100 - 104 2020.11

Point Grid Map-Based Mid-To-Mid Driving without Object Detection Reviewed

Shunya Seiya, Alexander Carballo, Eijiro Takeuchi, and Kazuya Takeda

2020 IEEE Intelligent Vehicles Symposium（IV） 2020.10

Masakiyo Teranishi, Keisuke Fujii, Kazuya Takeda

2020 IEEE 9th Global Conference on Consumer Electronics, GCCE 2020 page： 124 - 125 2020.10

Publishing type：Research paper (international conference proceedings)

© 2020 IEEE. In team sports, qualitative video analysis is mainly used for decision making of team tactics because of the complex interaction. Among the quantitative analyses, most of studies in team trajectory generation have been evaluated only on prediction errors and did not take tactical evaluations (e.g., good defense) into consideration. In this paper, we propose a trajectory prediction method incorporating the defensive evaluation (i.e., how well they protect the goal) into multi-agent imitation learning model. Although the proposed method had the similar prediction performance to the existing method, our method generated an improved trajectory in terms of defensive evaluation.

DOI： 10.1109/GCCE50665.2020.9291841

Extracting Human-Like Driving Behaviors from Expert Driver Data Using Deep Learning Reviewed

Other Link： https://dblp.uni-trier.de/db/conf/gcce/gcce2020.html#TeranishiFT20

Kyle Sama, Yoichi Morales, Hailong Liu, Naoki Akai, Alexander Carballo, Eijiro Takeuchi, Kazuya Takeda

IEEE Transactions on Vehicular Technology Vol. 69 ( 9 ) page： 9315 - 9329 2020.9

Language：Japanese Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

© 1967-2012 IEEE. This paper introduces a method to extract driving behaviors from a human expert driver which are applied to an autonomous agent to reproduce proactive driving behaviors. Deep learning techniques were used to extract latent features from the collected data. Extracted features were clustered into behaviors and used to create velocity profiles allowing an autonomous driving agent could drive in a human-like manner. By using proactive driving behaviors, the agent could limit potential sources of discomfort such as jerk and uncomfortable velocities. Additionally, we proposed a method to compare trajectories where not only the geometric similarity is considered, but also velocity, acceleration and jerk. Experimental results in a simulator implemented in ROS show that the autonomous agent built with the driving behaviors was capable of driving similarly to expert human drivers.

DOI： 10.1109/TVT.2020.2980197

Policy learning with partial observation and mechanical constraints for multi-person modeling Reviewed

Keisuke Fujii, Naoya Takeishi, Yoshinobu Kawahara, Kazuya Takeda

CoRR Vol. abs/2007.03155 2020.7

Publishing type：Research paper (scientific journal)

Extracting the rules of real-world biological multi-agent behaviors is a
current challenge in various scientific and engineering fields. Biological
agents generally have limited observation and mechanical constraints; however,
most of the conventional data-driven models ignore such assumptions, resulting
in lack of biological plausibility and model interpretability for behavioral
analyses in biological and cognitive science. Here we propose sequential
generative models with partial observation and mechanical constraints, which
can visualize whose information the agents utilize and can generate
biologically plausible actions. We formulate this as a decentralized
multi-agent imitation learning problem, leveraging binary partial observation
models with a Gumbel-Softmax reparameterization and policy models based on
hierarchical variational recurrent neural networks with physical and
biomechanical constraints. We investigate the empirical performances using
real-world multi-person motion datasets from basketball and soccer games.

Other Link： http://arxiv.org/pdf/2007.03155v1

Personalized Subjective Driving Risk: Analysis and Prediction Reviewed

Bao Naren, Carballo Alexander, Miyajima Chiyomi, Takeuchi Eijiro, Takeda Kazuya

JOURNAL OF ROBOTICS AND MECHATRONICS Vol. 32 ( 3 ) page： 503-519 2020.6

Convolution augmented transformer for semi-supervised sound event detection Reviewed

K Miyazaki, T Komatsu, T Hayashi, S Watanabe, T Toda, K Takeda

Proc. Workshop Detection Classification Acoust. Scenes Events (DCASE) page： 100 - 104 2020.6

Weakly-Supervised Sound Event Detection with Self-Attention Reviewed

Koichi Miyazaki, Tatsuya Komatsu, Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, Kazuya Takeda

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings Vol. 2020-May page： 66 - 70 2020.5

Publishing type：Research paper (international conference proceedings)

© 2020 IEEE. In this paper, we propose a novel sound event detection (SED) method that incorporates a self-attention mechanism of the Transformer for a weakly-supervised learning scenario. The proposed method utilizes the Transformer encoder, which consists of multiple self-attention modules, allowing to take both local and global context information of the input feature sequence into account. Furthermore, inspired by the great success of BERT in the natural language processing field, the proposed method introduces a special tag token into the input sequence for weak label prediction, which enables the aggregation of the whole sequence information. To demonstrate the performance of the proposed method, we conduct the experimental evaluation using the DCASE2019 Task4 dataset. The experimental results demonstrate that the proposed method outperforms the DCASE2019 Task4 baseline method, which is based on the convolutional recurrent neural network, and the self-attention mechanism effectively works for SED.

DOI： 10.1109/ICASSP40776.2020.9053609

Other Link： https://dblp.uni-trier.de/db/conf/icassp/icassp2020.html#MiyazakiKH0TT20

Characterization of Multiple 3D LiDARs for Localization and Mapping using Normal Distributions Transform Reviewed

Alexander Carballo, Abraham Monrroy, David Wong, Patiphon Narksri, Jacob Lambert, Yuki Kitsukawa, Eijiro Takeuchi, Shinpei Kato, Kazuya Takeda

arXiv preprint arXiv:2004.01374 page： - 2020.4

LIBRE: The multiple 3d lidar dataset Reviewed

Alexander Carballo, Jacob Lambert, Abraham Monrroy, David Wong, Patiphon Narksri, Yuki Kitsukawa, Eijiro Takeuchi, Shinpei Kato, Kazuya Takeda

arXiv preprint arXiv:2003.06129 page： - 2020.3

A Survey of Autonomous Driving: Common Practices and Emerging Technologies Reviewed

Ekim Yurtsever, Jacob Lambert, Alexander Carballo, Kazuya Takeda

IEEE Access Vol. 8 page： 58443 - 58469 2020

Language：Japanese Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

© 2013 IEEE. Automated driving systems (ADSs) promise a safe, comfortable and efficient driving experience. However, fatalities involving vehicles equipped with ADSs are on the rise. The full potential of ADSs cannot be realized unless the robustness of state-of-the-art is improved further. This paper discusses unsolved problems and surveys the technical aspect of automated driving. Studies regarding present challenges, high-level system architectures, emerging methodologies and core functions including localization, mapping, perception, planning, and human machine interfaces, were thoroughly reviewed. Furthermore, many state-of-the-art algorithms were implemented and compared on our own platform in a real-world driving setting. The paper concludes with an overview of available datasets and tools for ADS development.

DOI： 10.1109/ACCESS.2020.2983149

Other Link： http://arxiv.org/pdf/1906.05113v3

Performance Analysis of 10 Models of 3D LiDARs for Automated Driving Reviewed

Jacob Lambert, Alexander Carballo, Abraham Monrroy Cano, Patiphon Narksri, David Wong, Eijiro Takeuchi, Kazuya Takeda

IEEE Access Vol. 8 page： 131699 - 131722 2020

Language：Japanese Publishing type：Research paper (scientific journal)

© 2013 IEEE. Automated vehicle technology has recently become reliant on 3D LiDAR sensing for perception tasks such as mapping, localization and object detection. This has led to a rapid growth in the LiDAR manufacturing industry with several competing makers releasing new sensors regularly. With this increased variety of LiDARs, each with different properties such as number of laser emitters, resolution, field-of-view, and price tags, a more in-depth comparison of their characteristics and performance is required. This work compares 10 commonly used 3D LiDARs, establishing several metrics to assess their performance. Various outstanding issues with specific LiDARs were qualitatively identified. The accuracy and precision of individual LiDAR beams and accumulated point clouds are evaluated in a controlled environment at distances from 5 to 180 meters. Reflective targets were used to characterize intensity patterns and quantify the impact of surface reflectivity on accuracy and precision. A vehicle and pedestrian mannequin were also used as additional targets of interest. A thorough assessment of these LiDARs is given with their potential applicability for automated driving tasks. The data collected in these experiments and analysis tools are all shared openly.

DOI： 10.1109/ACCESS.2020.3009680

END-TO-END AUTOMATIC SPEECH RECOGNITION INTEGRATED WITH CTC-BASED VOICE ACTIVITY DETECTION Reviewed

Yoshimura Takenori, Hayashi Tomoki, Takeda Kazuya, Watanabe Shinji

2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING page： 6999 - 7003 2020

Point Grid Map-Based Mid-To-Mid Driving without Object Detection Reviewed

Shunya Seiya, Alexander Carballo, Eijiro Takeuchi, Kazuya Takeda

IEEE Intelligent Vehicles Symposium, Proceedings page： 2044 - 2051 2020

Publishing type：Research paper (international conference proceedings)

© 2020 IEEE. Teaching autonomous vehicles to imitate human driving in complex, urban traffic scenarios is a difficult task. 'End-to-end' autonomous driving systems, based on 'imitation learning', are an expecting approach. A model learns the relationships between sensing input and vehicle control signal outputs. These methods can successfully achieve driving in simple scenarios such as lane keeping. In contrast, the 'mid-to-mid' autonomous driving methods now being proposed. In such framework, the model learns the relationships between pre-processed feature maps from the model-based system as input and the future position of the ego vehicle as the output. Mid-to-mid driving methods can direct vehicles more robustly than end-to-end driving methods in some complex driving environments. However, mid-to-mid driving methods use the results of the object detection module to create the feature map. If object detection fails, or detection performance is poor due to changes in the driving environment, prediction performance may also be degraded. Our proposed method uses a prediction module that outputs point grid maps directly, without the use of an object detection module, which are then incorporated into the feature map. Point grid maps represent the locations of surrounding vehicles and obstacles directly, based on LiDAR point cloud data. Since the results of object detection are not used by the prediction module, detection performance does not affect prediction performance. In this study we conduct two experiments, an off-line evaluation using a Lyft dataset, and an on-line evaluation using the CARLA simulator. The results show that our model can achieve the same level of ego-vehicle position prediction performance as a model using annotated object location information.

DOI： 10.1109/IV47402.2020.9304809

Other Link： https://dblp.uni-trier.de/db/conf/ivs/ivs2020.html#SeiyaCTT20

Nagoya University at TRECVID 2014: The instance search task Reviewed

Cai Zhi Zhu, Yinqiang Zheng, Ichiro Ide, Shinichi Satoh, Kazuya Takeda

2014 TREC Video Retrieval Evaluation, TRECVID 2014 2020

Publishing type：Research paper (international conference proceedings)

© 2020 2014 TREC Video Retrieval Evaluation. All rights reserved. This paper presents our recent progress on a video object retrieval system that participated in the Instance Search (INS) task of the TRECVID 2014. Basically the system is a further extension of our previous Bag-of-Words (BOW) framework, with emphasis on pursuing a practical spatial re-ranking method scalable to large video database this year. We take the asymmetrical dissimilarities based system, which performed best in the INS2013 task, as the baseline, and re-rank with an improved spatial verification method. Experiments carried out the TRECVID INS2013 and INS2014 consistently show that, our re-ranking algorithm is able to further improve the baseline system at a rather fast speed.

Intervention Force-based Imitation Learning for Autonomous Navigation in Dynamic Environments. Reviewed

Tomoya Yokoyama, Shunya Seiya, Eijiro Takeuchi, Kazuya Takeda

page： 1679 - 1688 2020

Intelligibility enhancement based on speech waveform modification using hearing impairment Reviewed

Shu Hikosaka, Shogo Seki, Tomoki Hayashi, Kazuhiro Kobayashi, Kazuya Takeda, Hideki Banno, Tomoki Toda

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH Vol. 2020-October page： 4059 - 4063 2020

Publishing type：Research paper (international conference proceedings)

Copyright © 2020 ISCA In this paper, we propose a speech waveform modification method which incorporates a hearing impairment simulator, to improve speech intelligibility for the hearing-impaired. The settings of hearing aid devices usually need to be manually adjusted to suit the needs of each user, which creates a significant burden. To address this issue, the proposed method creates a spectral shaping filter, using a hearing impairment simulator capable of estimating speech signals as perceived by a specific hearing-impaired person. We conduct objective and subjective evaluations through simulations using the hearing impairment simulator. Our experimental results demonstrate that; 1) the proposed spectral shaping filter can significantly improve both speech intelligibility and quality, 2) the filter can be combined with a well-known speech intelligibility enhancement technique based on power compensation using dynamic range compression (DRC), and 3) speech intelligibility can be further improved by controlling the trade-off between filtering and DRC-based power compensation.

DOI： 10.21437/Interspeech.2020-2062

ITS+DM Hackathon (ITSC 2017): Lane Departure Prediction with Naturalistic Driving Data Reviewed

Other Link： https://dblp.uni-trier.de/db/conf/interspeech/interspeech2020.html#HikosakaSHKTBT20

Attention-Based Speech Recognition Using Gaze Information

Osamu Segawa, Tomoki Hayashi, Kazuya Takeda

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) page： 465-470 2019.12

Andrey Alekseenko, Hien Q. Dang, Gaurav Bansal, Javier Sanchez-Medina, Chiyomi Miyajima, Takatsugu Hirayama, Kazuya Takeda, Ichiro Ide

IEEE Intelligent Transportation Systems Magazine Vol. 11 ( 4 ) page： 78 - 93 2019.12

Language：English Publishing type：Research paper (scientific journal)

© 2009-2012 IEEE. On October 16th, 2017, in Yokohama, Japan, from 8:00 to 18:00, the first Intelligent Transportation Systems plus Data Mining challenge was organized under the umbrella of the 2017 IEEE Intelligent Transportation Systems Conference, the flagship conference of the IEEE Intelligent Transportation Systems Society. This activity was organized thanks to a three way collaboration between the ITS Society, Nagoya University, and the IEEE ITSC 2017 organizers. The twenty-three contestants, coming from eleven different countries, faced a classic Naturalistic Driving problem: Lane Departure detection. This paper presents the three best solutions produced. The solutions submitted by most of the participants were very diverse and interesting, but overall, the top ones concurred in the use of ensemble learning after a very interesting feature engineering phase. This hackathon formulation was complex in several ways. It was complex in terms of class imbalance, the challenge time duration and the fact that the provided dataset included only numerical measurements coming from the inertial unit in the testing car. That restriction made it difficult to expect outstanding results ? the best one was only slightly over 3% above baseline. However, the organizers thought that such complexities pushed participants to show their repertoire as data scientists, taking into consideration for example computer power load of the different algorithms tested, and overall yielding more interesting approaches to share with the community. Additionally, the most interesting learned lessons were shared, from both an organizational and technical point of view.

DOI： 10.1109/MITS.2018.2880264

An Empirical Study of Adaptive Training of Daily Activity Classifier and Chat Application Designed for the Elderly to Go Out Reviewed

Motion Analysis and Performance Improved Method for 3D LiDAR Sensor Data Compression

Chenxi Tu,Eijiro Takeuchi,Alexander Carballo,Chiyomi Miyajima,Kazuya Takeda

IEEE transction on ITS page： - 2019.11

Underdetermined Source Separation Based on Generalized Multichannel Variational Autoencoder Reviewed

Shogo Seki, Hirokazu Kameoka, Li Li, Tomoki Toda, and Kazuya Takeda

IEEE access page： - 2019.11

Estimating the Probabilities of Surrounding Vehicles' Intentions and Trajectories using a Behavior Planner

Hatem Darweesh, Eijiro Takeuchi, Kazuya Takeda

International journal of automotive engineering Vol. 10 ( 4 ) page： 299-308 2019.11

ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit Reviewed

Tomoki Hayashi, Ryuichi Yamamoto, Katsuki Inoue, Takenori Yoshimura, Shinji Watanabe, Tomoki Toda, Kazuya Takeda, Yu Zhang, Xu Tan

arXiv preprint arXiv:1910.10909 page： 00 2019.10

Effects on user perception of a'modified'speed experience through in-vehicle virtual reality Reviewed

Yusuke Sakai, Toshimitsu Watanabe, Yoshio Ishiguro, Takanori Nishino, Kazuya Takeda

Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings page： 166-170 2019.9

LeadingDisplay: a versatile, robotic display for infotainment in autonomous vehicles Reviewed

Yoshio Ishiguro, Kazuya Takeda

Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings page： 405-409 2019.9

Improving target selection accuracy for vehicle touch screens Reviewed

Kosuke Ito, Kento Ohtani, Yoshio Ishiguro, Takanori Nishino, Kazuya Takeda

Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings page： 176-180 2019.9

Pre-Trained Text Embeddings for Enhanced Text-to-Speech Synthesis Reviewed

Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, Kazuya Takeda, Shubham Toshniwal, Karen Livescu

Proc. Interspeech 2019 page： 4430-4434 2019.9

Robustness of Statistical Voice Conversion Based on Direct Waveform Modification Against Background Sounds Reviewed

Yusuke Kurita, Kazuhiro Kobayashi, Kazuya Takeda, Tomoki Toda

Proc. Interspeech 2019 page： 684-688 2019.9

Effects on the Perception of Speed and Normality When Virtual Reality Scenes Reviewed

Yusuke Sakai, Toshimitsu Watanabe, Yoshio Ishiguro, Takanori Nishino, Kazuya Takeda

11th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications (2019 Automotive User Interfaces) 2019.9

Real-time Streaming Point Cloud Compression for 3D LiDAR Sensor Using U-net

Chenxi Tu,Eijiro Takeuchi,Alexander Carballo,Kazuya Takeda

IEEE access page： - 2019.8

Overview of the five key research groups within the Behaviour Signal Processing Laboratory (Takeda Laboratory) at Nagoya University

Kazuya Takeda

Impact Vol. 2019 ( 5 ) page： 32-35 2019.6

A Survey of Autonomous Driving: Common Practices and Emerging Technologies Reviewed

Ekim Yurtsever, Jacob Lambert, Alexander Carballo, Kazuya Takeda

arXiv preprint arXiv:1906.05113 page： 00 2019.6

Risky action recognition in lane change video clips using deep spatiotemporal networks with segmentation mask transfer Reviewed

Ekim Yurtsever, Yongkang Liu, Jacob Lambert, Chiyomi Miyajima, Eijiro Takeuchi, Kazuya Takeda, John HL Hansen

arXiv page： 1906.02859 2019.6

A Predictive Reward Function for Human-Like Driving Based on a Transition Model of Surrounding Environment Reviewed

Daiki Hayashi, Yunfei Xu, Takashi Bando, Kazuya Takeda

2019 International Conference on Robotics and Automation (ICRA) page： 7618-7624 2019.5

Point cloud compression for 3D LiDAR sensor using recurrent neural network with residual blocks Reviewed

Chenxi Tu, Eijiro Takeuchi, Alexander Carballo, Kazuya Takeda

2019 International Conference on Robotics and Automation (ICRA) page： 3274-3280 2019.5

SCENE-DEPENDENT ANOMALOUS ACOUSTIC-EVENT DETECTION BASED ON CONDITIONAL WAVENET AND I-VECTOR

Komatsu Tatsuya, Hayashi Tomoki, Kondo Reishi, Toda Tomoki, Takeda Kazuya

2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP) page： 870-874 2019.5

A Predictive Reward Function for Human-like Driving based on a Transition Model of Surrounding Environment

Hayashi Daiki, Xu Yunfei, Bando Takashi, Takeda Kazuya

2019 INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA) page： 7618-7624 2019.5

Point Cloud Compression for 3D LiDAR Sensor using Recurrent Neural Network with Residual Blocks

Tu Chenxi, Takeuchi Eijiro, Carballo Alexander, Takeda Kazuya

2019 INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA) page： 3274-3280 2019.5

Akira Tamamori, Yoshio Ishiguro, Kei Hiroi, Nobuo Kawaguchi, Kazuya Takeda

IPSJ Transactions on Consumer Device & Systems (CDS) Vol. 9 ( 2 ) page： 33 - 46 2019.5

Environmental sound processing and its applications Reviewed

Koichi Miyazaki, Tomoki Toda, Tomoki Hayashi, Kazuya Takeda

IEEJ Transactions on Electrical and Electronic Engineering Vol. 14 ( 3 ) page： 340 - 351 2019.3

Language：English Publishing type：Research paper (scientific journal)

© 2019 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc. As part of the effort to develop techniques for understanding environments using sound, many studies in the field of computational auditory scene analysis have focused on using computers to perform functions carried out naturally by the human auditory system. Thanks to recent progress in machine-learning techniques, these environmental sound-processing techniques have significantly improved and a widening variety of applications has resulted in considerable interest in this field. In this review, we introduce the fundamental techniques of environmental sound processing, as well as recent advances in front-end and back-end processing and potential applications for these techniques. Prospects for further progress in the field of environmental sound processing and the challenges still to be overcome are also discussed. © 2019 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

DOI： 10.1002/tee.22868

Impact of driver behavior on fuel consumption: Classification, evaluation and prediction using machine learning Reviewed

A traffic flow simulation framework for learning driver heterogeneity from naturalistic driving data using autoencoders Reviewed

Ekim Yurtsever, Chiyomi Miyajima, Kazuya Takeda

International Journal of Automotive Engineering Vol. 10 ( 1 ) page： 86-93 2019.3

Real-Time Streaming Point Cloud Compression for 3D LiDAR Sensor Using U-Net

Tu Chenxi, Takeuchi Eijiro, Carballo Alexander, Takeda Kazuya

IEEE ACCESS Vol. 7 page： 113616-113625 2019

Peng Ping, Wenhu Qin, Yang Xu, Chiyomi Miyajima, Kazuya Takeda

IEEE Access Vol. 7 page： 78515 - 78532 2019

Language：Japanese Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

© 2013 IEEE. Driving behavior has a large impact on vehicle fuel consumption. Dedicated study on the relationship between the driving behavior and fuel consumption can contribute to decreasing the energy cost of transportation and the development of the behavior assessment technology for the ADAS system. Therefore, it is vital to evaluate this relationship in order to develop more ecological driving assistance systems and improve the vehicle fuel economy. However, modeling driving behavior under the dynamic driving conditions is complex, making a quantitative analysis of the relationship between the driving behavior and the fuel consumption difficult. In this paper, we introduce two kinds of machine learning methods for evaluating the fuel efficiency of driving behavior using the naturalistic driving data. In the first stage, we use an unsupervised spectral clustering algorithm to study the macroscopic relationship between driving behavior and fuel consumption, using the data collected during the natural driving process. In the second stage, the dynamic information from the driving environment and natural driving data is integrated to generate a model of the relationship between various driving behaviors and the corresponding fuel consumption features. The dynamic environment factors are coded into a processable, digital form using a deep learning-based object detection method so that the environmental data can be linked with the vehicle's operating signal data to provide the training data for the deep learning network. The training data are labeled according to its fuel consumption feature distribution, which is obtained from the road segment data and historical driving data. This deep learning-based model can then be used as a predictor of the fuel consumption associated with different driving behaviors. Our results show that the proposed method can effectively identify the relationship between the driving behavior and the fuel consumption on both macro and micro levels, allowing for end-to-end fuel consumption feature prediction, which can then be applied in the advanced driving assistance systems.

DOI： 10.1109/ACCESS.2019.2920489

Daily activity recognition based on recurrent neural network using multi-modal signals Reviewed

Training Engineers in Autonomous Driving Technologies using Autoware

Carballo Alexander, Wong David, Ninomiya Yoshiki, Kato Shinpei, Takeda Kazuya

2019 IEEE INTELLIGENT TRANSPORTATION SYSTEMS CONFERENCE (ITSC) page： 3347-3354 2019

Risky Action Recognition in Lane Change Video Clips using Deep Spatiotemporal Networks with Segmentation Mask Transfer

Yurtsever Ekim, Liu Yongkang, Lambert Jacob, Miyajima Chiyomi, Takeuchi Eijiro, Takeda Kazuya, Hansen John H. L.

2019 IEEE INTELLIGENT TRANSPORTATION SYSTEMS CONFERENCE (ITSC) page： 3100-3107 2019

Personalized Safety-focused Control by Minimizing Subjective Risk

Bao Naren, Yang Dongfang, Carballo Alexander, Ozguner Umit, Takeda Kazuya

2019 IEEE INTELLIGENT TRANSPORTATION SYSTEMS CONFERENCE (ITSC) page： 3853-3858 2019

Crossing Blind Intersections from a Full Stop Using Estimated Visibility of Approaching Vehicles

Narksri Patiphon, Takeuchi Eijiro, Ninomiya Yoshiki, Takeda Kazuya

2019 IEEE INTELLIGENT TRANSPORTATION SYSTEMS CONFERENCE (ITSC) page： 2427-2434 2019

ATTENTION-BASED SPEECH RECOGNITION USING GAZE INFORMATION

Segawa Osamu, Hayashi Tomoki, Takeda Kazuya

2019 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU 2019) page： 465-470 2019

Optimizing Learned Object Detection on Point Clouds from 3D Lidars Through Range and Sparsity Information

Lambert Jacob, Takeuchi Eijiro, Takeda Kazuya

2019 ASIA-PACIFIC SIGNAL AND INFORMATION PROCESSING ASSOCIATION ANNUAL SUMMIT AND CONFERENCE (APSIPA ASC) page： 407-413 2019

Akira Tamamori, Tomoki Hayashi, Tomoki Toda, Kazuya Takeda

APSIPA Transactions on Signal and Information Processing Vol. 7 ( e21 ) page： 1 - 11 2018.12

Language：English Publishing type：Research paper (scientific journal) Publisher：CAMBRIDGE UNIV PRESS

Copyright © 2018 The Authors. Our aim is to develop a smartphone-based life-logging system. Human activity recognition (HAR) is one of the core techniques to realize it. Recent studies reported the effectiveness of feed-forward neural network (FF-NN) and recurrent neural network (RNN) as a classifier for HAR task. However, there are still unresolved problems in those studies: (1) a life-logging system using only a smartphone for recording device has not been developed, (2) only indoor activities have been utilized for evaluation, (3) insufficient investigations/evaluations of RNN. In this study, we address these unresolved problems as follows: (1) we build a prototype system for life-logging and conduct data recording experiment on this system to include both indoor and outdoor activities. The experimental results of HAR on this new dataset showed that RNN-based classifier was still effective. (2) From the results of a HAR experiment, it was demonstrated that a multi-layered Simple Recurrent Unit with a non-linear transform at the bottom layer and a highway-connection was the most effective. (3) We could grasp the reason for the improvement of RNN from FF-NN by observing the posterior probabilities over test data.

DOI： 10.1017/ATSIP.2018.25

Stereophonic Music Separation Based on Non-Negative Tensor Factorization with Cepstral Distance Regularization Reviewed

BACK-TRANSLATION-STYLE DATA AUGMENTATION FOR END-TO-END ASR Reviewed

Hayashi Tomoki, Watanabe Shinji, Zhang Yu, Toda Tomoki, Hori Takaaki, Astudillo Ramon, Takeda Kazuya

2018 IEEE WORKSHOP ON SPOKEN LANGUAGE TECHNOLOGY (SLT 2018) page： 426-433 2018.12

Back-translation-style data augmentation for end-to-end ASR

Tomoki Hayashi, Shinji Watanabe, Yu Zhang, Tomoki Toda, Takaaki Hori, Ramon Astudillo, Kazuya Takeda

2018 IEEE Spoken Language Technology Workshop (SLT) page： 426-433 2018.12

End-to-End Navigation with Branch Turning Support using Convolutional Neural Network

Seiya Shunya, Carballo Alexander, Takeuchi Eijiro, Miyajima Chiyomi, Takeda Kazuya

2018 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO) page： 499-506 2018.12

ITS+ DM Hackathon (ITSC 2017): Lane departure prediction with naturalistic driving data

Andrey Alekseenko, Hien Q Dang, Gaurav Bansal, Javier J Sánchez-Medina, Chiyomi Miyajima, Takatsugu Hirayama, Kazuya Takeda, Ichiro Ide

IEEE Intelligent Transportation Systems Magazine page： 00 2018.11

Driving Feature Extraction and Behavior Classification Using an Autoencoder to Reproduce the Velocity Styles of Experts Reviewed

Sama Kyle, Morales Yoichi, Akai Naoki, Liu Hailong, Takeuchi Eijiro, Takeda Kazuya

2018 21ST INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC) page： 1337-1343 2018.11

SecretSign: A Method of Finding an Off-line Target Object without Revealing the Target to Observers

Sakai Yusuke, Morita Hiromi, Ishiguro Yoshio, Nishino Takanori, Takeda Kazuya

2018 21ST INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC) page： 3651-3656 2018.11

High Density Ground Maps using Low Boundary Height Estimation for Autonomous Vehicles Reviewed

Carballo Alexander, Takeuchi Eijiro, Takeda Kazuya

2018 21ST INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC) page： 3811-3818 2018.11

SecretSign: A Method of Finding an Off-Line Target Object without Revealing the Target to Observers

Yusuke Sakai, Hiromi Morita, Yoshio Ishiguro, Takanori Nishino, Kazuya Takeda

2018 21st International Conference on Intelligent Transportation Systems (ITSC) page： 3651-3656 2018.11

Generalized multichannel variational autoencoder for underdetermined source separation

Shogo Seki, Hirokazu Kameoka, Li Li, Tomoki Toda, Kazuya Takeda

arXiv preprint arXiv:1810.00223 page： 00 2018.9

Connectionist Temporal Classification-based Sound Event Encoder for Converting Sound Events into Onomatopoeic Representations

Miyazaki Koichi, Hayashi Tomoki, Toda Tomoki, Takeda Kazuya

2018 26TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO) page： 852-856 2018.9

Anomalous Sound Event Detection Based on WaveNet

Hayashi Tomoki, Komatsu Tatsuya, Kondo Reishi, Toda Tomoki, Takeda Kazuya

2018 26TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO) page： 2494-2498 2018.9

Tsukuba Challenge 2017 Dynamic Object Tracks Dataset for Pedestrian Behavior Analysis

Lambert Jacob, Liang Leslie, Morales Luis Yoichi, Akai Naoki, Carballo Alexander, Takeuchi Eijiro, Narksri Patiphon, Seiya Shunya, Takeda Kazuya

JOURNAL OF ROBOTICS AND MECHATRONICS Vol. 30 ( 4 ) page： 598-612 2018.8

End-to-End Autonomous Mobile Robot Navigation with Model-Based System Support

Carballo Alexander, Seiya Shunya, Lambert Jacob, Darweesh Hatem, Narksri Patiphon, Morales Luis Yoichi, Akai Naoki, Takeuchi Eijiro, Takeda Kazuya

JOURNAL OF ROBOTICS AND MECHATRONICS Vol. 30 ( 4 ) page： 563-583 2018.8

Shogo Seki, Tomoki Toda, Kazuya Takeda

IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES Vol. E101A ( 7 ) page： 1057 - 1064 2018.7

Language：English Publishing type：Research paper (scientific journal) Publisher：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG

This paper proposes a semi-supervised source separation method for stereophonic music signals containing multiple recorded or processed signals, where synthesized music is focused on the stereophonic music. As the synthesized music signals are often generated as linear combinations of many individual source signals and their respective mixing gains, phase or phase difference information between inter-channel signals, which represent spatial characteristics of recording environments, cannot be utilized as acoustic clues for source separation. Non-negative Tensor Factorization (NTF) is an effective technique which can be used to resolve this problem by decomposing amplitude spectrograms of stereo channel music signals into basis vectors and activations of individual music source signals, along with their corresponding mixing gains. However, it is difficult to achieve sufficient separation performance using this method alone, as the acoustic clues available for separation are limited. To address this issue, this paper proposes a Cepstral Distance Regularization (CDR) method for NTF-based stereo channel separation, which involves making the cepstrum of the separated source signals follow Gaussian Mixture Models (GMMs) of the corresponding the music source signal. These GMMs are trained in advance using available samples. Experimental evaluations separating three and four sound sources are conducted to investigate the effectiveness of the proposed method in both supervised and semi-supervised separation frameworks, and performance is also compared with that of a conventional NTF method. Experimental results demonstrate that the proposed method yields significant improvements within both separation frameworks, and that cepstral distance regularization provides better separation parameters.

DOI： 10.1587/transfun.E101.A.1057

Daily activity recognition with large-scaled real-life recording datasets based on deep neural network using multi-modal signals Reviewed

Integrating driving behavior and traffic context through signal symbolization for data reduction and risky lane change detection

Ekim Yurtever, Suguru Yamazaki, Chiyomi Miyajima, Chiyomi, Kazuya Takeda, Masataka Mori, Kentarou Hitomi, and Masumi Egawa

IEEE Transactions on Intelligent Vehicles page： 00 2018.6

Retrieving a Driving Model Based on Clustered Intersection Data

Sama Kyle, Morales Yoichi, Akai Naoki, Takeuchi Eijiro, Takeda Kazuya

2018 3RD INTERNATIONAL CONFERENCE ON CONTROL AND ROBOTICS ENGINEERING (ICCRE) page： 222-226 2018.4

畳み込み雑音除去自己符号化器と対数周波数領域スペクトル特徴を用いた楽曲音源強調 Reviewed

大谷健登, 丹羽健太, 西野隆典, 武田一哉

電子情報通信学会論文誌D Vol. J101-D ( 3 ) page： 00 2018.3

ITSS Technical Activities Spotlight: Getting to Know the Naturalistic Driving Data Analytics Technical Committee [Technical Activities]

Pujitha Gunaratne, Kazuya Takeda

IEEE Intelligent Transportation Systems Magazine page： 167-167 2018.1

Tomoki Hayashi, Masafumi Nishida, Norihide Kitaoka, Tomoki Toda, Kazuya Takeda

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences Vol. E101A ( 1 ) page： 199 - 210 2018.1

Language：English Publishing type：Research paper (international conference proceedings) Publisher：Institute of Electronics, Information and Communication, Engineers, IEICE

© 2018 The Institute of Electronics, Information and Communication Engineers. In this study, toward the development of smartphone-based monitoring system for life logging, we collect over 1,400 hours of data by recording including both the outdoor and indoor daily activities of 19 subjects, under practical conditions with a smartphone and a small camera. We then construct a huge human activity database which consists of an environmental sound signal, triaxial acceleration signals and manually annotated activity tags. Using our constructed database, we evaluate the activity recognition performance of deep neural networks (DNNs), which have achieved great performance in various fields, and apply DNN-based adaptation techniques to improve the performance with only a small amount of subject-specific training data. We experimentally demonstrate that; 1) the use of multi-modal signal, including environmental sound and triaxial acceleration signals with a DNN is effective for the improvement of activity recognition performance, 2) the DNN can discriminate specified activities from a mixture of ambiguous activities, and 3) DNN-based adaptation methods are effective even if only a small amount of subject-specific training data is available.

DOI： 10.1587/transfun.E101.A.199

Modeling Driver Risk Perception on City Roads Using Deep Learning Reviewed

Peng Ping, Yuan Sheng, Wenhu Qin, Chiyomi Miyajima, Kazuya Takeda

IEEE Access Vol. 6 page： 68850 - 68866 2018

Language：English Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

© 2013 IEEE. Research on how risk is perceived by drivers is vital to driving behavior research and driving safety. As risk can be divided into subjective and objective risk, in this paper, we focus on modeling subjective risk perception by drivers using a deep learning method. Different drivers often perceive different levels of subjective risk under the same driving conditions. In addition, different driving conditions or driving events will have different effects on drivers. Based on these two risk perception features, in this paper, we first design an experiment on a city road with two lanes to assess the level of subjective risk perceived by drivers belonging to different groups. We then use a deep learning network-based method to abstract features of the driving environment. These environmental features are integrated with driver risk perception data and this information is used as training and testing data for the learning network. Finally, a long-short-term memory-based method is adopted to model the subjective risk perception of individual drivers based on traffic conditions and vehicle operation data from the driver's vehicle. Our results show that the proposed method can effectively model the subjective risk perception behavior of drivers, allowing for end-to-end risk perception prediction in future driving assistance systems.

DOI： 10.1109/ACCESS.2018.2879887

Recognizing emotions from speech using a physical model Reviewed

Norihide Kitaoka, Shuhei Segawa, Ryota Nishimura, Kazuya Takeda

Acoustical Science and Technology Vol. 39 ( 2 ) page： 167 - 170 2018

Learning How to Drive in Blind Intersections from Human Data Reviewed

Kyle Sama, Yoichi Morales, Naoki Akai, Eijiro Takeuchi, Kazuya Takeda

Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 page： 317 - 324 2018

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

© 2018 IEEE. In this paper we present a method to learn how to drive in different types of blind intersections using expert driving data. We cluster different intersections based on the velocity of how drivers approach them, and train a linear SVM classifier for each class of intersection. Through clustering we found that there were three different classes of intersections in typical residential areas in Japan. We used inverse reinforcement learning (IRL) to build a driving model for each type of intersection. The models were trained from 308 trajectories traversed by 5 different drivers. The models and policies were implemented and evaluated in a ROS simulator where the agent is provided a global path, and upon it reaching an intersection, it selects the appropriate trained policy. By doing this, the simulated autonomous vehicle can perform proactive safe driving behaviors when approaching blind intersections.

DOI： 10.1109/SMC.2018.00064

Multi-head decoder for end-to-end speech recognition Reviewed

Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, Kazuya Takeda

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH Vol. 2018-September page： 801 - 805 2018

Language：English Publishing type：Research paper (international conference proceedings) Publisher：ISCA

© 2018 International Speech Communication Association. All rights reserved. This paper presents a new network architecture called multi-head decoder for end-to-end speech recognition as an extension of a multi-head attention model. In the multi-head attention model, multiple attentions are calculated, and then, they are integrated into a single attention. On the other hand, instead of the integration in the attention level, our proposed method uses multiple decoders for each attention and integrates their outputs to generate a final output. Furthermore, in order to make each head to capture the different modalities, different attention functions are used for each head, leading to the improvement of the recognition performance with an ensemble effect. To evaluate the effectiveness of our proposed method, we conduct an experimental evaluation using Corpus of Spontaneous Japanese. Experimental results demonstrate that our proposed method outperforms the conventional methods such as location-based and multi-head attention models, and that it can capture different speech/linguistic contexts within the attention-based encoder-decoder framework.

DOI： 10.21437/Interspeech.2018-1655

Other Link： http://arxiv.org/pdf/1804.08050v2

An investigation of multi-speaker training for WaveNet vocoder

T. Hayashi, A. Tamamori, K. Kobayashi, K. Takeda, T. Toda

Proc. ASRU page： 712-718 2017.12

Investigation of effectiveness on recurrent neural network for daily activity recognition using multi-modal signals

A. Tamamori, T. Hayashi, T. Toda, K. Takeda

Proc. APSIPA page： 7 2017.12

An Investigation of Recurrent Neural Network for Daily Activity Recognition using Multi-modal Signals

Tamamori Akira, Hayashi Tomoki, Toda Tomoki, Takeda Kazuya

2017 ASIA-PACIFIC SIGNAL AND INFORMATION PROCESSING ASSOCIATION ANNUAL SUMMIT AND CONFERENCE (APSIPA ASC 2017) page： 1334-1340 2017.12

Involvement of poly-rC binding proteins in posttranscriptional regulation of Sortilin, the cytokine trafficking mediator Reviewed

Toshiki Yabe-Wada, Shintaro Matsuba, Kazuya Takeda, Akira Nakamura, Caroline C. Philpott, Nobuyuki Onai

CYTOKINE Vol. 100 page： 145 - 146 2017.12

Duration-controlled LSTM for polyphonic sound event detection

T. Hayashi, S. Watanabe, T. Toda, T. Hori, J. Le Roux, K. Takeda.

IEEE/ACM Transactions on Audio, Speech, and Language Processing Vol. 25 ( 11 ) page： 2059-2070 2017.11

Duration-Controlled LSTM for Polyphonic Sound Event Detection Reviewed

Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, Takaaki Hori, Jonathan Le Roux, Kazuya Takeda

IEEE/ACM Transactions on Audio Speech and Language Processing Vol. 25 ( 11 ) page： 2059 - 2070 2017.11

Language：English Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

© 2014 IEEE. This paper presents a new hybrid approach called duration-controlled long short-term memory (LSTM) for polyphonic sound event detection (SED). It builds upon a state-of-the-art SED method that performs frame-by-frame detection using a bidirectional LSTM recurrent neural network (BLSTM), and incorporates a duration-controlled modeling technique based on a hidden semi-Markov model. The proposed approach makes it possible to model the duration of each sound event precisely and to perform sequence-by-sequence detection without having to resort to thresholding, as in conventional frame-by-frame methods. Furthermore, to effectively reduce sound event insertion errors, which often occur under noisy conditions, we also introduce a binary-mask-based postprocessing that relies on a sound activity detection network to identify segments with any sound event activity, an approach inspired by the well-known benefits of voice activity detection in speech recognition systems. We conduct an experiment using the DCASE2016 task 2 dataset to compare our proposed method with typical conventional methods, such as nonnegative matrix factorization and standard BLSTM. Our proposed method outperforms the conventional methods both in an event-based evaluation, achieving a 75.3% F1 score and a 44.2% error rate, and in a segment-based evaluation, achieving an 81.1% F1 score, and a 32.9% error rate, outperforming the best results reported in the DCASE2016 task 2 Challenge.

DOI： 10.1109/TASLP.2017.2740002

A single-dimensional interface for arranging multiple audio sources in three-dimensional space Reviewed

A Single-Dimensional Interface for Arranging Multiple Audio Sources in Three-Dimensional Space

Kento Ohtani, Kenta Niwa, Kazuya Takeda

IEICE TRANSACTIONS on Information and Systems (0.411), Vol. E100-D ( 10 ) page： pp. 2635-264 2017.10

Kento Ohtani, Kenta Niwa, Kazuya Takeda

IEICE Transactions on Information and Systems Vol. E100D ( 10 ) page： 2635 - 2643 2017.10

Language：English Publishing type：Research paper (scientific journal) Publisher：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG

Copyright © 2017 The Institute of Electronics, Information and Communication Engineers. A single-dimensional interface which enables users to obtain diverse localizations of audio sources is proposed. In many conventional interfaces for arranging audio sources, there are multiple arrangement parameters, some of which allow users to control positions of audio sources. However, it is difficult for users who are unfamiliar with these systems to optimize the arrangement parameters since the number of possible settings is huge. We propose a simple, single-dimensional interface for adjusting arrangement parameters, allowing users to sample several diverse audio source arrangements and easily find their preferred auditory localizations. To select subsets of arrangement parameters from all of the possible choices, auditory-localization space vectors (ASVs) are defined to represent the auditory localization of each arrangement parameter. By selecting subsets of ASVs which are approximately orthogonal, we can choose arrangement parameters which will produce diverse auditory localizations. Experimental evaluations were conducted using music composed of three audio sources. Subjective evaluations confirmed that novice users can obtain diverse localizations using the proposed interface.

DOI： 10.1587/transinf.2017EDP7028

Signal Processing for Smart Vehicle Technologies: Part 2 [From the Guest Editors] Reviewed

Prediction Method for .Continuous Point Cloud Data Compression Using SLAM Information

Chenxi Tu, Eijiro Takeuchi, Chiyomi Miyajima, and Kazuya Takeda

fast-zero page： SLAM Information," fast-zero, Sep., 2017 2017.9

Evaluation of Deep Learning-Based Driving Signal Generation Methods for Vehicle Control

Shunya Seiya, Daiki Hayashi, Eijiro Takeuchi, Chiyomi Miyajima and Kazuya Takeda

Fast-ZERO, 2017 page： 00 2017.9

Estimating Risk Levels Perceived by Individuals for Lane Change Scenes,"The fourth International Symposium on Future Active Safety Technology Toward zero traffic accidents

Naren Bao, Chiyomi Miyajima, Eijiro Takeuchi, Kazuya Takeda, Shinichiro Honda, Toshiya Yoshitani, and Masayoshi Ito

The fourth International Symposium on Future Active Safety Technology Toward zero traffic accidents page： 00 2017.9

Missing component restoration for masked speech signals based on time-domain spectrogram factorization

Shogo Seki, Hirokazu Kameoka, Tomoki Toda, Kazuya Takeda

MLSP2017 page： 6 2017.9

Estimation of driver's risk feeling toward driving environment using neural network

Yuan Sheng, Yiyang Li, Chiyomi Miyajima, Eijiro Takeuchi, Kazuya Takeda, Shinichiro Honda, Toshiya Yoshitani, and Masayoshi Ito

FAST-zero '17 page： 00 2017.9

Prediction method for continuous point cloud data compression using SLAM information

Chenxi Tu, Eijiro Takeuchi, Chiyomi Miyajima, and Kazuya Takeda

4th International Symposium on Future Active Safety Technology toward zero traffic accidents (FAST-zero '17) page： 00 2017.9

Speaker-dependent WaveNet vocoder

Akira Tamamori,Tomoki Hayashi,Kazuhiro Kobayashi,Kazuya Takeda,Tomoki Toda,

INTERSPEECH 2017 page： pp. 1118-1122 2017.8

Stereophonic Music Separation Based on Non-negative Tensor Factorization with Cepstrum Regularization

Shogo Seki, Tomoki Toda, Kazuya Takeda

EUSIPCO2017 page： pp. 1011-1015 2017.8

Stereophonic Music Separation Based on Non-negative Tensor Factorization with Cepstrum Regularization

Seki Shogo, Toda Tomoki, Takeda Kazuya

2017 25TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO) page： 981-985 2017.8

Open Source Integrated Planner for Autonomous Navigation in Highly Dynamic Environments

Darweesh Hatem, Takeuchi Eijiro, Takeda Kazuya, Ninomiya Yoshiki, Sujiwo Adi, Morales Luis Yoichi, Akai Naoki, Tomizawa Tetsuo, Kato Shinpei

JOURNAL OF ROBOTICS AND MECHATRONICS Vol. 29 ( 4 ) page： 668-684 2017.8

Continuous point cloud data compression using SLAM based prediction Reviewed

Chenxi Tu, Eijro Takeuchi, Chiyomi Miyajima, Kazuya Takeda

IEEE 2017 Intelligent Vehicles Symposium (IV '17) page： 1744–1751 2017.6

Music Staging AI

Kenta Niwa, Kento Ohtani, Kazuya Takeda

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017) page： 6588-6589 2017.5

BLSTM-HMM hybrid system combined with sound activity detection network for polyphonic sound event detection Reviewed

T. Hayashi, S. Watanabe, T. Toda, T. Hori, J. Le Roux, K. Takeda.

Proc. ICASSP page： 766-770 2017.3

John H.L. Hansen, Kazuya Takeda, Sanjeev M. Naik, Mohan M. Trivedi, Gerhard U. Schmidt, Yingying Jennifer Chen, Wade Trappe

IEEE Signal Processing Magazine Vol. 34 ( 2 ) page： 18 - 21 2017.3

MUSIC STAGING AI Reviewed

Niwa Kenta, Ohtani Kento, Takeda Kazuya

2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP) page： 6588-6589 2017

AN INVESTIGATION OF MULTI-SPEAKER TRAINING FORWAVENET VOCODER Reviewed

Hayashi Tomoki, Tamamori Akira, Kobayashi Kazuhiro, Takeda Kazuya, Toda Tomoki

2017 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU) page： 712-718 2017

BLSTM-HMM HYBRID SYSTEM COMBINED WITH SOUND ACTIVITY DETECTION NETWORK FOR POLYPHONIC SOUND EVENT DETECTION Reviewed

Hayashi Tomoki, Watanabe Shinji, Toda Tomoki, Hori Takaaki, Le Roux Jonathan, Takeda Kazuya

2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP) page： 766-770 2017

Continuous Point Cloud Data Compression Using SLAM Based Prediction Reviewed

Tu Chenxi, Takeuchi Eijiro, Miyajima Chiyomi, Takeda Kazuya

2017 28TH IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV 2017) page： 1744-1751 2017

MISSING COMPONENT RESTORATION FOR MASKED SPEECH SIGNALS BASED ON TIME-DOMAIN SPECTROGRAM FACTORIZATION Reviewed

Seki Shogo, Kameoka Hirokazu, Toda Tomoki, Takeda Kazuya

2017 IEEE 27TH INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING page： . 2017

Speaker-dependent WaveNet vocoder Reviewed

Akira Tamamori, Tomoki Hayashi, Kazuhiro Kobayashi, Kazuya Takeda, Tomoki Toda

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH Vol. 2017-August page： 1118 - 1122 2017

Language：Japanese Publishing type：Research paper (international conference proceedings) Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC

Copyright © 2017 ISCA. In this study, we propose a speaker-dependent WaveNet vocoder, a method of synthesizing speech waveforms with WaveNet, by utilizing acoustic features from existing vocoder as auxiliary features of WaveNet. It is expected that WaveNet can learn a sample-by-sample correspondence between speech waveform and acoustic features. The advantage of the proposed method is that it does not require (1) explicit modeling of excitation signals and (2) various assumptions, which are based on prior knowledge specific to speech. We conducted both subjective and objective evaluation experiments on CMU-ARCTIC database. From the results of the objective evaluation, it was demonstrated that the proposed method could generate high-quality speech with phase information recovered, which was lost by a mel-cepstrum vocoder. From the results of the subjective evaluation, it was demonstrated that the sound quality of the proposed method was significantly improved from mel-cepstrum vocoder, and the proposed method could capture source excitation information more accurately.

DOI： 10.21437/Interspeech.2017-314

Impact of acoustic similarity on efficiency of verbal information transmission via subtle prosodic cues Reviewed

Bohan Chen, Norihide Kitaoka, Kazuya Takeda

Eurasip Journal on Audio, Speech, and Music Processing Vol. 2016 ( 1 ) 2016.12

Language：English Publishing type：Research paper (scientific journal) Publisher：SPRINGER INTERNATIONAL PUBLISHING AG

© 2016, The Author(s). In this study, we investigate the effect of tiny acoustic differences on the efficiency of prosodic information transmission. Study participants listened to textually ambiguous sentences, which could be understood with prosodic cues, such as syllable length and pause length. Sentences were uttered in voices similar to the participant’s own voice and in voices dissimilar to their own voice. The participants then identified which of four pictures the speaker was referring to. Both the eye movement and response time of the participants were recorded. Eye tracking and response time results both showed that participants understood the textually ambiguous sentences faster when listening to voices similar to their own. The results also suggest that tiny acoustic features, which do not contain verbal meaning can influence the processing of verbal information.

DOI： 10.1186/s13636-016-0097-6

Investigation on Recurrent Neural Network Architectures for Daily Activity Recognition Reviewed

Akira Tamamori, Tomoki Hayashi, Tomoki Toda, Kazuya Takeda

The 3rd International Conference on Universal Village (UV2016) page： 00 2016.12

Sound enhancement system using selective binary filtering Reviewed

Tomomi Suzuki, Takanori Nishino, Yoshio Ishiguro, Kazuya Takeda

5th Joint Meeting of the ASA and ASJ page： 00 2016.12

Convolutional Biderectional LSTM-HMM Hybrid System for Polyphonic Sound Event Detection Reviewed

Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, Takaaki Hori, Jonathan Le Roux, Kazuya Takeda

5th Joint Meeting of the ASA and ASJ page： 00 2016.12

Signal Processing for Smart Vehicle Technologies [From the Guest Editors]

John H.L. Hansen, Kazuya Takeda, Sanjeev M. Naik, Mohan M. Trivedi, Gerhard U. Schmidt, Yingying Chen

IEEE Signal Processing Magazine Vol. 33 ( 6 ) page： 12 - 13 2016.11

Compressing Continuous Point Cloud Data Using Image Compression Methods Reviewed

Chenxi Tu, Eijiro Takeuchi, Chiyomi Miyajima, Kazuya Takeda

ITSC2016 page： 00 2016.11

AI framework to arrange audio objects according to listener preferences Reviewed

Kento Ohtani, Kenta Niwa, Kazuya Takeda

5th Joint Meeting of the Acoustical Society of America and Acoustical Society of Japan page： 00 2016.11

Stereo Channel Music Signal Separation Based on Nonnegative Tensor Factorization with Cepstrum Regularization Reviewed

Shogo Seki, Kento Ohtani, Tomoki Toda, Kazuya Takeda

5th Joint Meeting of the ASA and ASJ page： 00 2016.11

Driver-Behavior Modeling Using On-Road Driving Data: A new application for behavior signal processing Reviewed

Chiyomi Miyajima, Kazuya Takeda

IEEE Signal Processing Magazine Vol. 33 ( 6 ) page： 14 - 21 2016.11

Language：English Publishing type：Research paper (scientific journal) Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

© 2016 IEEE. This article reviews data-centric approaches for statistical modeling of driver behavior. Modeling driver behavior is challenging due to its stochastic nature and the high degree of inter- and intradriver variability. One way to deal with the highly variable nature of driving behavior is to employ a data-centric approach that models driver behavior using large amounts of driving data collected from numerous drivers in a variety of traffic conditions. To obtain large amounts of realistic driving data, several projects have collected real-world driving data. Statistical machine-learning techniques, such as hidden Markov models (HMMs) and deep learning, have been successfully applied to model driver behavior using large amounts of driving data. We have also collected on-road data recording hundreds of drivers over more than 15 years. We have applied statistical signal processing and machine-learning techniques to this data to model various aspects of driver behavior, e.g., driver pedal-operation, car-following, and lane-change behaviors for predicting driver behavior and detecting risky driver behavior and driver frustration. By reviewing related studies and providing concrete examples of our own research, this article is intended to illustrate the usefulness of such data-centric approaches for statistical driver-behavior modeling.

DOI： 10.1109/MSP.2016.2602377

Analysis of driver workload when using speech interfaces Reviewed

D Hayashi, C Miyajima, K Takeda

The Journal of the Acoustical Society of America Vol. 140 ( 4 ) page： 2961 - 2961 2016.10

Projection of a virtual speaker into a vehicle using sound field control Reviewed

T Yamamura, Y Ishiguro, T Nishino, K Takeda

The Journal of the Acoustical Society of America Vol. 140 ( 4 ) page： 3063 - 3063 2016.10

Emotion recognition from speech using a physical model Reviewed

Norihide Kitaoka, Shuhei Segawa, Kazuya Takeda

Proc. ICA2016 page： 00 2016.9

Bidirectional LSTM-HMM Hybrid System for Polyphonic Sound Event Detection Reviewed

Tomoki Hayashi, Shinji Watanabe, Tomoki Toda, Takaaki Hori, Jonathan Le Roux, Kazuya Takeda

DCASE2016 page： 00 2016.9

Investigation of DNN-based modeling for audio-visual speech recognition Reviewed

Satoshi Tamura, Hiroshi Ninomiya, Norihide Kitaoka, Shin Osuga, Yurie Iribe, Kazuya Takeda, Satoru Hayamizu

2015 First International Workshop on Spoken Language Processing (MLSLP2015) page： 00 2016.9

Investigation on Recurrent Neural Network Architectures for Daily Activity Recognition Reviewed

Akira Tamamori, Tomoki Hayashi, Tomoki Toda, Kazuya Takeda

UV2016 page： 00 2016.9

Recurrent Neural Networkに基づく日常生活行動認識

玉森聡、林知樹、戸田智基、武田一哉

電子情報通信学会技術報告 Vol. 116(189) ( 7 ) page： 12 2016.8

Accelerated Deformable Part Models on GPUs Reviewed

Manato Hirabayashi, Shinpei Kato, Masato Edahiro, Kazuya Takeda, Seiichi Mita

IEEE Transactions on Parallel and Distributed Systems Vol. 27 ( 6 ) page： 1589 - 1602 2016.6

Language：English Publishing type：Research paper (scientific journal)

© 2015 IEEE. Object detection is a fundamental challenge facing intelligent applications. Image processing is a promising approach to this end, but its computational cost is often a significant problem. This paper presents schemes for accelerating the deformable part models (DPM) on graphics processing units (GPUs). DPM is a well-known algorithm for image-based object detection, and it achieves high detection rates at the expense of computational cost. GPUs are massively parallel compute devices designed to accelerate data-parallel compute-intensive workload. According to an analysis of execution times, approximately 98 percent of DPM code exhibits loop processing, which means that DPM could be highly parallelized by GPUs. In this paper, we implement DPM on the GPU by exploiting multiple parallelization schemes. Results of an experimental evaluation of this GPU-accelerated DPM implementation demonstrate that the best scheme of GPU implementations using an NVIDIA GPU achieves a speed up of 8.6x over a naive CPU-based implementation.

DOI： 10.1109/TPDS.2015.2453962

Prediction of Individual Driving Behavior on Highway Curves

Naren Bao, Daiki Hayashi, Chiyomi Miyajima, and Kazuya Takeda

The third Workshop on Natural- istic Driving Data Analysitics, IEEE Intelligent Vehicles Symposium page： 00 2016.6

Integrating driving behavior and traffic context through signal symbolization. Reviewed

Yamazaki, S., Miyajima, C., Yurtsever, E., Takeda, K., Mori, M., Hitomi, K., & Egawa, M.

In Intelligent Vehicles Symposium (IV), 2016 IEEE. page： pp. 642-647 2016.6

Classification of driver's neutral and cognitive distraction states based on peripheral vehicle behavior in driver's gaze transition Reviewed

Takatsugu Hirayama, Kenji Mase, Chiyomi Miyajima, Kazuya Takeda

IEEE Transactions on Intelligent Vehicles Vol. 1 ( 2 ) page： 148 - 157 2016.6

Language：English Publishing type：Research paper (scientific journal)

© 2016 IEEE. To support safe driving, numerous methods of detecting distractions using measurements of a driver's gaze have been proposed. These methods empirically focused on certain driving contexts and analyzed gaze behavior under particular peripheral vehicle conditions; therefore, numerous driving situations were not considered. To address this problem with hypothesis-testing approaches, we turn the problem around and propose a data-mining approach that analyzes peripheral vehicle behavior during gaze transitions of drivers in order to compare their neutral driving state with a cognitive distraction state. This change in thinking is the first contribution of this paper. The analysis results show that under the neutral condition, drivers generally turned their gaze to peripheral vehicles to be focused on; however, they did not do this consistently under the distracted condition. As the second contribution, we propose a simple classifier to discriminate between the cognitive distraction and neutral states by analyzing the peripheral vehicle behavior. The proposed classifier can manage various situations and provide high classification accuracy by focusing on gaze transitions from the front view toward other directions.

DOI： 10.1109/TIV.2016.2599786

Daily activity recognition based on acoustic signals and acceleration signals estimated with Gaussian process Reviewed

Masafumi Nishida, Norihide Kitaoka, Kazuya Takeda

2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2015 page： 279 - 282 2016.2

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

© 2015 Asia-Pacific Signal and Information Processing Association. We have created corpus of daily activities using wearable sensors. The corpus consists of sound and image data from a camera and motion signals from a smartphone for both indoor and outdoor activities over 72 continuous hours. We propose a method that can interpolate acceleration signals to any sample points with a Gaussian process in order to recognize daily activities. We conducted recognition experiments of daily activities using our corpus. Experimental results showed that the proposed method can improve recognition accuracy compared to a conventional method. This demonstrates the effectiveness of estimating acceleration signals with a Gaussian process to recognize daily activities.

DOI： 10.1109/APSIPA.2015.7415520

Audio-visual speech recognition using deep bottleneck features and high-performance lipreading Reviewed

Satoshi Tamura, Hiroshi Ninomiya, Norihide Kitaoka, Shin Osuga, Yurie Iribe, Kazuya Takeda, Satoru Hayamizu

2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2015 page： 575 - 582 2016.2

Language：English Publishing type：Research paper (international conference proceedings) Publisher：IEEE

© 2015 Asia-Pacific Signal and Information Processing Association. This paper develops an Audio-Visual Speech Recognition (AVSR) method, by (1) exploring high-performance visual features, (2) applying audio and visual deep bottleneck features to improve AVSR performance, and (3) investigating effectiveness of voice activity detection in a visual modality. In our approach, many kinds of visual features are incorporated, subsequently converted into bottleneck features by deep learning technology. By using proposed features, we successfully achieved 73.66% lipreading accuracy in speaker-independent open condition, and about 90% AVSR accuracy on average in noisy environments. In addition, we extracted speech segments from visual features, resulting 77.80% lipreading accuracy. It is found VAD is useful in both audio and visual modalities, for better lipreading and AVSR.

DOI： 10.1109/APSIPA.2015.7415335

Tracking driver signage observation using local feature matching and optical flow Reviewed

Chiyomi Miyajima, Katsuya Sakoyama, Kazuya Takeda

2015 IEEE/SICE International Symposium on System Integration, SII 2015 page： 479 - 482 2016.2

Language：English Publishing type：Research paper (international conference proceedings)

© 2015 IEEE. We investigate a method for identifying objects observed by drivers. Here we focus on roadside signage as an example, and track the driver's observation of signage while driving. A gaze tracking system and a forward-directed video camera are used to determine the driver's region of interest (ROI). The driver's observation of signage is detected by tracking the driver's ROI using optical flow, and by matching the driver's ROI with template images of signboards in a signage database using local feature matching. Driver and signage location information are used to limit candidate signboards for reducing computational cost for image matching. We conduct an experiment to evaluate our method and achieve a 66.2% detection rate of drivers' signboard observation with a false positive rate of 6.6%.

DOI： 10.1109/SII.2015.7405026

Modeling and detecting excessive trust from behavior signals: Overview of research project and results Reviewed

Kazuya Takeda

Human-Harmonized Information Technology, Volume 1: Vertical Impact page： 57 - 75 2016.1

Language：English Publishing type：Part of collection (book)

© Springer Japan 2016. An approach which would allow us to better understand behavioral states inherent in observed behaviors is proposed, based on the development of a mathematical representation of driving behaviors signals using our large driving behavior signal corpus. In particular, the project is aimed at developing technologies for preventing excessive trust in users of automated systems. Misuse/disuse of automation is introduced as a cognitive model of excessive trust, and methods of quantitative measurement are devised. PWARX and GMM models are proposed to represent discrete and continuous information in the cognition/decision/action process. We also develop a method of modeling visual behavior aiming at understanding environmental awareness while driving. We showed the effectiveness of the model experimentally through risky lane change detection. Finally, we show the effectiveness of the method to quantify excessive trust based on developed technology.

DOI： 10.1007/978-4-431-55867-5_3

Robust example search using bottleneck features for example-based speech enhancement Reviewed

Atsunori Ogawa, Shogo Seki, Keisuke Kinoshita, Marc Delcroix, Takuya Yoshioka, Tomohiro Nakatani, Kazuya Takeda

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH Vol. 08-12-September-2016 page： 3733 - 3737 2016

Publishing type：Research paper (international conference proceedings) Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC

Copyright © 2016 ISCA. Example-based speech enhancement is a promising approach for coping with highly non-stationary noise. Given a noisy speech input, it first searches in noisy speech corpora for the noisy speech examples that best match the input. Then, it concatenates the clean speech examples that are paired with the matched noisy examples to obtain an estimate of the underlying clean speech component in the input. This framework works well if the noisy speech corpora contain the noise included in the input. However, it is impossible to prepare corpora that cover all types of noisy environments. Moreover, the example search is usually performed using noise sensitive mel-frequency cepstral coefficient features (MFCCs). Consequently, a mismatch between an input and the corpora is inevitable. This paper proposes using bottleneck features (BNFs) extracted from a deep neural network (DNN) acoustic model for the example search. Since BNFs have good noise robustness (invariance), the mismatch is mitigated and thus a more accurate example search can be performed. Experimental results on the Aurora4 corpus show that the example-based approach using BNFs greatly improves the enhanced speech quality compared with that using MFCCs. It also consistently outperforms a conventional DNN-based approach, i.e. a denoising autoencoder.

DOI： 10.21437/Interspeech.2016-671