Language：Japanese   Publisher：IEEE Intelligent Vehicles Symposium, Proceedings  

In this research, a two-stage risk object warning method is proposed to generate the auditory and visual warning information simultaneously from the driving scene. The auditory warning module (AWM) is designed as a classification task by combining the rough location and type information as warning sentences and treating each sentence as one class. The visual warning module (VWM) is designed as a weakly supervised method to save the labor-intensive bounding box marking of risk objects. To confirm the effectiveness of the proposed method, we also create a linguistic risk notification (LRN) dataset by describing the driving scenario as several different sentences. The average accuracy of auditory warning is 96.4% for generating the warning sentences. The average accuracy of the weakly supervised visual warning algorithm is 81.3% for getting the risk vehicle localization without any supervisory information.

DOI： 10.1109/IV51971.2022.9827382

Web of Science

Scopus

Publisher：UIST 2022 Adjunct - Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology  

Many people now listen to music with earphones while walking, and are less likely to notice approaching people, cars, etc. Many methods of detecting approaching objects and notifying pedestrians have been proposed, but few have focused on low urgency situations or music listeners, and many notification methods are unpleasant. Therefore, in this work, we propose methods of gently notifying pedestrians listening to music of approaching objects using environmental sound. We conducted experiments in a virtual environment to assess directional perception accuracy and comfort. Our results show the proposed method allows participants to detect the direction of approaching objects as accurately as explicit notification methods, with less discomfort.

DOI： 10.1145/3526114.3558728

Web of Science

Scopus

Publisher：BMVC 2022 - 33rd British Machine Vision Conference Proceedings  

Recent self-supervised models have demonstrated equal or better performance than supervised methods, opening for AI systems to learn visual representations from practically unlimited data. However, these methods are typically classification-based and thus ineffective for learning high-resolution feature maps that preserve precise spatial information. This work introduces superpixels to improve self-supervised learning of dense semantically rich visual concept embeddings. Decomposing images into a small set of visually coherent regions reduces the computational complexity by O(1000) while preserving detail. We experimentally show that contrasting over regions improves the effectiveness of contrastive learning methods, extends their applicability to high-resolution images, improves overclustering performance, superpixels are better than grids, and regional masking improves performance. The expressiveness of our dense embeddings is demonstrated by improving the SOTA unsupervised semantic segmentation benchmark on Cityscapes, and for convolutional models on COCO. Code is available at https://github.com/robin-karlsson0/vice.

Scopus

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： doi/10.20729/00213195

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

Language：Japanese   Publisher：IEEE Intelligent Vehicles Symposium, Proceedings  

Video captioning aims to generate textual descriptions according to the video contents. The risk assessment of autonomous driving vehicles has become essential for an insurance company for providing adequate insurance coverage, in particular, for emerging MaaS business. The insurers need to assess the risk of autonomous driving business plans with a fixed route by analyzing a large number of driving data, including videos recorded by dash cameras and sensor signals. To make the process more efficient, generating captions for driving videos can provide insurers concise information to understand the video contents quickly. A natural problem with driving video captioning is, since the absence of egovehicles in these egocentric videos, descriptions of latent driving behaviors are difficult to be grounded in specific visual cues. To address this issue, we focus on generating driving video captions with accurate behavior descriptions, and propose to incorporate in-vehicle sensors which encapsulate the driving behavior information to assist the caption generation. We evaluate our method on the Japanese driving video captioning dataset called City Traffic, where the results demonstrate the effectiveness of in-vehicle sensors on improving the overall performance of generated captions, especially on generating more accurate descriptions for the driving behaviors.

DOI： 10.1109/IVWorkshops54471.2021.9669259

Web of Science

Scopus

Language：Japanese   Publisher：Society of Automotive Engineers of Japan  

End-to-end driving refers to deep learning methods for generating control signals directly from external sensors. Previous methods use a direction vector towards the target to select and turn at intersections. However, the vector has a smaller dimension than the image, and thus it is ignored during training. In this study, we propose a learning method to emphasize that vector by using L2 regularization, which enables a robot to follow trajectories with branches. We validate the system's performance by conducting experiments using several driving scenarios. Our approach allowed an autonomous robot to successfully follow trajectories, including unknown outdoor trajectories.

DOI： 10.11351/jsaeronbun.52.1368

CiNii Research

Language：Japanese   Publisher：Adjunct Proceedings - 11th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2019  

When operating the touch screen in a car, the touch point can shift due to the vibration, resulting in selection errors. Using larger target is a possible solution, but this significantly limits the amount of content that can be displayed on the touch screen. Therefore, we propose a method for in-vehicle touch screen target selection that can be used with a variety of sensors to increase selection accuracy. In this method, the vibration feature is learned by Variational Auto-Encoder based model, and it is used for estimating touch point distribution. Our experimental results demonstrate that the proposed method allows users to achieve higher target selection accuracy than conventional methods.

DOI： 10.1145/3349263.3351327

Scopus

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

We propose a music source enhancement technique which uses a convolutional denoising autoencoder (CDAE) and the log-frequency scale amplitude spectral features of musical instrument signals. The structure of the CDAE includes amplitude spectral characteristics of the sounds created by musical instruments in its network in order to estimate the amplitude spectra of the target signals. Evaluation results show that the proposed network achieves better signal-to-interference ratios (SIRs) than conventional network/input feature structures. We also propose a complementary CDAE approach, which estimates target and noise amplitude spectra simultaneously and combines them. By using complementary CDAE, SIRs of the estimated music signals are further improved.

DOI： 10.14923/transinfj.2017pdp0021

CiNii Research

CiNii Research

Language：English   Publisher：The Institute of Electronics, Information and Communication Engineers  

<p>A single-dimensional interface which enables users to obtain diverse localizations of audio sources is proposed. In many conventional interfaces for arranging audio sources, there are multiple arrangement parameters, some of which allow users to control positions of audio sources. However, it is difficult for users who are unfamiliar with these systems to optimize the arrangement parameters since the number of possible settings is huge. We propose a simple, single-dimensional interface for adjusting arrangement parameters, allowing users to sample several diverse audio source arrangements and easily find their preferred auditory localizations. To select subsets of arrangement parameters from all of the possible choices, auditory-localization space vectors (ASVs) are defined to represent the auditory localization of each arrangement parameter. By selecting subsets of ASVs which are approximately orthogonal, we can choose arrangement parameters which will produce diverse auditory localizations. Experimental evaluations were conducted using music composed of three audio sources. Subjective evaluations confirmed that novice users can obtain diverse localizations using the proposed interface.</p>

DOI： 10.1587/transinf.2017EDP7028

Web of Science

Scopus

CiNii Research

Language：Japanese   Publisher：ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings  

Through smartphones, user enables to download/listen music anytime and anywhere. As a concept of a future audio player, we propose a framework of music staging artificial intelligence (AI). In that framework, audio object signals, e.g. vocal, guitar, bass, drums and keyboards, are assumed to be extracted from stereo music signals. To visualize music as if live performance is virtually conducted, playing motion sequence is estimated by using separated signals. After adjusting the spatial arrangement of audio objects so as to each user prefers it, audio/visual rendering is conducted. We constructed two types of demonstration systems for music staging AI. In the smartphone-based implementation, each user enables to change the spatial arrangement through sliderbar dragging. Since information of user preferable spatial arrangement can be sent from each smartphone to server, it would enable to predict/recommend the user preferable spatial arrangement. In another implementation, head mount display (HMD) was utilized to dive into virtual music live performance. Each user enables to walk/teleport anywhere and audio is then changing corresponding to the user view.

DOI： 10.1109/ICASSP.2017.8005294

Scopus