記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：IEEE  

Dialogue robots powered by large language models can generate advanced utterances. However, the interaction performance is not yet perfect, and the robots sometimes cause problems during interactions. In this study, to detect anomalies in human-robot interactions, we created a dataset and constructed anomaly detection models. For the dataset creation, we collected videos of human-robot interactions in a framework where humans intervene when a dialogue breakdown occurs and labeled the scenes where humans intervened as anomalies. Using this dataset, we built classification models and deep metric learning models utilizing encoders for video, audio, and multimodal information. The results showed that we could successfully train the models and achieve an accuracy and F1-score of over 80%. The performance of the deep metric learning models surpassed that of the classification models, thus demonstrating the importance of separating the differences between classes. We also clarified the importance of audio information.

DOI： 10.1109/ro-man60168.2024.10731184

researchmap

担当区分：責任著者   記述言語：英語   掲載種別：研究論文（学術雑誌）   出版者・発行元：Springer Science and Business Media LLC  

The demand for therapeutic robots to alleviate mental health problems is growing. Studies have shown that people’s mental health improves when they are touched. Consequently, therapeutic robots are designed to alleviate stress through robot’s autonomous touch. However, robot’s autonomous touch can sometimes cause discomfort to recipients. This paper proposes a semi-autonomous touch method that merges robot’s autonomous touch with user-operated touch to mitigate discomfort while maintaining the sensation of being touched by another person. We conducted an experiment involving 24 participants who were touched on the neck by robots under three conditions: robot’s autonomous touch, user-operated touch, and the proposed semi-autonomous touch method condition. Additionally, the study investigated participants’ impressions of the robot in each condition. The results showed that semi-autonomous touch condition mitigated discomfort more effectively compared with the robot’s autonomous touch method condition. It also enhanced the feeling of being touched by another person entity and suppressed interaction boredom compared with the user-operated touch method condition. Participants reported higher trustworthiness and perceived friendliness in robots utilizing the semi-autonomous touch method compared to those with autonomous touch method condition. These findings indicate that robots featuring the proposed semi-autonomous touch method can provide a comforting experience, leveraging the therapeutic benefits of being touched by another person, and underscore their potential in mental health applications.

DOI： 10.1038/s41598-024-67964-0

researchmap

担当区分：筆頭著者   記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：IEEE  

Interactive systems that enable people to easily access mental health care in their daily lives are being studied. Self-instruction training, which is a kind of cognitive-behavioral therapy, is expected to be effective for general-purpose mental health care, but the difficulty for users to create self-instructional texts by themselves has hindered its widespread use. Furthermore, it is not sufficient to simply use a computer-assisted method for the training in which the computer automatically generates the self-instructional texts. This is because previous studies have shown that users do not feel self-instructional texts that are not created by themselves are their thoughts, and consequently cannot internalize the content of the texts, reducing the effectiveness of the training. In this study, we focused on the interface design that facilitates users to internalize automatically generated self-instructional texts, toward the realization of a self-instructional training support system. We proposed an interface that shows the real-time captured video of a user's face like a mirror and superimposes a speech bubble containing a self-instructional text to be connected to the user's face and confirmed the interface facilitates the user's internalization of the text through online experiments. The contribution of this paper is to provide a novel interface design for computer-assisted self-instructional training.

DOI： 10.1109/compsac61105.2024.00334

researchmap

その他リンク： https://hdl.handle.net/2237/0002011511

担当区分：筆頭著者   記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：ACM  

While research and applications of operated social robots that can be used as operators’ avatars have been conducted, a para-operated social robot has also been proposed, in which the robot and its operator exist in the same space, and the operator, robot, and interlocutor can interact with each other. From the findings of human-human-robot interaction research, it is known that the robot’s behavior influences the interpersonal relationships of the people involved in the dialog. However, there is no knowledge of how the operator is perceived by the interlocutor when the operations by the operator are unveiled. This paper reports preliminary results from two online surveys conducted to investigate the following question considering the robot-operator distance; Does the impression of the operator improve in unveiled para-operation when the operator operates the robot and has it say favorable utterances? The results of the two video surveys conducted at different distances between the robot and the operator show that the impression of the operator was improved by the robot’s favorable humorous utterances when the distance was close, but not when the distance was further apart. We consider that the findings contribute to the potential application of unveiled para-operated robots and suggest the importance of considering the distance factor in the influence of robots on human-human relationships.

DOI： 10.1145/3623809.3623924

researchmap

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：ACM  

In recent years, a framework of parallel conversations has been proposed to facilitate efficient conversations through cooperation between humans and dialogue systems. This approach aims to enable simultaneous conversations with multiple users by enabling the system to handle basic conversation and human operators to intervene when problems arise in the system’s conversation. Previous studies on parallel conversations have primarily focused on delegating simple exchanges such as greetings and acknowledgments to the system, with humans taking over for more complex interactions like providing guidance. Recent advancements in large language models may change this situation, enabling dialogue systems to engage in more advanced interactions. In this study, to examine which interventions will be made when large language models are utilized, we placed six dialogue robots based on large language models in an actual facility and conducted a field experiment involving parallel conversations for about a month. Our analysis of the collected data on dialogues and interventions showed that the most frequent interventions were made for supporting interactions when the system failed to react to the user utterances, indicating the limitations of using large language models alone and clarifying our next steps for facilitating smoother parallel conversations.

DOI： 10.1145/3623809.3623863

researchmap

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：ACM  

Dialogue systems are widely utilized in chatbots and call centers. However, it is often difficult for such systems to deliver fully autonomous dialogue. For users to have a better dialogue experience, a framework for human-system collaborative dialogue is proposed in which a human operator takes over the dialogue when needed, engaging in conversation with the user instead of the system (we call this process intervention). Operators join the dialogue in the middle; therefore, it is believed that dialogue summarization can be helpful for interventions. However, it is currently unclear whether dialogue summarization is actually useful. Therefore, in this study, we aim to investigate the usefulness of dialogue summaries for interventions through a field experiment conducted at an actual facility combining an aquarium and a zoo. The results of the field experiment revealed that dialogue summaries were more useful for intervention than dialogue history. Furthermore, we found no differences in the word categories included in the operator utterances during interventions irrespective of whether the dialogue history or dialogue format summary was presented to the operators, suggesting that dialogue format summary has content similar to that of dialogue history but improves the usefulness in intervention.

DOI： 10.1145/3623809.3623889

researchmap

担当区分：筆頭著者,　責任著者   記述言語：英語   掲載種別：研究論文（学術雑誌）   出版者・発行元：Nature Portfolio  

Studies on social robots that can communicate with humans are increasingly important. In particular, semi-autonomous robots have shown potential for practical applications in which robot autonomy and human teleoperation are jointly used to accomplish difficult tasks. However, it is unknown how the attitude represented in the autonomous behavior of the robots affects teleoperators. Previous studies reported that when humans play a particular role, their attitudes align with that role. The teleoperators of semi-autonomous robots also play the role given to the robots and may assimilate their autonomous expression. We hypothesized that the attitude of teleoperators may align with that of robots through teleoperation. To verify this, we conducted an experiment with conditions under which a participant operated a part of the body of an android robot that autonomously expressed a preferential attitude toward a painting and a condition under which they did not. Experimental results demonstrated that the preferential attitude of participants who teleoperated the android aligned statistically significantly more with that of the robot in comparison to those who did not teleoperate it, thereby supporting our hypothesis. This finding is novel regarding attitude change in teleoperators of semi-autonomous robots and can support the implementation of effective human-robot collaboration systems.

DOI： 10.1038/s41598-022-13829-3

researchmap

担当区分：筆頭著者,　責任著者   記述言語：日本語   掲載種別：研究論文（学術雑誌）   出版者・発行元：ヒューマンインタフェース学会  

While the potential of adopting robots interacting with humans in stores was shown in the previous studies, however robots that can work with people in stores and provide services have not taken root in society yet. In this paper, we realized an android robot which can help the duties of the salespersons at a boutique as well as provide mental support by cooperating with them. We considered that it is important for such robots to not only serve the customers but also support the salespersons. Therefore, we first interviewed salespersons at a boutique to find out what they expect from the robot. The result of the survey showed that they mainly expect three points from the robot: 1) reducing the workload of salesperson, 2) providing mental support for salesperson, and 3) assisting salesperson to establish a good relationship with the customer. To deal with these issues, we implemented an onsite-operated android. Through an 11-day field experiment in a boutique, in which the salespersons used the android, it was observed that the android could contribute to the salespersons in some situation of the work, and more possibilities for the android to contribute to the salespersons in some different aspects was investigated.

DOI： 10.11184/his.22.3_275

researchmap

担当区分：筆頭著者,　責任著者   記述言語：日本語   掲載種別：研究論文（学術雑誌）   出版者・発行元：一般社団法人 人工知能学会  

In this study, we developed and evaluated a dialogue system which enables an android robot to have a chat with users on Niconico Live provided by Dwango Co., Ltd. which is a live streaming service. In Niconico Live, broadcasters can talk to users who write comments displayed on the video stream. Therefore, by using Niconico Live chat, we eliminated speech recognition errors which can occur during speech conversations. In addition, the dialogue system can keep consistency of conversation by selecting the comment to which it can correctly respond because many comments are shown simultaneously on the video stream. The dialogue system was designed as a retrieval-based one which finds the appropriate response to user's utterance from a dialogue corpus. Therefore, we collected a dialogue corpus containing 4,460 pairs of comments and robot responses by teleoperating the android robot talking with users, as a first step. In the next step, we completed the dialogue system on Niconico Live integrating the dialogue corpus into it. To evaluate the performance of the dialogue system, we recorded the conversation between the android and users while running the designed system. After that, we showed the recorded conversation to evaluators and asked them how they feel about the naturalness and consistency of the conversation. Results of the experiment indicate that Niconico Live users perceived the responses of the dialogue system to be natural and found the chat with the android entertaining. Through this study, we demonstrated the applicability of the dialogue system on Niconico Live. However, it is difficult to discuss its effectiveness when applying it to other situations or other communication media such as a humanoid robot or a virtual agent. Therefore, as a future work, conducting a comparative experiment might lead to better understanding of the effectiveness of the dialogue system for androids.

DOI： 10.1527/tjsai.DSH-G

researchmap

開催年月日： 2024年3月

記述言語：日本語   会議種別：ポスター発表  

開催地：静岡大学 浜松キャンパス   国名：日本国  

本研究では，人とロボットのインタラクションにおける異常検出を目的とし，データセットの作成および異常検出モデルの構築を行った．具体的には，人とロボットのインタラクションに問題が生じた際に人間が介入する枠組みにおいて収集されたインタラクション映像に対して，人間が介入したシーンを異常と見做すことでデータセットを作成した．また，異常検出モデルは，既存の動画・音声分類モデルのファインチューニングを行うことで構築した．一般的なクラス分類手法に加えて，深層距離学習を用いたモデルを構築し，その有効性を検証した．

researchmap

開催年月日： 2024年3月

記述言語：日本語   会議種別：口頭発表（一般）  

開催地：静岡大学 浜松キャンパス   国名：日本国  

セラピーロボットが自律して人に接触することで，人のストレスを緩和することが期待される。しかし，ロボットからの自律した接触は人に不快感を与える可能性がある。そこで，本研究ではロボットからの自律した接触とユーザの操作に従った接触を融合させる半自律接触手法を提案した。実験により本手法によって，ロボットに触られている感覚を維持したまま，自律した接触と比べてユーザの不快感を低減でき，操作に従った接触と比べて飽きやすさを低減できることを示した。

researchmap

開催年月日： 2023年9月

記述言語：日本語   会議種別：ポスター発表  

開催地：公立はこだて未来大学   国名：日本国  

本研究では，異なる基本感情を視聴覚で組み合わせることで高次感情が知覚されるか，文脈に応じて異なる高次感情が知覚されるかについて，アンドロイドロボットを用いて検討した．対話中にアンドロイドが表現した感情を評定する実験の結果，恐怖の身体表現と悲しみ声の組み合わせから高次感情「恥」が知覚された．一方，文脈の違いで読み取られる高次感情に違いはみられないことが示された．またこの結果からアンドロイドロボットが高次感情をも表現できることも示唆された．

researchmap

開催年月日： 2023年8月

記述言語：日本語   会議種別：口頭発表（一般）  

開催地：豊橋技術科学大学   国名：日本国  

researchmap

開催年月日： 2023年6月

記述言語：日本語   会議種別：口頭発表（一般）  

開催地：熊本城ホール＋オンライン   国名：日本国  

自律対話エージェントは、コミュニティを構成する複数人の対話者と対話する一対多対話状況における応用として、バーチャルYouTuberのようにエンターテイメントを提供したり、授業や講演を行うことが期待できる。これまで、エージェントが一対一対話において、対話者の特性に適応した対話を行う効果は示されている。しかし、コミュニティ構成員らとの一対多対話において、コミュニティの特性に適応した対話能力をもつエージェントを実現する手法は確立されていない。そこで本研究では、課題解決に向けて、コミュニティ構成員のうち複数人がエージェントを操作し、エージェントを介して他の構成員らと対話することで、構成員同士の対話事例を収集し、その事例を用いてエージェントの自律対話機能を構築する手法を提案した。本手法の妥当性を検証するために、画面上のエージェントと複数の対話者が、ブラウザ上で対話できるインタフェースを開発し、実験を行った。その結果、コミュニティの特性を反映した対話事例を収集でき、コミュニティの特性に一定程度適応した対話が可能なエージェントを実現できることを示した。

researchmap

その他リンク： https://www.jstage.jst.go.jp/article/pjsai/JSAI2023/0/JSAI2023_2K4GS903/_article/-char/ja

開催年月日： 2023年6月

記述言語：日本語   会議種別：口頭発表（一般）  

開催地：熊本城ホール＋オンライン   国名：日本国  

ロボットの自律機能または人による操作の一方だけでは遂行が難しいタスクに対して、両者が協働してタスクの達成を目指す半自律ロボットが注目されている。しかし、半自律ロボットを操作する人が、協働を通じてロボットをどのような存在だと認識するかについては不明な点が多く、このことは実世界で両者が協働する際に問題になると考える。例えば、操作者がロボットを単なる道具のような存在あるいは人らしい相棒のような存在として認識するのかは、操作者がタスクの成功や失敗の結果を自分とロボットのどちらに帰属するかという問題に影響する可能性がある。そこで本研究ではまず、操作者が半自律ロボットをどのような存在だと認識するのかを理解するため、ロボットに動作指令を出した際に、操作者がロボットに対して操作感（道具らしさ）あるいは指示感（人らしさ）を感じるのかを評価するための心理尺度を作成した。さらに、ロボットへの動作指令の抽象度の違いが、操作者にとっての操作感と指示感が変化する要因の一つであると考え、この点を検証するための実験を行うことで、抽象度が低い場合は操作感が、高い場合は指示感がそれぞれ喚起される可能性を検討した。

researchmap

その他リンク： https://www.jstage.jst.go.jp/article/pjsai/JSAI2023/0/JSAI2023_3I5OS4c04/_article/-char/ja

開催年月日： 2019年1月

記述言語：英語   会議種別：ポスター発表  

開催地：大阪   国名：日本国  

researchmap

開催年月日： 2017年5月

記述言語：日本語   会議種別：口頭発表（一般）  

開催地：愛知県名古屋市 ウインクあいち   国名：日本国  

本研究では、株式会社ドワンゴが提供するニコニコ生放送サービス上で、人と雑談対話エンジンが協調してアンドロイドを操作し、雑談対話コーパスを収集しながら雑談対話エンジンの漸進的な自律性の獲得を目指すシステムを提案する。ニコニコ生放送では配信された映像を不特定多数のユーザが視聴するため、効率的な対話コーパス収集が可能である。本稿では上記手法を実装・運用し、雑談対話エンジンの一定の自律性の獲得を確認した。

researchmap

その他リンク： https://www.jstage.jst.go.jp/article/pjsai/JSAI2017/0/JSAI2017_2N23/_article/-char/ja/