Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

Cybernetic avatars (CAs), robots or CG agents operated remotely by humans, are increasingly used for remote communication, offering potential for applications such as telework. However, the usability of these systems, particularly regarding the operator’s sense of agency (SoA) and sense of ownership (SoO), remains underexplored. We conducted an experiment using a commercially employed CA system. The results showed that operators can experience SoA and SoO over on-screen CG avatars, supporting the system’s basic usability. Additionally, the results suggest that limiting operator hand movements might better maintain SoA in environments with a delay in the reflection of operators’ movement on CAs. These findings have preliminary implications for CA system usage, highlighting the need to consider trade-offs between operation and system responsiveness.

DOI： 10.1145/3687272.3690882

researchmap

Other Link： https://hdl.handle.net/2237/0002011729

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

Teleoperated social robots have been widely studied and employed, yet their operation interfaces provide limited information to the operator, causing difficulties in grasping the dialogue situation compared to face-to-face interactions. Particularly with conventional interfaces, operators face challenges with vocal adaptation due to the inability to assess the distance between the robot and the interlocutor. Consequently, operators struggle to determine the appropriate volume level for clear communication, leading to insecurity about their vocalization’s effectiveness. In this study, we propose a novel approach to address this issue: a voice volume gauge that enables operators to adjust their voice volume by displaying the difference between their current volume and the estimated optimal volume on the interface. We implemented the gauge and conducted a subjective evaluation experiment, which demonstrated no usability concerns for operators and a reduction in their insecurity levels. Furthermore, preliminary objective evaluation results based on speech analysis suggest that utilizing the gauge may bring operators’ vocalizations closer to those in face-to-face dialogues. The contribution of this paper lies in providing a simple interface design method to aid in solving the vocal adaptation problem for teleoperated robot applications.

DOI： 10.1145/3687272.3688313

researchmap

Other Link： https://hdl.handle.net/2237/0002011728

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

Various customer service robots’ teleoperation interfaces (I/Fs) have been developed for human-robot collaboration. However, a previous study has indicated a lack of operator enjoyment. This study aims to design an I/F that elicits operator enjoyment and identifies the factors that contribute to this enjoyment. We developed two I/Fs: a high degree of freedom I/F and a gradual flexibility degree of freedom I/F based on gamification to elicit operator enjoyment. Using these I/Fs, we conducted a field study in a real shopping mall to investigate the operators’ experiences. As a result, the operators in this experiment greatly enjoyed using our I/Fs. The results showed that our I/Fs could elicit operator enjoyment with the following three I/F factors suggested as potentially important design guidelines: ease of use through anonymity, moderate restriction of freedom, and sharing experiences with a group of operators.

DOI： 10.1145/3687272.3690883

researchmap

Other Link： https://hdl.handle.net/2237/0002011730

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

This study investigates the effects of both operator and avatar gender on the perceived ease of communication and creepiness of the avatars. Utilizing a mixed-methods 2 by 2 approach (male operator, female operator; male avatar, female avatar), we conducted an experiment involving 148 participants. The results indicate significant variations in perceived avatar impressions influenced by the gender alignment between the operator and the avatar. Specifically, female-operator-female-avatar group rated as the easiest to talk to, most interesting and least in the creepiness metric. These results suggest that for low-friction conversational encounters using female-operator-female-avatars might be most suitable.

DOI： 10.1145/3687272.3690891

researchmap

Other Link： https://hdl.handle.net/2237/0002011731

Authorship：Lead author,　Corresponding author   Language：Japanese   Publishing type：Research paper (scientific journal)   Publisher：Japan Society for Fuzzy Theory and Intelligent Informatics  

As research and applications of operated social robots continue to advance, a para-operated social robot has also been proposed, in which the robot and its operator exist in the same space, and the robot, operator, and interlocutor can interact face-to-face. Contributing to the promotion of human-human interaction by such robots, we considered that the operator could improve their interpersonal impression by making the robot say favorable utterances through operation during the dialog. This paper reports the results from the online video survey conducted to investigate the following hypothesis considering the robot-operator distance; Operator’s interpersonal impression is improved by making the robot say favorable utterances through operating. The results of the surveys conducted at different distances between the robot and the operator show that the impression of the operator was improved by the robot’s favorable humorous utterances when they are in close, but not when they are far apart. We consider that the findings contribute to the potential of dialog promotion by para-operated robots and suggest the importance of focusing on the positional relationship factor in the influence of robots on human-human relationships.

DOI： 10.3156/jsoft.36.4_695

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Dialogue robots powered by large language models can generate advanced utterances. However, the interaction performance is not yet perfect, and the robots sometimes cause problems during interactions. In this study, to detect anomalies in human-robot interactions, we created a dataset and constructed anomaly detection models. For the dataset creation, we collected videos of human-robot interactions in a framework where humans intervene when a dialogue breakdown occurs and labeled the scenes where humans intervened as anomalies. Using this dataset, we built classification models and deep metric learning models utilizing encoders for video, audio, and multimodal information. The results showed that we could successfully train the models and achieve an accuracy and F1-score of over 80%. The performance of the deep metric learning models surpassed that of the classification models, thus demonstrating the importance of separating the differences between classes. We also clarified the importance of audio information.

DOI： 10.1109/ro-man60168.2024.10731184

researchmap

Authorship：Corresponding author   Language：English   Publishing type：Research paper (scientific journal)   Publisher：Springer Science and Business Media LLC  

The demand for therapeutic robots to alleviate mental health problems is growing. Studies have shown that people’s mental health improves when they are touched. Consequently, therapeutic robots are designed to alleviate stress through robot’s autonomous touch. However, robot’s autonomous touch can sometimes cause discomfort to recipients. This paper proposes a semi-autonomous touch method that merges robot’s autonomous touch with user-operated touch to mitigate discomfort while maintaining the sensation of being touched by another person. We conducted an experiment involving 24 participants who were touched on the neck by robots under three conditions: robot’s autonomous touch, user-operated touch, and the proposed semi-autonomous touch method condition. Additionally, the study investigated participants’ impressions of the robot in each condition. The results showed that semi-autonomous touch condition mitigated discomfort more effectively compared with the robot’s autonomous touch method condition. It also enhanced the feeling of being touched by another person entity and suppressed interaction boredom compared with the user-operated touch method condition. Participants reported higher trustworthiness and perceived friendliness in robots utilizing the semi-autonomous touch method compared to those with autonomous touch method condition. These findings indicate that robots featuring the proposed semi-autonomous touch method can provide a comforting experience, leveraging the therapeutic benefits of being touched by another person, and underscore their potential in mental health applications.

DOI： 10.1038/s41598-024-67964-0

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Interactive systems that enable people to easily access mental health care in their daily lives are being studied. Self-instruction training, which is a kind of cognitive-behavioral therapy, is expected to be effective for general-purpose mental health care, but the difficulty for users to create self-instructional texts by themselves has hindered its widespread use. Furthermore, it is not sufficient to simply use a computer-assisted method for the training in which the computer automatically generates the self-instructional texts. This is because previous studies have shown that users do not feel self-instructional texts that are not created by themselves are their thoughts, and consequently cannot internalize the content of the texts, reducing the effectiveness of the training. In this study, we focused on the interface design that facilitates users to internalize automatically generated self-instructional texts, toward the realization of a self-instructional training support system. We proposed an interface that shows the real-time captured video of a user's face like a mirror and superimposes a speech bubble containing a self-instructional text to be connected to the user's face and confirmed the interface facilitates the user's internalization of the text through online experiments. The contribution of this paper is to provide a novel interface design for computer-assisted self-instructional training.

DOI： 10.1109/compsac61105.2024.00334

researchmap

Other Link： https://hdl.handle.net/2237/0002011511

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

In recent years, a framework of parallel conversations has been proposed to facilitate efficient conversations through cooperation between humans and dialogue systems. This approach aims to enable simultaneous conversations with multiple users by enabling the system to handle basic conversation and human operators to intervene when problems arise in the system’s conversation. Previous studies on parallel conversations have primarily focused on delegating simple exchanges such as greetings and acknowledgments to the system, with humans taking over for more complex interactions like providing guidance. Recent advancements in large language models may change this situation, enabling dialogue systems to engage in more advanced interactions. In this study, to examine which interventions will be made when large language models are utilized, we placed six dialogue robots based on large language models in an actual facility and conducted a field experiment involving parallel conversations for about a month. Our analysis of the collected data on dialogues and interventions showed that the most frequent interventions were made for supporting interactions when the system failed to react to the user utterances, indicating the limitations of using large language models alone and clarifying our next steps for facilitating smoother parallel conversations.

DOI： 10.1145/3623809.3623863

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

Dialogue systems are widely utilized in chatbots and call centers. However, it is often difficult for such systems to deliver fully autonomous dialogue. For users to have a better dialogue experience, a framework for human-system collaborative dialogue is proposed in which a human operator takes over the dialogue when needed, engaging in conversation with the user instead of the system (we call this process intervention). Operators join the dialogue in the middle; therefore, it is believed that dialogue summarization can be helpful for interventions. However, it is currently unclear whether dialogue summarization is actually useful. Therefore, in this study, we aim to investigate the usefulness of dialogue summaries for interventions through a field experiment conducted at an actual facility combining an aquarium and a zoo. The results of the field experiment revealed that dialogue summaries were more useful for intervention than dialogue history. Furthermore, we found no differences in the word categories included in the operator utterances during interventions irrespective of whether the dialogue history or dialogue format summary was presented to the operators, suggesting that dialogue format summary has content similar to that of dialogue history but improves the usefulness in intervention.

DOI： 10.1145/3623809.3623889

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

While research and applications of operated social robots that can be used as operators’ avatars have been conducted, a para-operated social robot has also been proposed, in which the robot and its operator exist in the same space, and the operator, robot, and interlocutor can interact with each other. From the findings of human-human-robot interaction research, it is known that the robot’s behavior influences the interpersonal relationships of the people involved in the dialog. However, there is no knowledge of how the operator is perceived by the interlocutor when the operations by the operator are unveiled. This paper reports preliminary results from two online surveys conducted to investigate the following question considering the robot-operator distance; Does the impression of the operator improve in unveiled para-operation when the operator operates the robot and has it say favorable utterances? The results of the two video surveys conducted at different distances between the robot and the operator show that the impression of the operator was improved by the robot’s favorable humorous utterances when the distance was close, but not when the distance was further apart. We consider that the findings contribute to the potential application of unveiled para-operated robots and suggest the importance of considering the distance factor in the influence of robots on human-human relationships.

DOI： 10.1145/3623809.3623924

researchmap

Authorship：Lead author,　Corresponding author   Language：English   Publishing type：Research paper (scientific journal)   Publisher：Nature Portfolio  

Studies on social robots that can communicate with humans are increasingly important. In particular, semi-autonomous robots have shown potential for practical applications in which robot autonomy and human teleoperation are jointly used to accomplish difficult tasks. However, it is unknown how the attitude represented in the autonomous behavior of the robots affects teleoperators. Previous studies reported that when humans play a particular role, their attitudes align with that role. The teleoperators of semi-autonomous robots also play the role given to the robots and may assimilate their autonomous expression. We hypothesized that the attitude of teleoperators may align with that of robots through teleoperation. To verify this, we conducted an experiment with conditions under which a participant operated a part of the body of an android robot that autonomously expressed a preferential attitude toward a painting and a condition under which they did not. Experimental results demonstrated that the preferential attitude of participants who teleoperated the android aligned statistically significantly more with that of the robot in comparison to those who did not teleoperate it, thereby supporting our hypothesis. This finding is novel regarding attitude change in teleoperators of semi-autonomous robots and can support the implementation of effective human-robot collaboration systems.

DOI： 10.1038/s41598-022-13829-3

researchmap

Authorship：Lead author,　Corresponding author   Language：Japanese   Publishing type：Research paper (scientific journal)   Publisher：Human Interface Society  

While the potential of adopting robots interacting with humans in stores was shown in the previous studies, however robots that can work with people in stores and provide services have not taken root in society yet. In this paper, we realized an android robot which can help the duties of the salespersons at a boutique as well as provide mental support by cooperating with them. We considered that it is important for such robots to not only serve the customers but also support the salespersons. Therefore, we first interviewed salespersons at a boutique to find out what they expect from the robot. The result of the survey showed that they mainly expect three points from the robot: 1) reducing the workload of salesperson, 2) providing mental support for salesperson, and 3) assisting salesperson to establish a good relationship with the customer. To deal with these issues, we implemented an onsite-operated android. Through an 11-day field experiment in a boutique, in which the salespersons used the android, it was observed that the android could contribute to the salespersons in some situation of the work, and more possibilities for the android to contribute to the salespersons in some different aspects was investigated.

DOI： 10.11184/his.22.3_275

CiNii Research

researchmap

Authorship：Lead author,　Corresponding author   Language：Japanese   Publishing type：Research paper (scientific journal)   Publisher：The Japanese Society for Artificial Intelligence  

In this study, we developed and evaluated a dialogue system which enables an android robot to have a chat with users on Niconico Live provided by Dwango Co., Ltd. which is a live streaming service. In Niconico Live, broadcasters can talk to users who write comments displayed on the video stream. Therefore, by using Niconico Live chat, we eliminated speech recognition errors which can occur during speech conversations. In addition, the dialogue system can keep consistency of conversation by selecting the comment to which it can correctly respond because many comments are shown simultaneously on the video stream. The dialogue system was designed as a retrieval-based one which finds the appropriate response to user's utterance from a dialogue corpus. Therefore, we collected a dialogue corpus containing 4,460 pairs of comments and robot responses by teleoperating the android robot talking with users, as a first step. In the next step, we completed the dialogue system on Niconico Live integrating the dialogue corpus into it. To evaluate the performance of the dialogue system, we recorded the conversation between the android and users while running the designed system. After that, we showed the recorded conversation to evaluators and asked them how they feel about the naturalness and consistency of the conversation. Results of the experiment indicate that Niconico Live users perceived the responses of the dialogue system to be natural and found the chat with the android entertaining. Through this study, we demonstrated the applicability of the dialogue system on Niconico Live. However, it is difficult to discuss its effectiveness when applying it to other situations or other communication media such as a humanoid robot or a virtual agent. Therefore, as a future work, conducting a comparative experiment might lead to better understanding of the effectiveness of the dialogue system for androids.

DOI： 10.1527/tjsai.DSH-G

CiNii Research

researchmap

Event date： 2025.2 - 2025.3

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Kindai University   Country：Japan  

researchmap

Event date： 2025.2 - 2025.3

Language：Japanese   Presentation type：Poster presentation  

Venue：Kindai University   Country：Japan  

researchmap

Event date： 2024.9

Language：English   Presentation type：Poster presentation  

Venue：Kyoto University   Country：Japan  

researchmap

Event date： 2024.9

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Hokkaido Information University   Country：Japan  

researchmap

Event date： 2024.5

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：ACT CITY HAMAMATSU + online   Country：Japan  

A semi-autonomous robot is being researched and applied, in which the robot's autonomy and human operation cooperate to accomplish practical tasks. Because such robots are both autonomous and operability, when an operator operates the robot, they may feel as if they are operating a tool that serves as their other self (sense of operation), while at the same time, they may feel as if they are giving direction to another person, the agent (sense of direction). However, the factors to make a difference in how the operator feels senses of operation and direction are not clear, and there is a lack of understanding of how this difference affects cooperation with the robot. In this study, we clarified whether an operator feels a sense of operation or direction changes depending on the degrees of abstractness of the operation method, and that which sense is preferred depends on an operator and task content. Therefore, the results of this paper suggest the importance of interaction design between an operator and the robot, which focuses on how an operator feels when operating the robot, for the application of semi-autonomous robots.

researchmap

Other Link： https://www.jstage.jst.go.jp/article/pjsai/JSAI2024/0/JSAI2024_3T5OS6b02/_article/-char/ja

Event date： 2024.3

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Kobe International Conference Center   Country：Japan  

researchmap

Event date： 2024.3

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Shizuoka University   Country：Japan  

researchmap

Event date： 2024.3

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Shizuoka University   Country：Japan  

researchmap

Event date： 2024.3

Language：Japanese   Presentation type：Poster presentation  

Venue：Shizuoka University   Country：Japan  

researchmap

Event date： 2023.9

Language：Japanese   Presentation type：Poster presentation  

Venue：Future University Hakodate   Country：Japan  

researchmap

Event date： 2023.8

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Toyohashi University of Technology   Country：Japan  

researchmap

Event date： 2023.6

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：KUMAMOTO-JO HALL + Online   Country：Japan  

An autonomous dialogue agent is expected to engage in live chat like Virtual YouTubers or to give classes and lectures in application of one-to-many dialogue where the agent interacts with multiple interlocutors forming a community. In one-to-one dialogue, the effectiveness has been shown that an agent interacts with its interlocutor adapting to their characteristics. However, in one-to-many dialogue with community members, no method has been established to realize an agent with the dialogue ability adapted to the characteristics of the community. In this study, we proposed a two-step method to build such an agent: (1) collecting dialogue examples from chat in which community members operate an agent to interact with the other members, (2) developing the agent by using these examples. We conducted experiments to verify this method with our developed system enabling dialogue examples collection and agent implementation. The results suggested that it is possible to collect dialogue examples that reflect the characteristics of a community and to realize an agent capable of dialogue adapted to the characteristics.

researchmap

Other Link： https://www.jstage.jst.go.jp/article/pjsai/JSAI2023/0/JSAI2023_2K4GS903/_article/-char/ja

Event date： 2023.6

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：KUMAMOTO-JO HALL + Online   Country：Japan  

A semi-autonomous robot is becoming important because it can accomplish practical tasks through collaboration between the robot’s autonomy and human operation. However, it is still unclear how a semi-autonomous robot is recognized by its operator who collaborates with it, and understanding this point is critical for application of the robot.For example, the attribution of responsibility for task failure or success may be affected depending on whether an operator treats the robot as a sidekick or a mere tool.In this study, we developed a psychological scale to measure whether an operator feels a sense of operation (tool-likeness) or direction (human-likeness) when they command it. Then, we conducted the experiment to investigate the hypothesis that the degrees of abstractness of the command affect the operator's recognition of a semi-autonomous robot. The results showed that high and low abstractness tended to evoke a sense of direction and operation, respectively.

researchmap

Other Link： https://www.jstage.jst.go.jp/article/pjsai/JSAI2023/0/JSAI2023_3I5OS4c04/_article/-char/ja

Event date： 2019.1

Language：English   Presentation type：Poster presentation  

Venue：Osaka   Country：Japan  

researchmap

Event date： 2017.5

Language：Japanese   Presentation type：Oral presentation (general)  

Country：Japan  

researchmap

Other Link： https://www.jstage.jst.go.jp/article/pjsai/JSAI2017/0/JSAI2017_2N23/_article/-char/ja/

Author：Other 

researchmap

Author：Other 

researchmap

Author：Other 

researchmap