研究者詳細 - 長谷川　泰久

Toward Autonomous Robotic Minimally Invasive Surgery: A Hybrid Framework Combining Task-Motion Planning and Dynamic Behavior Trees 査読有り国際誌

Fozilov K., Colan J., Sekiyama K., Hasegawa Y.

IEEE Access 11 巻頁： 91206 - 91224 2023年8月

担当区分：最終著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

The growing need for high levels of autonomy in Autonomous Robotic Surgery Systems (ARSS) calls for innovative approaches to reduce surgeons' cognitive load, optimize hospital workflows, and ensure efficient task-level reasoning and adaptation during execution. This paper presents a novel hybrid framework that synergistically combines Task-Motion Planning and Dynamic Behavior Trees for ARSS in Minimally Invasive Surgery. Our approach is designed to address the challenges of coordinating multiple surgical tools within a small workspace, thereby making complex surgical tasks like multi-throw suturing feasible and efficient. Through an extensive evaluation in simulation across diverse initial conditions and noise scenarios, the proposed method demonstrates improved success rates, reduced execution times, and fewer regrasps compared to standalone approaches. Furthermore, it showcases robustness under increased noise conditions. By applying our framework to a complex multi-throw suturing task, we illustrate its capability to seamlessly handle comprehensive suturing tasks, including needle picking, insertion, extraction, and the handover of the needle between Patient Side Manipulators. The results suggest that our hybrid approach not only enhances ARSS autonomy but also adapts effectively to unexpected environmental changes, laying the groundwork for its potential applicability in real-world surgical robotics.

DOI： 10.1109/ACCESS.2023.3308619

A Shared Control Framework for Enhanced Grasping Performance in Teleoperation 査読有り国際誌

Zhu Y., Jiang B., Chen Q., Aoyama T., Hasegawa Y.

IEEE Access 11 巻頁： 69204 - 69215 2023年7月

担当区分：最終著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Remote teleoperation has shown significant advancements since the first teleoperation system was proposed by Goertz in the 1940s. In recent years, the research on shared control methodologies in which the robot assists the operators in accomplishing the desired tasks has gained extensive attention. One such important task in teleoperation is object grasping. In this paper, we propose a shared control framework to enhance the teleoperated grasping performance. The proposed framework is built upon a virtual reality device-based direct teleoperation system. In this framework, a template matching-based object point cloud compensation is introduced for multi-angle grasping pose generation. Then, the feasible grasping candidates are selected considering joint constraints-aware manipulability. Finally, the grasping assistance is achieved by trajectory blending with dynamic authority adjustment. To validate the performance of the proposed framework, we carried out experimental evaluations. The output results indicate improved grasping performance in terms of reduced task completion time, linear trajectory, and workload.

DOI： 10.1109/ACCESS.2023.3292410

Natural Grasp Intention Recognition Based on Gaze in Human-Robot Interaction 国際共著国際誌

Yang B., Huang J., Chen X., Li X., Hasegawa Y.

IEEE Journal of Biomedical and Health Informatics 27 巻 ( 4 ) 頁： 2059 - 2070 2023年4月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Journal of Biomedical and Health Informatics

Objective: While neuroscience research has established a link between vision and intention, studies on gaze data features for intention recognition are absent. The majority of existing gaze-based intention recognition approaches are based on deliberate long-term fixation and suffer from insufficient accuracy. In order to address the lack of features and insufficient accuracy in previous studies, the primary objective of this study is to suppress noise from human gaze data and extract useful features for recognizing grasp intention. Methods: We conduct gaze movement evaluation experiments to investigate the characteristics of gaze motion. The target-attracted gaze movement model (TAGMM) is proposed as a quantitative description of gaze movement based on the findings. A Kalman filter (KF) is used to reduce the noise in the gaze data based on TAGMM. We conduct gaze-based natural grasp intention recognition evaluation experiments to collect the subject's gaze data. Four types of features describing gaze point dispersion (f_{var}), gaze point movement (f_{gm}), head movement (f_{hm}), and distance from the gaze points to objects (f_{d_{j}}) are then proposed to recognize the subject's grasp intentions. With the proposed features, we perform intention recognition experiments, employing various classifiers, and the results are compared with different methods. Results: The statistical analysis reveals that the proposed features differ significantly across intentions, offering the possibility of employing these features to recognize grasp intentions. We demonstrated the intention recognition performance utilizing the TAGMM and the proposed features in within-subject and cross-subject experiments. The results indicate that the proposed method can recognize the intention with accuracy improvements of 44.26% (within-subject) and 30.67% (cross-subject) over the fixation-based method. The proposed method also consumes less time (34.87 ms) to recognize the intention than the fixation-based method (about 1 s). Conclusion: This work introduces a novel TAGMM for modeling gaze movement and a variety of practical features for recognizing grasp intentions. Experiments confirm the effectiveness of our approach. Significance: The proposed TAGMM is capable of modeling gaze movements and can be utilized to process gaze data, and the proposed features can reveal the user's intentions. These results contribute to the development of gaze-based human-robot interaction.

DOI： 10.1109/JBHI.2023.3238406

細胞の変形度を想起させる力触覚提示が可能な布状アクチュエータを用いた微細操作インタフェース査読有り

齋藤澄和, 青山忠義, 舟洞佑記, 竹内大, 長谷川泰久

日本ロボット学会誌 41 巻 ( 3 ) 頁： 322 - 325 2023年4月

A Concurrent Framework for Constrained Inverse Kinematics of Minimally Invasive Surgical Robots Open Access

Colan J., Davila A., Fozilov K., Hasegawa Y.

Sensors 23 巻 ( 6 ) 頁： 3328 2023年3月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Sensors

Minimally invasive surgery has undergone significant advancements in recent years, transforming various surgical procedures by minimizing patient trauma, postoperative pain, and recovery time. However, the use of robotic systems in minimally invasive surgery introduces significant challenges related to the control of the robot’s motion and the accuracy of its movements. In particular, the inverse kinematics (IK) problem is critical for robot-assisted minimally invasive surgery (RMIS), where satisfying the remote center of motion (RCM) constraint is essential to prevent tissue damage at the incision point. Several IK strategies have been proposed for RMIS, including classical inverse Jacobian IK and optimization-based approaches. However, these methods have limitations and perform differently depending on the kinematic configuration. To address these challenges, we propose a novel concurrent IK framework that combines the strengths of both approaches and explicitly incorporates RCM constraints and joint limits into the optimization process. In this paper, we present the design and implementation of concurrent inverse kinematics solvers, as well as experimental validation in both simulation and real-world scenarios. Concurrent IK solvers outperform single-method solvers, achieving a 100% solve rate and reducing the IK solving time by up to 85% for an endoscope positioning task and 37% for a tool pose control task. In particular, the combination of an iterative inverse Jacobian method with a hierarchical quadratic programming method showed the highest average solve rate and lowest computation time in real-world experiments. Our results demonstrate that concurrent IK solving provides a novel and effective solution to the constrained IK problem in RMIS applications.

DOI： 10.3390/s23063328

OpenRST: An Open Platform for Customizable 3D Printed Cable-Driven Robotic Surgical Tools Open Access

Colan J., Davila A., Zhu Y., Aoyama T., Hasegawa Y.

IEEE Access 11 巻頁： 6092 - 6105 2023年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Robot-assisted minimally invasive surgery (RMIS) has been shown to be effective in improving surgeon capabilities, providing magnified 3D vision, highly dexterous surgical tools, and intuitive human-robot interfaces for high-precision tool motion control. Robotic surgical tools (RST) are a critical component that defines the performance of an RMIS system. Current RSTs still represent a high cost, with few commercially available options, which limits general access and research on RMIS. We aim to take advantage of recent progress in biocompatible 3D printing and contribute to the development of RMIS technologies, presenting an open platform for low-cost, biocompatible, and customizable RSTs. The proposed design concept consists of a 3-DOF end-effector with a decoupled wrist mechanism, a tool interface module, and a tool drive unit. We validated our end-effector design using Finite Element Analysis (FEA) to confirm that stress generated by high grip forces is maintained below the material yield stress. Validation experiments showed that the proposed RST could provide up to 10N grip forces and up to 3N pulling forces. The proposed control framework exhibited a mean absolute positioning tracking error of approximately 0.1 rad. Finally, we also demonstrated the use of the proposed RST in two surgical training tasks: pick-and-place and stitching. The designs and software control framework are open-access and freely available for customization and fast development at https://github.com/jcolan/OpenRST.

DOI： 10.1109/ACCESS.2023.3236821

Micromanipulation System Capable of Simultaneously Presenting High-Resolution and Large Field-of-View Images in Real-Time Open Access

Aoyama T., Takeno S., Yokoe K., Hano K., Takasu M., Takeuchi M., Hasegawa Y.

IEEE Access 11 巻頁： 34274 - 34285 2023年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Microinjection technology is widely used in biotechnological processes such as gene manipulation and microinsemination. Generally, microinjection is performed under an optical microscope while viewing the video of the targets in real-time. In the microinjection process, multiple oocytes must be placed in the same droplet, for which the injection must be performed multiple times. Furthermore, the target must be observed at different magnifications during injection and the oocyte transfer operation. Hence, the operators are required to change the magnification and light intensity repeatedly. These tasks are complicated and are a burden for the operators. Thus, the accuracy, reproducibility, and productivity of microinjection depend on the operator's skill level. In this study, to reduce the burden on the operator and simplify the operation of microinjection, we propose a micromanipulation system that enables both wide-range and high-resolution video presentation with free viewpoint selection. The proposed micromanipulation system is based on a view-expansion microscope system with simultaneous multi-view imaging using a galvanometer mirror and a high-speed vision system. We verify the effectiveness of the proposed system by evaluating the resolution of images presented to the operator and through the experiments involving moving microbeads performed by inexperienced subjects. The experimental results indicate that our micromanipulation system streamlines the task of micromanipulation by eliminating the need to change the objective lens.

DOI： 10.1109/ACCESS.2023.3264785

Improvement in the Manipulability of Remote Touch Screens Based on Peri-Personal Space Transfer 国際誌 Open Access

Yamamoto K., Zhu Y., Aoyama T., Takeuchi M., Hasegawa Y.

IEEE Access 11 巻頁： 43962 - 43974 2023年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Current remote-control interfaces are difficult to operate intuitively while viewing the entire remote display and require familiarity with the operation. The space within an individual's reach, called the peri-personal space (PPS), assists them in planning their physical movements. Hence, manipulating objects outside one's PPS is more difficult than doing so inside it. Therefore, this study aimed to develop an interface that creates the illusion of an operator's finger being transferred to a remote display and transfers the PPS to the virtual finger in the display. To transfer the PPS to a manipulated object, it is necessary to enhance the user's sense of agency and ownership toward the manipulated target by ensuring it is similar in shape and motion to the user's body. Long-term input testing confirmed that by transferring the PPS to a virtual finger, there is no significant difference, as suggested by a t-test between the coefficients of the learning curves outside and inside the PPS. Furthermore, when users learned to manipulate the virtual finger outside the PPS under conditions that impaired their sense of agency, even after two weeks of learning, the PPS was not continuously transferred to the virtual finger for more than seven minutes from the start of the virtual finger operation. The experimental results demonstrated that the interface enabled the display located outside the PPS to be operated with the same degree of operability as inside the PPS from the start of the operation, even with a short learning period.

DOI： 10.1109/ACCESS.2023.3271003

Gaze and Environmental Context-Guided Deep Neural Network and Sequential Decision Fusion for Grasp Intention Recognition 国際共著国際誌 Open Access

Yang B., Chen X., Xiao X., Yan P., Hasegawa Y., Huang J.

IEEE Transactions on Neural Systems and Rehabilitation Engineering 31 巻頁： 3687 - 3698 2023年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Transactions on Neural Systems and Rehabilitation Engineering

Grasp intention recognition plays a crucial role in controlling assistive robots to aid older people and individuals with limited mobility in restoring arm and hand function. Among the various modalities used for intention recognition, the eye-gaze movement has emerged as a promising approach due to its simplicity, intuitiveness, and effectiveness. Existing gaze-based approaches insufficiently integrate gaze data with environmental context and underuse temporal information, leading to inadequate intention recognition performance. The objective of this study is to eliminate the proposed deficiency and establish a gaze-based framework for object detection and its associated intention recognition. A novel gaze-based grasp intention recognition and sequential decision fusion framework (GIRSDF) is proposed. The GIRSDF comprises three main components: gaze attention map generation, the Gaze-YOLO grasp intention recognition model, and sequential decision fusion models (HMM, LSTM, and GRU). To evaluate the performance of GIRSDF, a dataset named Invisible containing data from healthy individuals and hemiplegic patients is established. GIRSDF is validated by trial-based and subject-based experiments on Invisible and outperforms the previous gaze-based grasp intention recognition methods. In terms of running efficiency, the proposed framework can run at a frequency of about 22 Hz, which ensures real-Time grasp intention recognition. This study is expected to inspire additional gaze-related grasp intention recognition works.

DOI： 10.1109/TNSRE.2023.3314503

Auditory Feedback for Enhanced Sense of Agency in Shared Control 査読有り

Tomoya Morita, Yaonan Zhu, Tadayoshi Aoyama, Masaru Takeuchi, Kento Yamamoto, Yasuhisa Hasegawa

Sensors 22 巻 ( 24 ) 頁： 9779 2022年12月

An immersive micro-manipulation system using real-time 3D imaging microscope and 3D operation interface for high-speed and accurate micro-manipulation 査読有り国際誌

Yokoe K., Aoyama T., Fujishiro T., Takeuchi M., Hasegawa Y.

ROBOMECH Journal 9 巻 ( 1 ) 頁： 16 2022年12月

担当区分：最終著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：ROBOMECH Journal

The use of intracytoplasmic sperm injection (ICSI), an assisted reproductive technique (ART), is increasing widely. ICSI is currently performed by specially skilled embryologists. However, with the increasing demand for ART, the shortage of skilled embryologists has become a problem. Therefore, we propose an immersive micromanipulation system that requires no special skills for efficient and accurate micromanipulation. Our proposed system is composed of a real-time three-dimensional (3D) imaging microscope and 3D operation interfaces. The 3D operation interfaces are stationary pen-type or wearable glove-type interfaces. In this system, an operator wearing a head-mounted display (HMD) and using 3D operation interfaces is immersed in a virtual micromanipulation space. The operator can move the pipettes by 3D operation interface and freely change the viewpoint. We verified that the proposed system improves the speed and accuracy of operating a pipette through two types of experiments with subjects.

DOI： 10.1186/s40648-022-00228-6

Auditory Feedback for Enhanced Sense of Agency in Shared Control 査読有り国際誌

Morita T., Zhu Y., Aoyama T., Takeuchi M., Yamamoto K., Hasegawa Y.

Sensors 22 巻 ( 24 ) 頁： 9779 - 9779 2022年12月

担当区分：最終著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Sensors

There is a growing need for robots that can be remotely controlled to perform tasks of one’s own choice. However, the SoA (Sense of Agency: the sense of recognizing that the motion of an observed object is caused by oneself) is reduced because the subject of the robot motion is identified as external due to shared control. To address this issue, we aimed to suppress the decline in SoA by presenting auditory feedback that aims to blur the distinction between self and others. We performed the tracking task in a virtual environment under four different auditory feedback conditions, with varying levels of automation to manipulate the virtual robot gripper. Experimental results showed that the proposed auditory feedback suppressed the decrease in the SoA at a medium level of automation. It is suggested that our proposed auditory feedback could blur the distinction between self and others, and that the operator attributes the subject of the motion of the manipulated object to himself.

DOI： 10.3390/s22249779

A Review on Tactile Displays for Conventional Laparoscopic Surgery 査読有り

Jacinto Colan, Ana Davila, Yasuhisa Hasegawa

Surgeries (Switzerland) 3 巻 ( 4 ) 頁： 334 - 346 2022年11月

Intelligent Gait Analysis and Evaluation System Based on Cane Robot 査読有り国際共著

Qingyang Yan, Jian Huang, Dongrui Wu, Zhaohui Yang, Yujie Wang, Yasuhisa Hasegawa, Toshio Fukuda

IEEE Transactions on Neural Systems and Rehabilitation Engineering 30 巻頁： 2916 - 2926 2022年10月

Development of High-Cell-Density Tissue Method for Compressed Modular Bioactuator 査読有り国際共著

Takuto Nomura, Masaru Takeuchi, Eunhye Kim, Qiang Huang, Yasuhisa Hasegawa, Toshio Fukuda

Micromachines 13 巻 ( 10 ) 頁： 1725 2022年10月

A Wirelessly Powered 4-Channel Neurostimulator for Reconstructing Walking Trajectory 査読有り

Takeuchi M., Tokutake K., Watanabe K., Ito N., Aoyama T., Saeki S., Kurimoto S., Hirata H., Hasegawa Y.

Sensors 22 巻 ( 19 ) 頁： 7198 2022年9月

担当区分：最終著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Sensors

A wirelessly powered four-channel neurostimulator was developed for applying selective Functional Electrical Stimulation (FES) to four peripheral nerves to control the ankle and knee joints of a rat. The power of the neurostimulator was wirelessly supplied from a transmitter device, and the four nerves were connected to the receiver device, which controlled the ankle and knee joints in the rat. The receiver device had functions to detect the frequency of the transmitter signal from the transmitter coil. The stimulation site of the nerves was selected according to the frequency of the transmitter signal. The rat toe position was controlled by changing the angles of the ankle and knee joints. The joint angles were controlled by the stimulation current applied to each nerve independently. The stimulation currents were adjusted by the Proportional Integral Differential (PID) and feed-forward control method through a visual feedback control system, and the walking trajectory of a rat’s hind leg was reconstructed. This study contributes to controlling the multiple joints of a leg and reconstructing functional motions such as walking using the robotic control technology.

DOI： 10.3390/s22197198

Visual Sensing System to Investigate Self-Propelled Motion and Internal Color of Multiple Aqueous Droplets 査読有り

Aoyama T., Yamada S., Suematsu N.J., Takeuchi M., Hasegawa Y.

Sensors 22 巻 ( 16 ) 頁： 6309 2022年8月

担当区分：最終著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Sensors

This study proposes a visual sensing system to investigate the self-propelled motions of droplets. In the visual sensing of self-propelled droplets, large field-of-view and high-resolution images are both required to investigate the behaviors of multiple droplets as well as chemical reactions in the droplets. Therefore, we developed a view-expansive microscope system using a color camera head to investigate these chemical reactions; in the system, we implemented an image processing algorithm to detect the behaviors of droplets over a large field of view. We conducted motion tracking and color identification experiments on the self-propelled droplets to verify the effectiveness of the proposed system. The experimental results demonstrate that the proposed system is able to detect the location and color of each self-propelled droplet in a large-area image.

DOI： 10.3390/s22166309

A Therapeutic Strategy for Lower Motor Neuron Disease and Injury Integrating Neural Stem Cell Transplantation and Functional Electrical Stimulation in a Rat Model 国際誌 Open Access

Tokutake K., Takeuchi M., Kurimoto S., Saeki S., Asami Y., Onaka K., Saeki M., Aoyama T., Hasegawa Y., Hirata H.

International Journal of Molecular Sciences 23 巻 ( 15 ) 頁： 8760 - 8760 2022年8月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：International Journal of Molecular Sciences

Promising treatments for upper motor neuron disease are emerging in which motor function is restored by brain–computer interfaces and functional electrical stimulation. At present, such technologies and procedures are not applicable to lower motor neuron disease. We propose a novel therapeutic strategy for lower motor neuron disease and injury integrating neural stem cell transplantation with our new functional electrical stimulation control system. In a rat sciatic nerve transection model, we transplanted embryonic spinal neural stem cells into the distal stump of the peripheral nerve to reinnervate denervated muscle, and subsequently demonstrated that highly responsive limb movement similar to that of a healthy limb could be attained with a wirelessly powered two-channel neurostimulator that we developed. This unique technology, which can reinnervate and precisely move previously denervated muscles that were unresponsive to electrical stimulation, contributes to improving the condition of patients suffering from intractable diseases of paralysis and traumatic injury.

DOI： 10.3390/ijms23158760