研究者詳細 - 長尾　確

論文 - 長尾　確

分割表示 35 件中 1 - 20 件目／全件表示 >>

VR Dance Training System Capable of Human Motion Tracking and Automatic Dance Evaluation 査読有り

江嵜一優, 長尾　確

PRESENCE: Virtual and Augmented Reality 29 巻頁： 1 - 23 2023年

担当区分：最終著者,　責任著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：The MIT Press

In this paper, a method for 3D human body tracking using multiple cameras and an automatic evaluation method using machine learning are developed to construct a virtual reality (VR) dance self-training system for fast-moving hip-hop dance. Dancers’ movement data are input as time-series data of temporal changes in joint point positions and rotations and are categorized into instructional items that are frequently pointed out by coaches as areas for improvement in actual dance lessons. For automatic dance evaluation, contrastive learning is used to obtain better expression vectors with less data. As a result, the accuracy when using contrastive learning was 0.79, a significant improvement from 0.65 without contrastive learning. In addition, since each dance is modeled by a coach, the accuracy was slightly improved to 0.84 by using, as input, the difference between the expression vectors of the model’s and the user’s movement data. Eight subjects used the VR dance training system, and results of a questionnaire survey confirmed that the system is effective.

DOI： 10.1162/PRES_a_00383

VR Presentation Training System Using Machine Learning Techniques for Automatic Evaluation 招待有り査読有り国際誌

Yuto Yokoyama and Katashi Nagao

International Journal of Virtual and Augmented Reality 5 巻 ( 1 ) 2022年1月

担当区分：最終著者,　責任著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IGI Global

In this paper, we build an immersive training space using building-scale VR, a technology that makes a virtual space based on an entire building existing in the real world. The space is used for presentations, allowing students to self-train. The results of a presentation are automatically evaluated by using machine learning or the like and fed back to the user. In this space, users can meet their past selves (more accurately, their avatars), so they can objectively observe their presentations and recognize weak points. We developed a mechanism for recording and reproducing activities in virtual space in detail and a mechanism for applying machine learning to activity records. With these mechanisms, a system for recording, reproducing, and automatically evaluating presentations was developed.

DOI： 10.4018/IJVAR.290044

Impact Sound Generation for Audiovisual Interaction with Real-World Movable Objects in Building-Scale Virtual Reality 査読有り国際誌

Nagao Katashi, Kumon Kaho, Hattori Kodai

APPLIED SCIENCES-BASEL 11 巻 ( 16 ) 2021年8月

担当区分：筆頭著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Applied Sciences (Switzerland)

In building-scale VR, where the entire interior of a large-scale building is a virtual space that users can walk around in, it is very important to handle movable objects that actually exist in the real world and not in the virtual space. We propose a mechanism to dynamically detect such objects (that are not embedded in the virtual space) in advance, and then generate a sound when one is hit with a virtual stick. Moreover, in a large indoor virtual environment, there may be multiple users at the same time, and their presence may be perceived by hearing, as well as by sight, e.g., by hearing sounds such as footsteps. We, therefore, use a GAN deep learning generation system to generate the impact sound from any object. First, in order to visually display a real-world object in virtual space, its 3D data is generated using an RGB-D camera and saved, along with its position information. At the same time, we take the image of the object and break it down into parts, estimate its material, generate the sound, and associate the sound with that part. When a VR user hits the object virtually (e.g., hits it with a virtual stick), a sound is generated. We demonstrate that users can judge the material from the sound, thus confirming the effectiveness of the proposed method.

DOI： 10.3390/app11167546

Web of Science

Scopus

VR Training System to Help Improve Photography Skills

小林大貴, 長尾　確

APPLIED SCIENCES-BASEL 13 巻 ( 7817 ) 2023年7月

担当区分：最終著者,　責任著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Applied Sciences (Switzerland)

Featured Application: Virtual Reality Simulation and Training for Photography Skills Improvement. People aspiring to enhance their photography skills face multiple challenges, such as finding subjects and understanding camera-specific parameters. To address this, we present the VR Photo Training System. This system allows users to practice photography in a virtual space and provides feedback on user-taken photos using machine-learning models. These models, trained on datasets from the virtual environment, evaluate the aesthetics, composition, and color of photos. The system also includes a feature offering composition advice, which further aids in skill development. The evaluation and recommendation functions of our system have shown sufficient accuracy, proving its effectiveness for photography training.

DOI： 10.3390/app13137817

Web of Science

Scopus

Autonomous Stair Ascending and Descending by Quadruped Wheelchairs

赤見坂篤記, 長尾　確

2023 20TH INTERNATIONAL CONFERENCE ON UBIQUITOUS ROBOTS, UR 頁： 295 - 301 2023年6月

担当区分：最終著者,　責任著者記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：2023 20th International Conference on Ubiquitous Robots, UR 2023

Wheelchairs are an important means of transportation for people with disabilities. However, because wheelchairs move on wheels, it is impossible for wheelchair users to move along paths such as stairs that cannot be covered by wheels, thus imposing significant limitations on their mobility. Although some wheelchairs are now available with crawler mechanisms, they cannot handle spiral staircases, in which the length of the tread differs between the inside and outside of the staircase. This paper proposes a quadruped wheelchair that can be used on all types of stairs. The proposed quadruped wheelchair is a fusion of a quadruped robot and a wheelchair, and has the potential to achieve high mobility on both plain and uneven terrain. On the other hand, it is very complicated to carry a person on the wheelchair. In this paper, we show that a quadruped wheelchair with a person on board can ascend and descend stairs using a simulator and reinforcement learning, which is a milestone toward the realization of a quadruped wheelchair.

DOI： 10.1109/UR57808.2023.10202377

Web of Science

Scopus

Virtual Reality Campuses as New Educational Metaverses 招待有り

Nagao Katashi

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E106.D 巻 ( 2 ) 頁： 93 - 100 2023年2月

記述言語：英語出版者・発行元：一般社団法人電子情報通信学会

This paper focuses on the potential value and future prospects of using virtual reality (VR) technology in online education. In detailing online education and the latest VR technology, we focus on metaverse construction and artificial intelligence (AI) for educational VR use. In particular, we describe a virtual university campus in which on-demand VR lectures are conducted in virtual lecture halls, automated evaluations of student learning and training using machine learning, and the linking of multiple digital campuses.

DOI： 10.1587/transinf.2022eti0001

Web of Science

Scopus

CiNii Research

Effects of a new speech support application on intensive speech therapy and changes in functional brain connectivity in patients with post-stroke aphasia

Katsuno Yuta, Ueki Yoshino, Ito Keiichi, Murakami Satona, Aoyama Kiminori, Oishi Naoya, Kan Hirohito, Matsukawa Noriyuki, Nagao Katashi, Tatsumi Hiroshi

FRONTIERS IN HUMAN NEUROSCIENCE 16 巻頁： 870733 2022年9月

記述言語：英語出版者・発行元：Frontiers in Human Neuroscience

Aphasia is a language disorder that occurs after a stroke and impairs listening, speaking, reading, writing, and calculation skills. Patients with post-stroke aphasia in Japan are increasing due to population aging and the advancement of medical treatment. Opportunities for adequate speech therapy in chronic stroke are limited due to time constraints. Recent studies have reported that intensive speech therapy for a short period of time or continuous speech therapy using high-tech equipment, including speech applications (apps, can improve aphasia even in the chronic stage. However, its underlying mechanism for improving language function and its effect on other cognitive functions remains unclear. In the present study, we investigated whether intensive speech therapy using a newly developed speech support app could improve aphasia and other cognitive functions in patients with chronic stroke. Furthermore, we examined whether it can alter the brain network related to language and other cortical areas. Thus, we conducted a prospective, single-comparison study to examine the effects of a new speech support app on language and cognitive functions and used resting state functional MRI (rs-fMRI) regions of interest (ROI) to ROI analysis to determine changes in the related brain network. Two patients with chronic stroke participated in this study. They used the independent speech therapy system to perform eight sets of 20 randomly presented words/time (taking approximately 20 min), for 8 consecutive weeks. Their language, higher cognitive functions including attention function, and rs-fMRI, were evaluated before and after the rehabilitation intervention using the speech support app. Both patients had improved pronunciation, daily conversational situations, and attention. The rs-fMRI analysis showed increased functional connectivity of brain regions associated with language and attention related areas. Our results show that intensive speech therapy using this speech support app can improve language and attention functions even in the chronic stage of stroke, and may be a useful tool for patients with aphasia. In the future, we will conduct longitudinal studies with larger numbers of patients, which we hope will continue the trends seen in the current study, and provide even stronger evidence for the usefulness of this new speech support app.

DOI： 10.3389/fnhum.2022.870733

Web of Science

Scopus

PubMed

Recognition of Students' Mental States in Discussion Based on Multimodal Data and its Application to Educational Support 査読有り国際誌

Shimeng Peng, Katashi Nagao

IEEE Access 9 巻頁： 1 - 16 2021年2月

担当区分：最終著者,　責任著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE

Students will experience a complex mixture of mental states during discussion, including concentration, confusion, frustration, and boredom, which have been widely acknowledged as crucial components for revealing a student's learning states. In this study, we propose using multimodal data to design an intelligent monitoring agent that can assist teachers in effectively monitoring the multiple mental states of students during discussion. We firstly developed an advanced multi-sensor-based system and applied it in a real university's research lab to collect a multimodal “in-the-wild” teacher-student conversation dataset. Then, we derived a set of proxy features from facial, heart rate, and acoustic modalities and used them to train several supervised learning classifiers with different multimodal fusion approaches single-channellevel, feature-level, and decision-level fusion to recognize students' multiple mental states in conversations. We explored how to design multimodal analytics to augment the ability to recognize different mental states and found that fusing heart rate and acoustic modalities yields better recognize the states of concentration (AUC = 0.842) and confusion (AUC = 0.695), while fusing three modalities yield the best performance in recognizing the states of frustration (AUC = 0.737) and boredom (AUC = 0.810). Our results also explored the possibility of leveraging the advantages of the replacement capabilities between different modalities to provide human teachers with solutions for addressing the challenges with monitoring students in different real-world education environments.

DOI： 10.1109/ACCESS.2021.3054176

Using Presentation Slides and Adjacent Utterances for Post-editing of Speech Recognition Results for Meeting Recordings 査読有り国際誌

Kamiya K., Kawase T., Higashinaka R., Nagao K.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12848 LNAI 巻頁： 331 - 340 2021年

担当区分：最終著者記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

In recent years, the use of automatic speech recognition (ASR) systems in meetings has been increasing, such as for minutes generation and speaker diarization. The problem is that ASR systems often misrecognize words because there is domain-specific content in meetings. In this paper, we propose a novel method for automatically post-editing ASR results by using presentation slides that meeting participants use and utterances adjacent to a target utterance. We focus on automatic post-editing rather than domain adaptation because of the ease of incorporating external information, and the method can be used for arbitrary speech recognition engines. In experiments, we found that our method can significantly improve the recognition accuracy of domain-specific words (proper nouns). We also found an improvement in the word error rate (WER).

DOI： 10.1007/978-3-030-83527-9_28

Scopus

Recognition of Students' Multiple Mental States in Conversation Based on Multimodal Cues

Peng Shimeng, Ohira Shigeki, Nagao Katashi

COMPUTER SUPPORTED EDUCATION (CSEDU 2020) 1473 巻頁： 468 - 479 2021年

記述言語：日本語出版者・発行元：Communications in Computer and Information Science

Learning activities, especially face-to-face conversational coaching may invite students experience a set of learning-centered mental states including concentration, confusion, frustration, and boredom, those mental sates have been widely used as vital proxies for inferring their learning processes and are closely linked with learning outcomes. Recognizing students’ learning-centered mental states, particularly effectively detecting negative mental states such as confusion and frustration in teacher-student conversation could help teacher effectively monitor students’ learning situations in order to direct personalized and adaptive coaching resources to maximum students’ learning outcome. Most of research focused on analyzing students’ mental states using univariate modality when they completing pre-designed tasks in a computer related environment. It is still an open question on how to effectively measure students’ multiple mental states when they interacting with human teacher in coach-led conversations from various aspects. To achieve this goal, in this work, we developed an advanced multi-sensor-based system to record multi-modal conversational data of student-teacher conversations generated in a real university lab. We then attempt to derive a series of interpretable features from multiple perspectives including facial and physiological (heart rate) to characterize students’ multiple mental states. A set of supervised classifiers were built based on those features with different modality fusion methods to recognize multiple mental states of students. Our results have provided the experimental evidence to validate the outstanding predictive ability of our proposed features and the possibility of using multimodal data to recognize students’ multiple mental states in ‘in-the-wild’ student-teacher conversation.

DOI： 10.1007/978-3-030-86439-2_24

Web of Science

Scopus

Using Presentation Slides and Adjacent Utterances for Post-editing of Speech Recognition Results for Meeting Recordings

長尾確

The 24th International Conference of Text, Speech and Dialogue (TSD2021) vol. 12848 巻頁： 331 - 340 2021年

Automatic Generation of Multidestination Routes for Autonomous Wheelchairs 査読有り国際誌

Yusuke Mori, Katashi Nagao

Journal of Robotics and Mechatronics 32 巻 ( 6 ) 頁： 1121 - 1136 2020年12月

担当区分：最終著者,　責任著者記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：Fuji Technology Press Ltd.

To solve the problem of autonomously navigating multiple destinations, which is one of the tasks in the Tsukuba Challenge 2019, this paper proposes a method for automatically generating the optimal travel route based on costs associated with routes. In the proposed method, the route information is generated by playing back the acquired driving data to perform self-localization, and the self-localization log is stored. In addition, the image group of road surfaces is acquired fromthe driving data. The costs of routes are generated based on texture analysis of the road surface image group and analysis of the self-localization log. The cost-added route information is generated by combining the costs calculated by the two methods, and by assigning the combined costs to the route. The minimum-cost multidestination route is generated by conducting a route search using cost-added route information. Then, we evaluated the proposed method by comparing it with the method of generating the route using only the distance cost. The results confirmed that the proposed method generates travel routes that account for safety when the autonomous wheelchair is being driven.

DOI： 10.20965/jrm.2020.p1121

Automatic Reconstruction of Building-Scale Indoor 3D Environment with a Deep-Reinforcement-Learning-Based Mobile Robot 査読有り

Menglong Yang, Katashi Nagao

International Journal of Robotics and Automation Technology 6 巻 ( 1 ) 頁： 11-23 2019年6月

記述言語：英語掲載種別：研究論文（学術雑誌）

The aim of this paper is to digitize the environments in which humans live, at low cost, and reconstruct highly accurate three-dimensional environments that are based on those in the real world. This three-dimensional content can be used such as for virtual reality environments and three-dimensional maps for automatic driving systems.
In general, however, a three-dimensional environment must be carefully reconstructed by manually moving the sensors used to first scan the real environment on which the three-dimensional one is based. This is done so that every corner of an entire area can be measured, but time and costs increase as the area expands. Therefore, a system that creates three-dimensional content that is based on real-world large-scale buildings at low cost is proposed. This involves automatically scanning the indoors with a mobile robot that uses low-cost sensors and generating 3D point clouds.
When the robot reaches an appropriate measurement position, it collects the three-dimensional data of shapes observable from that position by using a 3D sensor and 360-degree panoramic camera. The problem of determining an appropriate measurement position is called the “next best view problem," and it is difficult to solve in a complicated indoor environment. To deal with this problem, a deep reinforcement learning method is employed. It combines reinforcement learning, with which an autonomous agent learns strategies for selecting behavior, and deep learning done using a neural network. As a result, 3D point cloud data can be generated with better quality than the conventional rule-based approach.

DOI： 10.15377/2409-9694.2019.06.2

Prediction of Students' Answer Relevance in Discussion Based on their Heart Rate Data 査読有り

Shimeng Peng, Shigeki Ohira, Katashi Nagao

International Journal of Innovation and Research in Educational Sciences 6 巻 ( 3 ) 頁： 414-424 2019年5月

記述言語：英語掲載種別：研究論文（学術雑誌）

Whether a discussion is executed effectively depends on the completion level of the question-and-answer segments (Q&A segments) generated during the discussion. Relevance of answers could be used as a clue for evaluating the Q&A segments' completion degree. In this study, we argue that discussion participants' heart rate (HR) and its variability (HRV), which have recently received increased attention for being a crucial indicator in cognitive task performance evaluation, can be used to predict participants' answer-relevance in Q&A segments of discussions. To validate our argument, we propose an intelligent system that acquires and visualizes the HR data with the help of a non-invasive device, e.g. an Apple Watch, for measuring and recording the HR data of participants which is being updated in real-time. We also developed a web-based human-scoring method for evaluating answer-relevance of Q&A segments and question-difficulty level. A total of 17 real lab-seminar-style discussion experiments were conducted, during which the Q&A segments and the HR of participants were recorded using our proposed system. We then experimented with three machine-learning classifiers, i.e. logistic regression, support vector machine, and random forest, to predict answer-relevance of Q&A segments using the extracted HR and HRV features. Area Under the ROC Curve (AUC) was used to evaluate classifier accuracy using leave-one-student-out cross validation. We achieved an AUC= 0.76 for logistic regression classifier, an AUC=0.77 for SVM classifier, and an AUC=0.79 for random forest classifier. We examined possibilities of using participants' HR data to predict their answer-statements' relevance in Q&A segments of discussions, which provides evidence of the potential utility of the presented tools in scaling-up analysis of this type to a large number of subjects and in implementing these tools to evaluate and improve discussion outcomes in higher education environment.

Building-Scale Virtual Reality: Reconstruction and Modification of Building Interior Extends Real World 査読有り

Katashi Nagao, Menglong Yang, Yusuke Miyakawa

International Journal of Multimedia Data Engineering and Management (IJMDEM) 10 巻 ( 1 ) 頁： 1-21 2019年4月

担当区分：筆頭著者記述言語：英語掲載種別：研究論文（学術雑誌）

A method is presented that extends the real world into all buildings. This building-scale virtual reality (VR) method differs from augmented reality (AR) in that it uses automatically generated 3D point cloud maps of building interiors. It treats an entire indoor area a pose tracking area by using data collected using an RGB-D camera mounted on a VR headset and using deep learning to build a model from the data. It modifies the VR space in accordance with its intended usage by using segmentation and replacement of the 3D point clouds. This is difficult to do with AR but is essential if VR is to be used for actual real-world applications, such as disaster simulation including simulation of fires and flooding in buildings. 3D pose tracking in the building-scale VR is more accurate than conventional RGB-D simultaneous localization and mapping.

DOI： 10.4018/IJMDEM.2019010101

Picognizer：電子音の認識のためのJavaScriptライブラリの開発と評価査読有り

栗原一貴, 植村あい子, 板谷あかり, 北原鉄朗, 長尾確

情報処理学会論文誌 60 巻 ( 2 ) 頁： 397-410 2019年2月

記述言語：日本語掲載種別：研究論文（学術雑誌）

われわれは今やさまざまな電子音に囲まれて生活しており、それらを検出・認識し情報処理を行うことはMaker文化の発展デジタルゲーム拡張およびゲーミフィケーションの構成のうえで有意義であるが、エンドユーザプログラマが個々の検出・認識対象について教師付き学習により認識器を構築するのは現状ではまだ容易ではない。そこで再生ごとの音響的変動が小さいという電子音の特徴を活かし、Dynamic Time Warpingなどの伝統的なテンプレートベースのパターンマッチングアルゴリズムにより電子音の検出・認識を行うJavaScriptライブラリを実装する。基礎的な性能評価を行うとともに、さまざまなユースケースを例示することでその有用性を示す。

Evidence-Based Education: Case Study of Educational Data Acquisition and Reuse

Katashi Nagao, Naoya Morita, Shigeki Ohira

Journal of Systemics, Cybernetics and Informatics 15 巻 ( 7 ) 頁： 77-84 2018年2月

記述言語：英語掲載種別：研究論文（学術雑誌）

There must be as many concrete indicators as possible in education, which will become signposts. People will not be confident about their learning and will become confused with tenuous instruction. It is necessary to clarify what they can do and what kinds of abilities they can improve. This paper describes a case of evidence-based education that acquires educational data from students' study activities and not only uses the data to enable instructors to check the students' levels of understanding but also improve their levels of performance. Our previous research called discussion mining was specifically used to collect various data on meetings (statements and their relationships, presentation materials such as slides, audio and video, and participants' evaluations of statements). This paper focuses on student presentations and discussions in laboratory seminars that are closely related to their research activities in writing their theses. We propose a system that supports tasks to be achieved in research activities and a machine-learning method to make the system sustainable for long-term operation by automatically extracting essential tasks. We conducted participant-based experiments that involved students and computer-simulation-based experiments to evaluate how efficiently our proposed machine-learning method updated the task extraction model. We confirmed from the participant-based experiments that informing responsible students of tasks that were automatically extracted on the system we developed improved their awareness of the tasks. Here, we also explain improvements in extraction accuracy and reductions in labeling costs with our method and how we confirmed its effectiveness through computer simulations.

Tools and evaluation methods for discussion and presentation skills training 査読有り

Katashi Nagao, Mehrdad Tehrani, Jovilyn B. Fajardo

Smart Learning Environments: a SpringerOpen Journal 2 巻 ( 5 ) 頁： doi:10.1186/s40561-015-0011-1 2015年2月

担当区分：筆頭著者記述言語：英語掲載種別：研究論文（学術雑誌）

Our university is currently developing an advanced physical-digital learning environment that can train students to enhance their discussion and presentation skills. The environment guarantees an efficient discussion among users with state-of-the-art technologies such as touch panel discussion tables, digital posters, and an interactive wall-sized whiteboard. It includes a data mining system that efficiently records, summarizes, and annotates discussions held inside our facility. We also developed a digital poster authoring tool, a novel tool for creating interactive digital posters displayed using our digital poster presentation system. Evaluation results show the efficiency of using our facilities: the data mining system and the digital poster authoring tool. In addition, our physical-digital learning environment will be further enhanced with a vision system that will detect interactions with the digital poster presentation system and the different discussion tools enabling a more automated skill evaluation and discussion mining.

DOI： doi:10.1186/s40561-015-0011-1

映像と論文へのアノテーションに基づく論文読解支援システム査読有り

石戸谷顕太朗，山本圭介，大平茂輝，長尾　確

映像情報メディア学会誌 66 巻 ( 11 ) 頁： J461-J470 2012年11月

小型無人移動体との連携による個人用知的移動体の安全自動走行査読有り

渡邉賢, 長尾確

情報処理学会論文誌 53 巻 ( 11 ) 頁： 2599-2611 2012年11月

　前のページ - 次のページ　

<長尾 確>

論文 - 長尾 確

<長尾　確>

論文 - 長尾　確