記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）  

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：2025 Asia Pacific Signal and Information Processing Association Annual Summit and Conference Apsipa ASC 2025  

This paper proposes a hierarchical symbolic music generation method based on autoregressive modeling of variational autoencoder (VAE)-based bar-wise features. The method represents music as a sequence of low-dimensional bar-level features, enabling efficient modeling of long-term structure while maintaining local coherence. It consists of a VAE-based encoder and decoder for bar-wise feature extraction and composition, and a Transformer-based feature sequence generator conditioned on chord progression. To evaluate global structural coherence, we introduce a new metric, Bar-wise Feature Similarity Distance (BFSD). Experimental results show that the proposed method improves long-term structure compared to baseline models and achieves local naturalness comparable to existing methods. The source code for BFSD is available at https://github.com/KateSawada/barwise_feature_similarity_distance.

DOI： 10.1109/apsipaasc65261.2025.11249414

Scopus

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：2025 Asia Pacific Signal and Information Processing Association Annual Summit and Conference Apsipa ASC 2025  

Speech quality assessment (SQA) refers to the evaluation of speech quality, and developing an accurate automatic SQA method that reflects human perception has become increasingly important, in order to keep up with the generative AI boom. In recent years, SQA has progressed to a point that researchers started to faithfully use automatic SQA in research papers as a rigorous measurement of goodness for speech generation systems. We believe that the scientific challenges and open-source activities of late have stimulated the growth in this field. In this paper, we review recent challenges as well as open-source implementations and toolkits for SQA, and highlight the importance of maintaining such activities to facilitate the development of not only SQA itself but also generative AI for speech.

DOI： 10.1109/apsipaasc65261.2025.11249197

Scopus

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：2025 Asia Pacific Signal and Information Processing Association Annual Summit and Conference Apsipa ASC 2025  

This paper evaluates the generalization performance of supervised virtual microphone estimation under reverberant environments. While the previous studies have shown that virtual microphone estimators trained in reverberant time conditions can operate effectively, detailed investigations on their performance have not been conducted from various perspectives. In this paper, we clarify the generalization capability of the estimator by examining both estimation performance and its impact on array signal processing under simulated environments with different reverberation times. Furthermore, we explored strategies to improve the estimator, including different model architectures and loss functions. Experimental results show that as the reverberation time increases, the estimation performance decreases and the enhancement performance by MVDR based on steering vectors degrades significantly. Furthermore, the results suggest that enhancement performance can be improved by properly designing the model architecture and loss function.

DOI： 10.1109/apsipaasc65261.2025.11249347

Scopus

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：2025 Asia Pacific Signal and Information Processing Association Annual Summit and Conference Apsipa ASC 2025  

Automatic piano rearrangement systems for controlling score difficulty can support players without musical rearrangement skills and help to increase their motivation to practice. While the existing systems allow difficulty adjustment in the predefined levels, more flexible control-such as local arrangement of only difficult passages and continuous or multifaceted difficulty scaling-is desirable. The conventional methods for designing difficulty measures and applying them to difficulty estimation rely on information dependent on specific score data formats and on whole score information, limiting their applicability for the data formats that can be used and making local difficulty assessment impossible. In this paper, we propose a music difficulty measure (DM) for estimating the difficulty of piano scores to address these issues. DM consists of seven features that are independent of data format and whole score information, making them applicable both as bar-level measures (DM-bar) and as whole-score measures (DM-whole). We also propose a linear regression-based music difficulty estimation method using DM: a linear regression model is trained on DM-whole to predict whole difficulty, and the same model is applied to DM-bar to estimate bar-level difficulty. In addition, statistics of the estimated bar-level difficulties over a whole score are further integrated into the model to improve the estimation accuracy. The experimental evaluation demonstrates the effectiveness of the proposed measure on the music difficulty estimation task.

DOI： 10.1109/apsipaasc65261.2025.11249163

Scopus

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：2025 Asia Pacific Signal and Information Processing Association Annual Summit and Conference Apsipa ASC 2025  

Spoken-text-style transfer (STST) aims to convert a given text to a desired style while preserving its semantic content. In this paper, we focus on modeling the disfluency in spoken text, which can be useful as a preprocessing step in spontaneous speech synthesis. Previous methods suffer from unknown words and confusion between normal and disfluent words. Our proposed solutions to the above-mentioned problems include a masked language model (MLM) approach to temporally replace unknown words, and a disfluency symbol representation (DSR) to increase the discrepancy against normal words. Experimental evaluation results show that MLM improves the robustness against unknown words, and the use of DSR achieves a higher transfer accuracy. Finally, we show the potential of speaker-modeling to achieve speaker-wise disfluency control in STST.

DOI： 10.1109/apsipaasc65261.2025.11248977

Scopus

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：2025 Asia Pacific Signal and Information Processing Association Annual Summit and Conference Apsipa ASC 2025  

We aim to establish a method to estimate the seating position of a speaker from monaural speech recorded by a driving recorder in a vehicle. There is a demand for identifying the seating position of a driver and passengers after the occurrence of a traffic accident. Since speech recorded with a microphone of the driving recorder is often available, we attempt to estimate the seating position using those speeches. If a person hears a sound directly with their ear in real space, it is possible to estimate the sound source direction to some extent from the characteristics of the pinna, even if it is one ear. However, the sound recorded with a mono microphone has a loss of information, making it difficult to estimate the direction. On the other hand, in a closed sound field, such as the inside of a vehicle, acoustic features of the monaural speech differ depending on the seating position from which the monaural speech reaches the microphone. Therefore, it is expected that the seating position estimation is still possible by modeling those subtle acoustic feature differences in the individual vehicles using machine learning techniques. In this paper, we investigate the possibility of estimating the seating position using the monaural speech by conducting experiments in a simulated sound field where the size of the vehicle interior and the arrangement of the microphone and seating positions are consistent during training and testing. The experimental results have demonstrated that 1) when the size of the vehicle interior and the arrangement of microphones and seats are almost the same during training and testing, the proposed method is effective for sound source localization using monaural audio, 2) if the microphone positions are different during training and testing, the performance degrades significantly, and 3) by using multiple utterances per person, the performance can be further improved.

DOI： 10.1109/apsipaasc65261.2025.11249106

Scopus

記述言語：英語   掲載種別：研究論文（国際会議プロシーディングス）  

掲載種別：研究論文（学術雑誌）   出版者・発行元：IEEE Journal on Selected Topics in Signal Processing  

We investigate the use of linguistic intermediates to resolve domain mismatches in the electrolaryngeal (EL) speech enhancement task. We first propose the use of linguistic encoders to produce bottleneck feature intermediates, and use a recognition, alignment, and synthesis framework, effectively improving performance due to the removal of the timbre mismatches between the pretraining (typical) and fine-tuning (EL) data. We then further improve this by introducing discrete text intermediates, which effectively alleviate temporal mismatches between the source (EL) and target (typical) data to improve prosody modeling. Our findings show that by simply using bottleneck feature intermediates, more intelligible and naturally sounding speech can already be synthesized, as shown by a significant 16% improvement in character error rate and 0.83 improvement in naturalness score compared to the baseline. Moreover, through the use of discrete phoneme-level intermediates, we can further improve the modeling of the temporal structure of typical speech and get another absolute improvement of 1.4% in character error rate and 0.2 in naturalness compared to the initially proposed system. Finally, we also verify these findings on a larger pseudo-EL dataset of 14 speakers and another set of 3 real-world EL speakers, which consistently show that using the phoneme-level intermediates is most effective approach in terms of phoneme error rate. We conclude the research by summarizing the advantages and disadvantages of each proposed technique.

DOI： 10.1109/JSTSP.2025.3584195

Open Access

Web of Science

Scopus

出版者・発行元：ICASSP IEEE International Conference on Acoustics Speech and Signal Processing Proceedings  

Singing voice synthesis (SVS) technology has progressed rapidly in recent years. However, vocal ensemble synthesis has not yet been widely explored. In this work, we focus on unison singing, which is to have several singers singing the same melody together. Our goal is to understand what acoustic properties affect the naturalness of the synthesized unison singing.We utilize NNSVS, an SVS toolkit that allows us to manipulate individual acoustic features, including timing, f0, and spectrum features, in a fully data-driven manner to investigate their effect in unison singing synthesis. Through listening tests, it was shown that the fluctuation in timing and f0 is an important factor in synthesizing natural unison singing. Furthermore, we discovered the potential to generate unison singing using an SVS model trained only with a single singer dataset.

DOI： 10.1109/ICASSP49660.2025.10889744

Scopus

出版者・発行元：European Signal Processing Conference  

We propose Serenade, a novel framework for the singing style conversion (SSC) task. Although singer identity conversion has made great strides in the previous years, converting the singing style of a singer has been an unexplored research area. We find three main challenges in SSC: modeling the target style, disentangling source style, and retaining the source melody. To model the target singing style, we use an audio infilling task by predicting a masked segment of the target mel-spectrogram with a flow-matching model using the complement of the masked target mel-spectrogram along with disentangled acoustic features. On the other hand, to disentangle the source singing style, we use a cyclic training approach, where we use synthetic converted samples as source inputs and reconstruct the original source mel-spectrogram as a target. Finally, to retain the source melody better, we investigate a post-processing module using a source-filter-based vocoder and resynthesize the converted waveforms using the original F0 patterns. Our results showed that the Serenade framework can handle generalized SSC tasks with the best overall similarity score, especially in modeling breathy and mixed singing styles. We also found that resynthesizing with the original F0 patterns alleviated out-of-tune singing and improved naturalness, but found a slight tradeoff in similarity due to not changing the F0 patterns into the target style.

DOI： 10.23919/EUSIPCO63237.2025.11226227

Scopus

掲載種別：研究論文（国際会議プロシーディングス）  

その他リンク： https://dblp.uni-trier.de/rec/conf/eusipco/2025

出版者・発行元：European Signal Processing Conference  

Source-filter HiFi-GAN (SiFi-GAN) is a neural vocoder offering fast, high-quality voice synthesis with fundamental frequency (F0) controllability. However, SiFi-GAN takes hand-crafted acoustic features from traditional signal processing as input, causing some limitations, such as sound quality degradation in F0 extrapolation. This paper proposes VAE-SiFiGAN, which learns latent representations from Mel-spectrograms via a variational autoencoder (VAE). The latent representations learned through the probabilistic framework enable SiFi-GAN to better model the stochastic components in speech signals, achieving sound quality improvements in F0 modification. Furthermore, to address the insufficient F0 controllability caused by the entanglement of Mel-spectrograms and F0 information, we propose to guide the latent representation learning process with hand-crafted features less affected by F0 and used only during training. Experimental results show that VAE-SiFiGAN achieves superior F0 controllability compared to SiFi-GAN.

DOI： 10.23919/EUSIPCO63237.2025.11226579

Scopus

掲載種別：研究論文（国際会議プロシーディングス）  

その他リンク： https://dblp.uni-trier.de/rec/conf/eusipco/2025

記述言語：英語   掲載種別：研究論文（学術雑誌）   出版者・発行元：Apsipa Transactions on Signal and Information Processing  

This paper proposes music similarity representation learning (MSRL) based on individual instruments (InMSRL) utilizing music source separation (MSS) and human preference without requiring clean instrument stems during inference. We propose three methods that effectively improve performance. First, we introduce end-to-end fine-tuning (E2E-FT) for the Cascade approach that sequentially performs MSS and music similarity feature extraction. E2E-FT allows the model to minimize the adverse effects of a separation error on the feature extraction. Second, we propose multi-task learning for the Direct approach that directly extracts disentangled music similarity features using a single music similarity feature extractor. Multi-task learning, which is based on the disentangled music similarity feature extraction and MSS based on reconstruction with disentangled music similarity features, further enhances instrument feature disentanglement. Third, we employ perception-aware fine-tuning (PAFT). PAFT utilizes human preference, allowing the model to perform InMSRL aligned with human perceptual similarity. We conduct experimental evaluations and demonstrate that 1) E2E-FT for Cascade significantly improves objective InMSRL performance, 2) the multi-task learning for Direct is also helpful to improve disentanglement performance in the feature extraction, 3) PAFT significantly enhances the perceptual InMSRL performance, and 4) Cascade with E2EFT and PAFT outperforms Direct with the multi-task learning and PAFT.

DOI： 10.1561/116.20250016

Open Access

Web of Science

Scopus

Web of Science

出版者・発行元：Asru 2025 2025 IEEE Automatic Speech Recognition and Understanding Workshop  

Modern speech quality prediction models are trained on audio data resampled to a specific sampling rate. When tested on audio with a higher sampling rate, these models can produce biased scores. We present HighRateMOS, the first non-intrusive mean opinion score (MOS) model that explicitly considers sampling rate. HighRateMOS ensembles three model variants that exploit the following information: (i) a learnable embedding of speech sampling rate, (ii) Wav2vec 2.0 selfsupervised embeddings, (iii) multi-scale CNN spectral features, and (iv) MFCC features. In AudioMOS 2025 Track 3, HighRateMOS ranked first in five of eight metrics. Our experiments confirm that modeling sampling rate leads to more robust and sampling-rate-agnostic speech quality predictions.

DOI： 10.1109/ASRU65441.2025.11434689

Scopus

出版者・発行元：Asru 2025 2025 IEEE Automatic Speech Recognition and Understanding Workshop  

We present VERSA-v2, a major upgrade of the Versatile Evaluation of Speech and Audio (VERSA) toolkit for standardized and scalable evaluation across speech, audio, and music tasks. It features a modular, object-oriented architecture that simplifies metric integration and now supports over 100 metrics, organized into curated task-specific packs. VERSA-v2 also introduces interactive visualizations, per-metric profiling, and prompt-based evaluation using both text- and audio-based large language models (LLMs). These advancements make VERSA-v2 a robust, extensible, and LLM-enabled platform for comprehensive and interpretable speech and audio evaluation.

DOI： 10.1109/ASRU65441.2025.11434734

Scopus

出版者・発行元：Asru 2025 2025 IEEE Automatic Speech Recognition and Understanding Workshop  

This is the summary paper for the AudioMOS Challenge 2025, the very first challenge for automatic subjective quality prediction for synthetic audio. The challenge consists of three tracks. The first track aims to assess text-to-music samples in terms of overall quality and textual alignment. The second track is based on the four evaluation dimensions of Meta Audiobox Aesthetics, and the test set consists of text-to-speech, text-to-audio, and text-to-music samples. The third track focuses on synthetic speech quality assessment in different sampling rates. The challenge attracted 24 unique teams from both academia and industry, and improvements over the baselines were confirmed. The outcome of this challenge is expected to facilitate development and progress in the field of automatic evaluation for audio generation systems.

DOI： 10.1109/ASRU65441.2025.11434660

Scopus

記述言語：英語   掲載種別：研究論文（学術雑誌）   出版者・発行元：一般社団法人 日本音響学会  

Evaluating synthetic speech generated by machines is a complicated process, as it involves judging along multiple dimensions including naturalness, intelligibility, and whether the intended purpose is fulfilled. While subjective listening tests conducted with human participants have been the gold standard for synthetic speech evaluation, its costly process design has also motivated the development of automated objective evaluation protocols. In this review, we first provide a historical view of listening test methodologies, from early in-lab comprehension tests to recent large-scale crowdsourcing mean opinion score (MOS) tests. We then recap the development of automatic measures, ranging from signal-based metrics to model-based approaches that utilize deep neural networks or even the latest self-supervised learning techniques. We also describe the VoiceMOS Challenge series, a scientific event we founded that aims to promote the development of data-driven synthetic speech evaluation. Finally, we provide insights into unsolved issues in this field as well as future prospects. This review is expected to serve as an entry point for early academic researchers to enrich their knowledge in this field, as well as speech synthesis practitioners to catch up on the latest developments.

DOI： 10.1250/ast.e24.12

Web of Science

Scopus

CiNii Research

記述言語：日本語   出版者・発行元：一般社団法人 日本音響学会  

DOI： 10.20697/jasj.80.7_381

CiNii Research

掲載種別：研究論文（学術雑誌）   出版者・発行元：IEEE/ACM Transactions on Audio Speech and Language Processing  

We investigate state-of-the-art automatic speech recognition (ASR) systems and provide thorough investigations on training methods to adapt them to low-resourced electrolaryngeal (EL) datasets. Transfer learning is often sufficient to resolve low-resourced problems; however, in EL speech, the domain shift between the pretraining and fine-tuning data is too large to overcome, limiting the ASR performance. We propose a method of reducing the domain shift gap during transfer learning between the healthy and EL datasets by conducting an intermediate fine-tuning task that uses imperfectly synthesized EL speech. Although using imperfect synthetic speech is non-intuitive, we proved the effectiveness of this method by decreasing the character error rate by up to 6.1% compared to the baselines using naive transfer learning. To further understand the model's behavior, we further analyze the produced latent spaces in each task through linguistic and identity proxy tasks and find that the intermediate fine-tuning focuses on identifying the voicing characteristics of the EL speakers. Moreover, we also show the differences between a simulated EL speaker from a real EL speaker and find that simulated EL data has pronunciation differences from real EL data, showing the huge domain gap between real EL and other speech data.

DOI： 10.1109/TASLP.2024.3402557

Web of Science

Scopus

掲載種別：研究論文（学術雑誌）   出版者・発行元：IEEE/ACM Transactions on Audio Speech and Language Processing  

The foundation model paradigm leverages a shared foundation model to achieve state-of-the-art (SOTA) performance for various tasks, requiring minimal downstream-specific data collection and modeling. This approach has proven crucial in the field of Natural Language Processing (NLP). However, the speech processing community lacks a similar setup to explore the paradigm systematically. To bridge this gap, we establish the Speech processing Universal PERformance Benchmark (SUPERB). SUPERB represents an ecosystem designed to evaluate foundation models across a wide range of speech processing tasks, facilitating the sharing of results on an online leaderboard and fostering collaboration through a community-driven benchmark database that aids in new development cycles. We present a unified learning framework for solving the speech processing tasks in SUPERB with the frozen foundation model followed by task-specialized lightweight prediction heads. Combining our results with community submissions, we verify that the framework is simple yet effective, as the best-performing foundation model shows competitive generalizability across most SUPERB tasks. Finally, we conduct a series of analyses to offer an in-depth understanding of SUPERB and speech foundation models, including information flows across tasks inside the models and the statistical significance and robustness of the benchmark.

DOI： 10.1109/TASLP.2024.3389631

Web of Science

Scopus

掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings  

We propose a novel framework for electrolaryngeal speech intelligibility enhancement through the use of robust linguistic encoders. Pretraining and fine-tuning approaches have proven to work well in this task, but in most cases, various mismatches, such as the speech type mismatch (electrolaryngeal vs. typical) or a speaker mismatch between the datasets used in each stage, can deteriorate the conversion performance of this framework. To resolve this issue, we propose a linguistic encoder robust enough to project both EL and typical speech in the same latent space, while still being able to extract accurate linguistic information, creating a unified representation to reduce the speech type mismatch. Furthermore, we introduce HuBERT output features to the proposed framework for reducing the speaker mismatch, making it possible to effectively use a large-scale parallel dataset during pretraining. We show that compared to the conventional framework using mel-spectrogram input and output features, using the proposed framework enables the model to synthesize more intelligible and naturally sounding speech, as shown by a significant 16% improvement in character error rate and 0.83 improvement in naturalness score.

DOI： 10.1109/ICASSP48485.2024.10447197

Web of Science

Scopus

その他リンク： https://dblp.uni-trier.de/db/conf/icassp/icassp2024.html#VioletaHMYKT24

掲載種別：研究論文（国際会議プロシーディングス）   出版者・発行元：Proceedings of 2024 IEEE Spoken Language Technology Workshop Slt 2024  

We present the third edition of the VoiceMOS Challenge, a scientific initiative designed to advance research into automatic prediction of human speech ratings. There were three tracks. The first track was on predicting the quality of 'zoomed-in' high-quality samples from speech synthesis systems. The second track was to predict ratings of samples from singing voice synthesis and voice conversion with a large variety of systems, listeners, and languages. The third track was semi-supervised quality prediction for noisy, clean, and enhanced speech, where a very small amount of labeled training data was provided. Among the eight teams from both academia and industry, we found that many were able to outperform the baseline systems. Successful techniques included retrieval-based methods and the use of non-self-supervised representations like spectrograms and pitch histograms. These results showed that the challenge has advanced the field of subjective speech rating prediction.

DOI： 10.1109/SLT61566.2024.10832295

Web of Science

Scopus

掲載種別：研究論文（学術雑誌）   出版者・発行元：IEEE Signal Processing Letters  

The trend of scaling up speech generation models poses the threat of biometric information leakage of the identities of the voices in the training data, raising privacy and security concerns. In this letter, we investigate the training of multi-speaker text-to-speech (TTS) models using data that underwent speaker anonymization (SA), a process that tends to hide the speaker identity of the input speech while maintaining other attributes. Two signal processing-based and three deep neural network-based SA methods were used to anonymize VCTK, a multi-speaker TTS dataset, which is further used to train an end-to-end TTS model, VITS, to perform unseen speaker TTS during the testing phase. We conducted extensive objective and subjective experiments to evaluate the anonymized training data, as well as the performance of the downstream TTS model trained using those data. Importantly, we found that UTMOS, a data-driven subjective rating predictor model, and GVD, a metric that measures the gain of voice distinctiveness, are good indicators of the downstream TTS performance. We summarize insights in the hope of helping future researchers determine the usefulness of the SA system for multi-speaker TTS training.

DOI： 10.1109/LSP.2024.3482701

Web of Science

Scopus

J-GLOBAL

J-GLOBAL

J-GLOBAL

J-GLOBAL

J-GLOBAL

J-GLOBAL