研究者詳細 - 駒水　孝裕

論文 - 駒水　孝裕

分割表示 >> ／全件表示 100 件中 1 - 100 件目

Towards Ensemble-Based Imbalanced Text Classification Using Metric Learning

Komamizu, T

DATABASE AND EXPERT SYSTEMS APPLICATIONS, DEXA 2023, PT II 14147 巻頁： 188 - 202 2023年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

This paper reports s series of ensemble approaches for imbalance text classification. All the approaches utilize a metric learning technique for obtaining better representations of texts to train weak classifiers. Each approach deals with the class imbalance problem with an undersampling-based ensemble approach, because metric learning techniques also suffer from this problem. In this paper, four ensemble approaches (namely, MLBagging, MLBoosting, MLStacking, and MLBoostacking) are proposed, three of which are corresponding to ensemble frameworks (namely, bagging, boosting, and stacking), and the other is a combination of boosting and stacking. MLBagging, MLBoosting, and MLStacking train metric learners on the individual undersampled dataset and combine them, while MLBoostacking trains metric learners in a step-by-step manner; that is, a metric learner learns a feature transformation so that failed-to-classify samples in the previous step should be correctly classified. The experimental evaluation on three imbalanced text classification datasets (namely, unfair statement classification in terms of service, hate speech detection in a forum, and hate speech tweet detection) shows that the proposed approaches lift classification performance from BERT-based approaches, by improving the representations of texts through metric learning.

DOI： 10.1007/978-3-031-39821-6_15

その他リンク： https://dblp.uni-trier.de/db/conf/dexa/dexa2023-2.html#Komamizu23

MMEnsemble: Imbalanced Classification Framework Using Metric Learning and Multi-sampling Ratio Ensemble

Komamizu, T

DATABASE AND EXPERT SYSTEMS APPLICATIONS, DEXA 2021, PT II 12924 巻頁： 176 - 188 2021年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

In classification, class imbalance is a factor that degrades the classification performance of many classification methods. Resampling is one widely accepted approach to the class imbalance; however, it still suffers from an insufficient data space, which also degrades performance. To overcome this, in this paper, an undersampling-based imbalanced classification framework, MMEnsemble, is proposed that incorporates metric learning into a multi-ratio undersampling-based ensemble. This framework also overcomes a problem with determining the appropriate sampling ratio in the multi-ratio ensemble method. It was evaluated by using 12 real-world datasets. It outperformed the state-of-the-art approaches of metric learning, undersampling, and oversampling in recall and ROC-AUC, and it performed comparably with them in terms of Gmean and F-measure metrics.

DOI： 10.1007/978-3-030-86475-0_18

その他リンク： https://dblp.uni-trier.de/db/conf/dexa/dexa2021-2.html#Komamizu21

Multi-proposal collaboration and multi-task training for weakly-supervised video moment retrieval

Zhang, BL; Yang, C; Jiang, B; Komamizu, T; Ide, I

INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS 2025年1月

出版者・発行元：International Journal of Machine Learning and Cybernetics

This study focuses on weakly-supervised Video Moment Retrieval (VMR), aiming to identify a moment semantically similar to the given query within an untrimmed video using only video-level correspondences, without relying on temporal annotations during training. Previous methods either aggregate predictions for all instances in the video, or indirectly address the task by proposing reconstructions for the query. However, these methods often produce low-quality temporal proposals, struggle with distinguishing misaligned moments in the same video, or lack stability due to a reliance on a single auxiliary task. To address these limitations, we present a novel weakly-supervised method called Multi-proposal Collaboration and Multi-task Training (MCMT). Initially, we generate multiple proposals and derive corresponding learnable Gaussian masks from them. These masks are then combined to create a high-quality positive sample mask, highlighting video clips most relevant to the query. Concurrently, we classify other clips in the same video as the easy negative sample and the entire video as the hard negative sample. During training, we introduce forward and inverse masked query reconstruction tasks to impose more substantial constraints on the network, promoting more robust and stable retrieval performance. Extensive experiments on two standard benchmarks affirm the effectiveness of the proposed method in VMR.

DOI： 10.1007/s13042-024-02520-w

Quantifying Image-Adjective Associations by Leveraging Large-Scale Pretrained Models

Matsuhira C., Kastner M.A., Komamizu T., Hirayama T., Ide I.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 15523 LNCS 巻頁： 428 - 441 2025年

出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Quantifying the associations between images and adjectives, i.e., how much the visual characteristics of an image are connected with a certain adjective, is important for better image understanding. For instance, the appearance of a kitten can be associated with adjectives such as “soft”, “small”, and “cute” rather than the opposite “hard”, “large”, and “scary”. Thus, giving scores for a kitten photo considering the degree of its association with each antonym adjective pair (termed adjective axis, e.g., “round” vs. “sharp”) aids in understanding the image content and its atmosphere. Existing methods rely on subjective human engagement, making it difficult to estimate the association of images with arbitrary adjective axes in a single framework. To enable the extension to arbitrary axes, we explore the use of large-scale pretrained models, including Large Language Models (LLMs) and Vision Language Models (VLMs). In the proposed training-free framework, users only need to specify a pair of antonym nouns that negatively and positively describe the target axis (e.g., “roundness” and “sharpness”). Evaluation confirms that the proposed framework can predict negative and positive associations between adjectives and images as correctly as the manually-assisted comparative. The result also highlights the pros and cons of utilizing the VLM’s textual or visual embedding for specific types of adjective axes. Furthermore, computing the similarities among four adjective axes unveils how the proposed framework connects them with each other, such as its tendency to regard a sharp object as being small, hard, and quick in motion.

DOI： 10.1007/978-981-96-2071-5_31

Towards Visual Storytelling by Understanding Narrative Context Through Scene-Graphs

Phueaksri I., Kastner M.A., Kawanishi Y., Komamizu T., Ide I.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 15523 LNCS 巻頁： 226 - 239 2025年

出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

VIsual STorytelling (VIST) is a task that transforms a sequence of images into narrative text stories. A narrative story requires an understanding of the contexts and relationships among images. Our study introduces a story generation process that emphasizes creating a coherent narrative by constructing both image and narrative contexts to control the coherence. First, the image contexts are generated from the content of individual images, using image features and scene graphs that detail the elements of the images. Second, the narrative context is generated by focusing on the overall image sequence. Ensuring that each caption fits within the overall story maintaining continuity and coherence. We also introduce a narrative concept summary, which is external knowledge represented as a knowledge graph. This summary encapsulates the narrative concept of an image sequence to enhance the understanding of its overall content. Following this, both image and narrative contexts are used to generate a coherent and engaging narrative. This framework is based on Long Short-Term Memory (LSTM) with an attention mechanism. We evaluate the proposed method using the VIST dataset, and the results highlight the importance of understanding the context of an image sequence in generating coherent and engaging stories. The study demonstrates the significance of incorporating narrative context into the generation process to ensure the coherence of the generated narrative.

DOI： 10.1007/978-981-96-2071-5_17

Feature Extraction for Claim Check-Worthiness Prediction Tasks Using LLM

Teramoto Y., Komamizu T., Matsushita M., Hatano K.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 15342 LNCS 巻頁： 53 - 58 2024年12月

掲載種別：論文集(書籍)内論文出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

This study explores the use of Large Language Models (LLMs) for Claim Check-Worthiness Prediction (CCWP), a crucial pre-screening task in fact-checking. We predict the time between a claim’s occurrence and verification by analyzing data from fact-checking organizations. The results show that validation time is the same between the top 25% and bottom 75% of total checklist condition fulfillment claims. That is, further optimization is needed for LLMs to perform effective CCWPs.

DOI： 10.1007/978-3-031-78090-5_5

Cross-modal recipe retrieval based on unified text encoder with fine-grained contrastive learning

Zhang, BL; Kyutoku, H; Doman, K; Komamizu, T; Ide, I; Qian, JB

KNOWLEDGE-BASED SYSTEMS 305 巻 2024年12月

出版者・発行元：Knowledge-Based Systems

Cross-modal recipe retrieval is vital for transforming visual food cues into actionable cooking guidance, making culinary creativity more accessible. Existing methods separately encode the recipe Title, Ingredient, and Instruction using different text encoders, then aggregate them to obtain recipe feature, and finally match it with encoded image feature in a joint embedding space. These methods perform well but require significant computational cost. In addition, they only consider matching the entire recipe and the image but ignore the fine-grained correspondence between recipe components and the image, resulting in insufficient cross-modal interaction. To this end, we propose Unified Text Encoder with Fine-grained Contrastive Learning (UTE-FCL) to achieve a simple but efficient model. Specifically, in each recipe, UTE-FCL first concatenates each of the Ingredient and Instruction texts composed of multiple sentences as a single text. Then, it connects these two concatenated texts with the original single-phrase Title to obtain the concatenated recipe. Finally, it encodes these three concatenated texts and the original Title by a Transformer-based Unified Text Encoder (UTE). This proposed structure greatly reduces the memory usage and improves the feature encoding efficiency. Further, we propose fine-grained contrastive learning objectives to capture the correspondence between recipe components and the image at Title, Ingredient, and Instruction levels by measuring the mutual information. Extensive experiments demonstrate the effectiveness of UTE-FCL compared to existing methods.

DOI： 10.1016/j.knosys.2024.112641

ICDAR'24 Workshop Chairs' Welcome Message

Dao M.S., Riegler M.A., Dang-Nguyen D.T., Tran H.N., Kiran R.U., Komamizu T.

ACM International Conference Proceeding Series 2024年6月

検出領域絞込みと検出履歴を考慮した広角映像中の鳥追跡

Tingwei Liu, 川西康友, 駒水孝裕, 井手一郎

人工知能学会第二種研究会資料 2024 巻 ( Challenge-064 ) 頁： 07 2024年3月

Computational measurement of perceived pointiness from pronunciation (vol 83, pg 26183, 2024)

Matsuhira, C; Kastner, MA; Komamizu, T; Ide, I; Hirayama, T; Kawanishi, Y; Doman, K; Deguchi, D

MULTIMEDIA TOOLS AND APPLICATIONS 83 巻 ( 9 ) 頁： 26211 - 26212 2024年3月

掲載種別：研究論文（学術雑誌）出版者・発行元：Multimedia Tools and Applications

The original publication of this article contains the following errors: missing ORCID of authors incorrect author contribution statement pronunciation symbols were not shown correctly in both online and PDF versionsthe gamma symbol "Γ" were incorrectly displayed as "0" in the PDF version missing ORCID of authors incorrect author contribution statement pronunciation symbols were not shown correctly in both online and PDF versions the gamma symbol "Γ" were incorrectly displayed as "0" in the PDF version The original article has been corrected.

DOI： 10.1007/s11042-023-17657-z

Computational measurement of perceived pointiness from pronunciation

Matsuhira, C; Kastner, MA; Komamizu, T; Ide, I; Hirayama, T; Kawanishi, Y; Doman, K; Deguchi, D

MULTIMEDIA TOOLS AND APPLICATIONS 83 巻 ( 9 ) 頁： 26183 - 26210 2024年3月

掲載種別：研究論文（学術雑誌）出版者・発行元：Multimedia Tools and Applications

Sound symbolism is a well-researched topic of psycholinguistics, which tries to comprehend the connection between the sound of a word and its meanings. The Bouba-Kiki effect, one form of sound symbolism, claims that people perceive the pronunciation of “Kiki” as pointier than that of “Bouba.” There is no research that focuses on modeling such perception, i.e., how pointy a pronunciation sounds to humans, through computational and data-driven approaches. To address this, this paper first proposes the novel concept of “phonetic pointiness” defined as how pointy a shape humans are most likely to associate with a given pronunciation. We then model this phonetic pointiness from computational and data-driven approaches to calculate a score for an arbitrary pronunciation. There are three proposed models: a referential model, an expressive model, and a combined model, which integrates the previous two. The idea comes from an existing psycholinguistic classification of two types of sound symbolisms: referential symbolism and expressive symbolism, where the former relates to vocabulary knowledge, while the latter is based on pure human intuition. The proposed models are constructed only with image and language data available on the Web, therefore not requiring task-specific human annotations. We evaluate these models through a crowd-sourced user study, finding a promising correlation between human perception and the phonetic pointiness calculated by the proposed models. The results indicate that human perception can be modeled better by combining both types of sound symbolisms. Furthermore, by observing the behaviors of the models, we show several possible use-cases, such as product naming and psycholinguistic research, which can be a useful insight to further studies and applications.

DOI： 10.1007/s11042-023-15732-z

Image-Collection Summarization Using Scene-Graph Generation With External Knowledge

Phueaksri, I; Kastner, MA; Kawanishi, Y; Komamizu, T; Ide, I

IEEE ACCESS 12 巻頁： 17499 - 17512 2024年

掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Summarization tasks aim to summarize multiple pieces of information into a short description or representative information. A text summarization task summarizes textual information into a short description, whereas an image collection summarization task summarizes an image collection into images or textual representation in which the challenge is to understand the relationship between images. In recent years, scene-graph generation has shown the advantage of describing the visual contexts of a single-image, and incorporating external knowledge into the scene-graph generation model has also given effective directions for unseen single-image scene-graph generation. While external knowledge has been implemented in related work, it is still challenging to use this information efficiently for relationship estimation during the summarization. Following this trend, in this paper, we propose a novel scene-graph-based image-collection summarization model that aims to generate a summarized scene-graph of an image collection. The key idea of the proposed method is to enhance the relation predictor toward relationships between images in an image collection incorporating knowledge graphs as external knowledge for training a model. With this approach, we build an end-to-end framework that can generate a summarized scene graph of an image collection. To evaluate the proposed method, we also build an extended annotated MS-COCO dataset for this task and introduce an evaluation process that focuses on estimating the similarity between a summarized scene graph and ground-truth scene graphs. Traditional evaluation focuses on calculating precision and recall scores, which involve true positive predictions without balancing precision and recall. Meanwhile, the proposed evaluation process focuses on calculating the F-score of the similarity between a summarized scene graph and ground-truth scene graphs, which aims to balance both false positives and false negatives. Experimental results show that using external knowledge to enhance the relation predictor achieves better results than existing methods.

DOI： 10.1109/ACCESS.2024.3360113

Zero-Shot Pill-Prescription Matching With Graph Convolutional Network and Contrastive Learning

Nguyen, TT; Nguyen, PL; Kawanishi, Y; Komamizu, T; Ide, I

IEEE ACCESS 12 巻頁： 55889 - 55904 2024年

出版者・発行元：IEEE Access

Patients' safety is paramount in the healthcare industry, and reducing medication errors is essential for improvement. A promising solution to this problem involves the development of automated systems capable of assisting patients in verifying their pill intake mistakes. This paper investigates a Pill-Prescription matching task that seeks to associate pills in a multi-pill photo with their corresponding names in the prescription. We specifically aim to overcome the limitations of existing pill detection methods when faced with unseen pills, a situation characteristic of zero-shot learning. We propose a novel method named Zero-PIMA (Zero-shot Pill-Prescription Matching), designed to match pill images with prescription names effectively, even for pills not included in the training dataset. Zero-PIMA is an end-to-end model that includes an object localization module to determine and extract features of pill images and a graph convolutional network to capture the spatial relationship of the pills' text in the prescription. After that, we leverage the contrastive learning paradigm to increase the distance between mismatched pill images and pill name pairs while minimizing the distance between matched pairs. In addition, to deal with the zero-shot pill detection problem, we leverage pills' metadata retrieved from the DrugBank database to fine-tune a pre-trained text encoder, thereby incorporating visual information about pills (e.g., shape, color) into their names, making them more informative and ultimately enhancing the pill image-name matching accuracy. Extensive experiments are conducted on our collected real-world VAIPEPP dataset of multi-pill photos and prescriptions. Through a series of comprehensive experiments, the proposed method outperforms other methods for both seen and unseen pills in terms of mean average precision. These results indicate that the proposed method could reduce medication errors and improve patients' safety.

DOI： 10.1109/ACCESS.2024.3390153

R-DiP: Re-ranking Based Diffusion Pre-computation for Image Retrieval

Kato, T; Komamizu, T; Ide, I

DATABASE AND EXPERT SYSTEMS APPLICATIONS, PT II, DEXA 2024 14911 巻頁： 233 - 247 2024年

出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

In image retrieval tasks, although efficient methods based on pre-computing information related to retrieval and effective methods utilizing re-ranking have been proposed, developing a method that achieves both efficiency and effectiveness at the same time, remains challenging. To develop an efficient and effective image retrieval method, we propose a simple-yet-effective novel image retrieval framework; R-DiP (Re-ranking based Diffusion Pre-computation). It incorporates an effective re-ranking model into the pre-computation step of an existing efficient method, namely, Offline Diffusion that pre-computes the diffusion process in the offline step and provides a simple linear combination-based retrieval in the online step. Experimental results on standard benchmarks shows that R-DiP performs comparable to the State-Of-The-Art (SOTA) image retrieval method, namely SuperGlobal, while maintaining the efficiency of Offline Diffusion. Notably, in million-scale datasets, R-DiP improves the mAP (mean Average Precision) by about 2.0%, and reduces the speed by about 75% on average, surpassing SOTA methods. These results indicate that R-DiP is a promising solution to the efficiency-effectiveness trade-off in image retrieval, that offers the flexibility to incorporate any advanced re-ranking method in the future.

DOI： 10.1007/978-3-031-68312-1_18

One-Stage Open-Vocabulary Temporal Action Detection Leveraging Temporal Multi-scale and Action Label Features

Nguyen, TT; Kawanishi, Y; Komamizu, T; Ide, I

2024 IEEE 18TH INTERNATIONAL CONFERENCE ON AUTOMATIC FACE AND GESTURE RECOGNITION, FG 2024 2024年

出版者・発行元：2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition, FG 2024

Open-vocabulary Temporal Action Detection (Open-vocab TAD) is an advanced video analysis approach that expands Closed-vocabulary Temporal Action Detection (Closed-vocab TAD) capabilities. Closed-vocab TAD is typically confined to localizing and classifying actions based on a predefined set of categories. In contrast, Open-vocab TAD goes further and is not limited to these predefined categories. This is particularly useful in real-world scenarios where the variety of actions in videos can be vast and not always predictable. The prevalent methods in Open-vocab TAD typically employ a 2-stage approach, which involves generating action proposals and then identifying those actions. However, errors made during the first stage can adversely affect the subsequent action identification accuracy. Additionally, existing studies face challenges in handling actions of different durations owing to the use of fixed temporal processing methods. Therefore, we propose a L-stage approach consisting of two primary modules: Multi-scale Video Analysis (MVA) and Video-Text Alignment (VTA). The MVA module captures actions at varying temporal resolutions, overcoming the challenge of detecting actions with diverse durations. The VTA module leverages the synergy between visual and textual modalities to precisely align video segments with corresponding action labels, a critical step for accurate action identification in Open-vocab scenarios. Evaluations on widely recognized datasets THUMOSl4 and ActivityNet-I.3, showed that the proposed method achieved superior results compared to the other methods in both Open-vocab and Closed-vocab settings. This serves as a strong demonstration of the effectiveness of the proposed method in the TAD task.

DOI： 10.1109/FG59268.2024.10581896

Interpolating the Text-to-Image Correspondence Based on Phonetic and Phonological Similarities for Nonword-to-Image Generation

Matsuhira, C; Kastner, MA; Komamizu, T; Hirayama, T; Doman, K; Kawanishi, Y; Ide, I

IEEE ACCESS 12 巻頁： 41299 - 41316 2024年

掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Text-to-Image (T2I) generation is the task of synthesizing images corresponding to a given text input. The recent innovations in artificial intelligence have enhanced the capacity of conventional T2I generation, yielding more and more powerful models day by day. However, their behavior is known to become unstable in the face of text inputs containing nonwords that have no definition within a language. This behavior not only results in situations where image generation does not match human expectations but also hinders these models from being utilized in psycholinguistic applications and simulations. This paper exploits the human nature of associating nonwords with their phonetically and phonologically similar words and uses it to propose a T2I generation framework robust against nonword inputs. The framework comprises a phonetics-aware language model as well as an adjusted T2I generation model. Our evaluations confirm that the proposed nonword-to-image generation synthesizes images that depict visual concepts of phonetically similar words more stably than comparative methods. We also assess how the image generation results match human expectations, showing a better agreement than the phonetics-blind baseline.

DOI： 10.1109/ACCESS.2024.3378095

ICDAR 24: Intelligent Cross-Data Analysis and Retrieval

Dao, MS; Riegler, MA; Dang-Nguyen, DT; Tran, HN; Kiran, RU; Komamizu, T

PROCEEDINGS OF THE 4TH ANNUAL ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, ICMR 2024 頁： 1332 - 1333 2024年

出版者・発行元：ICMR 2024 - Proceedings of the 2024 International Conference on Multimedia Retrieval

Our workshop aims to provide a platform for both academic and industrial professionals engaged in the analysis and retrieval of cross-data from diverse perspectives, with a particular emphasis on wearable and ambient sensors, lifelog cameras, social networks, and surrounding sensors. Despite numerous studies exploring individual viewpoints, there remains a significant gap in the analysis and retrieval of cross-data to maximize benefits for humanity. Additionally, challenges such as data security and distributed learning for cross-modal model training and inference arise when dealing with large and distributed datasets. We invite researchers to contribute to this initiative, with the overarching goal of fostering the development of a smart and sustainable society through the efficient utilization of intelligent cross-data analysis and retrieval techniques.

DOI： 10.1145/3652583.3659999

Do LLMs Agree with Humans on Emotional Associations to Nonsense Words?

Miyakawa Y., Matsuhira C., Kato H., Hirayama T., Komamizu T., Ide I.

CMCL 2024 - 13th Edition of the Workshop on Cognitive Modeling and Computational Linguistics, Proceedings of the Workshop 頁： 81 - 85 2024年

出版者・発行元：CMCL 2024 - 13th Edition of the Workshop on Cognitive Modeling and Computational Linguistics, Proceedings of the Workshop

Understanding human perception of nonsense words is helpful to devise product and character names that match their characteristics. Previous studies have suggested the usefulness of Large Language Models (LLMs) for estimating such human perception, but they did not focus on its emotional aspects. Hence, this study aims to elucidate the relationship of emotions evoked by nonsense words between humans and LLMs. Using a representative LLM, GPT-4, we reproduce the procedure of an existing study to analyze evoked emotions of humans for nonsense words. A positive correlation of 0.40 was found between the emotion intensity scores reproduced by GPT-4 and those manually annotated by humans. Although the correlation is not very high, this demonstrates that GPT-4 may agree with humans on emotional associations to nonsense words. Considering that the previous study reported that the correlation among human annotators was about 0.68 on average and that between a regression model trained on the annotations for real words and humans was 0.17, GPT-4’s agreement with humans is notably strong.

Investigating Conceptual Blending of a Diffusion Model for Improving Nonword-to-Image Generation

Matsuhira C., Kastner M.A., Komamizu T., Hirayama T., Ide I.

MM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia 頁： 7307 - 7315 2024年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：MM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia

Text-to-image diffusion models sometimes depict blended concepts in the generated images. One promising use case of this effect would be the nonword-to-image generation task which attempts to generate images intuitively imaginable from a non-existing word (nonword). To realize nonword-to-image generation, an existing study focused on associating nonwords with similar-sounding words. Since each nonword can have multiple similar-sounding words, generating images containing their blended concepts would increase intuitiveness, facilitating creative activities and promoting computational psycholinguistics. Nevertheless, no existing study has quantitatively evaluated this effect in either diffusion models or the nonword-to-image generation paradigm. Therefore, this paper first analyzes the conceptual blending in a pretrained diffusion model, Stable Diffusion. The analysis reveals that a high percentage of generated images depict blended concepts when inputting an embedding interpolating between the text embeddings of two text prompts referring to different concepts. Next, this paper explores the best text embedding space conversion method of an existing nonword-to-image generation framework to ensure both the occurrence of conceptual blending and image generation quality. We compare the conventional direct prediction approach with the proposed method that combines k-nearest neighbor search and linear regression. Evaluation reveals that the enhanced accuracy of the embedding space conversion by the proposed method improves the image generation quality, while the emergence of conceptual blending could be attributed mainly to the specific dimensions of the high-dimensional text embedding space.

DOI： 10.1145/3664647.3681202

その他リンク： https://dblp.uni-trier.de/db/conf/mm/mm2024.html#Matsuhira0KHI24

Lightweight Maize Disease Detection through Post-Training Quantization with Similarity Preservation

Padeiro C.V., Chen T.W., Komamizu T., Ide I.

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops 頁： 2111 - 2120 2024年

出版者・発行元：IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

Traditional crop disease diagnosis, reliant on expert visual observation, is expensive, time-consuming, and prone to error. While Convolutional Neural Networks (CNNs) offer promising alternatives, their high resource demands limit their accessibility to farmers, particularly those in resource-constrained settings. Lightweight models that operate on resource-limited devices without network access are crucial to address this gap. This paper proposes a Similarity-Preserving Quantization (SPQ) method to convert high-precision CNNs into lower-precision models while maintaining similar feature representations. While quantization offers a promising approach for building lightweight CNNs for crop disease detection, the quality of quantized models often suffers. SPQ addresses this challenge by ensuring equivalent activation patterns for similar crop images in both the original and quantized models. Experimental evaluation using MobileNetV2 and ResNet-50 demonstrates that SPQ improves throughput, inference, and memory footprint more than 3 times while preserving the detection performance.

DOI： 10.1109/CVPRW63382.2024.00216

L3Masking: Multi-task Fine-tuning for Language Models by Leveraging Lessons Learned from Vanilla Models

Yusuke Kimura, Takahiro Komamizu, Kenji Hatano

Proceedings of the 1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual (CustomNLP4U) 頁： 53 - 62 2024年

L3Masking: Multi-task Fine-tuning for Language Models by Leveraging Lessons Learned from Vanilla Models

Kimura Y., Komamizu T., Hatano K.

1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual, CustomNLP4U 2024 - Proceedings of the Workshop 頁： 53 - 62 2024年

出版者・発行元：1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual, CustomNLP4U 2024 - Proceedings of the Workshop

When distributional differences exist between pre-training and fine-tuning data, language models (LMs) may perform poorly on downstream tasks. Recent studies have reported that multi-task learning of downstream task and masked language modeling (MLM) task during the fine-tuning phase improves the performance of the downstream task. Typical MLM tasks (e.g., random token masking (RTM)) tend not to care tokens corresponding to the knowledge already acquired during the pre-training phase, therefore LMs may not notice the important clue or not effective to acquire linguistic knowledge of the task or domain. To overcome this limitation, we propose a new masking strategy for MLM task, called L3Masking 1, that leverages lessons (specifically, token-wise likelihood in a context) learned from the vanilla language model to be fine-tuned. L3Masking actively masks tokens with low likelihood on the vanilla model. Experimental evaluations on text classification tasks in different domains confirms a multi-task text classification method with L3Masking performed task adaptation more effectively than that with RTM. These results suggest the usefulness of assigning a preference to the tokens to be learned as the task or domain adaptation.

RecipeMeta: Metapath-enhanced Recipe Recommendation on Heterogeneous Recipe Network

Shi J., Komamizu T., Doman K., Kyutoku H., Ide I.

Proceedings of the 5th ACM International Conference on Multimedia in Asia, MMAsia 2023 2023年12月

出版者・発行元：Proceedings of the 5th ACM International Conference on Multimedia in Asia, MMAsia 2023

Recipe is a set of instructions that describes how to make food. It can help people from the preparation of ingredients, food cooking process, etc. to prepare the food, and increasingly in demand on the Web. To help users find the vast amount of recipes on the Web, we address the task of recipe recommendation. Due to multiple data types and relationships in a recipe, we can treat it as a heterogeneous network to describe its information more accurately. To effectively utilize the heterogeneous network, metapath was proposed to describe the higher-level semantic information between two entities by defining a compound path from peer entities. Therefore, we propose a metapath-enhanced recipe recommendation framework, RecipeMeta, that combines GNN (Graph Neural Network)-based representation learning and specific metapath-based information in a recipe to predict User-Recipe pairs for recommendation. Through extensive experiments, we demonstrate that the proposed model, RecipeMeta, outperforms state-of-the-art methods for recipe recommendation.

DOI： 10.1145/3595916.3626430

NarSUM 2023 Chairs Welcome

Kankanhalli M.S., Patras I., Liu J., Wong Y., Komamizu T.

NarSUM 2023 - Proceedings of the 2nd Workshop on User-centric Narrative Summarization of Long Videos, Co-located with: MM 2023 2023年10月

An Automatic Labeling Method for Subword-Phrase Recognition in Effective Text Classification

Kimura Y., Komamizu T., Hatano K.

Informatica (Slovenia) 47 巻 ( 3 ) 頁： 315 - 326 2023年8月

掲載種別：研究論文（学術雑誌）出版者・発行元：Informatica (Slovenia)

The deep learning-based text classification methods perform better than traditional ones. In addition to the success of the deep learning technique, multi-Task learning (MTL) has come to become a promising approach for text classification; for instance, an MTL approach in text classification employs named entity recognition as an auxiliary task and has showcased that the task helps to improve the text classification performance. Existing MTL-based text classification methods depend on the auxiliary tasks using supervised labels. Obtaining such supervision labels requires additional human and financial costs in addition to those for the main text classification task. To reduce these additional costs, we propose an MTL-based text classification framework on supervised label creation by automatically labeling phrases in texts for the auxiliary recognition task. A basic idea to realize the proposed framework is to utilize phrasal expressions consisting of subwords (called subword-phrases). To the best of our knowledge, no text classification approach has been designed on top of subword-phrases because subwords only sometimes express a coherent set of meanings. The novelty of the proposed framework is in adding subword-phrase recognition as an auxiliary task and utilizing subword-phrases for text classification. It extracts subword-phrases in an unsupervised manner using the statistics approach. To construct labels for effective subword-phrase recognition tasks, extracted subword-phrases are classified based on document classes to ensure that subword-phrases dedicated to some classes can be distinguishable. Experimental evaluation for text classification using five popular datasets showcased the effectiveness of the subword-phrase recognition as an auxiliary task. It also showed that comparing various labeling schemes in recent studies indicated insights for labeling common subword-phrases among several document classes.

DOI： 10.31449/inf.v47i3.4742

Image Impression Estimation by Clustering People with Similar Tastes

Kojima Banri, Komamizu Takahiro, Kawanishi Yasutomo, Doman Keisuke, Ide Ichiro

IEICE Proceeding Series 78 巻頁： P1-14 2023年7月

記述言語：英語出版者・発行元：The Institute of Electronics, Information and Communication Engineers

This paper proposes a method for estimating impressions received from images according to the personal attributes of users, so that they can find the desired images based on their tastes. A previous study taking into account gender and age as personal attributes showed promising results. However, it also showed that users sharing the same gender and age do not necessarily share similar tastes. Therefore, other attributes should be considered to well capture users' personal tastes. However, taking more attributes into account leads to a problem that insufficient amounts of data are served to classifiers, due to explosion of the number of combinations of attributes. To tackle this problem, we propose an aggregation-based method to condense training data for impression estimation while personal attribute information is taken into account. For evaluation, a dataset of 4,000 carpet images annotated with 24 impression words by crowd-workers was prepared, which contained 273k annotations. Experimental results showed that the use of combinations of personal attributes improved the accuracy of impression estimation. This indicates that combinations of personal attributes are helpful to estimate impressions of individual viewers to images.

DOI： 10.34385/proc.78.p1-14

MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results

Kondo Yuki, Ukita Norimichi, Yamaguchi Takayuki, Hou Hao-Yu, Shen Mu-Yi, Hsu Chia-Chi, Huang En-Ming, Huang Yu-Chen, Xia Yu-Cheng, Wang Chien-Yao, Lee Chun-Yi, Huo Da, Kastner Marc-A., Liu Tingwei, Kawanishi Yasutomo, Hirayama Takatsugu, Komamizu Takahiro, Ide Ichiro, Shinya Yosuke, Liu Xinyao, Liang Guang, Yasui Syusuke

IEICE Proceeding Series 78 巻頁： TE-1 2023年7月

記述言語：英語出版者・発行元：The Institute of Electronics, Information and Communication Engineers

Small Object Detection (SOD) is an important machine vision topic because (i) a variety of real-world applications require object detection for distant objects and (ii) SOD is a challenging task due to the noisy, blurred, and less-informative image appearances of small objects. This paper proposes a new SOD dataset consisting of 39,070 images including 137,121 bird instances, which is called the Small Object Detection for Spotting Birds (SOD4SB) dataset. The detail of the challenge with the SOD4SB dataset is introduced in this paper. In total, 223 participants joined this challenge. This paper briefly introduces the awardwinning methods. The dataset, the baseline code, and the website for evaluation on the public testset are publicly available.

DOI： 10.34385/proc.78.te-1

Small Object Detection for Birds with Swin Transformer

Huo Da, Kastner Marc-A., Liu Tingwei, Kawanishi Yasutomo, Hirayama Takatsugu, Komamizu Takahiro, Ide Ichiro

IEICE Proceeding Series 78 巻頁： TE-3 2023年7月

記述言語：英語出版者・発行元：The Institute of Electronics, Information and Communication Engineers

Object detection is the task of detecting objects in an image. In this task, the detection of small objects is particularly difficult. Other than the small size, it is also accompanied by difficulties due to blur, occlusion, and so on. Current small object detection methods are tailored to small and dense situations, such as pedestrians in a crowd or far objects in remote sensing scenarios. However, when the target object is small and sparse, there is a lack of objects available for training, making it more difficult to learn effective features. In this paper, we propose a specialized method for detecting a specific category of small objects; birds. Particularly, we improve the features learned by the neck; the sub-network between the backbone and the prediction head, to learn more effective features with a hierarchical design. We employ Swin Transformer to upsample the image features. Moreover, we change the shifted window size for adapting to small objects. Experiments show that the proposed Swin Transformerbased neck combined with CenterNet can lead to good performance by changing the window sizes. We further find that smaller window sizes (default 2) benefit mAPs for small object detection.

DOI： 10.34385/proc.78.te-3

Towards Achieving Lightweight Deep Neural Network for Precision Agriculture with Maize Disease Detection

Padeiro Carlos-Victorino, Komamizu Takahiro, Ide Ichiro

IEICE Proceeding Series 78 巻頁： P1-23 2023年7月

記述言語：英語出版者・発行元：The Institute of Electronics, Information and Communication Engineers

Agriculture is the pillar industry of human sur- vival. However, various crop diseases reduce the hu- man food supply and lead to starvation and death in the worst cases. Experts perform visual symptoms ob- servation for crop disease diagnosis. Which process is time-consuming and expensive. Also, the process has significant risk of human error due to subjective per- ception. Convolutional Neural Networks (CNN) use image processing techniques to show great potential in plant disease detection. However, it requires thou- sands of channels to learn rich features, resulting in large models requiring powerful computing, power sup- ply, and high bandwidth, making it more expensive and difficult for farmers to acquire. Therefore, deploying these solutions on resource-constrained devices is de- sirable to make them more accessible. Thus, we pro- pose a lightweight object detection CNN that can run on resource-constrained devices to detect crop diseases. Channel pruning is applied to optimize resource use by removing unimportant channels and filter weights to reduce network parameters, inference time, and the number of FLOPS. Experimental results with object de- tector, Faster R-CNN with two backbones, ResNet-50, and EfficientNet-B7, show significant improvement in model efficiency, keeping high accuracy.

DOI： 10.34385/proc.78.p1-23

[D21] 歴史情報としての法令データベースの構築

佐野智也, 外山勝彦, 駒水孝裕, 増田知子

デジタルアーカイブ学会誌 7 巻 ( s2 ) 頁： s142 - s145 2023年

記述言語：日本語出版者・発行元：デジタルアーカイブ学会

国家・社会制度に関する政策は、法令を通して制度化されるため、日本社会の動きは、法令情報を介して捉えることができる。本研究は、制定・改正などを通じた法令の連続的変遷を把握し、日本の国家・社会運営の長期的変化を調査するための研究基盤の確立を目指すものである。その最初の目標として、明治以降の全法令を検索可能なオープンデータベースシステムの構築を進めているが、現在、明治 19（1886）年から平成 29（2017）年までに公布された法律と勅令のXML文書化を完了し、それらの全文検索が可能なデータベースの構築を終えた。本報告では、既存のデータベースの問題点について述べた上で、構築したデータベースを説明する。

DOI： 10.24506/jsda.7.s2_s142

NarSUM'23: The 2ndWorkshop on User-Centric Narrative Summarization of Long Videos

Kankanhalli, M; Patras, I; Liu, JQ; Wong, YK; Komamizu, T; Yamazaki, S; Stephen, K; Kansal, K

PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023 頁： 9731 - 9733 2023年

出版者・発行元：MM 2023 - Proceedings of the 31st ACM International Conference on Multimedia

With video capture devices becoming widely popular, the amount of video data generated per day has seen a rapid increase over the past few years. Browsing through hours of video data to retrieve useful information is a tedious and boring task. Video Summarization technology has played a crucial role in addressing this issue. It is a well-researched topic in the multimedia community. However, the focus so far has been limited to creating summary to videos which are short (only a few minutes). This workshop aims to call for researchers on relevant background to focus on novel solutions for user-centric narrative summarization of long videos. This workshop will also cover important aspects of video summarization research like what is "important"in a video, how to evaluate the goodness of a created summary, open challenges in video summarization, etc.

DOI： 10.1145/3581783.3610946

MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results.

Yuki Kondo, Norimichi Ukita, Takayuki Yamaguchi, Hao-Yu Hou, Mu-Yi Shen, Chia-Chi Hsu, En-Ming Huang, Yu-Chen Huang, Yu-Cheng Xia, Chien-Yao Wang, Chun-Yi Lee, Da Huo, Marc A. Kastner 0001, Tingwei Liu, Yasutomo Kawanishi, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide, Yosuke Shinya, Xinyao Liu, Guang Liang, Syusuke Yasui

CoRR abs/2307.09143 巻 2023年

MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results

Kondo, Y; Ukita, N; Yamaguchi, T; Hou, HY; Shen, MY; Hsu, CC; Huang, EM; Huang, YC; Xia, YC; Wang, CY; Lee, CY; Da Huo; Kastner, MA; Liu, TW; Kawanishi, Y; Hirayama, T; Komamizu, T; Ide, I; Shinya, Y; Liu, XY; Liang, G; Yasui, S

2023 18TH INTERNATIONAL CONFERENCE ON MACHINE VISION AND APPLICATIONS, MVA 頁： 1 - 11 2023年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Proceedings of MVA 2023 - 18th International Conference on Machine Vision and Applications

Small Object Detection (SOD) is an important machine vision topic because (i) a variety of real-world applications require object detection for distant objects and (ii) SOD is a challenging task due to the noisy, blurred, and less-informative image appearances of small objects. This paper proposes a new SOD dataset consisting of 39,070 images including 137,121 bird instances, which is called the Small Object Detection for Spotting Birds (SOD4SB) dataset. The detail of the challenge with the SOD4SB dataset 1 is introduced in this paper. In total, 223 participants joined this challenge. This paper briefly introduces the award-winning methods. The dataset 2, the baseline code 3, and the website for evaluation on the public testset 4 are publicly available.

DOI： 10.23919/MVA57639.2023.10215935

その他リンク： https://dblp.uni-trier.de/db/conf/mva/mva2023.html#KondoUYHSHHHXWLHKLKHKISLLY23

IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretraining.

Chihaya Matsuhira, Marc A. Kastner 0001, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide

CoRR abs/2303.03144 巻 2023年

Image Impression Estimation by Clustering People with Similar Tastes

Kojima, B; Komamizu, T; Kawanishi, Y; Doman, K; Ide, I

2023 18TH INTERNATIONAL CONFERENCE ON MACHINE VISION AND APPLICATIONS, MVA 頁： 1 - 5 2023年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Proceedings of MVA 2023 - 18th International Conference on Machine Vision and Applications

This paper proposes a method for estimating impressions from images according to the personal attributes of users so that they can find the desired images based on their tastes. Our previous work, which considered gender and age as personal attributes, showed promising results, but it also showed that users sharing these attributes do not necessarily share similar tastes. Therefore, other attributes should be considered to capture the personal tastes of each user well. However, taking more attributes into account leads to a problem in which insufficient amounts of data are served to classifiers due to the explosion of the number of combinations of attributes. To tackle this problem, we propose an aggregation-based method to condense training data for impression estimation while considering personal attribute information. For evaluation, a dataset of 4,000 carpet images annotated with 24 impression words was prepared. Experimental results showed that the use of combinations of personal attributes improved the accuracy of impression estimation, which indicates the effectiveness of the proposed approach.

DOI： 10.23919/MVA57639.2023.10216055

その他リンク： https://dblp.uni-trier.de/db/conf/mva/mva2023.html#KojimaKKDI23

An Approach to Generate a Caption for an Image Collection Using Scene Graph Generation

Phueaksri, I; Kastner, MA; Kawanishi, Y; Komamizu, T; Ide, I

IEEE ACCESS 11 巻頁： 128245 - 128260 2023年

掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Summarization is a challenging task that aims to generate a summary by grasping common information of a given set of information. Text summarization is a popular task of determining the topic or generating a textual summary of documents. In contrast, image summarization aims to find a representative summary of a collection of images. However, current methods are still restricted to generating a visual scene graph, tags, and noun phrases, but cannot generate a fitting textual description of an image collection. Thus, we introduce a novel framework for generating a summarized caption of an image collection. Since scene graph generation shows advancement in describing objects and their relationships on a single image, we use it in the proposed method to generate a scene graph for each image in an image collection. Then, we find common objects and their relationships from all scene graphs and represent them as a summarized scene graph. For this, we merge all scene graphs and select part of it by estimating the most common objects and relationships. Finally, the summarized scene graph is input into a captioning model. In addition, we introduce a technique to generalize specific words in the final caption into common concept words incorporating external knowledge. To evaluate the proposed method, we construct a dataset for this task by extending the annotation of the MS-COCO dataset using an image retrieval method. The evaluation of the proposed method on this dataset showed promising performance compared to text summarization-based methods.

DOI： 10.1109/ACCESS.2023.3332098

Nonword-to-Image Generation Considering Perceptual Association of Phonetically Similar Words

Matsuhira, C; Kastner, MA; Komamizu, T; Hirayama, T; Doman, K; Ide, I

PROCEEDINGS OF THE 1ST INTERNATIONAL WORKSHOP ON MULTIMEDIA CONTENT GENERATION AND EVALUATION, MCGE 2023 頁： 115 - 125 2023年

Visual Passage Score Aggregation for Image Retrieval

Komamizu T.

Proceedings - 2023 IEEE 6th International Conference on Multimedia Information Processing and Retrieval, MIPR 2023 頁： 37 - 42 2023年

出版者・発行元：Proceedings - 2023 IEEE 6th International Conference on Multimedia Information Processing and Retrieval, MIPR 2023

This paper proposes an effective image retrieval method. Recent image retrieval approaches attempt to construct a single global feature, including local features of an image. In contrast, this paper proposes multiple features for each image. The basic idea is that a target object in a query image is not necessarily in a major part of a database image; therefore, its single feature may include noisy information from the surroundings of the target object. To deal with this, this paper proposes a Visual Passage Score Aggregation framework (VPSA). VPSA first decomposes an image into several pieces of images, called Visual Passages. Based on visual passages, VPSA aggregates relevance scores of visual passages for ranking. VPSA is efficient in the retrieval phase because an ordinary nearest neighbor search is used. The experiment revealed that VPSA showed superior or comparable performance to the state-of-the-art methods, and it takes a shorter time in the retrieval phase.

DOI： 10.1109/MIPR59079.2023.00021

Towards Captioning an Image Collection from a Combined Scene Graph Representation Approach

Phueaksri, I; Kastner, MA; Kawanishi, Y; Komamizu, T; Ide, I

MULTIMEDIA MODELING, MMM 2023, PT I 13833 巻頁： 178 - 190 2023年

掲載種別：論文集(書籍)内論文出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Most content summarization models from the field of natural language processing summarize the textual contents of a collection of documents or paragraphs. In contrast, summarizing the visual contents of a collection of images has not been researched to this extent. In this paper, we present a framework for summarizing the visual contents of an image collection. The key idea is to collect the scene graphs for all images in the image collection, create a combined representation, and then generate a visually summarizing caption using a scene-graph captioning model. Note that this aims to summarize common contents across all images in a single caption rather than describing each image individually. After aggregating all the scene graphs of an image collection into a single scene graph, we normalize it by using an additional concept generalization component. This component selects the common concept in each sub-graph with ConceptNet based on word embedding techniques. Lastly, we refine the captioning results by replacing a specific noun phrase with a common concept from the concept generalization component to improve the captioning results. We construct a dataset for this task based on the MS-COCO dataset using techniques from image classification and image-caption retrieval. An evaluation of the proposed method on this dataset shows promising performance.

DOI： 10.1007/978-3-031-27077-2_14

Towards Achieving Lightweight Deep Neural Network for Precision Agriculture with Maize Disease Detection

Padeiro, CV; Komamizu, T; Ide, I

2023 18TH INTERNATIONAL CONFERENCE ON MACHINE VISION AND APPLICATIONS, MVA 頁： 1 - 6 2023年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Proceedings of MVA 2023 - 18th International Conference on Machine Vision and Applications

Agriculture is the pillar industry of human survival. However, various crop diseases reduce the human food supply and lead to starvation and death in the worst cases. Experts perform visual symptoms observation for crop disease diagnosis. Which process is time-consuming and expensive. Also, the process has significant risk of human error due to subjective perception. Convolutional Neural Networks (CNN) use image processing techniques to show great potential in plant disease detection. However, it requires thousands of channels to learn rich features, resulting in large models requiring powerful computing, power supply, and high bandwidth, making it more expensive and difficult for farmers to acquire. Therefore, deploying these solutions on resource-constrained devices is desirable to make them more accessible. Thus, we propose a lightweight object detection CNN that can run on resource-constrained devices to detect crop diseases. Channel pruning is applied to optimize resource use by removing unimportant channels and filter weights to reduce network parameters, inference time, and the number of FLOPS. Experimental results with object detector, Faster R-CNN with two backbones, ResNet-50, and EfficientNet-B7, show significant improvement in model efficiency, keeping high accuracy.

DOI： 10.23919/MVA57639.2023.10215815

その他リンク： https://dblp.uni-trier.de/db/conf/mva/mva2023.html#PadeiroKI23

Small Object Detection for Birds with Swin Transformer

Huo, D; Kastner, MA; Liu, T; Kawanishi, Y; Hirayama, T; Komamizu, T; Ide, I

2023 18TH INTERNATIONAL CONFERENCE ON MACHINE VISION AND APPLICATIONS, MVA 頁： 1 - 5 2023年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Proceedings of MVA 2023 - 18th International Conference on Machine Vision and Applications

Object detection is the task of detecting objects in an image. In this task, the detection of small objects is particularly difficult. Other than the small size, it is also accompanied by difficulties due to blur, occlusion, and so on. Current small object detection methods are tailored to small and dense situations, such as pedestrians in a crowd or far objects in remote sensing scenarios. However, when the target object is small and sparse, there is a lack of objects available for training, making it more difficult to learn effective features. In this paper, we propose a specialized method for detecting a specific category of small objects; birds. Particularly, we improve the features learned by the neck; the sub-network between the backbone and the prediction head, to learn more effective features with a hierarchical design. We employ Swin Transformer to upsample the image features. Moreover, we change the shifted window size for adapting to small objects. Experiments show that the proposed Swin Transformer-based neck combined with CenterNet can lead to good performance by changing the window sizes. We further find that smaller window sizes (default 2) benefit mAPs for small object detection.

DOI： 10.23919/MVA57639.2023.10216093

その他リンク： https://dblp.uni-trier.de/db/conf/mva/mva2023.html#HuoKLKHKI23

Multi-Task Learning-based Text Classification with Subword-Phrase Extraction

Kimura Y., Komamizu T., Hatano K.

ACM International Conference Proceeding Series 頁： 23 - 30 2022年12月

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：ACM International Conference Proceeding Series

Text classification using deep learning, which is trained with a tremendous amount of text, has achieved superior performance than traditional methods. In addition to its success, multi-Task learning has become a promising approach for text classification; for instance, a multi-Task learning approach employs named entity recognition as an auxiliary task for text classification. The existing MTL-based text classification methods depend on auxiliary tasks using supervised labels, which require large human and/or financial efforts to create. To reduce these efforts, this paper proposes a multi-Task learning-based text classification framework which reduces the additional efforts on supervised label creation. A basic idea to realize this is that to utilize phrasal expressions consisting of subwords (called subword-phrase). To the best of our knowledge, there has been no text classification approach on top of subword-phrases, because subwords do not always express a coherent set of meanings. The proposed framework is new to add subword-phrase recognition as an auxiliary task, and to utilize subword-phrases for text classification. To realize the low-cost auxiliary recognition task, the framework extracts subword-phrases in an unsupervised manner. The experimental evaluation of the five popular datasets for text classification showcases the effectiveness of the involvement of the subword-phrase recognition as an auxiliary task. It also shows comparative results with the state-of-The-Art method.

DOI： 10.1145/3568562.3568635

Detection of Birds in a 3D Environment Referring to Audio-Visual Information

Kawanishi, Y; Ide, I; Chu, B; Matsuhira, C; Kastner, MA; Komamizu, T; Deguchi, D

2022 18TH IEEE INTERNATIONAL CONFERENCE ON ADVANCED VIDEO AND SIGNAL BASED SURVEILLANCE (AVSS 2022) 頁： 1 - 7 2022年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：AVSS 2022 - 18th IEEE International Conference on Advanced Video and Signal-Based Surveillance

We propose a method to detect birds in a 3D environment referring to both audio information observed from a microphone array and visual information observed from a panorama camera. In general, in panorama images, birds appear relatively too small to be detected accurately even with the state-of-the-art deep learning models. Thus, the proposed method takes a two step approach where the birds are first roughly located referring to audio information by Sound Source Localization (SSL), and then image detection is applied within its vicinity. Through evaluation on a dataset annotated with bounding boxes surrounding the birds, we show that the proposed method improves detection performance of birds that appear in relatively small sizes in the image, in both accuracy and processing speed.

DOI： 10.1109/AVSS56176.2022.9959510

その他リンク： https://dblp.uni-trier.de/db/conf/avss/avss2022.html#KawanishiICMKKD22

Action Semantic Alignment for Image Captioning

Huo D., Kastner M.A., Komamizu T., Ide I.

Proceedings - 5th International Conference on Multimedia Information Processing and Retrieval, MIPR 2022 頁： 194 - 197 2022年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Proceedings - 5th International Conference on Multimedia Information Processing and Retrieval, MIPR 2022

Image captioning is one of the main goals in vision and language processing, which aims to generate proper descriptions of images. Recently, the attention mechanisms became crucial in captioning tasks, as they can capture global dependencies between modalities. Moreover, some works have used objects detected from the input image as anchor points, so called object tags, to ease such alignments resulting in good performance for this task. In this paper, we newly introduce action information as a prior to further improve this, by adding action tags for training. The action tags can learn alignment at action semantic level and catch the previously ignored dimension of action, that could be very important in image captioning. We found that training with action tags can be used to describe images in a dynamic style. Furthermore, we found it can actually lead to a significant improvement compared with other methods in captioning performance measured by common metrics.

DOI： 10.1109/MIPR54900.2022.00041

その他リンク： https://dblp.uni-trier.de/db/conf/mipr/mipr2022.html#HuoKKI22

Towards Efficient Data Access Through Multiple Relationship in Graph-Structured Digital Archives

Kusu, K; Komamizu, T; Hatano, K

FROM BORN-PHYSICAL TO BORN-VIRTUAL: AUGMENTING INTELLIGENCE IN DIGITAL LIBRARIES, ICADL 2022 13636 巻頁： 377 - 391 2022年

掲載種別：論文集(書籍)内論文出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

The research field of digital libraries mainly deals with data with graph structure. Graph database management systems (GDBMSs) are suitable for managing data in the digital library because the data size is large and its structure is complex. However, when performing a non-simple search or analysis on a graph, GDBMSs cannot avoid reaching already-scanned nodes from different starting nodes by repeatedly traversing edges such as property paths pattern in SPARQL. Therefore, when a GDBMS reaches high degree nodes, the number of graph traversals increases in proportion to the number of its adjacent nodes. Consequently, the cost of traversing multiple paths extremely increases affected by nodes connected enormous the number of edges in conventional GDBMSs. In this paper, we propose a data access approach by repeatedly traversing edges belonging to a specific relationship or anything one while distinguishing between high degree nodes and low degree ones. Finally, a result of our experiment indicated our approach can increase the speed of repeat traversals by a factor of a maximum of ten.

DOI： 10.1007/978-3-031-21756-2_29

Intuitive Gait Modeling using Mimetic-Words for Gait Description and Generation

Kato H., Hirayama T., Doman K., Ide I., Kawanishi Y., Komamizu T., Deguchi D., Murase H.

Proceedings - 5th International Conference on Multimedia Information Processing and Retrieval, MIPR 2022 頁： 240 - 245 2022年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Proceedings - 5th International Conference on Multimedia Information Processing and Retrieval, MIPR 2022

Gait is one of the most familiar action for us, that is why we can distinguish slight difference of human gaits and perceive their impressions. However, the relationship has been never explored because of the absence of intuitive labels for the slight differences. In this paper, to solve this problem, we propose a intuitive gait model using Japanese mimetic-words. A mimetic-word has sound-symbolism, which means that there is an association between linguistic sounds and sensory experiences, and the phonemes of a mimetic-word is strongly related to the visual sensation. Thanks to the sound-symbolism, Japanese mimetic-words have a possibility of modeling gaits intuitively. Thus, we have previously proposed a method which describes gait with a mimetic-word. In this paper, in the opposite direction, we propose a method which generates gait from a mimetic-word, and confirm the effectiveness of the proposed intuitive gait model which consists of the phonetic-vector through evaluations of both the generation task and the description task.

DOI： 10.1109/MIPR54900.2022.00050

その他リンク： https://dblp.uni-trier.de/db/conf/mipr/mipr2022.html#KatoHDIKKDM22

An Ensemble Framework of Multi-ratio Undersampling-based Imbalanced Classification 査読有り

駒水孝裕

Journal of Data Intelligence 2 (1) 巻頁： 30 - 46 2021年

FPX-G: First Person Exploration for Graph

Komamizu T., Ito S., Ogawa Y., Toyama K.

Proceedings - 4th International Conference on Multimedia Information Processing and Retrieval, MIPR 2021 頁： 70 - 76 2021年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Proceedings - 4th International Conference on Multimedia Information Processing and Retrieval, MIPR 2021

Data exploration is a fundamental user task in the information seeking process. In data exploration, users have ambiguous information needs, and they traverse across the data for gathering information. In this paper, a novel data exploration system, called FPX-G, is proposed that uses virtual reality (VR) technology. VR-based data exploration (or immersive analytics) is a recent trend in data analytics, and the existing work approaches involve aggregated information in an interactive and 3D manner. However, exploration for individual pieces of data scarcely has been approached. Traditional data exploration is done on 2D displays, therefore space is limited, and there is no depth. FPX-G fully utilizes 3D space to make individual piece of data visible in the user’s line of sight. In this paper, the data structure in FPX-G is designed as a graph, and the data exploration process is modeled as graph traversal. To utilize the capability of VR, FPX-G provides a first person view-based interface from which users can look at individual pieces of data and can walk through the data (like walking in a library). In addition to the walking mechanism, to deal with limited physical space in a room, FPX-G introduces eye-tracking technology for traversing data through a graph. A simulation-based evaluation reveals that FPX-G provides a significantly efficient interface for exploring data compared with the traditional 2D interface.

DOI： 10.1109/MIPR51284.2021.00018

その他リンク： https://dblp.uni-trier.de/db/conf/mipr/mipr2021.html#KomamizuIOT21

Evaluation Scheme of Focal Translation for Japanese Partially Amended Statutes

Yamakoshi T., Komamizu T., Ogawa Y., Toyama K.

WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop 頁： 124 - 132 2021年

出版者・発行元：WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop

For updating the translations of Japanese statutes based on their amendments, we need to consider the translation “focality;” that is, we should only modify expressions that are relevant to the amendment and retain the others to avoid misconstruing its contents. In this paper, we introduce an evaluation metric and a corpus to improve focality evaluations. Our metric is called an Inclusive Score for DIfferential Translation: (ISDIT). ISDIT consists of two factors: (1) the n-gram recall of expressions unaffected by the amendment and (2) the n-gram precision of the output compared to the reference. This metric supersedes an existing one for focality by simultaneously calculating the translation quality of the changed expressions in addition to that of the unchanged expressions. We also newly compile a corpus for Japanese partially amendment translation that secures the focality of the post-amendment translations, while an existing evaluation corpus does not. With the metric and the corpus, we examine the performance of existing translation methods for Japanese partially amendment translations.

Combining Multi-ratio Undersampling and Metric Learning for Imbalanced Classification.

Takahiro Komamizu

Journal of Data Intelligence 2 巻 ( 4 ) 頁： 462 - 474 2021年

Random walk-based entity representation learning and re-ranking for entity search

Komamizu, T

KNOWLEDGE AND INFORMATION SYSTEMS 62 巻 ( 8 ) 頁： 2989 - 3013 2020年8月

掲載種別：研究論文（学術雑誌）出版者・発行元：Knowledge and Information Systems

Linked Data (LD) has become a valuable source of factual records, and entity search is a fundamental task in LD. The task is, given a query consisting of a set of keywords, to retrieve a set of relevant entities in LD. The state-of-the-art approaches for entity search are based on information retrieval techniques. This paper first examines these approaches with a traditional evaluation metric, recall@k, to reveal their potential for improvement. To obtain evidence for the potentials, an investigation is carried out on the relationship between queries and answer entities in terms of path lengths on a graph of LD. On the basis of the investigation, learning representations of entities are dealt with. The existing methods of entity search are based on heuristics that determine relevant fields (i.e., predicates and related entities) to constitute entity representations. Since the heuristics require burdensome human decisions, this paper is aimed at removing the burden with a graph proximity measurement. To this end, in this paper, RWRDoc is proposed. It is an RWR (random walk with restart)-based representation learning method that learns representations of entities by using weighted combinations of representations of reachable entities w.r.t. RWR. RWRDoc is mainly designed to improve recall scores; therefore, as shown in experiments, it lacks capability in ranking. In order to improve the ranking qualities, this paper proposes a personalized PageRank-based re-ranking method, PPRSD (Personalized PageRank-based Score Distribution), for the retrieved results. PPRSD distributes relevance scores calculated by text-based entity search methods in a personalized PageRank manner. Experimental evaluations showcase that RWRDoc can improve search qualities in terms of recall@1000 and PPRSD can compensate for RWRDoc’s insufficient ranking capability, and the evaluations confirmed this compensation.

DOI： 10.1007/s10115-020-01445-4

ランダムフォレストを用いた法令用語の校正

山腰貴大, 小川泰弘, 駒水孝裕, 外山勝彦

人工知能学会論文誌 35 巻 ( 1 ) 頁： H-J53_1 - 14 2020年1月

記述言語：日本語掲載種別：研究論文（学術雑誌）出版者・発行元：一般社団法人人工知能学会

We propose a method that assists legislation drafters in finding inappropriate use of Japanese legal terms and their corrections from Japanese statutory sentences. In particular, we focus on sets of similar legal terms whose usages are strictly defined in legislation drafting rules that have been established over the years. In this paper, we first define input and output of legal term correction task. We regard it as a special case of sentence completion test with multiple choices. Next, we describe a legal term correction method for Japanese statutory sentences. Our method predicts suitable legal terms using Random Forest classifiers. The classifiers in our method use adjacent words to a target legal term as input features, and are optimized in various parameters including the number of adjacent words to be used for each legal term set. We conduct an experiment using actual statutory sentences from 3,983 existing acts and cabinet orders that consist of approximately 47M words in total. As for legal term sets, we pick 27 sets from legislation drafting manuals. The experimental result shows that our method outperformed existing modern word prediction methods using neural language models and that each Random Forest classifier utilizes characteristics of its corresponding legal term set.

DOI： 10.1527/tjsai.h-j53

事前学習モデルBERTによる法令用語の校正

山腰貴大, 駒水孝裕, 小川泰弘, 外山勝彦

人工知能学会全国大会論文集 2020 巻 ( 0 ) 頁： 4P3OS805 - 4P3OS805 2020年

出版者・発行元：一般社団法人人工知能学会

<p>法令文書には，「者」「物」「もの」や「規定」「規程」のように互いに類似している法令用語が出現する．このような法令用語は，法制執務（法令の起草・制定・改廃など法令文書の作成・管理に関する業務）の慣習や規則によって使用法とともに定義されている．法令において，これらの法令用語はそれに従い，厳密に書き分ける必要がある．契約書や約款などの広義の法令文書においても，誤解を防ぐために，法令に準じて正しく書き分けることが望ましい．そこで，本研究では，与えられた法令文から法令用語を検出し，誤用と思われるものに対してその修正案を出力することにより，法令文書の作成を支援する手法を提案する．本手法では，このタスクを選択肢付き穴埋め問題とみなし，分類器により解決する．分類器は，一般文によって事前学習したBERTモデルから構築する．このとき，（１）法令文によるドメイン適応，（２）訓練データのアンダーサンプリング，（３）分類器の統一の三つの工夫を施すことにより性能向上を図る．実験の結果，ランダムフォレストやニューラル言語モデルによる分類器よりも本手法の方が高い性能を発揮することを明らかにした．</p>

DOI： 10.11517/pjsai.JSAI2020.0_4P3OS805

Japanese mistakable legal term correction using infrequency-aware bert classifier

Yamakoshi T., Komamizu T., Ogawa Y., Toyama K.

Transactions of the Japanese Society for Artificial Intelligence 35 巻 ( 4 ) 頁： 1 - 17 2020年

記述言語：英語出版者・発行元：Transactions of the Japanese Society for Artificial Intelligence

We propose a method to assist legislative drafters that locates inappropriate legal terms in Japanese statutory sentences and suggests corrections. We focus on sets of mistakable legal terms whose usages are defined in legislation drafting rules. Our method predicts suitable legal terms using a classifier based on BERT (Bidirectional Encoder Representations from Transformers). The BERT classifier is pretrained with a huge number of whole sentences; thus, it contains abundant linguistic knowledge. Classifiers for predicting legal terms suffer from two-level infrequency: term-level infrequency and set-level infrequency. The former causes a class imbalance problem and the latter causes an underfitting problem; both degrade classification performance. To overcome these problems, we apply three techniques, namely, preliminary domain adaptation, repetitive soft undersampling, and classifier unification. The preliminary domain adaptation improves overall performance by providing prior knowledge of statutory sentences, the repetitive soft undersampling overcomes term-level infrequency, and the classifier unification overcomes set-level infrequency while saving storage consumption. Our experiments show that our classifier outperforms conventional classifiers using Random Forest or language models, and that all three training techniques improve performance.

DOI： 10.1527/tjsai.E-K25

Exploring Relevant Parts Between Legal Documents Using Substructure Matching 査読有り

Komamizu, T; Fujioka, K; Ogawa, Y; Toyama, K

NEW FRONTIERS IN ARTIFICIAL INTELLIGENCE, JSAI-ISAI 2019 12331 巻頁： 5 - 19 2020年

記述言語：英語掲載種別：研究論文（その他学術会議資料等）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Legal documents are typically hierarchically structured. This paper focuses on ordinances and rules (OR documents for short) in the local governments, which are designed for social lives under the governments. OR documents are composed of provisions for social lives in various aspects such as healthy development of youths and landscape preservation. OR documents in different local governments share common provisions but also include different provisions depending on their social situations. There is a large demand on helping governmental officers draft OR documents, especially searching “relevant parts” of OR documents. To help drafting OR documents, this paper designs the relevancy of OR documents with two basic measurements; matching ratio and provision commonality. Based on the relevancy, this paper develops a structured document search algorithm for OR documents. Experimental evaluation on real OR documents in Japan demonstrates that the proposed algorithm successfully discovers relevant parts of OR documents.

DOI： 10.1007/978-3-030-58790-1_1

その他リンク： https://dblp.uni-trier.de/db/conf/jsai/jsai2019w.html#KomamizuFOT19

SPARQL with XQuery-based Filtering.

Takahiro Komamizu

CoRR abs/2009.06194 巻 2020年

SPARQL with XQuery-based filtering

Komamizu T.

CEUR Workshop Proceedings 2721 巻頁： 69 - 73 2020年

出版者・発行元：CEUR Workshop Proceedings

Linked Open Data (LOD) has been proliferated over various domains, however, there are still lots of open data in various format other than RDF. Document-centric XML data are such open data that are connected with entities in LOD as supplemental documents for these entities. To utilize document-centric XML data linked from entities in LOD, in this paper, a SPARQL-based seamless access method on RDF and XML data is proposed. In particular, an extension to SPARQL, XQueryFILTER, which enables XQuery as a filter in SPARQL is proposed. For efficient query processing of the combination of SPARQL and XQuery, a query optimization is proposed. Experimental scenarios using real-world data showcase the effectiveness of XQueryFILTER and optimization efficiency.

MUEnsemble: Multi-ratio Undersampling-Based Ensemble Framework for Imbalanced Data

Komamizu, T; Uehara, R; Ogawa, Y; Toyama, K

DATABASE AND EXPERT SYSTEMS APPLICATIONS, DEXA 2020, PT II 12392 巻頁： 213 - 228 2020年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Class imbalance is commonly observed in real-world data, and it is still problematic in that it hurts classification performance due to biased supervision. Undersampling is one of the effective approaches to the class imbalance. The conventional undersampling-based approaches involve a single fixed sampling ratio. However, different sampling ratios have different preferences toward classes. In this paper, an undersampling-based ensemble framework, MUEnsemble, is proposed. This framework involves weak classifiers of different sampling ratios, and it allows for a flexible design for weighting weak classifiers in different sampling ratios. To demonstrate the principle of the design, in this paper, three quadratic weighting functions and a Gaussian weighting function are presented. To reduce the effort required by users in setting parameters, a grid search-based parameter estimation automates the parameter tuning. An experimental evaluation shows that MUEnsemble outperforms undersampling-based methods and oversampling-based state-of-the-art methods. Also, the evaluation showcases that the Gaussian weighting function is superior to the fundamental weighting functions. In addition, the parameter estimation predicted near-optimal parameters, and MUEnsemble with the estimated parameters outperforms the state-of-the-art methods.

DOI： 10.1007/978-3-030-59051-2_14

その他リンク： https://dblp.uni-trier.de/db/conf/dexa/dexa2020-2.html#KomamizuUOT20

Japanese Mistakable Legal Term Correction using Infrequency-aware BERT Classifier" 査読有り

Takahiro Yamakoshi, Takahiro Komamizu, Yasuhiro Ogawa, Katsuhiko Toyama

the 3rd Annual Workshop on Applications of Artificial Intelligence in the Legal Industry (LegalAI 2019) 頁： 4342-4351 2019年12月

Japanese Mistakable Legal Term Correction using Infrequency-aware BERT Classifier Matching 査読有り

Takahiro Yamakoshi, Takahiro Komamizu, Yasuhiro Ogawa, Katsuhiko Toyama

Proc. 3rd Annual Workshop on Applications of Artificial Intelligence in the Legal Industry 頁： - - 4351 2019年12月

記述言語：英語掲載種別：研究論文（その他学術会議資料等）出版者・発行元：Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019

We propose a method that assists legislative drafters in locating inappropriate legal terms in Japanese statutory sentences and suggests corrections. We focus on sets of mistakable legal terms whose usages are defined in legislation drafting rules. Our method predicts suitable legal terms using a classifier based on a BERT (Bidirectional Encoder Representations from Transformers) model. We apply three techniques in training the BERT classifier, specifically, preliminary domain adaptation, repetitive soft undersampling, and classifier unification. These techniques cope with two levels of infrequency: legal term-level infrequency that causes class imbalance and legal term set-level infrequency that causes underfitting. Concretely, preliminary domain adaptation improves overall performance by providing prior knowledge of statutory sentences, repetitive soft undersampling improves performance on infrequent legal terms without sacrificing performance on frequent legal terms, and classifier unification improves performance on infrequent legal term sets by sharing common knowledge among legal term sets. Our experiments show that our classifier outperforms conventional classifiers using Random Forest or a language model, and that all three training techniques contribute to performance improvement.

DOI： 10.1109/BigData47090.2019.9006511

その他リンク： https://dblp.uni-trier.de/db/conf/bigdataconf/bigdataconf2019.html#YamakoshiKOT19

Exploring Relevant Parts between Legal Documents using Substructure Matching 査読有り

Takahiro Komamizu, Kazuya Fujioka, Yasuhiro Ogawa, Katsuhiko Toyama

the Thirteenth International Workshop on Juris-informatics (JURISIN 2019) 頁： 16-28 2019年11月

言い換えによる自然言語－SPARQL対訳コーパスの拡張

李偉嘉, 小川泰弘, 駒水孝裕, 外山勝彦

第17回情報学ワークショップ論文集頁： - 2019年11月

Analyzing Japanese Law History through Modeling Multi-versioned Entity 査読有り

Takahiro Komamizu, Yushi Uchida, Yasuhiro Ogawa, Katsuhiko Toyama

the 2nd International Workshop on Contextualized Knowledge Graphs (CKG 2019) 頁： - 2019年10月

利用規約中の不公平文の自動検出

青山恵子, 小川泰弘, 駒水孝裕, 外山勝彦

第15回テキストアナリティクス・シンポジウム NLC2019-8(2019-9) 頁： 1-6 2019年9月

Thai Legal Term Correction using Random Forests with Outside-the-sentence Features 査読有り

Takahiro Yamakoshi, Vee Satayamas, Hutchatai Chanlekha, Yasuhiro Ogawa, Takahiro Komamizu, Asanee Kawtrakul, Katsuhiko Toyama

the 33rd Pacific Asia Conference on Language, Information and Computation (PACLIC 33) 頁： 161-170 2019年9月

弱分類器の調整に基づく不均衡データ向けアンサンブル・フレームワーク査読有り

植原リサ, 駒水孝裕, 小川泰弘, 外山勝彦

第12回Webとデータベースに関するフォーラム頁： 81-84 2019年9月

共通BERT分類器による紛らわしい法令用語の校正

山腰貴大, 駒水孝裕, 小川泰弘, 外山勝彦

言語処理学会NLP若手の会第14回シンポジウム頁： - 2019年8月

nagoy Team's Summarization System at the NTCIR-14 QA-Lab PoliInfo 査読有り

Yasuhiro Ogawa, Michiaki Satou, Takahiro Komamizu, Katsuhiko Toyama

the Fourteenth NTCIR conference (NTCIR-14) , Revised Selected Papers 頁： 110-121 2019年6月

法律の要約のためのランダムフォレストを用いた重要文抽出

小川泰弘, 佐藤充晃, 駒水孝裕, 外山勝彦

人工知能学会全国大会(第33回)論文集 JSAI2019 巻 ( 0 ) 頁： 4E2OS7a02 - 4E2OS7a02 2019年6月

記述言語：日本語掲載種別：研究論文（学術雑誌）出版者・発行元：一般社団法人人工知能学会

本研究の目標は，日本法令の要約を提供することである．そのためにランダムフォレストによる重要文抽出に基づく自動要約を提案する．従来の自動要約に関する研究においては，原文書の情報のみが用いられてきた．近年では機械学習に基づく手法なども提案されている．しかし，そうした機械学習において利用される学習データの量は，特に日本語においては，充分でなかった．それに対し，本研究の法令の要約においては，政府が作成する「法令のあらまし」を利用することにより，この問題を解決する．さらに，従来利用されてきた決定木やSVMを使った手法に代えて，ランダムフォレストを用いた重要文抽出を提案し，その性能が従来手法を上回ることを示す．本論文の貢献は，従来よりもサイズの大きな要約用コーパスを作成した点と，重要文抽出におけるランダムフォレストの有効性を確認した点にある．

DOI： 10.11517/pjsai.jsai2019.0_4e2os7a02

Detecting Communities and Correlated Attribute Clusters on Multi-Attributed Graphs 査読有り

Ito, H; Komamizu, T; Amagasa, T; Kitagawa, H

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS E102D 巻 ( 4 ) 頁： 810 - 820 2019年4月

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：IEICE Transactions on Information and Systems

Multi-attributed graphs, in which each node is characterized by multiple types of attributes, are ubiquitous in the real world. Detection and characterization of communities of nodes could have a significant impact on various applications. Although previous studies have attempted to tackle this task, it is still challenging due to difficulties in the integration of graph structures with multiple attributes and the presence of noises in the graphs. Therefore, in this study, we have focused on clusters of attribute values and strong correlations between communities and attribute-value clusters. The graph clustering methodology adopted in the proposed study involves Community detection, Attribute-value clustering, and deriving Relationships between communities and attribute-value clusters (CAR for short). Based on these concepts, the proposed multi-attributed graph clustering is modeled as CAR-clustering. To achieve CAR-clustering, a novel algorithm named CARNMF is developed based on non-negative matrix factorization (NMF) that can detect CAR in a cooperative manner. Results obtained from experiments using real-world datasets show that the CARNMF can detect communities and attribute-value clusters more accurately than existing comparable methods. Furthermore, clustering results obtained using the CARNMF indicate that CARNMF can successfully detect informative communities with meaningful semantic descriptions through correlations between communities and attribute-value clusters.

DOI： 10.1587/transinf.2018DAP0022

Japanese Mistakable Legal Term Correction using Infrequency-aware BERT Classifier

Yamakoshi, T; Komamizu, T; Ogawa, Y; Toyama, K

2019 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA) 頁： 4342 - 4351 2019年

Thai legal term correction using random forests with outside-the-sentence features

Yamakoshi T., Satayamas V., Chanlekha H., Ogawa Y., Komamizu T., Kawtrakul A., Toyama K.

Proceedings of the 33rd Pacific Asia Conference on Language, Information and Computation, PACLIC 2019 頁： 279 - 287 2019年

出版者・発行元：Proceedings of the 33rd Pacific Asia Conference on Language, Information and Computation, PACLIC 2019

We propose a method for finding and correct- ing misused Thai legal terms in Thai statu- tory sentences. Our method predicts legal terms using Random Forest classifiers, each of which is optimized for each set of similar legal terms. Each classifier utilizes outside- the-sentence features, namely, promulgation year, title keywords, and section keywords of statutes, in addition to words adjacent to the targeted legal term. Our experiment shows that our method outperformed not only a Ran- dom Forest method without the outside-the- sentence features, but also BERT (Bidirec- tional Encoder Representations from Trans- formers), a powerful language representation model, in overall accuracy.

nagoy Team's Summarization System at the NTCIR-14 QA Lab-PoliInfo 査読有り

Yasuhiro Ogawa, Michiaki Satou, Takahiro Komamizu, Katsuhiko Toyama

Post-conference Proceedings of the 14th NTCIR Conference on Evaluation of Information Access Technologies 11966 LNCS 巻頁： to appear - 121 2019年

記述言語：英語掲載種別：研究論文（その他学術会議資料等）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

The nagoy team participated in the NTCIR-14 QA Lab-PoliInfo’s summarization subtask. This paper describes our summarization system for assembly member speeches using random forest classifiers. Since we encountered an imbalance in the data, we were unable to achieve good results in this subtask when training on all data. To solve this problem, we developed a new summarization system that applies multiple random forest classifiers training on different-sized data sets step by step. As a result, our system achieved good performance, especially in the evaluation by ROUGE scores. In this paper, we also compare our system with a single random forest classifier using probability.

DOI： 10.1007/978-3-030-36805-0_9

その他リンク： https://dblp.uni-trier.de/db/conf/ntcir/ntcir2019.html#OgawaSKT19

Graph Analytical Re-ranking for Entity Search

Komamizu T.

CEUR Workshop Proceedings 2482 巻 2019年

出版者・発行元：CEUR Workshop Proceedings

Entity search is a fundamental task in Linked Data (LD). The task is, given a keyword search query, to retrieve a set of entities in LD which are relevant to the query. The state-of-the-art approaches for entity search are based on information retrieval technologies such as TF-IDF vectorization and ranking models. This paper examines the approaches by applying a traditional evaluation metrics, recall@k, and shows ranking qualities still room left for improvements. In order to improve the ranking qualities, this paper explores possibilities of graph analytical methods. LD is regarded as a large graph, graph analytical approaches are therefore appropriate for this purpose. Since query-based graph analytical approaches fit to entity search tasks, this paper proposes a personalized PageRank-based re-ranking method, PPRSD (Personalized PageRank based Score Distribution), for retrieved results by the state-of-the-art. The experimental evaluation recognizes improvements but its results are not satisfactory, yet. For further improvements, this paper reports investigations about relationship between queries and entities in terms of path lengths on the graph, and discusses future directions for graph analytical approaches.

Analyzing Japanese law history through modeling multi-versioned entity

Komamizu T., Uchida Y., Ogawa Y., Toyama K.

CEUR Workshop Proceedings 2599 巻 2019年

出版者・発行元：CEUR Workshop Proceedings

As law is a blueprint of a society and is changed over time as social environments changed, analyzing histories (change provenances) of laws can reveal important facts such as legislative facts and critical events for the society. Linked Open Data (LOD) has emerged as a preferred method for publishing and sharing open data, however, there is an ontological barrier for publishing law history data as LOD. To break through the barrier, this paper proposes an ontology for law history data of the Japanese statute law. The ontology is inspired from PROV-O and SIOC ontologies. The LOD dataset based on the proposed ontology enables wide variety of analyses on the law history data by simple SPARQL queries. The analyses include simple search, visualization, temporal analysis, data mining, etc. This paper displays parts of the analyses which indicate several legislative facts behind changes of laws. The analyses demonstrate the proposed ontology and LOD dataset are useful for legal data analysis. The proposed ontology is comparable with ELI (European Legislation Identifier) which is designed for EU laws, this paper thus discusses the comparability and future directions of the proposed ontology.

Graph Analytical Re-ranking for Entity Search

Takahiro Komamizu

Proceedings of the 1st International Workshop on EntitY REtrieval (EYRE 2018) 頁： (to appear) 2018年10月

Learning Interpretable Entity Representation in Linked Data 査読有り

Takahiro Komamizu

Proc. the 29th International Conference on Database and Expert Systems Applications 頁： 153-168 2018年9月

Community Detection and Correlated Attribute Cluster Analysis on Multi-Attributed Graphs 査読有り

Hiroyoshi Ito, Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 2nd International workshop on Data Analytics solutions for Real-LIfe APplications (DARLI-AP 2018) co-located with the 21st International Conference on Extending Database Technology (EDBT 2018) 2018年3月

Network-Word Embedding for Dynamic Text Attributed Networks 査読有り

Hiroyoshi Ito, Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 6th International Workshop on Semantic Computing for Social Networks and Organization Sciences (SCSN 2018) co-located with the 12th IEEE International Conference on Semantic Computing (ICSC 2018) 2018年1月

Analytical toolbox for smart city applications: Garbage collection log use case 査読有り

Takahiro Komamizu, Jin Nakazawa, Toshiyuki Amagasa, Hiroyuki Kitagawa, Hideyuki Tokuda

Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017 2018- 巻頁： 4105 - 4110 2018年1月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Institute of Electrical and Electronics Engineers Inc.

Analyzing and feeding back the results on real-world services are important missions in the Big Data era to realize smart city. However, analyzing real-world data is still challenging because of dirtiness of data and large variety of analytic requirements. To cope with the challenges, this paper proposes and develops an analytical toolbox for smart city applications. The analytical toolbox consists of three phases: preparation, analysis, and visualization. The preparation phase deals with the dirtiness of the data by including fundamental data cleansing techniques and data integration techniques. The analysis phase is responsible for ETL (extract, transform and load) process and analytical query processing from the next phase. The visualization phase deals with analytical requirements from users and visualization of analytical results. This paper showcases a real-world use case of the proposed analytical toolbox. The use case is now open in public with help of Fujisawa city, Japan, and this fact indicates that the proposed analytical toolbox is feasible for real-world data analysis and feeding back to citizens.

DOI： 10.1109/BigData.2017.8258429

Implicit order join: Joining log data with property data by discovering implicit order-oriented keys with human assistance 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017 2018- 巻頁： 4400 - 4406 2018年1月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Institute of Electrical and Electronics Engineers Inc.

Data integration is still laboursome task when integrating data are not consistently managed. Such inconsistency can happen easily in real-world situations, such as properties of objects are managed by a central organization and trajectories (or logs) of the objects are recorded by other peripheral organizations. This paper deals with a case of missing ordering information. Integrating property data and log data without ordering information causes duplicated results. In order to solve this problem, this paper proposes a join algorithm, called implicit order join, which discovers implicit ordering information from both property data and log data with help of partial true integrated results from human assistance. With the discovered ordering information, the implicit order join enables to integrate the property data and log data. In order to discover the implicit ordering information, ordering correlation between attribute sequences of property data and log data should be found from comprehensive examination of possible attribute sequence pairs. The potential number of sequence pairs is as high as factorial order of the number of attributes. Therefore, this paper develops a heuristic approach to prune unnecessary examinations based on ordering dependency between attribute sequences. Experimental evaluation in this paper indicates that implicit order join can reduce 77% labouring tasks for integration and the pruning method reduces the number of attribute sequences in orders of magnitude.

DOI： 10.1109/BigData.2017.8258474

Japanese Legal Term Correction Using Random Forests 査読有り

Yamakoshi, T; Komamizu, T; Ogawa, Y; Toyama, K

LEGAL KNOWLEDGE AND INFORMATION SYSTEMS (JURIX 2018) 313 巻頁： 161 - 170 2018年

記述言語：英語出版者・発行元：Frontiers in Artificial Intelligence and Applications

We propose a method that assists legislation officers in finding inappropriate Japanese legal terms in Japanese statutory sentences and suggests corrections. In particular, we focus on sets of similar legal terms whose usages are defined in legislation drafting rules. Our method predicts suitable legal terms in statutory sentences using Random Forest classifiers, each of which is optimized for each set of similar legal terms. Our experiment shows that our method outperformed existing modern word prediction methods using neural language models.

DOI： 10.3233/978-1-61499-935-5-161

Learning Interpretable Entity Representation in Linked Data

Komamizu, T

DATABASE AND EXPERT SYSTEMS APPLICATIONS, DEXA 2018, PT I 11029 巻頁： 153 - 168 2018年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Linked Data has become a valuable source of factual records. However, because of its simple representations of records (i.e., a set of triples), learning representations of entities is required for various applications such as information retrieval and data mining. Entity representations can be roughly classified into two categories; (1) interpretable representations, and (2) latent representations. Interpretability of learned representations is important for understanding relationship between two entities, like why they are similar. Therefore, this paper focuses on the former category. Existing methods are based on heuristics which determine relevant fields (i.e., predicates and related entities) to constitute entity representations. Since the heuristics require laboursome human decisions, this paper aims at removing the labours by applying a graph proximity measurement. To this end, this paper proposes RWRDoc, an RWR (random walk with restart)-based representation learning method which learns representations of entities by weighted combinations of minimal representations of whole reachable entities w.r.t. RWR. Comprehensive experiments on diverse applications (such as ad-hoc entity search, recommender system using Linked Data, and entity summarization) indicate that RWRDoc learns proper interpretable entity representations.

DOI： 10.1007/978-3-319-98809-2_10

その他リンク： https://dblp.uni-trier.de/db/conf/dexa/dexa2018-1.html#Komamizu18

CROISSANT: Centralized Relational Interface for Web-scale SPARQL Endpoints 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 19th International Conference on Information Integration and Web-based Applications & Services (iiWAS2017) 2017年12月

SOLA: Stream OLAP-based Analytical Framework for Roadway Maintenance 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Salman Ahmed Shaikh, Hiroaki Shiokawa, Hiroyuki Kitagawa

Proc. the 9th International Conference on Management of Digital EcoSystems (MEDES 2017) 2017年11月

GitHubとStack Overflowの開発者の活動記録を併用したリポジトリ推薦査読有り

永野真知, 早瀬康裕, 駒水孝裕, 北川博之

ソフトウェアエンジニアリングシンポジウム 2017論文集 2017年8月

Exploring identical users on GitHub and stack overflow 査読有り

Takahiro Komamizu, Yasuhiro Hayase, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proceedings of the International Conference on Software Engineering and Knowledge Engineering, SEKE 頁： 584 - 589 2017年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Knowledge Systems Institute Graduate School

Analyzing behaviours of developers in different platforms (in particular, GitHub and Stack Overflow in this paper) can reveal interesting facts related to development activities. There are only few datasets for analysing crossplatform user behaviours, especially across GitHub and Stack Overflow. Users on GitHub and Stack Overflow are identifiable by equivalences of email addresses. In order to increase the number of identifiable users on these datasets, this paper retrieves potentially identifiable users between GitHub and Stack Overflow not relying only on email addresses. This paper employs a classification-based link prediction, which design the user identification problem as a link prediction problem on the bipartite graph consisting of users of GitHub and those of Stack Overflow. With the identification method, this paper generates a probabilistic dataset containing pairs of users with probabilities (or confidences). This paper, as well, publishes the identification tool in order to enable further data generation on appearing datasets of GitHub, Stack Overflow and others. The generated dataset and tool are highly helpful to accelerate researches on mining software repositories.

DOI： 10.18293/SEKE2017-109

FORK: Feedback-Aware ObjectRank-Based Keyword Search over Linked Data 査読有り

Takahiro Komamizu, Sayami Okumura, Toshiyuki Amagasa, Hiroyuki Kitagawa

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10648 巻頁： 58 - 70 2017年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Springer Verlag

Ranking quality for keyword search over Linked Data (LD) is crucial when users look for entities from LD, since datasets in LD have complicated structures as well as much contents. This paper proposes a keyword search method, FORK, which ranks entities in LD by ObjectRank, a well-known link-structure analysis algorithm that can deal with different types of nodes and edges. The first attempt of applying ObjectRank to LD search reveals that ObjectRank with inappropriate settings gives worse ranking results than PageRank which is equivalent to ObjectRank with all the same authority transfer weights. Therefore, deriving appropriate authority transfer weights is the most important issue for encouraging ObjectRank in LD search. FORK involves a relevance feedback algorithm to modify the authority transfer weights according with users’ relevance judgements for ranking results. The experimental evaluation of ranking qualities using an entity search benchmark showcases the effectiveness of FORK, and it proves ObjectRank is more feasible raking method for LD search than PageRank and other comparative baselines including information retrieval techniques and graph analytic methods.

DOI： 10.1007/978-3-319-70145-5_5

Towards Real-time Analysis of Smart City Data: A Case Study on City Facility Utilizations 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Salman Ahmed Shaikh, Hiroaki Shiokawa, Hiroyuki Kitagawa

Proc. the 14th IEEE International Conference on Smart City (SmartCity 2016) 頁： 1357-1364 2016年12月

Interleaving Clustering of Classes and Properties for Disambiguating Linked Data 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 18th International Conference on Asia-Pacific Digital Libraries (ICADL 2016) 2016年12月

Visual Spatial-OLAP for Vehicle Recorder Data on Micro-sized Electric Vehicles 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 20th International Database Engineering & Applications Symposium (IDEAS 2016) 頁： 358-363 2016年7月

H-SPOOL: A SPARQL-based ETL Framework for OLAP over Linked Data with Dimension Hierarchy Extraction 招待有り査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

12 巻 ( 3 ) 頁： 359-378 2016年6月

SPOOL: A SPARQL-based ETL Framework for OLAP over Linked Data 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 17th International Conference on Information Integration and Web-based Applications & Services (iiWAS 2015) ( 49 ) 頁： 1-10 2015年12月

Facet-value Extraction Scheme from Textual Contents in XML Data 招待有り査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

11 巻 ( 3 ) 頁： 270-290 2015年6月

Extracting Facets from Textual Contents for Faceted Search over XML Data 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 16th International Conference on Information Integration and Web-based Applications & Services (iiWAS 2014) 頁： 420-429 2014年12月

Frequent-Pattern based Facet Extraction from Graph Data 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 17th International Conference on Network-Based Information Systems (NBiS 2014) 頁： 318-323 2014年9月

A Scheme of Automated Object and Facet Extraction for Faceted Search over XML Data 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 18th International Database Engineering & Applications Symposium (IDEAS 2014) 頁： 338-341 2014年7月

A Scheme of Fragment-Based Faceted Image Search 査読有り

Takahiro Komamizu, Mariko Kamie, Kazuhiro Fukui, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 23rd International Conference on Database and Expert Systems Applications (DEXA 2012) 頁： 450-457 2012年9月

Faceted Navigation Framework for XML Data 招待有り査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

8 巻 ( 4 ) 頁： 348-370 2012年8月

A Framework of Faceted Navigation for XML Data 査読有り

Takahiro Komamizu, Toshiyuki Amagasa, Hiroyuki Kitagawa

Proc. the 13th International Conference on Information Integration and Web-based Applications & Services (iiWAS 2011) 頁： 28-35 2011年12月

　前のページ - 次のページ　

1