研究者詳細 - 駒水　孝裕

論文 - 駒水　孝裕

分割表示 100 件中 1 - 20 件目／全件表示 >>

Towards Ensemble-Based Imbalanced Text Classification Using Metric Learning

Komamizu, T

DATABASE AND EXPERT SYSTEMS APPLICATIONS, DEXA 2023, PT II 14147 巻頁： 188 - 202 2023年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

This paper reports s series of ensemble approaches for imbalance text classification. All the approaches utilize a metric learning technique for obtaining better representations of texts to train weak classifiers. Each approach deals with the class imbalance problem with an undersampling-based ensemble approach, because metric learning techniques also suffer from this problem. In this paper, four ensemble approaches (namely, MLBagging, MLBoosting, MLStacking, and MLBoostacking) are proposed, three of which are corresponding to ensemble frameworks (namely, bagging, boosting, and stacking), and the other is a combination of boosting and stacking. MLBagging, MLBoosting, and MLStacking train metric learners on the individual undersampled dataset and combine them, while MLBoostacking trains metric learners in a step-by-step manner; that is, a metric learner learns a feature transformation so that failed-to-classify samples in the previous step should be correctly classified. The experimental evaluation on three imbalanced text classification datasets (namely, unfair statement classification in terms of service, hate speech detection in a forum, and hate speech tweet detection) shows that the proposed approaches lift classification performance from BERT-based approaches, by improving the representations of texts through metric learning.

DOI： 10.1007/978-3-031-39821-6_15

その他リンク： https://dblp.uni-trier.de/db/conf/dexa/dexa2023-2.html#Komamizu23

MMEnsemble: Imbalanced Classification Framework Using Metric Learning and Multi-sampling Ratio Ensemble

Komamizu, T

DATABASE AND EXPERT SYSTEMS APPLICATIONS, DEXA 2021, PT II 12924 巻頁： 176 - 188 2021年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

In classification, class imbalance is a factor that degrades the classification performance of many classification methods. Resampling is one widely accepted approach to the class imbalance; however, it still suffers from an insufficient data space, which also degrades performance. To overcome this, in this paper, an undersampling-based imbalanced classification framework, MMEnsemble, is proposed that incorporates metric learning into a multi-ratio undersampling-based ensemble. This framework also overcomes a problem with determining the appropriate sampling ratio in the multi-ratio ensemble method. It was evaluated by using 12 real-world datasets. It outperformed the state-of-the-art approaches of metric learning, undersampling, and oversampling in recall and ROC-AUC, and it performed comparably with them in terms of Gmean and F-measure metrics.

DOI： 10.1007/978-3-030-86475-0_18

その他リンク： https://dblp.uni-trier.de/db/conf/dexa/dexa2021-2.html#Komamizu21

Multi-proposal collaboration and multi-task training for weakly-supervised video moment retrieval

Zhang, BL; Yang, C; Jiang, B; Komamizu, T; Ide, I

INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS 2025年1月

出版者・発行元：International Journal of Machine Learning and Cybernetics

This study focuses on weakly-supervised Video Moment Retrieval (VMR), aiming to identify a moment semantically similar to the given query within an untrimmed video using only video-level correspondences, without relying on temporal annotations during training. Previous methods either aggregate predictions for all instances in the video, or indirectly address the task by proposing reconstructions for the query. However, these methods often produce low-quality temporal proposals, struggle with distinguishing misaligned moments in the same video, or lack stability due to a reliance on a single auxiliary task. To address these limitations, we present a novel weakly-supervised method called Multi-proposal Collaboration and Multi-task Training (MCMT). Initially, we generate multiple proposals and derive corresponding learnable Gaussian masks from them. These masks are then combined to create a high-quality positive sample mask, highlighting video clips most relevant to the query. Concurrently, we classify other clips in the same video as the easy negative sample and the entire video as the hard negative sample. During training, we introduce forward and inverse masked query reconstruction tasks to impose more substantial constraints on the network, promoting more robust and stable retrieval performance. Extensive experiments on two standard benchmarks affirm the effectiveness of the proposed method in VMR.

DOI： 10.1007/s13042-024-02520-w

Quantifying Image-Adjective Associations by Leveraging Large-Scale Pretrained Models

Matsuhira C., Kastner M.A., Komamizu T., Hirayama T., Ide I.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 15523 LNCS 巻頁： 428 - 441 2025年

出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Quantifying the associations between images and adjectives, i.e., how much the visual characteristics of an image are connected with a certain adjective, is important for better image understanding. For instance, the appearance of a kitten can be associated with adjectives such as “soft”, “small”, and “cute” rather than the opposite “hard”, “large”, and “scary”. Thus, giving scores for a kitten photo considering the degree of its association with each antonym adjective pair (termed adjective axis, e.g., “round” vs. “sharp”) aids in understanding the image content and its atmosphere. Existing methods rely on subjective human engagement, making it difficult to estimate the association of images with arbitrary adjective axes in a single framework. To enable the extension to arbitrary axes, we explore the use of large-scale pretrained models, including Large Language Models (LLMs) and Vision Language Models (VLMs). In the proposed training-free framework, users only need to specify a pair of antonym nouns that negatively and positively describe the target axis (e.g., “roundness” and “sharpness”). Evaluation confirms that the proposed framework can predict negative and positive associations between adjectives and images as correctly as the manually-assisted comparative. The result also highlights the pros and cons of utilizing the VLM’s textual or visual embedding for specific types of adjective axes. Furthermore, computing the similarities among four adjective axes unveils how the proposed framework connects them with each other, such as its tendency to regard a sharp object as being small, hard, and quick in motion.

DOI： 10.1007/978-981-96-2071-5_31

Towards Visual Storytelling by Understanding Narrative Context Through Scene-Graphs

Phueaksri I., Kastner M.A., Kawanishi Y., Komamizu T., Ide I.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 15523 LNCS 巻頁： 226 - 239 2025年

出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

VIsual STorytelling (VIST) is a task that transforms a sequence of images into narrative text stories. A narrative story requires an understanding of the contexts and relationships among images. Our study introduces a story generation process that emphasizes creating a coherent narrative by constructing both image and narrative contexts to control the coherence. First, the image contexts are generated from the content of individual images, using image features and scene graphs that detail the elements of the images. Second, the narrative context is generated by focusing on the overall image sequence. Ensuring that each caption fits within the overall story maintaining continuity and coherence. We also introduce a narrative concept summary, which is external knowledge represented as a knowledge graph. This summary encapsulates the narrative concept of an image sequence to enhance the understanding of its overall content. Following this, both image and narrative contexts are used to generate a coherent and engaging narrative. This framework is based on Long Short-Term Memory (LSTM) with an attention mechanism. We evaluate the proposed method using the VIST dataset, and the results highlight the importance of understanding the context of an image sequence in generating coherent and engaging stories. The study demonstrates the significance of incorporating narrative context into the generation process to ensure the coherence of the generated narrative.

DOI： 10.1007/978-981-96-2071-5_17

Feature Extraction for Claim Check-Worthiness Prediction Tasks Using LLM

Teramoto Y., Komamizu T., Matsushita M., Hatano K.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 15342 LNCS 巻頁： 53 - 58 2024年12月

掲載種別：論文集(書籍)内論文出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

This study explores the use of Large Language Models (LLMs) for Claim Check-Worthiness Prediction (CCWP), a crucial pre-screening task in fact-checking. We predict the time between a claim’s occurrence and verification by analyzing data from fact-checking organizations. The results show that validation time is the same between the top 25% and bottom 75% of total checklist condition fulfillment claims. That is, further optimization is needed for LLMs to perform effective CCWPs.

DOI： 10.1007/978-3-031-78090-5_5

Cross-modal recipe retrieval based on unified text encoder with fine-grained contrastive learning

Zhang, BL; Kyutoku, H; Doman, K; Komamizu, T; Ide, I; Qian, JB

KNOWLEDGE-BASED SYSTEMS 305 巻 2024年12月

出版者・発行元：Knowledge-Based Systems

Cross-modal recipe retrieval is vital for transforming visual food cues into actionable cooking guidance, making culinary creativity more accessible. Existing methods separately encode the recipe Title, Ingredient, and Instruction using different text encoders, then aggregate them to obtain recipe feature, and finally match it with encoded image feature in a joint embedding space. These methods perform well but require significant computational cost. In addition, they only consider matching the entire recipe and the image but ignore the fine-grained correspondence between recipe components and the image, resulting in insufficient cross-modal interaction. To this end, we propose Unified Text Encoder with Fine-grained Contrastive Learning (UTE-FCL) to achieve a simple but efficient model. Specifically, in each recipe, UTE-FCL first concatenates each of the Ingredient and Instruction texts composed of multiple sentences as a single text. Then, it connects these two concatenated texts with the original single-phrase Title to obtain the concatenated recipe. Finally, it encodes these three concatenated texts and the original Title by a Transformer-based Unified Text Encoder (UTE). This proposed structure greatly reduces the memory usage and improves the feature encoding efficiency. Further, we propose fine-grained contrastive learning objectives to capture the correspondence between recipe components and the image at Title, Ingredient, and Instruction levels by measuring the mutual information. Extensive experiments demonstrate the effectiveness of UTE-FCL compared to existing methods.

DOI： 10.1016/j.knosys.2024.112641

ICDAR'24 Workshop Chairs' Welcome Message

Dao M.S., Riegler M.A., Dang-Nguyen D.T., Tran H.N., Kiran R.U., Komamizu T.

ACM International Conference Proceeding Series 2024年6月

検出領域絞込みと検出履歴を考慮した広角映像中の鳥追跡

Tingwei Liu, 川西康友, 駒水孝裕, 井手一郎

人工知能学会第二種研究会資料 2024 巻 ( Challenge-064 ) 頁： 07 2024年3月

Computational measurement of perceived pointiness from pronunciation (vol 83, pg 26183, 2024)

Matsuhira, C; Kastner, MA; Komamizu, T; Ide, I; Hirayama, T; Kawanishi, Y; Doman, K; Deguchi, D

MULTIMEDIA TOOLS AND APPLICATIONS 83 巻 ( 9 ) 頁： 26211 - 26212 2024年3月

掲載種別：研究論文（学術雑誌）出版者・発行元：Multimedia Tools and Applications

The original publication of this article contains the following errors: missing ORCID of authors incorrect author contribution statement pronunciation symbols were not shown correctly in both online and PDF versionsthe gamma symbol "Γ" were incorrectly displayed as "0" in the PDF version missing ORCID of authors incorrect author contribution statement pronunciation symbols were not shown correctly in both online and PDF versions the gamma symbol "Γ" were incorrectly displayed as "0" in the PDF version The original article has been corrected.

DOI： 10.1007/s11042-023-17657-z

Computational measurement of perceived pointiness from pronunciation

Matsuhira, C; Kastner, MA; Komamizu, T; Ide, I; Hirayama, T; Kawanishi, Y; Doman, K; Deguchi, D

MULTIMEDIA TOOLS AND APPLICATIONS 83 巻 ( 9 ) 頁： 26183 - 26210 2024年3月

掲載種別：研究論文（学術雑誌）出版者・発行元：Multimedia Tools and Applications

Sound symbolism is a well-researched topic of psycholinguistics, which tries to comprehend the connection between the sound of a word and its meanings. The Bouba-Kiki effect, one form of sound symbolism, claims that people perceive the pronunciation of “Kiki” as pointier than that of “Bouba.” There is no research that focuses on modeling such perception, i.e., how pointy a pronunciation sounds to humans, through computational and data-driven approaches. To address this, this paper first proposes the novel concept of “phonetic pointiness” defined as how pointy a shape humans are most likely to associate with a given pronunciation. We then model this phonetic pointiness from computational and data-driven approaches to calculate a score for an arbitrary pronunciation. There are three proposed models: a referential model, an expressive model, and a combined model, which integrates the previous two. The idea comes from an existing psycholinguistic classification of two types of sound symbolisms: referential symbolism and expressive symbolism, where the former relates to vocabulary knowledge, while the latter is based on pure human intuition. The proposed models are constructed only with image and language data available on the Web, therefore not requiring task-specific human annotations. We evaluate these models through a crowd-sourced user study, finding a promising correlation between human perception and the phonetic pointiness calculated by the proposed models. The results indicate that human perception can be modeled better by combining both types of sound symbolisms. Furthermore, by observing the behaviors of the models, we show several possible use-cases, such as product naming and psycholinguistic research, which can be a useful insight to further studies and applications.

DOI： 10.1007/s11042-023-15732-z

Image-Collection Summarization Using Scene-Graph Generation With External Knowledge

Phueaksri, I; Kastner, MA; Kawanishi, Y; Komamizu, T; Ide, I

IEEE ACCESS 12 巻頁： 17499 - 17512 2024年

掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Summarization tasks aim to summarize multiple pieces of information into a short description or representative information. A text summarization task summarizes textual information into a short description, whereas an image collection summarization task summarizes an image collection into images or textual representation in which the challenge is to understand the relationship between images. In recent years, scene-graph generation has shown the advantage of describing the visual contexts of a single-image, and incorporating external knowledge into the scene-graph generation model has also given effective directions for unseen single-image scene-graph generation. While external knowledge has been implemented in related work, it is still challenging to use this information efficiently for relationship estimation during the summarization. Following this trend, in this paper, we propose a novel scene-graph-based image-collection summarization model that aims to generate a summarized scene-graph of an image collection. The key idea of the proposed method is to enhance the relation predictor toward relationships between images in an image collection incorporating knowledge graphs as external knowledge for training a model. With this approach, we build an end-to-end framework that can generate a summarized scene graph of an image collection. To evaluate the proposed method, we also build an extended annotated MS-COCO dataset for this task and introduce an evaluation process that focuses on estimating the similarity between a summarized scene graph and ground-truth scene graphs. Traditional evaluation focuses on calculating precision and recall scores, which involve true positive predictions without balancing precision and recall. Meanwhile, the proposed evaluation process focuses on calculating the F-score of the similarity between a summarized scene graph and ground-truth scene graphs, which aims to balance both false positives and false negatives. Experimental results show that using external knowledge to enhance the relation predictor achieves better results than existing methods.

DOI： 10.1109/ACCESS.2024.3360113

Zero-Shot Pill-Prescription Matching With Graph Convolutional Network and Contrastive Learning

Nguyen, TT; Nguyen, PL; Kawanishi, Y; Komamizu, T; Ide, I

IEEE ACCESS 12 巻頁： 55889 - 55904 2024年

出版者・発行元：IEEE Access

Patients' safety is paramount in the healthcare industry, and reducing medication errors is essential for improvement. A promising solution to this problem involves the development of automated systems capable of assisting patients in verifying their pill intake mistakes. This paper investigates a Pill-Prescription matching task that seeks to associate pills in a multi-pill photo with their corresponding names in the prescription. We specifically aim to overcome the limitations of existing pill detection methods when faced with unseen pills, a situation characteristic of zero-shot learning. We propose a novel method named Zero-PIMA (Zero-shot Pill-Prescription Matching), designed to match pill images with prescription names effectively, even for pills not included in the training dataset. Zero-PIMA is an end-to-end model that includes an object localization module to determine and extract features of pill images and a graph convolutional network to capture the spatial relationship of the pills' text in the prescription. After that, we leverage the contrastive learning paradigm to increase the distance between mismatched pill images and pill name pairs while minimizing the distance between matched pairs. In addition, to deal with the zero-shot pill detection problem, we leverage pills' metadata retrieved from the DrugBank database to fine-tune a pre-trained text encoder, thereby incorporating visual information about pills (e.g., shape, color) into their names, making them more informative and ultimately enhancing the pill image-name matching accuracy. Extensive experiments are conducted on our collected real-world VAIPEPP dataset of multi-pill photos and prescriptions. Through a series of comprehensive experiments, the proposed method outperforms other methods for both seen and unseen pills in terms of mean average precision. These results indicate that the proposed method could reduce medication errors and improve patients' safety.

DOI： 10.1109/ACCESS.2024.3390153

R-DiP: Re-ranking Based Diffusion Pre-computation for Image Retrieval

Kato, T; Komamizu, T; Ide, I

DATABASE AND EXPERT SYSTEMS APPLICATIONS, PT II, DEXA 2024 14911 巻頁： 233 - 247 2024年

出版者・発行元：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

In image retrieval tasks, although efficient methods based on pre-computing information related to retrieval and effective methods utilizing re-ranking have been proposed, developing a method that achieves both efficiency and effectiveness at the same time, remains challenging. To develop an efficient and effective image retrieval method, we propose a simple-yet-effective novel image retrieval framework; R-DiP (Re-ranking based Diffusion Pre-computation). It incorporates an effective re-ranking model into the pre-computation step of an existing efficient method, namely, Offline Diffusion that pre-computes the diffusion process in the offline step and provides a simple linear combination-based retrieval in the online step. Experimental results on standard benchmarks shows that R-DiP performs comparable to the State-Of-The-Art (SOTA) image retrieval method, namely SuperGlobal, while maintaining the efficiency of Offline Diffusion. Notably, in million-scale datasets, R-DiP improves the mAP (mean Average Precision) by about 2.0%, and reduces the speed by about 75% on average, surpassing SOTA methods. These results indicate that R-DiP is a promising solution to the efficiency-effectiveness trade-off in image retrieval, that offers the flexibility to incorporate any advanced re-ranking method in the future.

DOI： 10.1007/978-3-031-68312-1_18

One-Stage Open-Vocabulary Temporal Action Detection Leveraging Temporal Multi-scale and Action Label Features

Nguyen, TT; Kawanishi, Y; Komamizu, T; Ide, I

2024 IEEE 18TH INTERNATIONAL CONFERENCE ON AUTOMATIC FACE AND GESTURE RECOGNITION, FG 2024 2024年

出版者・発行元：2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition, FG 2024

Open-vocabulary Temporal Action Detection (Open-vocab TAD) is an advanced video analysis approach that expands Closed-vocabulary Temporal Action Detection (Closed-vocab TAD) capabilities. Closed-vocab TAD is typically confined to localizing and classifying actions based on a predefined set of categories. In contrast, Open-vocab TAD goes further and is not limited to these predefined categories. This is particularly useful in real-world scenarios where the variety of actions in videos can be vast and not always predictable. The prevalent methods in Open-vocab TAD typically employ a 2-stage approach, which involves generating action proposals and then identifying those actions. However, errors made during the first stage can adversely affect the subsequent action identification accuracy. Additionally, existing studies face challenges in handling actions of different durations owing to the use of fixed temporal processing methods. Therefore, we propose a L-stage approach consisting of two primary modules: Multi-scale Video Analysis (MVA) and Video-Text Alignment (VTA). The MVA module captures actions at varying temporal resolutions, overcoming the challenge of detecting actions with diverse durations. The VTA module leverages the synergy between visual and textual modalities to precisely align video segments with corresponding action labels, a critical step for accurate action identification in Open-vocab scenarios. Evaluations on widely recognized datasets THUMOSl4 and ActivityNet-I.3, showed that the proposed method achieved superior results compared to the other methods in both Open-vocab and Closed-vocab settings. This serves as a strong demonstration of the effectiveness of the proposed method in the TAD task.

DOI： 10.1109/FG59268.2024.10581896

Interpolating the Text-to-Image Correspondence Based on Phonetic and Phonological Similarities for Nonword-to-Image Generation

Matsuhira, C; Kastner, MA; Komamizu, T; Hirayama, T; Doman, K; Kawanishi, Y; Ide, I

IEEE ACCESS 12 巻頁： 41299 - 41316 2024年

掲載種別：研究論文（学術雑誌）出版者・発行元：IEEE Access

Text-to-Image (T2I) generation is the task of synthesizing images corresponding to a given text input. The recent innovations in artificial intelligence have enhanced the capacity of conventional T2I generation, yielding more and more powerful models day by day. However, their behavior is known to become unstable in the face of text inputs containing nonwords that have no definition within a language. This behavior not only results in situations where image generation does not match human expectations but also hinders these models from being utilized in psycholinguistic applications and simulations. This paper exploits the human nature of associating nonwords with their phonetically and phonologically similar words and uses it to propose a T2I generation framework robust against nonword inputs. The framework comprises a phonetics-aware language model as well as an adjusted T2I generation model. Our evaluations confirm that the proposed nonword-to-image generation synthesizes images that depict visual concepts of phonetically similar words more stably than comparative methods. We also assess how the image generation results match human expectations, showing a better agreement than the phonetics-blind baseline.

DOI： 10.1109/ACCESS.2024.3378095

ICDAR 24: Intelligent Cross-Data Analysis and Retrieval

Dao, MS; Riegler, MA; Dang-Nguyen, DT; Tran, HN; Kiran, RU; Komamizu, T

PROCEEDINGS OF THE 4TH ANNUAL ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, ICMR 2024 頁： 1332 - 1333 2024年

出版者・発行元：ICMR 2024 - Proceedings of the 2024 International Conference on Multimedia Retrieval

Our workshop aims to provide a platform for both academic and industrial professionals engaged in the analysis and retrieval of cross-data from diverse perspectives, with a particular emphasis on wearable and ambient sensors, lifelog cameras, social networks, and surrounding sensors. Despite numerous studies exploring individual viewpoints, there remains a significant gap in the analysis and retrieval of cross-data to maximize benefits for humanity. Additionally, challenges such as data security and distributed learning for cross-modal model training and inference arise when dealing with large and distributed datasets. We invite researchers to contribute to this initiative, with the overarching goal of fostering the development of a smart and sustainable society through the efficient utilization of intelligent cross-data analysis and retrieval techniques.

DOI： 10.1145/3652583.3659999

Do LLMs Agree with Humans on Emotional Associations to Nonsense Words?

Miyakawa Y., Matsuhira C., Kato H., Hirayama T., Komamizu T., Ide I.

CMCL 2024 - 13th Edition of the Workshop on Cognitive Modeling and Computational Linguistics, Proceedings of the Workshop 頁： 81 - 85 2024年

出版者・発行元：CMCL 2024 - 13th Edition of the Workshop on Cognitive Modeling and Computational Linguistics, Proceedings of the Workshop

Understanding human perception of nonsense words is helpful to devise product and character names that match their characteristics. Previous studies have suggested the usefulness of Large Language Models (LLMs) for estimating such human perception, but they did not focus on its emotional aspects. Hence, this study aims to elucidate the relationship of emotions evoked by nonsense words between humans and LLMs. Using a representative LLM, GPT-4, we reproduce the procedure of an existing study to analyze evoked emotions of humans for nonsense words. A positive correlation of 0.40 was found between the emotion intensity scores reproduced by GPT-4 and those manually annotated by humans. Although the correlation is not very high, this demonstrates that GPT-4 may agree with humans on emotional associations to nonsense words. Considering that the previous study reported that the correlation among human annotators was about 0.68 on average and that between a regression model trained on the annotations for real words and humans was 0.17, GPT-4’s agreement with humans is notably strong.

Investigating Conceptual Blending of a Diffusion Model for Improving Nonword-to-Image Generation

Matsuhira C., Kastner M.A., Komamizu T., Hirayama T., Ide I.

MM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia 頁： 7307 - 7315 2024年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：MM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia

Text-to-image diffusion models sometimes depict blended concepts in the generated images. One promising use case of this effect would be the nonword-to-image generation task which attempts to generate images intuitively imaginable from a non-existing word (nonword). To realize nonword-to-image generation, an existing study focused on associating nonwords with similar-sounding words. Since each nonword can have multiple similar-sounding words, generating images containing their blended concepts would increase intuitiveness, facilitating creative activities and promoting computational psycholinguistics. Nevertheless, no existing study has quantitatively evaluated this effect in either diffusion models or the nonword-to-image generation paradigm. Therefore, this paper first analyzes the conceptual blending in a pretrained diffusion model, Stable Diffusion. The analysis reveals that a high percentage of generated images depict blended concepts when inputting an embedding interpolating between the text embeddings of two text prompts referring to different concepts. Next, this paper explores the best text embedding space conversion method of an existing nonword-to-image generation framework to ensure both the occurrence of conceptual blending and image generation quality. We compare the conventional direct prediction approach with the proposed method that combines k-nearest neighbor search and linear regression. Evaluation reveals that the enhanced accuracy of the embedding space conversion by the proposed method improves the image generation quality, while the emergence of conceptual blending could be attributed mainly to the specific dimensions of the high-dimensional text embedding space.

DOI： 10.1145/3664647.3681202

その他リンク： https://dblp.uni-trier.de/db/conf/mm/mm2024.html#Matsuhira0KHI24

Lightweight Maize Disease Detection through Post-Training Quantization with Similarity Preservation

Padeiro C.V., Chen T.W., Komamizu T., Ide I.

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops 頁： 2111 - 2120 2024年

出版者・発行元：IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

Traditional crop disease diagnosis, reliant on expert visual observation, is expensive, time-consuming, and prone to error. While Convolutional Neural Networks (CNNs) offer promising alternatives, their high resource demands limit their accessibility to farmers, particularly those in resource-constrained settings. Lightweight models that operate on resource-limited devices without network access are crucial to address this gap. This paper proposes a Similarity-Preserving Quantization (SPQ) method to convert high-precision CNNs into lower-precision models while maintaining similar feature representations. While quantization offers a promising approach for building lightweight CNNs for crop disease detection, the quality of quantized models often suffers. SPQ addresses this challenge by ensuring equivalent activation patterns for similar crop images in both the original and quantized models. Experimental evaluation using MobileNetV2 and ResNet-50 demonstrates that SPQ improves throughput, inference, and memory footprint more than 3 times while preserving the detection performance.

DOI： 10.1109/CVPRW63382.2024.00216

　前のページ - 次のページ　

1
2
3
4
5