Would you like to see your presentation here, made available to a global audience of researchers?
Add your own presentation or have us affordably record your next conference.
Zero-Shot Composed Image Retrieval (ZS-CIR) involves diverse tasks with a broad range of visual content manipulation intent across domains, scenes, objects, and attributes. A key challenge in ZS-CIR is training models on datasets with limited intention-relevant data to accurately interpret human intent, as implicitly expressed through textual modifications, to retrieve the desired images. In this paper, we introduce an intention-based image-text dataset generated through reasoning by a Multimodal Large Language Model (MLLM) to enhance ZS-CIR model training for interpreting human manipulation intents. Leveraging this dataset, we propose De-MINDS, a novel framework that distills the MLLM’s reasoning capabilities to capture human intentions, thereby improving ZS-CIR models’ comprehension of manipulation text. Specifically, a simple mapping network translates image information into language space, forming a query with the manipulation text. De-MINDS then extracts intention-relevant information from the query, converting it into pseudo-word tokens for accurate ZS-CIR. De-MINDS demonstrates robust generalization and significant performance improvements across four ZS-CIR tasks, outperforming existing methods by 2.15% to 4.05% and establishing new state-of-the-art results with comparable inference times. Our code is available at https://anonymous.4open.science/r/De-MINDS/.