← All publications
ICLR 2025 · International Conference on Learning Representations

Multi-Perspective Data Augmentation for Few-shot Object Detection

Anh-Khoa Nguyen Vu, Quoc-Truong Truong, Vinh-Tiep Nguyen, Thanh Duc Ngo, Thanh-Toan Do, Tam V. Nguyen

Abstract

Recent few-shot object detection (FSOD) methods have focused on augmenting synthetic samples for novel classes, showing promising results with the rise of diffusion models. However, the diversity of such datasets is often limited because they lack awareness of typical and hard samples, especially regarding foreground and background relationships. To tackle this, we propose a Multi-Perspective Data Augmentation (MPAD) framework. For foreground-foreground relationships, we propose in-context learning for object synthesis (ICOS) with bounding box adjustments to enhance the detail and spatial information of synthetic samples. Inspired by the large margin principle, we design a Harmonic Prompt Aggregation Scheduler (HPAS) to mix prompt embeddings at each timestep of the diffusion generation process, producing hard novel samples. For foreground-background relationships, we introduce a Background Proposal method (BAP) to sample typical and hard backgrounds. Our framework significantly outperforms traditional methods, achieving an average increase of 17.5% in nAP50 over the baseline on PASCAL VOC.

Overview

MPAD (Multi-Perspective Data Augmentation) improves few-shot object detection by generating better synthetic training data with diffusion models. Existing diffusion-based augmentation lacks awareness of typical vs. hard samples and of foreground–background relationships, which limits diversity.

MPAD attacks this from three angles. For foreground–foreground relationships, In-Context learning for Object Synthesis (ICOS) with bounding-box adjustment sharpens the detail and spatial information of synthesized instances. Inspired by the large-margin principle, a Harmonic Prompt Aggregation Scheduler (HPAS) mixes prompt embeddings at each diffusion timestep to produce hard novel samples that tighten class boundaries. For foreground–background relationships, a Background Proposal method (BAP) samples both typical and hard backgrounds.

Figures
MPAD generates hard novel-class samples with diffusion: the Harmonic Prompt Aggregation Scheduler (HPAS) mixes prompt embeddings across denoising steps, turning noise into a detection-ready instance.
Fig. 1. MPAD generates hard novel-class samples with diffusion: the Harmonic Prompt Aggregation Scheduler (HPAS) mixes prompt embeddings across denoising steps, turning noise into a detection-ready instance.
A synthesized novel-class instance with its bounding box — augmented training data produced by MPAD's in-context object synthesis (ICOS).
Fig. 2. A synthesized novel-class instance with its bounding box — augmented training data produced by MPAD's in-context object synthesis (ICOS).
Results

Across multiple few-shot object detection benchmarks, MPAD significantly outperforms traditional augmentation, with an average gain of 17.5% in nAP50 over the baseline on PASCAL VOC. The paper was published at ICLR 2025; code is publicly available.