← All publications
FeaturedThe IEEE/CVF Winter Conference on Applications of Computer Vision 2027

Evidence Placement Matters: Pixel-Grounded Re-Ranking for Training-Free Composed Video Retrieval

Long Le, Khang Nguyen, Bao Tran, Thanh Duc Ngo

Abstract

Training-free composed video retrieval (CoVR) often converts a reference video into language, rewrites the generated caption according to an edit instruction, and retrieves with the resulting target description. This creates an information bottleneck because visual evidence omitted during generation cannot be recovered by the final matcher. We instead ask what evidence should occupy the final compositional decision. PiL-RANK is a training-free framework that keeps direct visual evidence in the loop until that decision. A frozen retriever first builds a top-k shortlist, then a frozen MLLM listwise re-ranks it from raw query and candidate frames, while generated descriptions serve only as verifiable priors. On WebVid-CoVR-Test, pixel-grounded re-ranking alone improves R@1 from 54.73 to 62.52 with BLIP and from 54.81 to 64.55 with CLIP-L. The prior adds a smaller increment, reaching 64.20 and 65.81. Its effect depends on the pixels available for verification. Replacing query and candidate visual evidence with generated captions flips the identical prior's sign and drops performance below the first-stage baseline. Shortlist expansion to the largest candidate set supported by the MLLM image context (k = 23), combined with rank fusion, raises R@10 from 86.15 to 90.41. The same pipeline also transfers to CIRR, lifting a simple frozen CLIP-L baseline from 13.44 to 42.02 R@1. Generated language is therefore best treated as a prior to be checked against pixels rather than as evidence to retrieve with.