Publications

When Helpful Text Hurts: Option-Redirecting Bias in Vision–Language Models

Tam Le Thi Thanh, Hoang Tran Van, Thanh Duc Ngo

In tri-modal visual question answering (VQA), auxiliary text — captions, OCR outputs, retrieved descriptions — is commonly added to complement image and question, yet its reliability is often uncontrolled. We show that the most harmful auxiliary text is not the most factually incorrect, but the text that aligns with the question while contradicting the image and favoring a specific distractor, systematically redirecting predictions. To isolate this, we introduce the Textual Reliability Ladder, a controlled diagnostic that decomposes auxiliary text along three axes — image consistency, question relevance and option support. Across ScienceQA, VCR, A-OKVQA and Causal-VidQA and several recent VLMs, such distractor-supporting text induces the largest accuracy drops (up to 53.1%) and concentrates errors on specific incorrect options. We further propose a training-free, inference-time intervention that counteracts this redirection via noise-stability steering and dynamic grounding, reducing redirected errors while largely preserving performance under faithful text. The results argue that auxiliary-text reliability must be judged at the decision level, not by factual correctness alone.

From Dialogue to Evidence: Retrieval-State-Conditioned Interaction for Text-Based Person Retrieval

Bao Tran, Thanh Duc Ngo

Interactive text-based person retrieval resolves ambiguity in an underspecified initial query through iterative clarification. Existing methods rely on free-form dialogue, so acquired evidence is only implicitly encoded in dialogue history, remaining uncertainty is not explicitly represented, and generated questions are often weakly grounded in the visual differences that determine ranking — a problem amplified by the long-tailed distribution of visual attributes. We reformulate interactive retrieval as a retrieval-state-conditioned evidence-acquisition problem over a grounded discrete concept space of reusable concept atoms, a shared representation for the initial query, candidate-side evidence and user feedback. At each round the system selects the next concept atom conditioned on the current candidate set using a redundancy-aware utility that jointly favors ambiguity reduction, candidate discrimination and high-quality partitioning of the candidate pool. Natural language serves only as the interaction interface, while decision-making is performed over structured concept-level evidence — replacing unconstrained dialogue generation with structured, utility-driven clarification and requiring no additional question-generation or interaction-policy training. On Interactive-PEDES under a unified multi-round protocol, the method consistently outperforms language-centric baselines, with R@1 improving by 31.01% over five rounds.

Towards Scalable and Context-Aware Multimodal Interactive Video Retrieval

Bao Tran, Khiem Le, Thanh Duc Ngo

Interactive video retrieval (IVR) is challenging due to its multimodal nature, ambiguous user intent and strict real-time constraints. We introduce a unified, multimodal, context-aware framework for large-scale IVR. A fast, scalable design leverages GPU-accelerated indexing, parallel inference and lightweight fusion for sub-second responsiveness at million-scale; a multimodal architecture integrates vision-language embeddings, scene text, speech transcripts and object-level cues through adaptive score fusion to mitigate modality bias; and a multi-stage sequential retrieval paradigm handles ambiguous or partially ordered queries via coarse-to-fine expansion and context-aware alignment.

ITSELF: Attention Guided Fine-Grained Alignment for Vision-Language Retrieval

Tien-Huy Nguyen, Huu-Loc Tran, Thanh Duc Ngo

Vision Language Models (VLMs) have rapidly advanced and show strong promise for text-based person search (TBPS), a task that requires capturing fine-grained relationships between images and text to distinguish individuals. Previous methods address these challenges through local alignment, yet they are often prone to shortcut learning and spurious correlations, yielding misalignment. Motivated by the finding that encoder attention surfaces spatially precise evidence from the earliest training epochs, we introduce ITSELF, an attention-guided framework for implicit local alignment. At its core, Guided Representation with Attentive Bank (GRAB) converts the model's own attention into an Attentive Bank of high-saliency tokens and applies local objectives on this bank, learning fine-grained correspondences without extra supervision. Multi-Layer Attention for Robust Selection (MARS) aggregates attention across layers with diversity-aware top-k selection, and an Adaptive Token Scheduler (ATS) schedules the retention budget from coarse to fine over training.

NII-UIT at VBS2026: Towards Effective Visual Question Answering for Interactive and Multimodal Video Retrieval

Bao Tran, Tien Do, Thanh Duc Ngo, Duy-Dinh Le, Shin'ichi Satoh

Interactive video retrieval increasingly requires answering targeted questions about fleeting moments in long videos, under ambiguous queries and tight time limits. This paper presents the NII-UIT system for the Video Browser Showdown (VBS) 2026, addressing three limitations of current systems: localizing brief answer-bearing segments, the high verification effort demanded of users under time pressure, and limited integration of diverse modalities such as visual content and speech. The system introduces an Answer Span Prediction module to highlight candidate temporal regions, a Candidate Answer Suggestion mechanism that aggregates multimodal cues into verifiable answer options, and a dedicated In-Video Retrieval component.

MVDGC: Joint 3D and 2D Multi-view Pedestrian Detection via Dual Geometric Constraints

Thinh Phan, Hao Vo, Khoa Vo, Cuong Pham, Thanh Duc Ngo, Ngan Le

Holistic Feature Fusion for Fine-Grained Pen Classification

Huy Nguyen Pham Gia, Dang Ngo Viet Tue, Hien Pham Duy, Bao Ta Cao Nguyen, Thuyen Tran Doan, Thanh Duc Ngo

Pen classification is a fine-grained forensic document-analysis task that identifies the writing instrument behind an ink trace, supporting document authentication and forgery investigation. It is challenging because visually similar pens produce nearly indistinguishable strokes, while scanning artifacts, paper background and subtle ink variations obscure discriminative cues. We propose a dual-stream holistic feature-fusion framework that integrates deep visual representations (a ConvNeXt-Base backbone initialized from DINOv3) with a 215-dimensional handcrafted descriptor of color, texture, edge and stroke-morphology statistics. A radially biased Circle Spatial Attention module suppresses peripheral background and focuses on the centered circular sample, and the fused embedding is optimized with ArcFace-based margin supervision to separate highly confusable pen classes. On the ICDAR 2026 CircleID Pen Classification benchmark, the system reaches 0.92726 Top-1 accuracy on the private test set, securing joint first place.

MGTE: A Modular Multi-Granularity Text Ensemble for Interactive Image Retrieval

Khiem Le, Bao Tran, Duy-Dinh Le, Thanh Duc Ngo

Interactive text-to-image retrieval identifies a target image through multi-turn dialogue, where users progressively refine an ambiguous query. Existing methods encode the full dialogue as a single concatenated sequence, which is vulnerable to token-budget truncation and can overlook late-turn refinements or negated attributes. We propose Multi-Granularity Text Ensemble (MGTE), a training-free scoring module that represents dialogue context from three complementary views — the full dialogue, the latest turn and the initial caption — combined with turn-adaptive weights, so it drops into existing retrieval pipelines without retraining. A companion Negative-Aware Scoring (NAS) mechanism penalizes candidates that match negated concepts. On VisDial, MGTE consistently improves three representative retrieval pipelines with Hits@10 gains of up to +4.31, and with NAS the DAR pipeline reaches 85.42 Hits@10.

Super-Image Reranking: Lightweight Temporal Context Aggregation for Ad-hoc Video Search

Thuyen Tran Doan, Tien Do, Thanh Duc Ngo

Ad-hoc video search retrieves relevant video shots from large collections using free-form text queries. Pretrained vision-language models give strong frame-level matching, but max-similarity aggregation can miss semantic evidence spread across neighboring frames. We propose a lightweight retrieve-then-rerank framework that augments a frame-level image-text baseline with super-image reranking: for top-ranked candidate shots, consecutive frames are arranged into 3×3 multi-frame grid images and matched with the query using the same backbone, then fused with frame-level refinement and merged with the original score via query-wise normalization. On TRECVID AVS 2023 it improves infAP from 0.2160 to 0.2414 over a BEiT-3 frame-level max-similarity baseline, and zero-shot experiments on MSR-VTT with CLIP backbones show consistent gains — without any dedicated video-text training.

Track-Level Aggregation for Low-Resolution License Plate Recognition

Khanh V. Vu Nguyen, Sang T. Pham, Duc N. N. Phung, Trong P. Le, Vy N. Vo Tran, Tien Do, Duy-Dinh Le

Low-resolution license plate recognition from surveillance video is difficult when plate crops captured at long range are extremely small, blurred, compressed and noisy. Restoration-first pipelines may improve perceptual quality without preserving character-discriminative structure, while single-frame recognizers ignore complementary evidence available across consecutive observations. We propose a track-level framework that aggregates cues across multiple low-resolution frames before format-constrained decoding: a shared ViT encoder and temporal Transformer contextualize frames, quality-weighted score fusion favors sharper observations, regional grammar masks (e.g. Brazilian and Mercosur formats) constrain beam search, and a confidence-gated specialist ensemble resolves layout-specific errors. On the ICPR 2026 Low-Resolution License Plate Recognition benchmark it achieves 79.83% exact-match accuracy on the official hidden test set.