This survey examines multi-modal active prompting that integrates spatial and textual cues for interactive segmentation. It analyzes Bayesian active learning strategies for spatial prompts, vision-language integration for referring expressions, and uncertainty quantification in foundation models like SAM. The paper presents a comprehensive taxonomy, reviews 40+ key papers, and identifies five open research questions regarding uncertainty fusion across modalities.
Key findings
Spatial active learning has achieved 30-50% reduction in interaction rounds but ignores textual information completely
Current vision-language approaches focus on one-shot segmentation rather than iterative refinement
Integration of textual cues into active prompt selection remains largely unexplored despite semantic ambiguity challenges in medical imaging and autonomous driving
Foundation models like SAM provide powerful backbones but require optimized multi-modal prompt strategies
Five critical evidence gaps identified in uncertainty fusion, dynamic strategy selection, and evaluation standardization
Limitations & open questions
Survey focuses primarily on theoretical frameworks and lacks empirical validation experiments
Publication date of May 2026 suggests this may be a speculative or preprint document not yet peer-reviewed
Limited discussion of computational efficiency trade-offs in multi-modal fusion methods