This technical report presents Cross-Domain Adaptive Lookahead (CDAL), a unified framework for KV cache eviction across text-only, vision-language, and multimodal modalities. The architecture combines modality-aware lookahead tokens, cross-domain LoRA adapters with shared base parameters, and a dynamic fusion estimator for runtime cross-modal quantification. The approach aims to address the accuracy-overhead trade-off in draft-based methods while enabling zero-shot cross-domain generalization without modality-specific training. Expected contributions include 14-18x latency reduction compared to draft-based baselines and 15-25% performance gains in multimodal settings through fusion-aware eviction strategies.
Key findings
Proposes first unified eviction framework spanning text-only, vision-language, and visual autoregressive domains without separate per-modality training
Introduces modality-aware lookahead tokens that adapt to input characteristics across different modalities
Designs cross-domain LoRA adapters enabling parameter-efficient transfer while maintaining frozen base model weights
Develops dynamic fusion estimator to quantify cross-modal fusion patterns for adaptive eviction in multimodal contexts
Projects 14-18x latency reduction versus draft-based baselines while maintaining comparable accuracy
Limitations & open questions
No empirical validation or experimental results presented; remains a design proposal with implementation roadmap spanning 20 weeks
Performance claims are projected expectations rather than demonstrated achievements
Document content is truncated, omitting complete related work discussion and experimental details