This research proposal introduces AdaHybrid-ViT, an adaptive hybrid vision architecture that dynamically combines CNNs, local window transformers, and global transformers through a learnable gating mechanism. The architecture adapts computational pathways based on input complexity and available resources, achieving 90.2% ImageNet-1K accuracy with 4× fewer FLOPs than ViT-B/16. The proposed method includes a resource-aware controller that adjusts pathway weights dynamically, enabling efficient deployment on mobile and edge devices. Comprehensive evaluation protocols are outlined for ImageNet classification, COCO detection, and mobile deployment scenarios.
Key findings
AdaHybrid-ViT achieves 90.2% ImageNet-1K accuracy with 4× fewer FLOPs than standard ViT-B/16 through adaptive pathway selection
The three-branch architecture combines CNN local features, local transformer window attention, and global transformer dependencies via learnable gating
Resource-aware controller enables dynamic accuracy-efficiency trade-offs at inference time by adjusting pathway weights based on computational budget
Projected results suggest AdaHybrid-S achieves 82.5% accuracy with 22M parameters and 3.2G FLOPs, outperforming comparable baselines
Limitations & open questions
Experimental results are preliminary/projected rather than validated, as this is a research proposal without completed experiments
Effectiveness on extremely resource-constrained devices and training stability with three parallel pathways remain unverified
Document is truncated and missing complete risk analysis and mitigation details