Inhibited Self-Attention: Sharpening Focus in Vision Transformers

Vision Transformers (ViTs) have demonstrated remarkable performance in computer vision tasks.

Vision Transformers (ViTs) have demonstrated remarkable performance in computer vision tasks. However, their self-attention mechanism often diffuses focus across background regions, relying on spurious correlations rather than object-relevant cues. Inspired by inhibitory…