一作NeurIPS 2026Poster
Why Transformers Struggle with Distribution-Independent In-Context Learning
Omar Naim · Jerome Bolte · Nicholas Asher
理论与学习动力学OpenReview
首次获取该候选人的完整档案及 StartLux 评估共 20 积分；每位用户单独解锁，之后查看免费。主动刷新 20 积分。
实时查找电子邮箱，费用不超过20积分；未找到邮箱不扣费。
Omar Naim · Jerome Bolte · Nicholas Asher
Omar Naim · Swarnadeep Bhar · Jerome Bolte · Nicholas Asher
While transformer models exhibit strong in-context learning (ICL) abilities, they often fail to generalize under simple distribution shifts. We analyze these failures and identify Softmax, the scoring function in the attention mechanism, as a contributing factor. We propose Scaled Signed Averagin...
Omar Naim · Krish Sharma · Niyar R Barman · Nicholas Asher
Large Language Models (LLMs) typically come with a fixed architecture, despite growing evidence that not all layers contribute equally to every downstream task. We introduce TALE (Task-Aware Layer Elimination), an inference-time method that improves task performance by selectively removing layers...