作者位次 #2/4ACL 2026Poster· Natural Language Processing
SSA : Improving Performance With a Better Scoring Function
Omar Naim · Swarnadeep Bhar · Jerome Bolte · Nicholas Asher
摘要
While transformer models exhibit strong in-context learning (ICL) abilities, they often fail to generalize under simple distribution shifts. We analyze these failures and identify Softmax, the scoring function in the attention mechanism, as a contributing factor. We propose Scaled Signed Averagin...