Prism-$\Delta$: Differential Subspace Steering for Prompt Highlighting in Large Language Models
Yuyao Ge · Shenghua Liu · Yiwei Wang · Baolong Bi · Lingrui Mei · Jiayu Yao · Tianyu Liu · Jiafeng Guo · Xueqi Cheng
PRISM-Delta steers LLM attention by decomposing differential cross-covariance matrices, isolating discriminative directions with per-head softplus weighting across Key and Value channels, achieving up to +10.6% gains over prior methods.
摘要
Prompt highlighting steers a large language model to prioritize user-specified text spans during generation. A key challenge of existing Key-editing approaches is extracting steering directions that capture the difference between relevant and irrelevant contexts, rather than shared structural patterns common to both. We propose PRISM-$\Delta$ (Projection-based Relevance-Informed Steering Method), which decomposes the difference between positive and negative cross-covariance matrices to maximize discriminative energy while eliminating shared directions. Each attention head receives a continuous softplus importance weight, letting weak-but-useful heads contribute at reduced strength. The framework extends naturally to Value representations, capturing content-channel signal that Key-only methods leave unused. Across four benchmarks and five models, our methods match or exceed the best existing method on 19 of 20 configurations, with relative gains up to +13.0%, while halving the fluency cost of steering. Our methods also scale to long-context retrieval, outperforming the best existing method by up to +5.1% relative gain. PRISM-$\Delta$ is compatible with FlashAttention and adds negligible memory overhead.