一作ACL 2026Poster· Natural Language Processing
Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings
Tomomasa Hara · Hiroto Kurita · Masaaki Imaizumi · Kentaro Inui · Sho Yokoi
摘要
For constructing text embeddings, mean pooling, which averages token embeddings, is the standard approach. This paper examines whether mean pooling actually works well in real models. First, we note that mean pooling can collapse information beyond the first-order statistics of the token embeddin...