Does Self-Consistency Improve the Recall of Encyclopedic Knowledge?
Sho Hoshino · Ukyo Honda · Peinan Zhang
摘要
While self-consistency is known to improve performance on symbolic reasoning, its effect on the recall of encyclopedic knowledge is unclear due to a lack of targeted evaluation grounds. To address this, we establish such a knowledge recall split for the popular MMLU benchmark by applying a data-d...