一作SIGIR 2026Poster· Information Retrieval (Short)
When LLM Judges Inflate Scores: Exploring Overrating in Relevance Assessment
Chuting Yu · Hang Li · Shengyao Zhuang · Joel Mackenzie · Teerapong Leelanupab
摘要
Human relevance assessment is time-consuming and cognitively intensive, limiting the scalability of Information Retrieval evaluation. This has led to growing interest in using large language models (LLMs) as proxies for human judges. However, it remains an open question whether LLM-based relevanc...