作者位次 #8/10SIGIR 2026Poster· Information Retrieval (Resource)
Auto-Judge: A Cross-Task Benchmark for Comparing LLM Judges for Citation-Grounded RAG Systems
Naghmeh Farzi · Tim Hagen · Eugene Yang · Maik Fröbe · Ronak Pradeep · Hossein A. Rahmani · Xi Wang · Oleg Zendel · Martin Potthast · Laura Dietz
摘要
We present the Auto-Judge resource for the meta-evaluation of automated LLM judges, especially judges that evaluate Retrieval-Augmented Generation (RAG) systems that ground their response with citations. The resource couples (i) a data release of topics, pooled RAG responses, and human judgments,...