Benno Stein

Conversational search systems are typically evaluated using a fixed reference collection of conversations or through user studies with a live system. However, fixed-reference conversations can cover only a few plausible conversations, and user studies are costly, time-consuming, and often hard to...

Shared tasks offer excellent opportunities for advancing scientific fields and standardizing evaluation methods through collaborative community efforts. With the recent paradigm shift in information retrieval from keyword-based to conversational search, shared tasks can serve as promising sources...