Dawn Lawrie

While ColBERT is an effective neural retrieval architecture, it requires a heavy index structure to support candidate set retrieval based on approximated token embeddings, gathering and decompressing document token embeddings, and applying the MaxSim operation. Indexes in PLAID and similar ColBER...

We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results.Information coverage is a central aspect for retrieval, especially when the retrieval system is integrated with generativ...

Nugget-based LLM judges evaluate Retrieval-Augmented Generation (RAG) systems using a bank of questions that capture the key facts and criteria an answer should address. These nugget banks are typically constructed through a combination of human input and LLM generation. System outputs are graded...

Generation of citation-backed reports is a primary use case for retrieval-augmented generation (RAG) systems. While open-source evaluation tools exist for various RAG tasks, tools designed for report generation are lacking. Accordingly, we introduce Auto-ARGUE, a robust LLM-based implementation o...