Hidetaka Kamigaito

Understanding what kinds of factual knowledge large language models (LLMs) memorize is essential for evaluating their reliability and limitations.Entity-based QA is a common framework for analyzing non-verbatim memorization, but typical evaluations query each entity using a single canonical surfa...

Recently, we have often observed hallucinated citations or references that do not correspond to any existing work in papers under review, preprints, or published papers. Such hallucinated citations pose a serious concern to scientific reliability. When they appear in accepted papers, they may als...

The text-to-table task aims to generate structured data in tabular formats from unstructured text. While the integration of large language models (LLMs) has significantly enhanced the comprehensiveness and flexibility of generation, challenges regarding inconsistent output quality persist, such a...

We introduce VLURes , a multilingual benchmark for evaluating Vision-Language Models (VLMs) under long-text grounding : selecting and reasoning over the image-relevant subset of article-length text that contains distractors and ungrounded claims. VLURes contains 4,000 web-curated image + long-tex...

Recently, watermarking has attracted growing attention as a practical technique for source attribution of machine-generated text. However, most prior work studies watermarking under benign prompts, while its behavior under jailbreaking prompts remains underexplored. This gap matters because jailb...

Automatic evaluation metrics are essential for building multilingual translation systems. The common practice of evaluating these systems is averaging metric scores across languages, yet this is suspicious since metrics may suffer from cross-lingual scoring bias, where translations of equal quali...

Improving the quality of model-generated summaries, especially factuality, the accuracy of a summary with respect to its source content, remains a challenge. While reranking could select the optimal output from multiple generated candidates, it is limited to only using the source as guidance, res...

In simultaneous interpretation (SI), interpreters perform real-time translation by segmenting the source speech into chunks and translating them in the order they appear.Since surface-matching metrics such as BLEU correlate poorly with human evaluations, translation quality is often evaluated usi...