Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation
Jesse St. Amand · Callum Canavan · Sohaib Imran · Joseph Hewson · Aaron Lutz · Shi Feng · Puria Radmard · Lennie Wells
Self-generated text recognition accuracy depends on evaluation operationalization, reconciling conflicting prior findings; quality judgments are a dominant confound; and SGTR training transfers broadly, biasing LLM judges toward their own text.
摘要
Self-Generated Text Recognition (SGTR) — the ability of an LLM to identify its own outputs — poses risks to AI safeguards that rely on LLMs as evaluators or monitors: an LLM may recognize outputs from other copies of the same model and make biased judgments or collude outright. Prior work has drawn conflicting conclusions about whether current models possess significant SGTR capabilities. We explain these disagreements by identifying key experimental design choices — which we term operationalizations — that drive divergent results. Evaluating 13–21 models across six presentation operationalizations and four task-domain operationalizations, we find that accuracy varies substantially with evaluation format (pairwise vs. individual assessments of text), conversation format (presenting candidate text in user tags vs. assistant tags), and the domain of the task used to generate candidate text (e.g., coding vs. summarization). We corroborate previous observations that a quality heuristic — models attributing authorship to text they perceive as higher quality — is a dominant confound. We also find that improving a model's SGTR performance via supervised fine-tuning (SFT) on one operationalization can generalize to others, and can increase the model's preference for its own outputs when it acts as a judge in the AlpacaEval framework. Our results suggest that, despite confounds, some models possess practical SGTR capabilities, and that SGTR should be monitored and considered in the design of safety-critical AI applications.