Can Large Language Models Match Human Diversity in Educational Content Generation?
Mei Tan · HyunJi Nam · Dorottya Demszky
LLMs generate more homogeneous math problems and essay feedback than humans, amplifying the most common instructional routines while underrepresenting ambitious teaching.
摘要
Large language models (LLMs) are known to lack diversity in open-ended text generation, a limitation that is especially consequential in education, where homogeneity can narrow the kinds of thinking and disciplinary work students are invited to do. We study diversity in two educational generation tasks: math items and essay feedback. We compare frontier and open-weight models with human-authored reference distributions using lexical, semantic, and domain-grounded measures. For math, we examine the mathematical activities students are asked to perform; for feedback, we examine the revision work requested and the rhetorical strategies used to communicate it. We evaluate direct prompting, two diversity prompting methods, and post-training. Across both tasks, directly prompted model outputs are markedly less diverse than human-authored content---78% higher on lexical (Self-BLEU) and 51% higher on semantic (embedding) similarity---and represent a narrower range of instructional forms. Diversity prompting and post-training increase variation largely through surface features without broadening the instructional repertoire. Models inherit and amplify the most normative routines of human instruction. Overcoming mode collapse in education will require richer instructional data and investment in teachers’ creative capacity and professional judgment.