Cultural Benchmarking of LLM s in Standard and Dialectal A rabic Dialogues
Muhammad Dehan Al Kautsar · Saeed Almheiri · Momina Ahsan · Bilal Elbouardi · Younes Samih · Sarfraz Ahmad · Amr Keleg · Omar El Herraoui · Kareem Elzeky · Abed Alhakim Freihat · Mohamed Anwar · Zhuohan Xie · Junhong Liang · Mohammad Rustom Al Nasar · Preslav Nakov · Fajri Koto
摘要
There is a significant gap in evaluating cultural reasoning in LLMs using conversational datasets that capture culturally rich and dialectal contexts. Most Arabic benchmarks focus on short text snippets in Modern Standard Arabic (MSA), overlooking the cultural nuances that naturally arise in dial...