语言大模型 / LLM

ELMES: An Automated Framework for Evaluating Large Language Models in Educational Scenarios

Shou'ang Wei, Xinyun Wang, Shuzhen Bi, Jian Chen, Ruijia Li, Bo Jiang, Xin Lin, Min Zhang, Yu Song, BingDong Li, Aimin Zhou, Hao Hao

机构 * Shanghai Innavation Institute（上海创新研究院） ； Shanghai Institute of AI for Education（上海人工智能教育研究院） ； Department of Educational Information Technology（教育信息技术系） ； School of Computer Science and Technology（计算机科学与技术学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Prompt Engineering Techniques for Mitigating Cultural Bias Against Arabs and Muslims in Large Language Models: A Systematic Review

Bushra Asseri, Estabrag Abdelaziz, Areej Al-Wabil

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments Research is incomplete

BALSAM: A Platform for Benchmarking Arabic Large Language Models

Rawan Al-Matham, Kareem Darwish, Raghad Al-Rasheed, Waad Alshammari, Muneera Alhoshan, Amal Almazrua, Asma Al Wazrah, Mais Alheraki, Firoj Alam, Preslav Nakov, Norah Alzahrani, Eman alBilali, Nizar Habash, Abdelrahman El-Sheikh, Muhammad Elmallah, Haonan Li, Hamdy Mubarak, Mohamed Anwar, Zaid Alyafeai, Ahmed Abdelali, Nora Altwairesh, Maram Hasanain, Abdulmohsen Al Thubaity, Shady Shehata, Bashar Alhafni, Injy Hamed, Go Inoue, Khalid Elmadani, Ossama Obeid, Fatima Haouari, Tamer Elsayed, Emad Alghamdi, Khalid Almubarak, Saied Alshahrani, Ola Aljarrah, Safa Alajlan, Areej Alshaqarawi, Maryam Alshihri, Sultana Alghurabi, Atikah Alzeghayer, Afrah Altamimi, Abdullah Alfaifi, Abdulrahman AlOsaimy

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Large Language Model Agent for Structural Drawing Generation Using ReAct Prompt Engineering and Retrieval Augmented Generation

Xin Zhang, Lissette Iturburu, Juan Nicolas Villamizar, Xiaoyu Liu, Manuel Salmeron, Shirley J. Dyke, Julio Ramirez

机构 * Lyles School of Civil and Construction Engineering（莱尔斯土木与建设工程学院） ； Purdue University（普渡大学） ； School of Mechanical Engineering（机械工程学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

HIVMedQA: Benchmarking large language models for HIV medical decision support

Gonzalo Cardenal-Antolin, Jacques Fellay, Bashkim Jaha, Roger Kouyos, Niko Beerenwinkel, Diane Duroux

机构 * Department of Biosystems Science and Engineering, ETH Zurich, Basel, Switzerland（1 生物系统科学与工程系，苏黎世联邦理工学院，巴塞尔，瑞士） ； School of Life Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland（2 生命科学学院，洛桑联邦理工学院，洛桑，瑞士） ； Swiss Institute of Bioinformatics, Lausanne, Switzerland（3 瑞士生物信息学研究所，洛桑，瑞士） ； Biomedical Data Science Center, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland（4 生物医学数据科学中心，洛桑大学医院和洛桑大学，洛桑，瑞士） ； Institute of Medical Virology, University of Zurich, Zurich, Switzerland（5 医学病毒学研究所，苏黎世大学，苏黎世，瑞士） ； Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, Zurich, Switzerland（6 感染病与医院流行病学系，苏黎世大学医院，苏黎世，瑞士） ； ETH AI Center, ETH Zurich, Zurich, Switzerland（7 ETH人工智能中心，苏黎世联邦理工学院，苏黎世，瑞士） ； Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland（8 量化生物医学系，苏黎世大学，苏黎世，瑞士） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models

Giuseppe Russo, Debora Nozza, Paul Röttger, Dirk Hovy

机构 * EPFL（苏黎世联邦理工学院） ； Bocconi University（博科尼大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 13 pages, 4 figures

Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language

Kristin Gnadt, David Thulke, Simone Kopeinik, Ralf Schlüter

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Accepted at the 6th Workshop on Gender Bias in Natural Language Processing (GeBNLP) at ACL 2025

A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios

Xiachong Feng, Longxu Dou, Ella Li, Qinghao Wang, Haochuan Wang, Yu Guo, Chang Ma, Lingpeng Kong

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Automated Novelty Evaluation of Academic Paper: A Collaborative Approach Integrating Human and Large Language Model Knowledge

Wenqing Wu, Chengzhi Zhang, Yi Zhao

机构 * Department of Information Management, Nanjing University of Science and Technology（南京理工大学信息管理学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Journal of the Association for Information Science and Technology, 2025

A Code Comprehension Benchmark for Large Language Models for Code

Jayant Havare, Saurav Chaudhary, Ganesh Ramakrishnan, Kaushik Maharajan, Srikanth Tamilselvam

机构 * IIT Bombay（印度理工学院博伊斯分校） ； IBM Research（IBM研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments 10 Pages, 5 Figures

Quantum-Accelerated Neural Imputation with Large Language Models (LLMs)

Hossein Jamali

机构 * Department of Computer Science and Engineering University of Nevada, Reno（计算机科学与工程系 华盛顿大学内华达分校） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testing

Peiyan Zhang, Haibo Jin, Liying Kang, Haohan Wang

机构 * University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments 24 pages

Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions

Kun Zhang, Le Wu, Kui Yu, Guangyi Lv, Dacao Zhang

机构 * Hefei University of Technology（合肥工业大学） ； AI Laboratory, Lenovo Research（联想研究院人工智能实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 33 pages, 5 figures

CodeMirage: Hallucinations in Code Generated by Large Language Models

Vibhor Agarwal, Yulong Pei, Salwa Alamir, Xiaomo Liu

机构 * University of Surrey（萨里大学） ； JP Morgan AI Research（摩根大通AI研究） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted at AutoMates @ IJCAI 2024

A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models

Shuliang Liu, Hongyi Liu, Aiwei Liu, Bingchen Duan, Qi Zheng, Yibo Yan, He Geng, Peijie Jiang, Jia Liu, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； The Hong Kong University of Science and Technology（香港科技大学） ； Harbin Institute of Technology（哈尔滨工业大学） ； Ant Group, Alibaba（蚂蚁集团，阿里巴巴） ； Northeast Forest University（东北林业大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2025 Findings

Chat2SPaT: A Large Language Model Based Tool for Automating Traffic Signal Control Plan Management

Yue Wang, Miao Zhou, Guijing Huang, Rui Zhuo, Chao Yi, Zhenliang Ma

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactions

Yubo Li, Yidi Miao, Xueying Ding, Ramayya Krishnan, Rema Padman

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 5 figures

Journal ref Published at ACL 2025

From Intentions to Techniques: A Comprehensive Taxonomy and Challenges in Text Watermarking for Large Language Models

Harsh Nishant Lalai, Aashish Anantha Ramakrishnan, Raj Sanjay Shah, Dongwon Lee

机构 * Birla Institute of Technology and Science, Pilani（比拉理工学院和科学学院，比里尼） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments NAACL Findings 2025

A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications

Wenbo Shang, Xin Huang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 23 pages including references, 14 figures

Integrating Large Language Models in Financial Investments and Market Analysis: A Survey

Sedigheh Mahdavi, Jiating, Chen, Pradeep Kumar Joshi, Lina Huertas Guativa, Upmanyu Singh

机构 * AI Research Lab, Blend360, Columbia, USA（Blend360人工智能研究实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions

Ola Shorinwa, Zhiting Mei, Justin Lidard, Allen Z. Ren, Anirudha Majumdar

机构 * Princeton University（普林斯顿大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Divergent Creativity in Humans and Large Language Models

Antoine Bellemare-Pepin, François Lespinasse, Philipp Thölke, Yann Harel, Kory Mathewson, Jay A. Olson, Yoshua Bengio, Karim Jerbi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments First two and last listed authors are corresponding authors. The first two listed authors contributed equally to this work

Benchmarking the Pedagogical Knowledge of Large Language Models

Maxime Lelièvre, Amy Waldock, Meng Liu, Natalia Valdés Aspillaga, Alasdair Mackintosh, María José Ogando Portela, Jared Lee, Paul Atherton, Robin A. A. Ince, Oliver G. B. Garrod

机构 * Fab Inc（Fab公司） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Potemkin Understanding in Large Language Models

Marina Mancoridis, Bec Weeks, Keyon Vafa, Sendhil Mullainathan

机构 * Massachusetts Institute of Technology（麻省理工学院） ； Harvard University（哈佛大学） ； University of Chicago（芝加哥大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Zero-Shot Attribution for Large Language Models: A Distribution Testing Approach

Clément L. Canonne, Yash Pote, Uddalok Sarkar

机构 * University of Sydney（悉尼大学） ； National University of Singapore（新加坡国立大学） ； Indian Statistical Institute（印度统计研究所） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments 16 pages, 4 figures

CogniBench: A Legal-inspired Framework and Dataset for Assessing Cognitive Faithfulness of Large Language Models

Xiaqiang Tang, Jian Li, Keyu Hu, Du Nan, Xiaolong Li, Xi Zhang, Weigao Sun, Sihong Xie

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； Hunyuan AI Digital Human, Tencent（文心一言数字人，腾讯） ； Beijing University of Posts and Telecommunications（北京邮电大学） ； Shanghai AI Laboratory（上海人工智能实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025

Large language models for automated scholarly paper review: A survey

Zhenzhen Zhuang, Jiandong Chen, Hongfeng Xu, Yuwen Jiang, Jialiang Lin

机构 * School of Computer Science and Engineering, Guangzhou Institute of Science and Technology（计算机科学与工程学院，科学与技术研究院） ； School of Economics and Management, Guizhou Normal University（经济学与管理学院，贵州师范大学） ； School of Artificial Intelligence, Guangzhou Institute of Science and Technology（人工智能学院，科学与技术研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Please cite the version of Information Fusion

Journal ref Information Fusion, Vol. 124, 103332 (2025)

RWESummary: A Framework and Test for Choosing Large Language Models to Summarize Real-World Evidence (RWE) Studies

Arjun Mukerji, Michael L. Jackson, Jason Jones, Neil Sanghavi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 2 figures

Supernova Event Dataset: Interpreting Large Language Models' Personality through Critical Event Analysis

Pranav Agarwal, Ioana Ciucă

机构 * Stanford University（斯坦福大学） ； Mila, Quebec AI Institute（魁北克AI研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted at Actionable Interpretability Workshop at ICML 2025

DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models

Jinxiang Xie, Yilin Li, Xunjian Yin, Xiaojun Wan

机构 * Peking University（北京大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Journal ref Proceedings of the AAAI Conference on Artificial Intelligence 39 (24), 25561-25569, 2025