语言大模型 / LLM

Large Language Models as Neurolinguistic Subjects: Discrepancy between Performance and Competence

Linyang He, Ercong Nie, Helmut Schmid, Hinrich Schütze, Nima Mesgarani, Jonathan Brennan

机构 * Columbia University（哥伦比亚大学） ； University of Michigan（密歇根大学） ； Munich Center for Machine Learning（慕尼黑机器学习中心） ； LMU Munich, Germany（慕尼黑大学, 德国） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；instruction tuning(abstract)

Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap

Tong Nie, Jian Sun, Wei Ma

机构 * Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China（香港理工大学土木与环境工程系） ； Department of Traffic Engineering, Tongji University, Shanghai, 201804, China（同济大学交通工程系） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments Published at Artificial Intelligence for Transportation, Inaugural Issue

Journal ref Artificial Intelligence for Transportation, Volume 1, July 2025, 100003

LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research

Shuo Yan, Ruochen Li, Ziming Luo, Zimu Wang, Daoyang Li, Liqiang Jing, Kaiyu He, Peilin Wu, George Michalopoulos, Yue Zhang, Ziyang Zhang, Mian Zhang, Zhiyu Chen, Xinya Du

机构 * University of Texas at Dallas（德克萨斯大学达拉斯分校） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；language model(title,abstract)；large language model(abstract)；prompting(abstract)

Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognition

Yuan Tseng, Titouan Parcollet, Rogier van Dalen, Shucong Zhang, Sourav Bhattacharya

机构 * Samsung AI Center Cambridge（三星AI研究中心剑桥） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

HPSS: Heuristic Prompting Strategy Search for LLM Evaluators

Bosi Wen, Pei Ke, Yufei Sun, Cunxiang Wang, Xiaotao Gu, Jinfeng Zhou, Jie Tang, Hongning Wang, Minlie Huang

机构 * The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University（清华大学对话人工智能（CoAI）小组） ； University of Electronic Science and Technology of China（电子科技大学） ； Zhipu AI（智谱AI） ； The Knowledge Engineering Group (KEG), Tsinghua University（清华大学知识工程小组（KEG）） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；prompting(title,abstract)；large language model(abstract)；language model(abstract)

Comments ACL 2025 Findings

Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning

Rongzhe Wei, Mufei Li, Mohsen Ghassemi, Eleonora Kreačić, Yifan Li, Xiang Yue, Bo Li, Vamsi K. Potluru, Pan Li, Eli Chien

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Personality-Guided Code Generation Using Large Language Models

Yaoqi Guo, Zhenpeng Chen, Jie M. Zhang, Yang Liu, Yun Ma

机构 * Peking University（北京大学） ； Nanyang Technological University（南洋理工大学） ； King’s College London（伦敦大学国王学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Accepted by the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025) Main Track

Characterizing Bias: Benchmarking Large Language Models in Simplified versus Traditional Chinese

Hanjia Lyu, Jiebo Luo, Jian Kang, Allison Koenecke

机构 * University of Rochester（罗切斯特大学） ； Cornell University（康奈尔大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments To appear in the 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT '25)

MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models

Boyang Xue, Hongru Wang, Rui Wang, Sheng Wang, Zezhong Wang, Yiming Du, Bin Liang, Wenxuan Zhang, Kam-Fai Wong

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Accepted in ACL2025 Findings

Can Large Language Models Design Biological Weapons? Evaluating Moremi Bio

Gertrude Hattoh, Jeremiah Ayensu, Nyarko Prince Ofori, Solomon Eshun, Darlington Akogo

机构 * Gertrude Hattoh（第一作者） ； Jeremiah Ayensu（第一作者） ； Nyarko Prince Ofori（第一作者） ； Solomon Eshun（第一作者） ； Darlington Akogo（第一作者） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

FMDLlama: Financial Misinformation Detection based on Large Language Models

Zhiwei Liu, Xin Zhang, Kailai Yang, Qianqian Xie, Jimin Huang, Sophia Ananiadou

机构 * The University of Manchester（曼彻斯特大学） ； The Fin AI Singapore（Fin AI新加坡） ； Archimedes RC Athena（阿基米德RC雅典） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；instruction tuning(abstract)

Comments Accepted by The Web Conference (WWW) 2025 Short Paper Track

Exploring the Impact of Personality Traits on Conversational Recommender Systems: A Simulation with Large Language Models

Xiaoyan Zhao, Yang Deng, Wenjie Wang, Hongzhan lin, Hong Cheng, Rui Zhang, See-Kiong Ng, Tat-Seng Chua

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Redefining Machine Translation on Social Network Services with Large Language Models

Hongcheng Guo, Fei Zhao, Shaosheng Cao, Xinze Lyu, Ziyan Liu, Yue Wang, Boyang Wang, Zhoujun Li, Chonggang Lu, Zhe Xu, Yao Hu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

Should AI Optimize Your Code? A Comparative Study of Classical Optimizing Compilers Versus Current Large Language Models

Miguel Romero Rosas, Miguel Torres Sanchez, Rudolf Eigenmann

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 12 pages, 7 figures, Accepted at SupercomputingAsia 2025 (SCA'25), March 10 to 13, 2025, Singapore, Singapore

Code Evolution Graphs: Understanding Large Language Model Driven Design of Algorithms

Niki van Stein, Anna V. Kononova, Lars Kotthoff, Thomas Bäck

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Accepted at GECCO 2025

A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?

QiHong Chen, Jiachen Yu, Jiawei Li, Jiecheng Deng, Justin Tian Jin Chen, Iftekhar Ahmed

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Automated Non-Functional Requirements Generation in Software Engineering with Large Language Models: A Comparative Study

Jomar Thomas Almonte, Santhosh Anitha Boominathan, Nathalia Nascimento

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 11 pages

Ontology Generation using Large Language Models

Anna Sofia Lippolis, Mohammad Javad Saeedizade, Robin Keskisärkkä, Sara Zuppiroli, Miguel Ceriani, Aldo Gangemi, Eva Blomqvist, Andrea Giovanni Nuzzolese

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 2 figures and 3 tables. 20 pages

Use Me Wisely: AI-Driven Assessment for LLM Prompting Skills Development

Dimitri Ognibene, Gregor Donabauer, Emily Theophilou, Cansu Koyuturk, Mona Yavari, Sathya Bursic, Alessia Telari, Alessia Testa, Raffaele Boiano, Davide Taibi, Davinia Hernandez-Leo, Udo Kruschwitz, Martin Ruskov

专题命中 评测与基准 ：LLM(title,abstract)；prompting(title,abstract)；large language model(abstract)；language model(abstract)

Comments Preprint accepted for Publication in Educational Technology & Society (ET&S)

Laurel: Unblocking Automated Verification with Large Language Models

Eric Mugnier, Emmanuel Anaya Gonzalez, Ranjit Jhala, Nadia Polikarpova, Yuanyuan Zhou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 34 pages, accepted at OOPSLA 25

Data Quality Control in Federated Instruction-tuning of Large Language Models

Yaxin Du, Rui Ye, Fengting Yuchi, Wanru Zhao, Jingjing Qu, Yanfeng Wang, Siheng Chen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；instruction tuning(abstract)

Comparing Large Language Model AI and Human-Generated Coaching Messages for Behavioral Weight Loss

Zhuoran Huang, Michael P. Berry, Christina Chwyl, Gary Hsieh, Jing Wei, Evan M. Forman

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 12 pages, 5 figures

Journal ref Journal of Technology in Behavioral Science (2025)

Large Language Models for Persian $ \leftrightarrow $ English Idiom Translation

Sara Rezaeimanesh, Faezeh Hosseini, Yadollah Yaghoobzadeh

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

ASTRAL: Automated Safety Testing of Large Language Models

Miriam Ugarte, Pablo Valle, José Antonio Parejo, Sergio Segura, Aitor Arrieta

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Journal ref The 6th ACM/IEEE International Conference on Automation of Software Test (AST 2025)

Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraints

Jonathan Nöther, Adish Singla, Goran Radanović

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments This is an extended version of a paper published at AAAI 25

LitSumm: Large language models for literature summarisation of non-coding RNAs

Andrew Green, Carlos Ribas, Nancy Ontiveros-Palacios, Sam Griffiths-Jones, Anton I. Petrov, Alex Bateman, Blake Sweeney

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

End-to-End Bangla AI for Solving Math Olympiad Problem Benchmark: Leveraging Large Language Model Using Integrated Approach

H. M. Shadman Tabib, Jaber Ahmed Deedar

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Journal ref IJNLC,vol:13, Issue:5/6, page 49-59,2024

Large Language Models for Mental Health Diagnostic Assessments: Exploring The Potential of Large Language Models for Assisting with Mental Health Diagnostic Assessments -- The Depression and Anxiety Case

Kaushik Roy, Harshul Surana, Darssan Eswaramoorthi, Yuxin Zi, Vedant Palit, Ritvik Garimella, Amit Sheth

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation

Minzhi Li, Zhengyuan Liu, Shumin Deng, Shafiq Joty, Nancy F. Chen, Min-Yen Kan

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments COLING2025

Automated Multi-Label Annotation for Mental Health Illnesses Using Large Language Models

Abdelrahaman A. Hassan, Radwa J. Hanafy, Mohammed E. Fouda

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)