语言大模型 / LLM

To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models

Jessica Y. Bo, Sophia Wan, Ashton Anderson

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Journal ref Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems

Automated Program Repair Based on REST API Specifications Using Large Language Models

Katsuki Yamagishi, Norihiro Yoshida, Erina Makihara, Katsuro Inoue

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

LaMP-Val: Large Language Models Empower Personalized Valuation in Auction

Jie Sun, Tianyu Zhang, Houcheng Jiang, Kexin Huang, Xiang Shu, Zhibo Zhu, Lintao Ma, Xingyu Lu, Jun Zhou, Junkang Wu, Chi Luo, An Zhang, Junkang Wu, Jiancan Wu, Xiang Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 17 pages, 5 figures, 8tables

SBAN: A Framework & Multi-Dimensional Dataset for Large Language Model Pre-Training and Software Code Mining

Hamed Jelodar, Mohammad Meymani, Samita Bai, Roozbeh Razavi-Far, Ali A. Ghorbani

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization

Zhen-Song Chen, Hong-Wei Ding, Xian-Jia Wang, Witold Pedrycz

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments This work has been submitted to the IEEE for possible publication

Do Large Language Models Know How Much They Know?

Gabriele Prato, Jerry Huang, Prasanna Parthasarathi, Shagun Sodhani, Sarath Chandar

机构 * Chandar Research Lab（昌达尔研究实验室） ； Mila – Quebec AI Institute（魁北克人工智能研究所） ； Université de Montréal（蒙特利尔大学） ； Meta FAIR ； Polytechnique Montréal（蒙特利尔理工学院） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a long paper at the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). Official version of paper within conference proceedings is available at https://aclanthology.org/2024.emnlp-main.348/

MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, Ran He

机构 * State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室） ； School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院） ； Tencent Youtu Lab（腾讯优图实验室） ； Xiamen University（厦门大学） ； CASIA（中国科学院自动化研究所） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments NeurIPS DB 2025 Spotlight, Project Page: https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Evaluation

Large Language Models in Thematic Analysis: Prompt Engineering, Evaluation, and Guidelines for Qualitative Software Engineering Research

Cristina Martinez Montes, Robert Feldt, Cristina Miguel Martos, Sofia Ouhbi, Shweta Premanandan, Daniel Graziotin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models

Xianzheng Ma, Brandon Smart, Yash Bhalgat, Shuai Chen, Xinghui Li, Jian Ding, Jindong Gu, Dave Zhenyu Chen, Songyou Peng, Jia-Wang Bian, Philip H Torr, Marc Pollefeys, Matthias Nießner, Ian D Reid, Angel X. Chang, Iro Laina, Victor Adrian Prisacariu

机构 * University of Oxford（牛津大学） ； King Abdullah University of Science and Technology（国王 Abdullah 科学与技术大学） ； Technical University of Munich（慕尼黑技术大学） ； Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） ； Simon Fraser University（西蒙·弗雷泽大学） ； ETH Zurich（苏黎世联邦理工学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 2nd version update to Jun.2025

Measuring how changes in code readability attributes affect code quality evaluation by Large Language Models

Igor Regis da Silva Simoes, Elaine Venson

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Journal ref 2025: Proceedings of the XXXIX Brazilian Symposium on Software Engineering

DRO-InstructZero: Distributionally Robust Prompt Optimization for Large Language Models

Yangyang Li

机构 * Department of Electrical Engineering and Computer Science（电气工程与计算机科学系） ； Massachusetts Institute of Technology（麻省理工学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Preprint. Under review at ICLR 2026. 11 pages, 2 figures

MindBenchAI: An Actionable Platform to Evaluate the Profile and Performance of Large Language Models in a Mental Healthcare Context

Bridget Dwyer, Matthew Flathers, Akane Sano, Allison Dempsey, Andrea Cipriani, Asim H. Gazi, Carla Gorban, Carolyn I. Rodriguez, Charles Stromeyer, Darlene King, Eden Rozenblit, Gillian Strudwick, Jake Linardon, Jiaee Cheong, Joseph Firth, Julian Herpertz, Julian Schwarz, Margaret Emerson, Martin P. Paulus, Michelle Patriquin, Yining Hua, Soumya Choudhary, Steven Siddals, Laura Ospina Pinillos, Jason Bantjes, Steven Scheuller, Xuhai Xu, Ken Duckworth, Daniel H. Gillison, Michael Wood, John Torous

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Cancer Diagnosis Categorization in Electronic Health Records Using Large Language Models and BioBERT: Model Performance Evaluation Study

Soheil Hashtarkhani, Rezaur Rashid, Christopher L Brett, Lokesh Chinthala, Fekede Asefa Kumsa, Janet A Zink, Robert L Davis, David L Schwartz, Arash Shaban-Nejad

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 8 Pages

Journal ref JMIR Cancer, 2025 Oct 2:11:e72005

An Empirical Study of Python Library Migration Using Large Language Models

Md Mohayeminul Islam, Ajay Kumar Jha, May Mahmoud, Ildar Akhmetov, Sarah Nadi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis

Zeeshan Rasheed, Muhammad Waseem, Aakash Ahmad, Kai-Kristian Kemell, Wang Xiaofeng, Anh Nguyen Duc, Pekka Abrahamsson

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 34 pages and 2 figures

Survey of Large Language Models in Extended Reality: Technical Paradigms and Application Frontiers

Jingyan Wang, Yang Zhao, Haotian Mao, Xubo Yang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments Withdrawn by the authors after identifying improper reuse of illustrative materials and partial omission of citations in certain sections. A revised version will be prepared to ensure accurate attribution, and comprehensive citation coverage

Diagnosing Shoulder Disorders Using Multimodal Large Language Models and Consumer-Grade Cameras

Jindong Hong, Wencheng Zhang, Shiqin Qiao, Jianhai Chen, Jianing Qiu, Chuanyang Zheng, Qian Xu, Yun Ji, Qianyue Wen, Weiwei Sun, Hao Li, Huizhen Li, Huichao Wang, Kai Wu, Meng Li, Yijun He, Lingjie Luo, Jiankai Sun

机构 * Bytedance（字节跳动） ； Peking University（北京大学） ； Peking University People’s Hospital（北京大学人民医院） ； Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Reducing Large Language Model Safety Risks in Women's Health using Semantic Entropy

Jahan C. Penny-Dimri, Magdalena Bachmann, William R. Cooke, Sam Mathewlynn, Samuel Dockree, John Tolladay, Jannik Kossen, Lin Li, Yarin Gal, Gabriel Davis Jones

机构 * Oxford Digital Health Labs, Nuffield Department of Women’s and Reproductive Health（牛津数字健康实验室，女性与生殖健康系） ； University of Oxford（牛津大学） ； Oxford University Hospitals NHS Foundation Trust（牛津大学医院 NHS 基础信托） ； OATML, Department of Computer Science（OATML，计算机科学系） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 15 pages, 6 tables

The New Quant: A Survey of Large Language Models in Financial Prediction and Trading

Weilong Fu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)

Comments 21 pages

Test Case Generation from Bug Reports via Large Language Models: A Cognitive Layered Evaluation Framework

Irtaza Sajid Qureshi, Zhen Ming, Jiang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

MapIQ: Evaluating Multimodal Large Language Models for Map Question Answering

Varun Srivastava, Fan Lei, Srija Mukhopadhyay, Vivek Gupta, Ross Maciejewski

机构 * School of Computing and Augmented Intelligence（计算与增强智能学院） ； Arizona State University（亚利桑那州立大学） ； Department of Computer Science（计算机科学系） ； International Institute of Information Technology（国际信息科技研究所） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a conference paper at COLM 2025

A Multi-Language Object-Oriented Programming Benchmark for Large Language Models

Shuai Wang, Liang Ding, Li Shen, Yong Luo, Han Hu, Lefei Zhang, Fu Lin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)

Comments 20 pages, 12 figures

HAMMER: Hamiltonian Curiosity Augmented Large Language Model Reinforcement

Ming Yang, Xiaofan Li, Zhiyuan Ma, Dengliang Shi, Jintao Du, Yu Cheng, Weiguo Zheng

机构 * Fudan University（复旦大学） ； Tiansuan Lab, Ant Group Co., Ltd.（蚂蚁集团天算实验室） ； East China Normal University（华东师范大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 20 pages, 7 figures, 4 tables

Finetuning Large Language Model as an Effective Symbolic Regressor

Yingfan Hua, Ruikun Li, Jun Yao, Guohang Zhuang, Shixiang Tang, Bin Liu, Wanli Ouyang, Yan Lu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Binary Diff Summarization using Large Language Models

Meet Udeshi, Venkata Sai Charan Putrevu, Prashanth Krishnamurthy, Prashant Anantharaman, Sean Carrick, Ramesh Karri, Farshad Khorrami

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

DivLogicEval: A Framework for Benchmarking Logical Reasoning Evaluation in Large Language Models

Tsz Ting Chung, Lemao Liu, Mo Yu, Dit-Yan Yeung

机构 * The Hong Kong University of Science and Technology（香港科技大学） ； Fudan University（复旦大学） ； WeChat AI, Tencent（腾讯微信AI） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by EMNLP 2025. Project Page: https://ttchungc.github.io/projects/divlogiceval/

Safety and Security Analysis of Large Language Models: Benchmarking Risk Profile and Harm Potential

Charankumar Akiri, Harrison Simpson, Kshitiz Aryal, Aarav Khanna, Maanak Gupta

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

SPELUNKER: Item Similarity Search Using Large Language Models and Custom K-Nearest Neighbors

Ana Rodrigues, João Mata, Rui Rego

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 6 pages, 4 figures

EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysis

Shengyuan Liu, Boyun Zheng, Wenting Chen, Zhihao Peng, Zhenfei Yin, Jing Shao, Jiancong Hu, Yixuan Yuan

机构 * Chinese University of Hong Kong（中国香港大学） ； City University of Hong Kong（香港城市大学） ； University of Oxford（牛津大学） ； Shanghai AI Laboratory（上海人工智能实验室） ； The Sixth Affiliated Hospital, Sun Yat-sen University（中山大学第六附属医院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)

Comments 40 pages, 22 figures; Accepted by NeurIPS 2025 Dataset and Benchmark Track

V-GameGym: Visual Game Generation for Code Large Language Models

Wei Zhang, Jack Yang, Renshuai Tao, Lingzheng Chai, Shawn Guo, Jiajun Wu, Xiaoming Chen, Ganqu Cui, Ning Ding, Xander Xu, Hu Wei, Bowen Zhou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)