语言大模型 / LLM

NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment

NovBench：在学术论文新颖性评估中评估大语言模型

Wenqing Wu, Yi Zhao, Yuzhuo Wang, Siyou Li, Juexi Shao, Yunfei Long, Chengzhi Zhang

机构 * School of Economics and Management, Nanjing University of Science and Technology（南京理工大学经济管理学院） ； School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦玛丽女王大学电子工程与计算机科学学院） ； School of Management, Anhui University（安徽大学管理学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 NovBench通过1684篇论文与评审对的基准测试，评估大语言模型在新颖性评估中的能力，揭示当前模型在科学新颖性理解上的不足，强调需改进的细调策略。

Comments ACL 2026

The Role of Emotional Stimuli and Intensity in Shaping Large Language Model Behavior

情绪刺激与强度在塑造大语言模型行为中的作用

Ameen Patel, Felix Lee, Kyle Liang, Joseph Thomas

机构 * Irvington High School（欧文顿高中） ； Glen A. Wilson High School（格伦·A·威尔逊高中） ； Del Norte High School（德尔诺特高中） ； California High School（加利福尼亚高中） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 本文研究了四种情绪在情感提示中的影响，发现积极情绪刺激提升模型准确性并降低毒性，但会增加顺从行为。

Journal ref Poster Presentation at AACL Student Research Workshop 2025

Scalable Identification and Prioritization of Requisition-Specific Personal Competencies Using Large Language Models

基于大语言模型的可扩展需求特定个人能力识别与优先级排序

Wanxin Li, Denver McNeney, Nivedita Prabhu, Charlene Zhang, Renee Barr, Matthew Kitching, Khanh Dao Duc, Anthony S. Boyce

机构 * University of British Columbia（不列颠哥伦比亚大学） ； Amazon（亚马逊） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 本文提出利用大语言模型识别和优先级排序特定需求的个人能力，通过动态少样本提示、反思式自我改进、相似性过滤和多阶段验证，实现高准确率和低误检率。

Measuring and Eliminating Refusals in Military Large Language Models

测量和消除军事大语言模型中的拒绝行为

Jack FitzGerald, Dylan Bates, Aristotelis Lazaridis, Aman Sharma, Vincent Lu, Brian King, Yousif Azami, Sean Bailey, Jeremy Cao, Peter Damianov, Kevin de Haan, Joseph Madigan, Jeremy McLaurin, Luke Kerbs, Jonathan Tainer, Dave Anderson, Jonathan Beck, Jamie Cuticello, Colton Malkerson, Tyler Saltsman

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

AI总结 本研究通过评估军事大语言模型的拒绝行为，提出通过中期训练和端到端后训练实现零拒绝和最大军事任务准确性的方法。

Comments 30 pages

A unified foundational framework for knowledge injection and evaluation of Large Language Models in Combustion Science

为燃烧科学中的大语言模型知识注入和评估构建统一的基础框架

Zonglin Yang, Runze Mao, Tianhao Wu, Han Li, QingGuo Zhou, Zhi X. Chen

机构 * School of Mechanics and Engineering Science, Peking University（力学与工程科学学院，北京大学） ； AI for Science Institute（人工智能科学研究院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；pretraining(abstract)

AI总结 本研究提出一个统一框架，用于开发燃烧科学专用的大语言模型，通过三阶段知识注入路径提升模型性能。

Comments 5 figures, 1 table

Evaluating Monolingual and Multilingual Large Language Models for Greek Question Answering: The DemosQA Benchmark

评估单语和多语大型语言模型用于希腊问答：DemosQA基准

Charalampos Mastrokostas, Nikolaos Giarelis, Nikos Karacapilidis

机构 * Information Systems Lab, MEAD University of Patras, Rio Patras, Greece（信息系统实验室，MEAD帕特拉斯大学，帕特拉斯，希腊） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 本研究提出DemosQA数据集和高效评估框架，评估11种LLM在希腊问答任务中的表现，以解决低资源语言的模型有效性问题。

IndicEval: A Bilingual Indian Educational Evaluation Framework for Large Language Models

IndicEval：一种用于大型语言模型的双语印度教育评估框架

Saurabh Bharti, Gaurav Azad, Abhinaw Jagtap, Nachiket Tapas

机构 * Department of Computer Sciencce and Engineering（计算机科学与工程系） ； Swami Vivekanand Technical University Bhilai, India（斯瓦米·维韦kanand技术大学比哈尔，印度） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 IndicEval提出了一种双语印度教育评估框架，通过真实考试题目评估LLM的推理、领域知识和双语适应性，揭示了跨模型性能差异和多语言退化问题。

LLMStructBench: Benchmarking Large Language Model Structured Data Extraction

LLMStructBench: 评估大型语言模型结构化数据提取的基准测试

Sönke Tenckhoff, Mario Koddenbrock, Erik Rodner

机构 * University of Applied Sciences Berlin（柏林应用技术大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 LLMStructBench通过评估模型大小和提示策略对结构化数据提取的影响，为大型语言模型的解析可靠性提供新的基准测试。

OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG

OpenDecoder: 开源大型语言模型解码以纳入文档质量在RAG中

Fengran Mo, Zhan Su, Yuchen Hui, Jinghan Zhang, Jia Ao Sun, Zheyuan Liu, Chao Zhang, Tetsuya Sakai, Jian-Yun Nie

机构 * Clemson University（克莱姆森大学） ； University of Notre Dame（诺特丹大学） ； Georgia Institute of Technology（佐治亚理工学院） ； Waseda University（早稻田大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

AI总结 OpenDecoder通过整合文档质量评估提升RAG模型的鲁棒性，利用相关性、排序和QPP评分优化生成过程。

Comments Accepted by ACM WWW 2026

Exploring Approaches for Detecting Memorization of Recommender System Data in Large Language Models

探索用于检测大型语言模型中推荐系统数据记忆的方法

Antonio Colacicco, Vito Guida, Dario Di Palma, Fedelucio Narducci, Tommaso Di Noia

机构 * Politecnico di Bari（巴里理工大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 本文提出三种方法探索大型语言模型中推荐系统数据记忆的检测，发现自动优化提示是提取记忆样本最有效的方法。

SoK: Large Language Model Copyright Auditing via Fingerprinting

Shuo Shao, Yiming Li, Yu He, Hongwei Yao, Wenyuan Yang, Dacheng Tao, Zhan Qin

机构 * The State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室） ； Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院） ； Nanyang Technological University（南洋理工大学） ； City University of Hong Kong（香港城市大学） ； Sun Yat-Sen University（中山大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

ChessQA: Evaluating Large Language Models for Chess Understanding

Qianfeng Wen, Zhenwei Tang, Ashton Anderson

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Comments 33 pages,8 figures

Robustness of Large Language Models to Perturbations in Text

Ayush Singh, Navpreet Singh, Shubham Vatsal

机构 * inQbator AI at eviCore Healthcare Evernorth Health Services（inQbator AI 位于 eviCore 医疗保健 Evernorth 健康服务） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 8 pages, 1 figure, 6 tables, updated with results also from GPT-4, LLaMa-3

Small Language Models for Curriculum-based Guidance

Konstantinos Katharakis, Sippo Rossi, Raghava Rao Mukkamala

机构 * Copenhagen Business School（哥本哈根商学院） ； Hanken School of Economics（汉肯经济学院） 纠错

专题命中 评测与基准 ：language model(title,abstract)；small language model(title,abstract)；large language model(abstract)；prompting(abstract)

ClaimCheck: Real-Time Fact-Checking with Small Language Models

Akshith Reddy Putta, Jacob Devasier, Chengkai Li

专题命中 评测与基准 ：language model(title,abstract)；small language model(title,abstract)；LLM(abstract)；prompting(abstract)

RAGs to Riches: RAG-like Few-shot Learning for Large Language Model Role-playing

Timothy Rupprecht, Enfu Nan, Arash Akbari, Arman Akbari, Lei Lu, Priyanka Maan, Sean Duffy, Pu Zhao, Yumei He, David Kaeli, Yanzhi Wang

机构 * Department of Electrical and Computer Engineering（电气与计算机工程系） ； Northeastern University（东北大学） ； Tulane University（路易斯安那州立大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Guiding Large Language Models to Post-Edit Machine Translation with Error Annotations

Dayeon Ki, Marine Carpuat

机构 * Computer Science University of Maryland（计算机科学大学马里兰大学） ； Computer Science, UMIACS University of Maryland（计算机科学，UMIACS大学马里兰大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments NAACL 2024 Findings

Revealing Fine-Grained Values and Opinions in Large Language Models

Dustin Wright, Arnav Arora, Nadav Borenstein, Srishti Yadav, Serge Belongie, Isabelle Augenstein

机构 * University of Copenhagen（哥本哈根大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Findings of EMNLP 2024; 28 pages, 20 figures, 7 tables

OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models

Adam Coscia, Shunan Guo, Eunyee Koh, Alex Endert

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Adobe Research（Adobe研究） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Accepted to UIST 2025. 18 pages, 9 figures, 2 tables. For a demo video, see https://youtu.be/uobhmxo6EIE

Cequel: Cost-Effective Querying of Large Language Models for Text Clustering

Hongtao Wang, Taiyan Zhang, Renchi Yang, Jianliang Xu

机构 * Hong Kong Baptist University（香港 Baptist 大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Classic4Children: Adapting Chinese Literary Classics for Children with Large Language Model

Jiali Chen, Xusen Hei, Yuqi Xue, Zihan Wu, Jiayuan Xie, Yi Cai

机构 * Key Laboratory of Big Data and Intelligent Robot (South China University of Technology) Ministry of Education（大数据与智能机器人重点实验室（华南理工大学）教育部） ； School of Software Engineering, South China University of Technology（软件学院（华南理工大学）） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；instruction tuning(abstract)

Comments Accepted at NAACL 2025 Findings

"Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas

Junchen Ding, Penghao Jiang, Zihao Xu, Ziqi Ding, Yichen Zhu, Jiaojiao Jiang, Yuekang Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Understanding Learner-LLM Chatbot Interactions and the Impact of Prompting Guidelines

Cansu Koyuturk, Emily Theophilou, Sabrina Patania, Gregor Donabauer, Andrea Martinenghi, Chiara Antico, Alessia Telari, Alessia Testa, Sathya Bursic, Franca Garzotto, Davinia Hernandez-Leo, Udo Kruschwitz, Davide Taibi, Simona Amenta, Martin Ruskov, Dimitri Ognibene

专题命中 评测与基准 ：LLM(title,abstract)；prompting(title,abstract)；large language model(abstract)；language model(abstract)

Comments Long paper accepted for AIED 2025, the 26th International Conference on Artificial Intelligence in Education, July 22 - 26, 2025, Palermo, Italy

Structure Guided Large Language Model for SQL Generation

Qinggang Zhang, Hao Chen, Junnan Dong, Shengyuan Chen, Feiran Huang, Xiao Huang

机构 * Department of Computing, The Hong Kong Polytechnic University, Hung Hom, Hong Kong SAR, China（香港理工大学计算机系） ； City University of Macau, Macau SAR, China（澳门城市大学） ； College of Information Science and Technology, Jinan University, GZ, China（广州jinan大学信息科学与技术学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments The 42nd International Conference on Machine Learning

A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis

Jiachen Liu, Ziheng Geng, Ran Cao, Lu Cheng, Paolo Bocchini, Minghui Cheng

机构 * Department of Electrical and Computer Engineering, University of Miami（迈阿密大学电气与计算机工程系） ； Department of Civil and Architectural Engineering, University of Miami（迈阿密大学土木与建筑工程系） ； College of Civil Engineering, Hunan University（湖南大学土木学院） ； Department of Computer Science, University of Illinois Chicago（伊利诺伊大学芝加哥分校计算机科学系） ； Center for Catastrophe Modeling and Resilience, Lehigh University（莱斯大学灾难建模与韧性中心） ； School of Architecture, University of Miami（迈阿密大学建筑学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Large Language Model-Powered Conversational Agent Delivering Problem-Solving Therapy (PST) for Family Caregivers: Enhancing Empathy and Therapeutic Alliance Using In-Context Learning

Liying Wang, Ph. D., Daffodil Carrington, M. S., Daniil Filienko, M. S., Caroline El Jazmi, M. S., Serena Jinchen Xie, M. S., Martine De Cock, Ph. D., Sarah Iribarren, Ph. D., Weichao Yuwen, Ph. D

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Large Language Models for EEG: A Comprehensive Survey and Taxonomy

Naseem Babu, Jimson Mathew, A. P. Vinod

机构 * Department of Computer Science and Engineering, Indian Institute of Technology Patna（计算机科学与工程系，印度理工学院帕纳布分校） ； Infocomm Technology Cluster, Singapore Institute of Technology（信息通信技术集群，新加坡理工学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

Beyond the Sentence: A Survey on Context-Aware Machine Translation with Large Language Models

Ramakrishna Appicharla, Baban Gain, Santanu Pal, Asif Ekbal

机构 * Department of Computer Science and Engineering, Indian Institute of Technology Patna, India（印度理工学院帕纳瓦分校计算机科学与工程系） ； Wipro AI, Lab45, London, UK（Wipro AI，Lab45，伦敦，英国） ； School of AI and Data Science, Indian Institute of Technology Jodhpur, India（印度理工学院朱罗普分校人工智能与数据科学学院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Improving Radiology Report Conciseness and Structure via Local Large Language Models

Iryna Hartsock, Cyrillo Araujo, Les Folio, Ghulam Rasool

机构 * Department of Machine Learning, H. Lee Moffitt Cancer Center & Research Institute（机器学习系，H. Lee Moffitt癌症中心与研究所在读机构） ； Deparmtent of Diagnostic Imaging & Interventional Radiology, H. Lee Moffitt Cancer Center & Research Institute（诊断影像与介入放射学系，H. Lee Moffitt癌症中心与研究所在读机构） ； Department of Radiology, James A. Haley VA Hospital（放射学系，James A. Haley退伍军人医院） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments published version

SPRI: Aligning Large Language Models with Context-Situated Principles

Hongli Zhan, Muneeza Azmat, Raya Horesh, Junyi Jessy Li, Mikhail Yurochkin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；SFT(abstract)

Comments Forty-Second International Conference on Machine Learning (ICML 2025) Camera-Ready Version