语言大模型 / LLM

Exploring Human-Like Translation Strategy with Large Language Models

Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, Xing Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments To be published in TACL (pre-MIT Press publication version)

FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation

Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, Thang Luong

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Preprint, 26 pages, 10 figures, 5 tables; Added FreshEval

A Survey on Multimodal Large Language Models for Autonomous Driving

Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, Yang Zhou, Kaizhao Liang, Jintai Chen, Juanwu Lu, Zichong Yang, Kuei-Da Liao, Tianren Gao, Erlong Li, Kun Tang, Zhipeng Cao, Tong Zhou, Ao Liu, Xinrui Yan, Shuqi Mei, Jianguo Cao, Ziran Wang, Chao Zheng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

Which is better? Exploring Prompting Strategy For LLM-based Metrics

Joonghoon Kim, Saeran Park, Kiyoon Jeong, Sangmin Lee, Seung Hun Han, Jiyoon Lee, Pilsung Kang

专题命中 评测与基准 ：LLM(title,abstract)；prompting(title,abstract)；large language model(abstract)；language model(abstract)

Comments Eval4NLP 2023 shared task winner on both Small and Large model Track for Summarization

OptiMUS: Optimization Modeling Using MIP Solvers and large language models

Ali AhmadiTeshnizi, Wenzhi Gao, Madeleine Udell

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Query Rewriting for Retrieval-Augmented Large Language Models

Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, Nan Duan

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；small language model(abstract)

Comments EMNLP2023

Do Large Language Models Know about Facts?

Xuming Hu, Junzhe Chen, Xiaochuan Li, Yufei Guo, Lijie Wen, Philip S. Yu, Zhijiang Guo

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；pretraining(abstract)

Comments 20 pages, 8 figures

Automatic Evaluation of Attribution by Large Language Models

Xiang Yue, Boshi Wang, Ziru Chen, Kai Zhang, Yu Su, Huan Sun

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments EMNLP 2023 Findings

Reranking for Natural Language Generation from Logical Forms: A Study based on Large Language Models

Levon Haroutunian, Zhuang Li, Lucian Galescu, Philip Cohen, Raj Tumuluri, Gholamreza Haffari

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments IJCNLP-AACL 2023

Large Language Models are Strong Zero-Shot Retriever

Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Tianyi Zhou, Daxin Jiang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Work in progress

Sentiment Analysis in the Era of Large Language Models: A Reality Check

Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Jialin Pan, Lidong Bing

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；small language model(abstract)

Creating Large Language Model Resistant Exams: Guidelines and Strategies

Simon kaare Larsen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Safety Assessment of Chinese Large Language Models

Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng, Minlie Huang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Benchmark website: http://coai.cs.tsinghua.edu.cn/leaderboard/ ; SafetyPrompts repo: https://github.com/thu-coai/Safety-Prompts

Crowd Score: A Method for the Evaluation of Jokes using Large Language Model AI Voters as Judges

Fabricio Goes, Zisen Zhou, Piotr Sawicki, Marek Grzes, Daniel G. Brown

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 11 pages, 3 figures

Agentic Large Language Models, a survey

代理大语言模型：综述

Aske Plaat, Max van Duijn, Niki van Stein, Mike Preuss, Peter van der Putten, Kees Joost Batenburg

机构 * Leiden University Leiden Netherlands ； Leiden University \& AI Lab, Pegasystems Leiden Netherlands ； Leiden University ； Leiden University \& AI Lab, Pegasystems 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.AI、cs.LG

AI总结 本文综述了代理大语言模型的研究现状，探讨了其在医疗诊断、物流和金融分析等领域的应用，并提出通过推理、行动和交互提升大语言模型能力的未来研究方向。

Comments Website: https://askeplaat.github.io/agentic-llm-survey-site/

Journal ref JAIR volume 84, article 29, December 2025

Large Language Models for Time Series: A Survey

Xiyuan Zhang, Ranak Roy Chowdhury, Rajesh K. Gupta, Jingbo Shang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments GitHub repository: https://github.com/xiyuanzh/awesome-llm-time-series

HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models

Chen Chen, Yuchen Hu, Chao-Han Huck Yang, Sabato Macro Siniscalchi, Pin-Yu Chen, Eng Siong Chng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to NeurIPS 2023, 24 pages. Datasets and Benchmarks Track. Added the first Mandarin and code-switching (zh-cn and en-us) results from the LLM-based generative ASR error correction to Table 8 on Page 21

Characterizing Rhetorical Misalignment in Decision-Making with Language Models

表征语言模型决策中的修辞失配问题

Zirui Cheng, Joey Chan, Simo Du, Chenhao Tan, Yue Guo, Hao Peng

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； NYC Health + Hospitals/Jacobi Medical Center（纽约市健康与医院公司雅各比医学中心） ； University of Chicago（芝加哥大学） 纠错

专题命中 评测与基准 ：LLM(summary_cn,abstract)；language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

AI总结 该研究开发决策理论框架研究LLM的修辞失配问题，通过临床实验发现其会诱导平均2.81%的有害决策翻转，还提出用LLM模拟决策者实现可扩展评估，揭示了高风险领域的新安全隐患。

PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows

PolyWorkBench：多语言长视野语言模型智能体基准测试

Hongliang Li, Yijin Liu, Zhiwei Zhang, Zihe Liu, Xinyue Lou, Jinan Xu, Fandong Meng, Kaiyu Huang

机构 * Beijing Jiaotong University（北京交通大学） ； Weixin AI, Tencent Inc（腾讯微云人工智能实验室） 纠错

专题命中 评测与基准 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 研究多语言长视野工作流程中LLM智能体的表现，提出PolyWorkBench基准测试及结合多种评估方式的混合框架，发现最先进LLM智能体在多语言设置中性能降，强调联合建模语言变化和程序决策对智能体评估的重要性。

Comments 17 Pages, 5 figures

A Graph-Based Reinforcement Learning Framework for Structured Drift Diagnosis and Recovery in Autonomous LLM Agents

面向自主大语言模型智能体中结构化漂移诊断与恢复的基于图的强化学习框架

Ismail El Hamraoui, Sagar Jose, Nicolas Bureau, Robert Plana

专题命中 评测与基准 ：LLM(title,summary_cn)；language model(abstract)；small language model(abstract)；分类 cs.AI、cs.LG

AI总结 本研究针对自主LLM智能体的运行时行为漂移问题，提出基于图的强化学习即插即用恢复框架，经AppWorld基准验证，该框架可利用漂移信息做出正确恢复决策且输出符合要求。

LigBench: A Unified and Human-Aligned Benchmark for LLM-based Research Idea Generation

LigBench：面向基于大语言模型的研究创意生成的统一且符合人类偏好的基准

Chenrun Wang, Mingxuan Zhu, Tiancheng Huang, Wenjie Li, Yujie Zhang, Zichen Zhu, Zhiying Zou, Kai Yu, Lu Chen

机构 * Shanghai Jiao Tong University（上海交通大学） ； Shanghai Innovation Institution（上海创新研究院） 纠错

专题命中 评测与基准 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 该研究针对现有LLM研究创意生成评估零散、缺乏统一标准的问题，提出LigBench基准及PAIR-IQ数据集，实验表明LigBench评估稳定可解释且与专家判断一致性高，PAIR-IQ训练的模型排名准确性和鲁棒性更强

Comments 17 pages

LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence

LODESTAR：可信赖熵是被引导的，而非仅被测量——强化偏振器防止冻结型大语言模型（LLM）被错误证据误导而自信出错

Po-Jen Ko, Che-Cheng Wu, Hung-Chun Hsu, Li-Yang Chang, Chuan-Ju Wang

专题命中 评测与基准 ：LLM(title,title_cn)；分类 cs.CL、cs.LG

AI总结 LODESTAR是首个通过强化学习训练偏振器、依据第三方冻结型LLM的不确定性评分文本干预的方法，可提升检索增强问答的F1值等指标，减少模型读取误导性段落的概率。

Comments 28 pages, 3 figures

Benchmarking LLM Judges for Mobile Agent Evaluation

面向移动智能体评估的LLM评判基准测试

Ziqiang Wang, Li Gu, Zhixiang Chi, Zhi Liu, Seyed Mehdi Ayyoubzadeh, Yuanhao Yu, Yang Wang

机构 * Mila – Québec AI Institute（米拉-魁北克人工智能研究所） ； Concordia University（康考迪亚大学） ； University of Toronto（多伦多大学） ； Shanghai University（上海大学） ； McMaster University（麦克马斯特大学） 纠错

专题命中 评测与基准 ：LLM(title,title_cn)；分类 cs.CL、cs.AI

AI总结 该研究推出MobileJudgeBench基准，评估6种LLM评判器方法在移动智能体轨迹上的可靠性，发现简单基线评判器具竞争力、基准质量指标可预测评判器效用，且不同LLM后端故障特征相反。

ReLTEx: Reliable LLM-based Taxonomy Expansion

ReLTEx：基于大语言模型的可靠分类体系扩展

Zeinab Ghamlouch, Mehwish Alam

机构 * Télécom Paris（巴黎电信学院） ； Institut Polytechnique de Paris（巴黎综合理工学院） 纠错

专题命中 评测与基准 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 ReLTEx是结合LLM候选生成、结构感知验证与递归扩展控制的框架，可减少幻觉，在基准分类体系的掩码扩展任务中，能生成更可靠、语义连贯的分类体系扩展。

Autorubric: A Unifying Framework for Rubric-Based LLM Evaluation on Non-Verifiable Tasks

Autorubric：统一基于评分标准的LLM评估

Delip Rao, Chris Callison-Burch

机构 * University of Pennsylvania（宾夕法尼亚大学） 纠错

专题命中 评测与基准 ：LLM(title,title_cn)；分类 cs.CL、cs.AI

AI总结 Autorubric通过整合多种评分标准评估技术，提供统一的评分框架和默认设置，提升LLM评估的可靠性与一致性，并在多个基准测试中展示了其有效性。

Comments 60 pages; COLM 2026 camera ready copy

QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture

QuArch：评估计算机体系结构中大型语言模型推理的基准

Shvetank Prakash, Andrew Cheng, Mark Mazumder, Arya Tschand, Varun Gohil, Jeffrey Ma, Jason Yik, Zishen Wan, Jessica Quaye, Elisavet Lydia Alvanaki, Avinash Kumar, Chandrashis Mazumdar, Tuhin Khare, Alexander Ingare, Ikechukwu Uchendu, Radhika Ghosal, Abhishek Tyagi, Chenyu Wang, Andrea Mattia Garavagno, Sarah Gu, Alice Guo, Grace Hur, Luca P. Carloni, Tushar Krishna, Ankita Nayak, Amir Yazdanbakhsh, Vijay Janapa Reddi

专题命中 评测与基准 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结 该研究提出QuArch基准，用于评估LLM在计算机体系结构中的推理能力。它包含2671个问答对，发现前沿模型在高阶思维技能上有差距。经微调可提升内存层次结构设计任务性能，为构建和衡量LLM能力提供基础，推动计算系统创新。

Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory

人工智能利维坦：通过霍布斯社会契约理论视角探索大语言模型智能体的社会演化

Gordon Dai, Weijia Zhang, Jinhan Li, Siqi Yang, Chidera Onochie lbe, Srihas Rao, Arthur Caetano, Misha Sra

专题命中 评测与基准 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 本研究基于LLM智能体构建模拟社会，通过霍布斯社会契约理论验证其演化契合该理论，证实LLMs可模拟复杂社会动态，有望助力人类对社会系统的理解。

EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners

EduClaw-Bench：面向教学大型语言模型智能体的长周期基准测试集（含模拟学习者）

Unggi Lee, Sookbun Lee, Yeil Jeong, Eunjoo Lee, Minchul Shin, Hoilym Kwon

专题命中 评测与基准 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 研究人员推出EduClaw-Bench长周期基准测试集，结合模拟学习者评估LLM智能体辅导效果，发现辅导质量取决于基础模型与智能体适配器的结合，且多数组合无法维持全程良好辅导。

Real-Time Detection and Repair of LLM Agent Failures

LLM智能体故障的实时检测与修复

Sunny Dubey

专题命中 评测与基准 ：LLM(title,title_cn)；分类 cs.AI、cs.LG

AI总结 该研究提出基于LLM智能体步骤遥测的实时故障检测与修复系统，结合单类回声状态网络集成与确定性验证层，可高效检测并修复故障，提升任务成功率且成本极低。

Comments 16 pages, 5 figures. Code, data and demo: github.com/sunnydubey1111/agent-trajectory-sentinel Walkthrough: youtu.be/a05n_000klE

Right Answer, Wrong Method: Shortcut Hacking Misleads the Evaluation of LLM Reasoning on Frontier Science Benchmarks

正确答案，错误方法：捷径作弊误导前沿科学基准上的大语言模型推理评估

Xuan Ren, Weiqi Zhai, Tianle Pu, Yihua Zhu, Yihua Zhu, Hu Wei, Bing Zhao

机构 * Alibaba Group（阿里巴巴集团） ； Alibaba DAMO Academy（阿里巴巴达摩院） 纠错

专题命中 评测与基准 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 该研究指出前沿LLM在科学推理基准中存在解题作弊问题，仅答案评估会高估其推理能力，开发的反作弊策略可抑制该问题。

Comments working in progress