大模型推理能力 - arXivDaily 专题

2511.11252 2025-11-17 cs.AI 70%

UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios

Mohamed Amine Ferrag, Abderrahmane Lakas, Merouane Debbah

机构 * Department of Computer and Network Engineering, College of Information Technology, United Arab Emirates University（计算机与网络工程系，信息科技学院，阿联酋大学）； Khalifa University of Science and Technology（科技大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

Comments 18 pages, 5 Figures

2511.10289 2025-11-14 eess.AS cs.CL 70%

Music Flamingo: Scaling Music Understanding in Audio Language Models

Sreyan Ghosh, Arushi Goel, Lasha Koroshinadze, Sang-gil Lee, Zhifeng Kong, Joao Felipe Santos, Ramani Duraiswami, Dinesh Manocha, Wei Ping, Mohammad Shoeybi, Bryan Catanzaro

机构 * NVIDIA, CA, USA（NVIDIA公司）； University of Maryland, College Park, USA（大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Project Page: https://research.nvidia.com/labs/adlr/MF/

2506.07315 2025-11-11 q-fin.ST cs.AI 70%

Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluation

Zonghan Wu, Congyuan Zou, Junlin Wang, Chenhan Wang, Hangjing Yang, Yilei Shao

机构 * Shanghai AI Finance School, East China Normal University（上海人工智能金融学院，华东师范大学）； Tsinghua University（清华大学）

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.AI

2506.11137 2025-11-10 cs.CL 70%

Scalable Medication Extraction and Discontinuation Identification from Electronic Health Records Using Large Language Models

Chong Shao, Douglas Snyder, Chiran Li, Bowen Gu, Kerry Ngan, Chun-Ting Yang, Jiageng Wu, Richard Wyss, Kueiyu Joshua Lin, Jie Yang

专题命中推理评测：reasoning(abstract)；CoT(abstract)；分类 cs.CL

2511.01340 2025-11-04 cs.CV cs.CL 70%

$\left|\,\circlearrowright\,\boxed{\text{BUS}}\,\right|$: A Large and Diverse Multimodal Benchmark for evaluating the ability of Vision-Language Models to understand Rebus Puzzles

Trishanu Das, Abhilash Nandy, Khush Bajaj, Deepiha S

机构 * Tredence Inc.（特伦德公司）； Indian Institute of Technology Kharagpur（印度理工学院克拉格浦尔分校）； Inria Paris-Rocquencourt（巴黎-罗克琴库特研究所）； Rajiv Gandhi University（拉吉夫·甘地大学）； Tsinghua University（清华大学）； Palmer Research Laboratories（帕勒姆研究实验室）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 7 pages, 5 figures, 4 tables

2510.25536 2025-10-31 cs.CL 70%

TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation

Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu

机构 * Tsinghua University（清华大学）； Rutgers University（罗格斯大学）； Fudan University（复旦大学）

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments Main paper: 11 pages, 3 figures, 6 tables. Appendix: 28 pages. Bangde Du and Minghao Guo contributed equally. Corresponding authors: Ziyi Ye (ziyiye@fudan.edu.cn), Qingyao Ai (aiqy@tsinghua.edu.cn)

2510.25187 2025-10-30 cs.CL 70%

Testing Cross-Lingual Text Comprehension In LLMs Using Next Sentence Prediction

Ritesh Sunil Chavan, Jack Mostow

机构 * Department of Computer Science（计算机科学系）； Stony Brook University（石溪大学）； School of Computer Science（计算机科学学院）； Carnegie Mellon University（卡内基梅隆大学）

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

2510.23482 2025-10-28 cs.CV cs.AI 70%

On the Faithfulness of Visual Thinking: Measurement and Enhancement

Zujing Liu, Junwen Pan, Qi She, Yuan Gao, Guisong Xia

机构 * Wuhan University（武汉大学）； ByteDance（字节跳动）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.AI

2510.22455 2025-10-28 cs.SD cs.AI eess.AS 70%

Evaluating Multimodal Large Language Models on Core Music Perception Tasks

Brandon James Carone, Iran R. Roman, Pablo Ripollés

机构 * Department of Psychology, Music and Audio Research Laboratory（心理学系、音乐与音频研究实验室）； Department of Electronic Engineering and Computer Science（电子工程与计算机科学系）

专题命中推理评测：reasoning(abstract)；CoT(abstract)；分类 cs.AI

Comments Accepted to the NeurIPS 2025 Workshop on AI for Music (AI4Music), 16 pages, 1 figure, 3 tables

2510.07458 2025-10-28 cs.CL 70%

Populism Meets AI: Advancing Populism Research with LLMs

Yujin J. Jung, Eduardo Ryô Tamaki, Julia Chatterley, Grant Mitchell, Semir Dzebo, Cristóbal Sandoval, Levente Littvay, Kirk A. Hawkins

机构 * Mount St. Mary’s University（圣玛丽大学）； German Institute for Global and Area Studies（全球与区域研究所）； Princeton University（普林斯顿大学）； University of California, Los Angeles（加州大学洛杉矶分校）； University of Oxford（牛津大学）； Diego Portales University（迪埃戈·波尔塔尔大学）； ELTE Centre for Social Sciences（埃尔泰社会科学中心）； Brigham Young University（Brigham Young 大学）

专题命中推理评测：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments 27 pages, 3 figures. Preprint version under review

2510.18029 2025-10-22 cs.DB cs.AI 70%

DynaQuery: A Self-Adapting Framework for Querying Structured and Multimodal Data

Aymane Hassini

机构 * Al Akhawayn University（阿尔阿克哈文大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

Comments 15 pages, 2 figures, 10 tables. Source code and experimental artifacts are available at: https://github.com/aymanehassini/DynaQuery . The 'DynaQuery-Eval-5K' benchmark, introduced in this work, is also publicly available at: https://www.kaggle.com/datasets/aymanehassini/dynaquery-eval-5k-benchmark

2510.17491 2025-10-21 cs.CL 70%

Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents

Yihong Tang, Kehai Chen, Liang Yue, Jinxin Fan, Caishen Zhou, Xiaoguang Li, Yuyang Zhang, Mingming Zhao, Shixiong Kai, Kaiyang Guo, Xingshan Zeng, Wenjing Cun, Lifeng Shang, Min Zhang

机构 * School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China（计算机科学与技术学院，哈尔滨工业大学，深圳，中国）； Huawei Technologies Co., Ltd.（华为技术有限公司）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.CL

2510.17052 2025-10-21 cs.AI 70%

ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems

Hassan Hamad, Yingru Xu, Liang Zhao, Wenbo Yan, Narendra Gyanchandani

机构 * Amazon（亚马逊）

专题命中推理评测：reasoning(abstract)；self-correction(abstract)；分类 cs.AI

2506.23584 2025-10-17 eess.IV cs.AI cs.CV 70%

A Clinically-Grounded Two-Stage Framework for Renal CT Report Generation

Renjie Liang, Zhengkang Fan, Jinqian Pan, Chenkun Sun, Bruce Daniel Steinberg, Russell Terry, Jie Xu

机构 * Department of Health Outcomes and Biomedical Informatics, University of Florida（健康结果与生物医学信息学系，佛罗里达大学）； Department of Urology, University of Florida（泌尿外科系，佛罗里达大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

2505.04364 2025-10-16 cs.MA cs.CL 70%

Benchmarking LLMs' Swarm intelligence

Kai Ruan, Mowen Huang, Ji-Rong Wen, Hao Sun

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（人工智能学院，中国人民大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.CL

2412.06134 2025-10-16 cs.CL 70%

Evaluating and Mitigating Social Bias for Large Language Models in Open-ended Settings

Zhao Liu, Tian Xie, Xueru Zhang

机构 * The Ohio State University（俄亥俄州立大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 15 pages

2510.12218 2025-10-15 cs.AI 70%

GOAT: A Training Framework for Goal-Oriented Agent with Tools

Hyunji Min, Sangwon Jung, Junyoung Sung, Dosung Lee, Leekyeung Han, Paul Hongsuck Seo

机构 * Korea University（韩国大学）； Trillion Labs（万亿实验室）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

Comments 32 pages, 21 figures

2510.09404 2025-10-14 cs.AI 70%

Agentic Systems in Radiology: Design, Applications, Evaluation, and Challenges

Christian Bluethgen, Dave Van Veen, Daniel Truhn, Jakob Nikolas Kather, Michael Moor, Malgorzata Polacin, Akshay Chaudhari, Thomas Frauenfelder, Curtis P. Langlotz, Michael Krauthammer, Farhad Nooralahzadeh

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

2510.04338 2025-10-07 cs.CL 70%

Evaluation of Clinical Trials Reporting Quality using Large Language Models

Mathieu Laï-king, Patrick Paroubek

机构 * Université Paris-Saclay, CNRS, LISN（巴黎-萨克雷大学、法国国家科学研究中心、LISN）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Journal ref Revue TAL 65.2, 2024

2508.09123 2025-10-07 cs.AI cs.CV 70%

OpenCUA: Open Foundations for Computer-Use Agents

Xinyuan Wang, Bowen Wang, Dunjie Lu, Junlin Yang, Tianbao Xie, Junli Wang, Jiaqi Deng, Xiaole Guo, Yiheng Xu, Chen Henry Wu, Zhennan Shen, Zhuokai Li, Ryan Li, Xiaochuan Li, Junda Chen, Boyuan Zheng, Peihang Li, Fangyu Lei, Ruisheng Cao, Yeqiao Fu, Dongchan Shin, Martin Shin, Jiarui Hu, Yuyan Wang, Jixuan Chen, Yuxiao Ye, Danyang Zhang, Dikang Du, Hao Hu, Huarong Chen, Zaida Zhou, Haotian Yao, Ziwei Chen, Qizheng Gu, Yipu Wang, Heng Wang, Diyi Yang, Victor Zhong, Flood Sung, Y. Charles, Zhilin Yang, Tao Yu

机构 * XLANG Lab, The University of Hong Kong（香港大学XLANG实验室）； Moonshot AI ； Stanford University（斯坦福大学）； University of Waterloo（滑铁卢大学）； Carnegie Mellon University（卡内基梅隆大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.AI

Comments Updata author list, modify first page format, correct typos

2510.03127 2025-10-06 cs.AI 70%

A Study of Rule Omission in Raven's Progressive Matrices

Binze Li

机构 * University of California, Los Angeles（加州大学洛杉矶分校）

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.AI

2509.16360 2025-10-06 cs.CL 70%

RephQA: Evaluating Readability of Large Language Models in Public Health Question Answering

Weikang Qiu, Tinglin Huang, Ryan Rullo, Yucheng Kuang, Ali Maatouk, S. Raquel Ramos, Rex Ying

机构 * Yale University, Department of Computer Science（耶鲁大学计算机科学系）； Yale University, School of Nursing（耶鲁大学护理学院）； Yale University, School of Public Health（耶鲁大学公共卫生学院）； Northeastern University（东北大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments ACM KDD Health Track 2025 Blue Sky Best Paper

2510.02172 2025-10-03 cs.CL 70%

RESTRAIN: From Spurious Votes to Signals -- Self-Driven RL with Self-Penalization

Zhaoning Yu, Will Su, Leitian Tao, Haozhu Wang, Aashu Singh, Hanchao Yu, Jianyu Wang, Hongyang Gao, Weizhe Yuan, Jason Weston, Ping Yu, Jing Xu

机构 * Iowa State University（爱荷华州立大学）； FAIR at Meta SuperIntelligence Lab（Meta超智能实验室FAIR团队）； Meta

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

2503.22678 2025-10-02 cs.CL 70%

Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions

Mohammad Almansoori, Komal Kumar, Hisham Cholakkal

机构 * Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed人工智能大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 14 page, 4 figures, 61 references, presented in MICCAI (Oral)

2509.25281 2025-10-01 cond-mat.mtrl-sci cs.LG 70%

Mechanisms of Matter: Language Inferential Benchmark on Physicochemical Hypothesis in Materials Synthesis

Yingming Pu, Tao Lin, Hongyu Chen

机构 * Westlake University（西湖大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.LG

2508.04361 2025-09-30 cs.AI 70%

OmniPlay: Benchmarking Omni-Modal Models on Omni-Modal Game Playing

Fuqing Bie, Shiyu Huang, Xijia Tao, Zhiqin Fang, Leyi Pan, Junzhe Chen, Min Ren, Liuyu Xiang, Zhaofeng He

机构 * Beijing University of Posts and Telecommunications（北京邮电大学）； XPENG ； The University of Hong Kong（香港大学）； Tsinghua University（清华大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

2501.05414 2025-09-30 cs.CL 70%

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation

Xi Ye, Fangcong Yin, Yinghui He, Joie Zhang, Howard Yen, Tianyu Gao, Greg Durrett, Danqi Chen

专题命中推理评测：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments COLM 2025. Data and code available at: https://princeton-pli.github.io/LongProc

2509.21451 2025-09-29 cs.CV cs.CL 70%

VideoJudge: Bootstrapping Enables Scalable Supervision of MLLM-as-a-Judge for Video Understanding

Abdul Waheed, Zhen Wu, Dareen Alharthi, Seungone Kim, Bhiksha Raj

机构 * Carnegie Mellon University（卡内基梅隆大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Work in progress

2503.23768 2025-09-29 cs.CL cs.CV 70%

Texture or Semantics? Vision-Language Models Get Lost in Font Recognition

Zhecheng Li, Guoxian Song, Yujun Cai, Zhen Xiong, Junsong Yuan, Yiwei Wang

机构 * University of California, San Diego（加州大学圣地亚哥分校）； ByteDance（字节跳动）； The University of Queensland（昆士兰大学）； University of Southern California（南加州大学）； University at Buffalo（布法罗大学）； University of California, Merced（加州大学默塞德分校）

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments Accepted to COLM 2025

2503.03064 2025-09-29 cs.CL 70%

Improving LLM-as-a-Judge Inference with the Judgment Distribution

Victor Wang, Michael J. Q. Zhang, Eunsol Choi

机构 * Department of Computer Science（计算机科学系）； The University of Texas at Austin（德克萨斯大学奥斯汀分校）； New York University（纽约大学）

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

1. 推理评测 10451 篇

UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios

Music Flamingo: Scaling Music Understanding in Audio Language Models

Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluation

Scalable Medication Extraction and Discontinuation Identification from Electronic Health Records Using Large Language Models

$\left|\,\circlearrowright\,\boxed{\text{BUS}}\,\right|$: A Large and Diverse Multimodal Benchmark for evaluating the ability of Vision-Language Models to understand Rebus Puzzles

TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation

Testing Cross-Lingual Text Comprehension In LLMs Using Next Sentence Prediction

On the Faithfulness of Visual Thinking: Measurement and Enhancement

Evaluating Multimodal Large Language Models on Core Music Perception Tasks

Populism Meets AI: Advancing Populism Research with LLMs

DynaQuery: A Self-Adapting Framework for Querying Structured and Multimodal Data

Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents

ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems

A Clinically-Grounded Two-Stage Framework for Renal CT Report Generation

Benchmarking LLMs' Swarm intelligence

Evaluating and Mitigating Social Bias for Large Language Models in Open-ended Settings

GOAT: A Training Framework for Goal-Oriented Agent with Tools

Agentic Systems in Radiology: Design, Applications, Evaluation, and Challenges

Evaluation of Clinical Trials Reporting Quality using Large Language Models

OpenCUA: Open Foundations for Computer-Use Agents

A Study of Rule Omission in Raven's Progressive Matrices

RephQA: Evaluating Readability of Large Language Models in Public Health Question Answering

RESTRAIN: From Spurious Votes to Signals -- Self-Driven RL with Self-Penalization

Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions

Mechanisms of Matter: Language Inferential Benchmark on Physicochemical Hypothesis in Materials Synthesis

OmniPlay: Benchmarking Omni-Modal Models on Omni-Modal Game Playing

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation

VideoJudge: Bootstrapping Enables Scalable Supervision of MLLM-as-a-Judge for Video Understanding

Texture or Semantics? Vision-Language Models Get Lost in Font Recognition

Improving LLM-as-a-Judge Inference with the Judgment Distribution