大模型推理能力

CHEER-Ekman: Fine-grained Embodied Emotion Classification

Phan Anh Duong, Cat Luong, Divyesh Bommana, Tianyu Jiang

机构 * University of Cincinnati（辛辛那提大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments ACL 2025

DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture

Arijit Maji, Raghvendra Kumar, Akash Ghosh, Anushka, Nemil Shah, Abhilekh Borah, Vanshika Shah, Nishant Mishra, Sriparna Saha

机构 * Indian Institute of Technology Patna（印度理工学院帕纳巴分校） ； Banasthali Vidyapeeth University（班纳萨利大学） ； Pandit Deendayal Energy University（德英德能源大学） ； Manipal University Jaipur（马哈拉施特拉邦大学贾伊普尔分校） ； Dwarkadas J. Sanghvi College of Engineering（德瓦尔卡斯J.桑格维工程学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments EMNLP MAINS 2025

WISE: Weak-Supervision-Guided Step-by-Step Explanations for Multimodal LLMs in Image Classification

Yiwen Jiang, Deval Mehta, Siyuan Yan, Yaling Shen, Zimu Wang, Zongyuan Ge

机构 * Faculty of Engineering, Monash University（墨尔本大学工程学院） ； AIM for Health Lab, Faculty of IT, Monash University（墨尔本大学信息技术学院健康人工智能实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Accepted at EMNLP 2025 (Main)

Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts

Xuyang Wu, Yuan Wang, Hsin-Tai Wu, Zhiqiang Tao, Yi Fang

机构 * Santa Clara University（圣克拉拉大学） ； DOCOMO Innovations, Inc.（DOCOMO创新公司） ； Rochester Institute of Technology（罗切斯特理工学院） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments EMNLP Findings

Diagnostics of cognitive failures in multi-agent expert systems using dynamic evaluation protocols and subsequent mutation of the processing context

Andrejs Sorstkins, Josh Bailey, Dr Alistair Baron

机构 * School of Computing and Communications（计算与通讯学院） ； Lancaster University（兰卡斯特大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

Comments Dissertation and research project created in collaboration with JobFair LTD

Evaluating LLM Alignment on Personality Inference from Real-World Interview Data

Jianfeng Zhu, Julina Maharjan, Xinyu Li, Karin G. Coifman, Ruoming Jin

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 8 pages, 3 figures

Towards Understanding Visual Grounding in Visual Language Models

Georgios Pantazopoulos, Eda B. Özyiğit

机构 * The Alan Turing Institute（艾伦·图灵研究所） ； Heriot-Watt University（赫瑞-沃德大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.AI

MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning

Liujian Tang, Shaokang Dong, Yijia Huang, Minqi Xiang, Hongtao Ruan, Bin Wang, Shuo Li, Zhiheng Xi, Zhihui Cao, Hailiang Pang, Heng Kong, He Yang, Mingxu Chai, Zhilin Gao, Xingyu Liu, Yingnan Fu, Jiaming Liu, Xuanjing Huang, Yu-Gang Jiang, Tao Gui, Qi Zhang, Kang Wang, Yunke Zhang, Yuran Wang

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

Evaluation of Large Language Models for Anomaly Detection in Autonomous Vehicles

Petros Loukas, David Bassir, Savvas Chatzichristofis, Angelos Amanatiadis

机构 * Democritus University of Thrace（德米特里乌斯大学） ； Dongguan University of Technology（东莞理工大学） ； ENS-Paris-Saclay University（巴黎-萨克雷大学） ； Neapolis University Pafos（纳皮奥斯大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference

Qi Chen, Jingxuan Wei, Zhuoya Yao, Haiguang Wang, Gaowei Wu, Bihui Yu, Siyuan Li, Cheng Tan

机构 * University of Chinese Academy of Sciences（中国科学院大学） ； Zhejiang University（浙江大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.CL

Comments Accepted to ACM MM 2025

FActBench: A Benchmark for Fine-grained Automatic Evaluation of LLM-Generated Text in the Medical Domain

Anum Afzal, Juraj Vladika, Florian Matthes

机构 * Technical University of Munich（慕尼黑技术大学） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision

Yi Xu, Yuxin Hu, Zaiwei Zhang, Gregory P. Meyer, Siva Karthik Mustikovela, Siddhartha Srinivasa, Eric M. Wolff, Xin Huang

机构 * Cruise LLC (GM)（Cruise LLC（GM）） ； Northeastern University（东北大学） ； OpenAI（开放人工智能研究所） ； Meta ； University of Washington（华盛顿大学） ； Waymo LLC 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.LG

Comments Accepted by CoRL 2025

Rule Synergy Analysis using LLMs: State of the Art and Implications

Bahar Bateni, Benjamin Pratt, Jim Whitehead

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments Submitted for publication at the IEEE Transactions on Games 2024, Special Issue on Large Language Models and Games (10 pages excluding appendix, 3 figures)

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs

Yao Fu, Xianxuan Long, Runchao Li, Haotian Yu, Mu Sheng, Xiaotian Han, Yu Yin, Pan Li

机构 * Case Western Reserve University（凯斯西储大学） ； Hangzhou Dianzi University（杭州电子科技大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.AI

Comments Accepted to EMNLP2025 main conference (poster)

WebSight: A Vision-First Architecture for Robust Web Agents

Tanvir Bhathal, Asanshay Gupta

机构 * Stanford University（斯坦福大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

InPars+: Supercharging Synthetic Data Generation for Information Retrieval Systems

Matey Krastev, Miklos Hamar, Danilo Toapanta, Jesse Brouwers, Yibin Lei

机构 * University of Amsterdam（阿姆斯特丹大学） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.AI

PersonaEval: Are LLM Evaluators Human Enough to Judge Role-Play?

Lingfeng Zhou, Jialing Zhang, Jin Gao, Mohan Jiang, Dequan Wang

专题命中 推理评测 ：reasoning(abstract)；test-time compute(abstract)；分类 cs.CL

Comments Accepted by COLM 2025

URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models

Ruiqi Yan, Xiquan Li, Wenxi Chen, Zhikang Niu, Chen Yang, Ziyang Ma, Kai Yu, Xie Chen

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Sahel Sharifymoghaddam, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, Jimmy Lin

机构 * University of Waterloo（滑铁卢大学） ； CSIRO（澳大利亚联邦科学与工业研究组织） ； Independent（独立研究者） ； Carnegie Mellon University（卡内基梅隆大学） ； The University of Queensland（昆士兰大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.CL

Hide and Seek with LLMs: An Adversarial Game for Sneaky Error Generation and Self-Improving Diagnosis

Rui Zou, Mengqi Wei, Yutao Zhu, Jirong Wen, Xin Zhao, Jing Chen

专题命中 推理评测 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.AI

Not All Tokens Are What You Need In Thinking

Hang Yuan, Bin Yu, Haotian Li, Shijun Yang, Christina Dan Wang, Zhou Yu, Xueyin Xu, Weizhen Qi, Kai Chen

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments 11 pages, 7 figures and 3 tables

SafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generation

Hao Ye, Mengshi Qi, Zhaohong Liu, Liang Liu, Huadong Ma

机构 * State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（网络与交换技术国家重点实验室，北京邮电大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents

Xuehui Wang, Zhenyu Wu, JingJing Xie, Zichen Ding, Bowen Yang, Zehao Li, Zhaoyang Liu, Qingyun Li, Xuan Dong, Zhe Chen, Weiyun Wang, Xiangyu Zhao, Jixuan Chen, Haodong Duan, Tianbao Xie, Chenyu Yang, Shiqian Su, Yue Yu, Yuan Huang, Yiqian Liu, Xiao Zhang, Yanting Zhang, Xiangyu Yue, Weijie Su, Xizhou Zhu, Wei Shen, Jifeng Dai, Wenhai Wang

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； Shanghai Jiao Tong University（上海交通大学） ； Xiamen University（厦门大学） ； University of Science and Technology of China（中国科学技术大学） ； The Hong Kong University of Science and Technology（香港科技大学） ； Harbin Institute of Technology（哈尔滨工业大学） ； Tsinghua University（清华大学） ； Nanjing University（南京大学） ； Fudan University（复旦大学） ； University of Hong Kong（香港大学） ； Donghua University（东华大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.CL

Comments in progress

Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering

William Jurayj, Jeffrey Cheng, Benjamin Van Durme

机构 * Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；test-time compute(abstract)；分类 cs.CL

Comments Accepted to ACL 2025. Code: https://github.com/wjurayj/final_answer

ReviewAgents: Bridging the Gap Between Human and AI-Generated Paper Reviews

Xian Gao, Jiacheng Ruan, Zongyun Zhang, Jingsheng Gao, Ting Liu, Yuzhuo Fu

机构 * Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments Work in progress

GraphRunner: A Multi-Stage Framework for Efficient and Accurate Graph-Based Retrieval

Savini Kashmira, Jayanaka L. Dantanarayana, Krisztián Flautner, Lingjia Tang, Jason Mars

机构 * University of Michigan（密歇根大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale

Haoyang Li, Shang Wu, Xiaokang Zhang, Xinmei Huang, Jing Zhang, Fuxin Jiang, Shuai Wang, Tieying Zhang, Jianjun Chen, Rui Shi, Hong Chen, Cuiping Li

机构 * Renmin University of China（中国人民大学） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Rethinking Verification for LLM Code Generation: From Generation to Testing

Zihan Ma, Taolin Zhang, Maosong Cao, Junnan Liu, Wenwei Zhang, Minnan Luo, Songyang Zhang, Kai Chen

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院） ； MOE KLINNS Lab, Xi’an Jiaotong University（西安交通大学教育部KLINNS实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；verifier(abstract)；分类 cs.CL

DLaVA: Document Language and Vision Assistant for Answer Localization with Enhanced Interpretability and Trustworthiness

Ahmad Mohammadshirazi, Pinaki Prasad Guha Neogi, Ser-Nam Lim, Rajiv Ramnath

机构 * Department of Computer Science（计算机科学系） ； Engineering, Ohio State University, Ohio, US（工程系，俄亥俄州立大学，俄亥俄，美国） ； Department of Computer Science, University of Central Florida, Florida, US（计算机科学系，中央佛罗里达大学，佛罗里达，美国） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.AI

CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation

Guangya Yu, Yanhao Li, Zongying Jiang, Yuxiong Jin, Li Dai, Yupian Lin, Ruihui Hou, Weiyan Zhang, Yongqi Fan, Qi Ye, Jingping Liu, Tong Ruan

机构 * School of Information Science and Engineering, East China University of Science and Technology（信息科学与工程学院，东华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 2025 ACL Findings