大模型推理能力

NoLiMa: Long-Context Evaluation Beyond Literal Matching

Ali Modarressi, Hanieh Deilamsalehy, Franck Dernoncourt, Trung Bui, Ryan A. Rossi, Seunghyun Yoon, Hinrich Schütze

机构 * LMU（莱茵-穆尔大学） ； Center for Information and Language Processing（信息与语言处理中心） ； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心） ； Adobe（Adobe公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments Accepted at ICML 2025

CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMs

Haoxi Li, Sikai Bai, Jie Zhang, Song Guo

机构 * HKUST（香港科技大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.LG

Comments 9 pages, 6 figures

LLMs are Introvert

Litian Zhang, Xiaoming Zhang, Bingyu Yan, Ziyi Zhou, Bo Zhang, Zhenyu Guan, Xi Zhang, Chaozhuo Li

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） ； Beihang University（北京航空航天大学） ； Nanjing Normal University（南京师范大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.AI

An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability

Yusuke Yamauchi, Taro Yano, Masafumi Oyamada

机构 * The University of Tokyo（东京大学） ； NEC Corporation（日本电讯公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Prompt engineering and framework: implementation to increase code reliability based guideline for LLMs

Rogelio Cruz, Jonatan Contreras, Francisco Guerrero, Ezequiel Rodriguez, Carlos Valdez, Citlali Carrillo

机构 * IBM Technologies（IBM技术） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.AI

ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs

Xiyao Wang, Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li, Chung-Ching Lin, Kevin Lin, Linjie Li, Furong Huang, Lijuan Wang

机构 * University of Maryland College Park（马里兰大学学院市分校） ； Microsoft（微软） ； University of Michigan（密歇根大学） ； Cardiff University（卡迪夫大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.LG

Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training

Shurui Gui, Shuiwang Ji

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.LG

Hidden in Plain Sight: Evaluation of the Deception Detection Capabilities of LLMs in Multimodal Settings

Md Messal Monem Miah, Adrita Anika, Xi Shi, Ruihong Huang

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Accepted to ACL 2025 Main Conference

Detecting Voice Phishing with Precision: Fine-Tuning Small Language Models

Ju Yong Sim, Seong Hwan Kim

机构 * Korea National University of Transportation（韩国交通大学） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments 15 pages, 4 figures, 8 tables, journal submission

MiMo-VL Technical Report

Core Team, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, Bowen Shen, Zihan Zhang, Zihan Jiang, Zhixian Zheng, Zhichao Song, Zhenbo Luo, Yue Yu, Yudong Wang, Yuanyuan Tian, Yu Tu, Yihan Yan, Yi Huang, Xu Wang, Xinzhe Xu, Xingchen Song, Xing Zhang, Xing Yong, Xin Zhang, Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, Weiji Zhuang, Wei Liu, Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, Peng Wang, Nuo Chen, Menghang Zhu, Kangyang Zhou, Kang Zhou, Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, Jiaming Xu, Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, Dong Zhang, Di Zhang, Chong Ma, Chang Liu, Can Cai, Bingquan Xia

机构 * LLM-Core ； Xiaomi（小米） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 32 pages

Climate Finance Bench

Rafik Mankour, Yassine Chafai, Hamada Saleh, Ghassen Ben Hassine, Thibaud Barreau, Peter Tankov

机构 * Institut Louis Bachelier（路易·巴舍利尔研究所） ； CREST, ENSAE, Institut Polytechnique de Paris（CREST、ENSAE、巴黎理工学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments Dataset is available at https://github.com/Pladifes/climate_finance_bench

From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications

Feibo Jiang, Cunhua Pan, Li Dong, Kezhi Wang, Octavia A. Dobre, Merouane Debbah

机构 * Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing, Hunan Normal University（智能计算与语言信息处理湖南省重点实验室，湖南师范大学） ； National Mobile Communications Research Laboratory, Southeast University（国家移动通信研究实验室，东南大学） ； Changsha Social Laboratory of Artificial Intelligence, Hunan University of Technology and Business（长沙人工智能社会实验室，湖南工业大学） ； Department of Computer Science, Brunel University London（伦敦布鲁内尔大学计算机科学系） ； Faculty of Engineering and Applied Science, Memorial University（纪念大学工程与应用科学学院） ； G Research Center, Khalifa University of Science and Technology（科技大学6G研究中心） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

Towards Reliable Large Audio Language Model

Ziyang Ma, Xiquan Li, Yakun Song, Wenxi Chen, Chenpeng Du, Jian Wu, Yuanzhe Chen, Zhuo Chen, Yuping Wang, Yuxuan Wang, Xie Chen

机构 * X-LANCE Lab, School of Computer Science, MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University（X-LANCE实验室，计算机科学学院，人工智能国家重点实验室，上海交通大学） ； ByteDance（字节跳动） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments ACL 2025 Findings

CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space

Yong Zhao, Kai Xu, Zhengqiu Zhu, Yue Hu, Zhiheng Zheng, Yingfeng Chen, Yatai Ji, Chen Gao, Yong Li, Jincai Huang

机构 * National University of Defense Technology（国防科技大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark

Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, Graham Neubig

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments ACL 2025 Main

RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Reward

Zongsheng Wang, Kaili Sun, Bowen Wu, Qun Yu, Ying Li, Baoxun Wang

机构 * Platform and Content Group, Tencent（腾讯平台与内容部） ； School of Software & Microelectronics, Peking University（北京大学软件与微电子学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Towards Autonomous UAV Visual Object Search in City Space: Benchmark and Agentic Methodology

Yatai Ji, Zhengqiu Zhu, Yong Zhao, Beidan Liu, Chen Gao, Yihao Zhao, Sihang Qiu, Yue Hu, Quanjun Yin, Yong Li

机构 * State Key Lab of Digital-Intelligent Modeling and Simulation（数字智能建模与仿真国家重点实验室） ； Department of Electronic Engineering, Tsinghua University（清华大学电子工程系） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applications

Chunjing Gan, Dan Yang, Binbin Hu, Ziqi Liu, Yue Shen, Zhiqiang Zhang, Jian Wang, Jun Zhou

机构 * Ant Group（蚂蚁集团） 纠错

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.LG

OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs

John Murzaku, Owen Rambow

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Submitted to COLM 2025. Preprint

Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving

Yue Li, Meng Tian, Zhenyu Lin, Jiangtong Zhu, Dechang Zhu, Haiqiang Liu, Zining Wang, Yueyi Zhang, Zhiwei Xiong, Xinhai Zhao

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.CL

LongViTU: Instruction Tuning for Long-Form Video Understanding

Rujie Wu, Xiaojian Ma, Hai Ci, Yue Fan, Yuxuan Wang, Haozhe Zhao, Qing Li, Yizhou Wang

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.LG

(G)I-DLE: Generative Inference via Distribution-preserving Logit Exclusion with KL Divergence Minimization for Constrained Decoding

Hanwool Lee

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments preprint

CoKe: Customizable Fine-Grained Story Evaluation via Chain-of-Keyword Rationalization

Brihi Joshi, Sriram Venkatapathy, Mohit Bansal, Nanyun Peng, Haw-Shiuan Chang

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Bias Testing and Mitigation in LLM-based Code Generation

Dong Huang, Jie M. Zhang, Qingwen Bu, Xiaofei Xie, Junjie Chen, Heming Cui

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.AI

Comments Accepted by TOSEM

Goal2Story: A Multi-Agent Fleet based on Privately Enabled sLLMs for Impacting Mapping on Requirements Elicitation

Xinkai Zou, Yan Liu, Xiongbo Shi, Chen Yang

专题命中 推理评测 ：CoT(abstract)；planning(abstract)；分类 cs.AI

CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models

Zihui Cheng, Qiguang Chen, Jin Zhang, Hao Fei, Xiaocheng Feng, Wanxiang Che, Min Li, Libo Qin

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.AI

Comments Accepted at AAAI 2025; Project Page: https://github.com/czhhzc/CoMT

RocketEval: Efficient Automated LLM Evaluation via Grading Checklist

Tianjun Wei, Wei Wen, Ruizhi Qiao, Xing Sun, Jianghong Ma

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Accepted by ICLR 2025: https://openreview.net/forum?id=zJjzNj6QUe

Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling

Keqi Chen, Zekai Sun, Yuhua Wen, Huijun Lian, Yingming Gao, Ya Li

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysis

Jiahao Gai, Hao Mark Chen, Zhican Wang, Hongyu Zhou, Wanru Zhao, Nicholas Lane, Hongxiang Fan

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.LG

Comments Paper accepted by ASP-DAC'25

A Survey on LLM-powered Agents for Recommender Systems

Qiyao Peng, Hongtao Liu, Hua Huang, Qing Yang, Minglai Shao

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI