大模型推理能力

Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment

Hongbin Zhang, Kehai Chen, Xuefeng Bai, Yang Xiang, Min Zhang

机构 * Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China（计算与智能研究所，哈尔滨工业大学，深圳，中国） ； Peng Cheng Laboratory, Shenzhen, China（鹏城实验室，深圳，中国） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Under review;Work in progress;

RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration

Xiuyuan Chen, Jian Zhao, Yuchen Yuan, Tianle Zhang, Huilin Zhou, Zheng Zhu, Ping Hu, Linghe Kong, Chi Zhang, Weiran Huang, Xuelong Li

机构 * Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信） ； School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院） ； University of Science and Technology of China（中国科学技术大学） ； GigaAI ； School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

CLEVER: A Curated Benchmark for Formally Verified Code Generation

Amitayush Thakur, Jasper Lee, George Tsoukalas, Meghana Sistla, Matthew Zhao, Stefan Zetzsche, Greg Durrett, Yisong Yue, Swarat Chaudhuri

机构 * UT Austin（得克萨斯大学） ； Amazon（亚马逊） ； Caltech（加州理工学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Face-Human-Bench: A Comprehensive Benchmark of Face and Human Understanding for Multi-modal Assistants

Lixiong Qin, Shilong Ou, Miaoxuan Zhang, Jiangning Wei, Yuhang Zhang, Xiaoshuai Song, Yuchen Liu, Mei Wang, Weiran Xu

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） ； Beijing Normal University（北京师范大学） 纠错

专题命中 推理评测 ：CoT(abstract)；分类 cs.CL、cs.AI

Comments 50 pages, 14 figures, 42 tables. NeurIPS 2025 Datasets and Benchmarks Track

Can They Dixit? Yes they Can! Dixit as a Playground for Multimodal Language Model Capabilities

Nishant Balepur, Dang Nguyen, Dayeon Ki

机构 * University of Maryland（马里兰大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted as a Spotlight paper at the EMNLP 2025 Wordplay Workshop

Metadata Extraction Leveraging Large Language Models

Cuize Han, Sesh Jalagam

机构 * Box AI Platform（Box人工智能平台） 纠错

专题命中 推理评测 ：CoT(abstract)；分类 cs.AI、cs.LG

X-Ego: Acquiring Team-Level Tactical Situational Awareness via Cross-Egocentric Contrastive Video Representation Learning

Yunzhe Wang, Soham Hans, Volkan Ustun

机构 * University of Southern California（南加州大学） ； USC Institute for Creative Technologies（USC创意技术研究所） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 5 figures

Training-Free Spectral Fingerprints of Voice Processing in Transformers

Valentin Noël

机构 * Devoteam 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Preprint under review (2025). 12 pages, 8 figures

Who's Asking? Investigating Bias Through the Lens of Disability Framed Queries in LLMs

Vishnu Hari, Kalpana Panda, Srikant Panda, Amit Agarwal, Hitesh Laxmichand Patel

机构 * Birla Institute of Technology and Science (BITS)（巴拉·技术与科学学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at ICCV 2025

DanmakuTPPBench: A Multi-modal Benchmark for Temporal Point Process Modeling and Understanding

Yue Jiang, Jichu Li, Yang Liu, Dingkang Yang, Feng Zhou, Quyu Kong

机构 * Fudan University（复旦大学） ； Center for Applied Statistics and School of Statistics, Renmin University of China（应用统计中心和中国人民大学统计学院） ； Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（北京未来区块链与隐私计算高级创新中心） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Accepted by Neural Information Processing Systems (NeurIPS 2025)

Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variations

Bo-Han Feng, Chien-Feng Liu, Yu-Hsuan Li Liang, Chih-Kai Yang, Szu-Wei Fu, Zhehuai Chen, Ke-Han Lu, Sung-Feng Huang, Chao-Han Huck Yang, Yu-Chiang Frank Wang, Yun-Nung Chen, Hung-yi Lee

机构 * National Taiwan University（国立台湾大学） ； NVIDIA 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Submitted to ICASSP 2026

Neuronal Group Communication for Efficient Neural representation

Zhengqi Pei, Qingming Huang, Shuhui Wang

机构 * Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所） ； School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 28 pages, 2 figures

LC-Eval: A Bilingual Multi-Task Evaluation Benchmark for Long-Context Understanding

Sheikh Jubair, Arwa Omayrah, Amal Alshammari, Alhanoof Althnian, Abdulhamed Alothaimen, Norah A. Alzahrani, Shahad D. Alzaidi, Nora Al-Twairesh, Abdulmohsen Al-Thubaity

机构 * HUMAIN ； Saudi Data and AI Authority（沙特数据与人工智能管理局） ； King Saud University（沙特国王大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 1 figure, 15 tables, 10 main pages

ConsintBench: Evaluating Language Models on Real-World Consumer Intent Understanding

Xiaozhe Li, TianYi Lyu, Siyi Yang, Yuxi Gong, Yizhao Yang, Jinxuan Huang, Ligao Zhang, Zhuoyi Huang, Qingwen Liu

机构 * Tongji University（同济大学） ； Stanford University（斯坦福大学） ； Currents AI 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

A Systematic Approach to Predict the Impact of Cybersecurity Vulnerabilities Using LLMs

Anders Mølmen Høst, Pierre Lison, Leon Moonen

机构 * Simula \& University of Oslo Oslo, Norway ； Norwegian Computing Center Oslo, Norway ； Simula Research Laboratory Oslo, Norway 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted for publication in the 24th IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom 2025)

Efficient Large Language Model Inference with Neural Block Linearization

Mete Erdogan, Francesco Tonin, Volkan Cevher

机构 * Laboratory for Information and Inference Systems（信息与推断系统实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Towards Evaluating Proactive Risk Awareness of Multimodal Language Models

Youliang Yuan, Wenxiang Jiao, Yuejin Xie, Chihao Shen, Menghan Tian, Wenxuan Wang, Jen-tse Huang, Pinjia He

机构 * School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院） ； Xiaohongshu Inc.（小红书公司） ； Renmin University of China（中国人民大学） ； Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NeurIPS 2025 (Track on Datasets and Benchmarks)

Attention to Non-Adopters

Kaitlyn Zhou, Kristina Gligorić, Myra Cheng, Michelle S. Lam, Vyoma Raman, Boluwatife Aminu, Caeley Woo, Michael Brockman, Hannah Cha, Dan Jurafsky

机构 * Stanford University（斯坦福大学） ； Together AI ； Johns Hopkins University（约翰霍普金斯大学） ； Cornell University（康奈尔大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers

Ming Hu, Chenglong Ma, Wei Li, Wanghan Xu, Jiamin Wu, Jucheng Hu, Tianbin Li, Guohang Zhuang, Jiaqi Liu, Yingzhou Lu, Ying Chen, Chaoyang Zhang, Cheng Tan, Jie Ying, Guocheng Wu, Shujian Gao, Pengcheng Chen, Jiashi Lin, Haitao Wu, Lulu Chen, Fengxiang Wang, Yuanyuan Zhang, Xiangyu Zhao, Feilong Tang, Encheng Su, Junzhi Ning, Xinyao Liu, Ye Du, Changkai Ji, Pengfei Jiang, Cheng Tang, Ziyan Huang, Jiyao Liu, Jiaqi Wei, Yuejin Yang, Xiang Zhang, Guangshuai Wang, Yue Yang, Huihui Xu, Ziyang Chen, Yizhou Wang, Chen Tang, Jianyu Wu, Yuchen Ren, Siyuan Yan, Zhonghua Wang, Zhongxing Xu, Shiyan Su, Shangquan Sun, Runkai Zhao, Zhisheng Zhang, Dingkang Yang, Jinjie Wei, Jiaqi Wang, Jiahao Xu, Jiangtao Yan, Wenhao Tang, Hongze Zhu, Yu Liu, Fudi Wang, Yiqing Shen, Yuanfeng Ji, Yanzhou Su, Tong Xie, Hongming Shan, Chun-Mei Feng, Zhi Hou, Diping Song, Lihao Liu, Yanyan Huang, Lequan Yu, Bin Fu, Shujun Wang, Xiaomeng Li, Xiaowei Hu, Yun Gu, Ben Fei, Benyou Wang, Yuewen Cao, Minjie Shen, Jie Xu, Haodong Duan, Fang Yan, Hongxia Hao, Jielan Li, Jiajun Du, Yanbo Wang, Imran Razzak, Zhongying Deng, Chi Zhang, Lijun Wu, Conghui He, Zhaohui Lu, Jinhai Huang, Wenqi Shao, Yihao Liu, Siqi Luo, Yi Xin, Xiaohong Liu, Fenghua Ling, Yuqiang Li, Aoran Wang, Siqi Sun, Qihao Zheng, Nanqing Dong, Tianfan Fu, Dongzhan Zhou, Yan Lu, Wenlong Zhang, Jin Ye, Jianfei Cai, Yirong Chen, Wanli Ouyang, Yu Qiao, Zongyuan Ge, Shixiang Tang, Junjun He, Chunfeng Song, Lei Bai, Bowen Zhou

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions

Weijie Xu, Shixian Cui, Xi Fang, Chi Xue, Stephanie Eckman, Chandan K. Reddy

机构 * Amazon（亚马逊） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 40 pages, 13 figures

GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images

Xiang Lan, Feng Wu, Kai He, Qinghao Zhao, Shenda Hong, Mengling Feng

机构 * National University of Singapore（新加坡国立大学） ； Peking University People’s Hospital（北京大学人民医院） ； Peking University（北京大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2025 Camera-Ready

KITE: A Benchmark for Evaluating Korean Instruction-Following Abilities in Large Language Models

Dongjun Kim, Chanhee Park, Chanjun Park, Heuiseok Lim

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 13 pages, 3 figures, 5 tables

MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics

Yuxing Lu, Xukai Zhao, J. Ben Tamo, Micky C. Nnamdi, Rui Peng, Shuang Zeng, Xingyu Hu, Jinzhuo Wang, May D. Wang

机构 * Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University（沃克生物医学工程部门，佐治亚理工学院和埃默里大学） ； College of Future of Technology, Peking University（未来技术学院，北京大学） ； School of Architecture, Tsinghua University（建筑学院，清华大学） ； School of Electrical and Computer Engineering, Georgia Institute of Technology（电气与计算机工程学院，佐治亚理工学院） ； School of Computer Science, Georgia Institute of Technology（计算机科学学院，佐治亚理工学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 6 figures, 4 tables

COIG-Writer: A High-Quality Dataset for Chinese Creative Writing with Thought Processes

Yunwen Li, Shuangshuang Ying, Xingwei Qu, Xin Li, Sheng Jin, Minghao Liu, Zhoufutu Wen, Tianyu Zheng, Xeron Du, Qiguang Chen, Jiajun Shi, Wangchunshu Zhou, Jiazhan Feng, Wanjun Zhong, Libo Qin, Stephen Huang, Wanxiang Che, Chenghua Lin, Eli Zhang

机构 * M-A-P 2077AI 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

HALF: Harm-Aware LLM Fairness Evaluation Aligned with Deployment

Ali Mekky, Omar El Herraoui, Preslav Nakov, Yuxia Wang

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

InfoDet: A Dataset for Infographic Element Detection

Jiangning Zhu, Yuxing Zhou, Zheng Wang, Juntao Yao, Yima Gu, Yuhui Yuan, Shixia Liu

机构 * BNRist, Tsinghua University（清华大学信息与技术研究院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Submitted to ICLR 2026

MAFA: A multi-agent framework for annotation

Mahmood Hegazy, Aaron Rodrigues, Azzam Naeem

机构 * JP Morgan Chase CCB ML Team（摩根大通CCB机器学习团队） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Journal ref ECAI 2025 Workshop on AI in Finance

The simulation of judgment in LLMs

Edoardo Loru, Jacopo Nudo, Niccolò Di Marco, Alessandro Santirocchi, Roberto Atzeni, Matteo Cinelli, Vincenzo Cestari, Clelia Rossi-Arnaud, Walter Quattrociocchi

机构 * Department of Computer, Control and Management Engineering, Sapienza University of Rome（计算机、控制与管理工程系，罗马萨皮恩扎大学） ； Department of Computer Science, Sapienza University of Rome（计算机科学系，罗马萨皮恩扎大学） ； Department of Legal, Social, and Educational Sciences, Tuscia University（法律、社会与教育科学系，图斯西亚大学） ； Department of Psychology, Sapienza University of Rome（心理学系，罗马萨皮恩扎大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Please refer to published version: https://doi.org/10.1073/pnas.2518443122

Journal ref Proc. Natl. Acad. Sci. U.S.A. 122 (42) e2518443122, 2025

Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems

Karthik Avinash, Nikhil Pareek, Rishav Hada

机构 * FutureAGI Inc.（未来人工智能公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learning

Mahdi Cherakhloo, Arash Abbasi, Mohammad Saeid Sarafraz, Bijan Vosoughi Vahdat

机构 * Department of Medical Engineering（医学工程系） ； Electrical Engineering Department（电气工程系） ； Sharif University of Technology（谢里夫技术大学） ； Department of Electrical Engineering（电气工程系） ； University of Tehran（德黑兰大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI