大模型推理能力 - arXivDaily 专题

2410.06526 2025-03-04 cs.DB 82%

KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Tasks

Kaijing Ma, Xinrun Du, Yunran Wang, Haoran Zhang, Zhoufutu Wen, Xingwei Qu, Jian Yang, Jiaheng Liu, Minghao Liu, Xiang Yue, Wenhao Huang, Ge Zhang

专题命中推理评测：reasoning(title,abstract)；self-correction(abstract)

2502.19868 2025-02-28 cs.CV 82%

C-Drag: Chain-of-Thought Driven Motion Controller for Video Generation

Yuhao Li, Mirana Claire Angel, Salman Khan, Yu Zhu, Jinqiu Sun, Yanning Zhang, Fahad Shahbaz Khan

专题命中推理评测：chain-of-thought(title,abstract)；reasoning(abstract)

2502.15854 2025-02-25 cs.LG cs.AI cs.CL 82%

Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models

Aryan Jadon, Avinash Patil, Shashank Kumar

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 8 Pages

2404.09077 2025-02-19 cs.CL cs.AI cs.IR cs.LG 82%

CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoning

Zukang Yang, Zixuan Zhu, Xuan Zhu

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted for publication in NAACL 2025. The official version will be available in the ACL Anthology

2411.06387 2025-02-07 cs.LG cs.AI cs.CL 82%

Self-Training Meets Consistency: Improving LLMs' Reasoning with Consistency-Driven Rationale Evaluation

Jaehyeok Lee, Keisuke Sakaguchi, JinYeong Bak

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to NAACL 2025

2501.06186 2025-01-13 cs.CV 82%

LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs

Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shah, Fahad Shahbaz Khan, Salman Khan

专题命中推理评测：reasoning(title,abstract)；CoT(abstract)

Comments 15 pages, 5 Figures

2305.13168 2024-12-30 cs.CL cs.AI cs.DB cs.IR cs.LG 82%

LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities

Yuqi Zhu, Xiaohan Wang, Jing Chen, Shuofei Qiao, Yixin Ou, Yunzhi Yao, Shumin Deng, Huajun Chen, Ningyu Zhang

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments World Wide Web Journal

2412.15547 2024-12-23 cs.CL cs.AI cs.LG 82%

NGQA: A Nutritional Graph Question Answering Benchmark for Personalized Health-aware Nutritional Reasoning

Zheyuan Zhang, Yiyang Li, Nhi Ha Lan Le, Zehong Wang, Tianyi Ma, Vincent Galassi, Keerthiram Murugesan, Nuno Moniz, Werner Geyer, Nitesh V Chawla, Chuxu Zhang, Yanfang Ye

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2412.09168 2024-12-13 cs.SD cs.CV cs.MM eess.AS 82%

YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls

Zihao Chen, Haomin Zhang, Xinhan Di, Haoyu Wang, Sizhe Shan, Junjie Zheng, Yunming Liang, Yihan Fan, Xinfa Zhu, Wenjie Tian, Yihua Wang, Chaofan Ding, Lei Xie

专题命中推理评测：chain-of-thought(title,abstract)；CoT(abstract)

Comments 16 pages, 4 figures

2412.03904 2024-12-06 cs.AI cs.CL cs.LG 82%

MISR: Measuring Instrumental Self-Reasoning in Frontier Models

Kai Fronsdal, David Lindner

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 10 pages, 65 page appendix, 5 figures

2407.05399 2024-11-27 cs.CL cs.AI cs.LG 82%

IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning

Abhinav Joshi, Shounak Paul, Akshat Sharma, Pawan Goyal, Saptarshi Ghosh, Ashutosh Modi

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ACL 2024 Main Conference; 40 Pages (9 Pages + References + Appendix)

2410.20178 2024-11-13 cs.AI cs.CL cs.CV cs.LG 82%

LLMs Can Evolve Continually on Modality for X-Modal Reasoning

Jiazuo Yu, Haomiao Xiong, Lu Zhang, Haiwen Diao, Yunzhi Zhuge, Lanqing Hong, Dong Wang, Huchuan Lu, You He, Long Chen

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2410.08126 2024-11-01 cs.LG cs.AI cs.CL 82%

Mars: Situated Inductive Reasoning in an Open-World Environment

Xiaojuan Tang, Jiaqi Li, Yitao Liang, Song-chun Zhu, Muhan Zhang, Zilong Zheng

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by NeurIPS 2024 Track Datasets and Benchmarks. Project page: https://marscrafter.github.io/

2410.03117 2024-10-07 cs.AI cs.CL cs.LG 82%

ProcBench: Benchmark for Multi-Step Reasoning and Following Procedure

Ippei Fujisawa, Sensho Nobe, Hiroki Seto, Rina Onda, Yoshiaki Uchida, Hiroki Ikoma, Pei-Chun Chien, Ryota Kanai

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2410.02763 2024-10-04 cs.CV cs.AI cs.CL cs.LG 82%

Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videos

Jianrui Zhang, Mu Cai, Yong Jae Lee

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Project Page: https://vinoground.github.io

2406.14425 2024-09-18 cs.CL cs.AI cs.LG 82%

SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages

Gayane Ghazaryan, Erik Arakelyan, Pasquale Minervini, Isabelle Augenstein

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2408.11879 2024-08-23 cs.CL cs.AI cs.LG 82%

Beyond Labels: Aligning Large Language Models with Human-like Reasoning

Muhammad Rafsan Kabir, Rafeed Mohammad Sultan, Ihsanul Haque Asif, Jawad Ibn Ahad, Fuad Rahman, Mohammad Ruhul Amin, Nabeel Mohammed, Shafin Rahman

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted in ICPR 2024

2305.00050 2024-08-21 cs.AI cs.CL cs.CY cs.HC cs.LG stat.ME 82%

Causal Reasoning and Large Language Models: Opening a New Frontier for Causality

Emre Kıcıman, Robert Ness, Amit Sharma, Chenhao Tan

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Added three novel datasets. To be published in TMLR. Authors listed alphabetically

2312.05562 2024-08-06 cs.SE 82%

Chain-of-Thought in Neural Code Generation: From and For Lightweight Language Models

Guang Yang, Yu Zhou, Xiang Chen, Xiangyu Zhang, Terry Yue Zhuo, Taolue Chen

专题命中推理评测：chain-of-thought(title)；reasoning(abstract)；CoT(abstract)

Comments ACCEPT IN TSE

2406.04229 2024-06-07 cs.LG cs.AI cs.CL cs.DS stat.ML 82%

The CLRS-Text Algorithmic Reasoning Language Benchmark

Larisa Markeeva, Sean McLeish, Borja Ibarz, Wilfried Bounsi, Olga Kozlova, Alex Vitvitskyi, Charles Blundell, Tom Goldstein, Avi Schwarzschild, Petar Veličković

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Preprint, under review. Comments welcome

2405.15690 2024-05-27 cs.SE 82%

A Case Study of LLM for Automated Vulnerability Repair: Assessing Impact of Reasoning and Patch Validation Feedback

Ummay Kulsum, Haotian Zhu, Bowen Xu, Marcelo d'Amorim

专题命中推理评测：reasoning(title,abstract)；chain-of-thought(abstract)

Comments Code, data and artifacts are available: http://tinyurl.com/vrpilot-artifacts

2403.13271 2024-03-21 cs.SE 82%

Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs

Zhihong Sun, Chen Lyu, Bolun Li, Yao Wan, Hongyu Zhang, Ge Li, Zhi Jin

专题命中推理评测：reasoning(title,abstract)；chain-of-thought(abstract)

Comments Accepted for LREC-COLING 2024

2312.14890 2024-02-13 cs.AI cs.CC cs.CL cs.LG 82%

NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes

Lizhou Fan, Wenyue Hua, Lingyao Li, Haoyang Ling, Yongfeng Zhang

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages, 7 figures, 2 tables

2401.10529 2024-01-26 cs.CV cs.AI cs.CL cs.LG 82%

Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences

Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jaehong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, Huaxiu Yao, Furong Huang

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages, 23 figures

2312.17532 2024-01-01 cs.CL cs.AI cs.LG 82%

Enhancing Quantitative Reasoning Skills of Large Language Models through Dimension Perception

Yuncheng Huang, Qianyu He, Jiaqing Liang, Sihang Jiang, Yanghua Xiao, Yunwen Chen

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted in the 40th IEEE International Conference on Data Engineering (ICDE 2024)

2312.14184 2023-12-25 cs.CL cs.AI cs.LG 82%

Large Language Models in Medical Term Classification and Unexpected Misalignment Between Response and Reasoning

Xiaodan Zhang, Sandeep Vemulapalli, Nabasmita Talukdar, Sumyeong Ahn, Jiankun Wang, Han Meng, Sardar Mehtab Bin Murtaza, Aakash Ajay Dave, Dmitry Leshchiner, Dimitri F. Joseph, Martin Witteveen-Lane, Dave Chesla, Jiayu Zhou, Bin Chen

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2311.11567 2023-12-06 cs.CV 82%

InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models

Xiaotian Han, Quanzeng You, Yongfei Liu, Wentao Chen, Huangjie Zheng, Khalil Mrini, Xudong Lin, Yiqi Wang, Bohan Zhai, Jianbo Yuan, Heng Wang, Hongxia Yang

专题命中推理评测：reasoning(title,abstract)；logical reasoning(abstract)

2308.00002 2023-11-17 cs.AI cs.CL cs.LG 82%

An Overview Of Temporal Commonsense Reasoning and Acquisition

Georg Wenzel, Adam Jatowt

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages, 7 figures, 6 tables

2310.06803 2023-10-11 cs.CL cs.AI cs.LG 82%

Advancing Transformer's Capabilities in Commonsense Reasoning

Yu Zhou, Yunqiu Han, Hanyu Zhou, Yulun Wu

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2306.02520 2023-07-18 cs.CL cs.AI cs.LG 82%

A Study of Situational Reasoning for Traffic Understanding

Jiarui Zhang, Filip Ilievski, Kaixin Ma, Aravinda Kollaa, Jonathan Francis, Alessandro Oltramari

专题命中推理评测：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 11 pages, 6 figures, 5 tables, camera ready version of SIGKDD 2023

1. 推理评测 10379 篇

KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Tasks

C-Drag: Chain-of-Thought Driven Motion Controller for Video Generation

Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models

CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoning

Self-Training Meets Consistency: Improving LLMs' Reasoning with Consistency-Driven Rationale Evaluation

LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs

LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities

NGQA: A Nutritional Graph Question Answering Benchmark for Personalized Health-aware Nutritional Reasoning

YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls

MISR: Measuring Instrumental Self-Reasoning in Frontier Models

IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning

LLMs Can Evolve Continually on Modality for X-Modal Reasoning

Mars: Situated Inductive Reasoning in an Open-World Environment

ProcBench: Benchmark for Multi-Step Reasoning and Following Procedure

Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videos

SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages

Beyond Labels: Aligning Large Language Models with Human-like Reasoning

Causal Reasoning and Large Language Models: Opening a New Frontier for Causality

Chain-of-Thought in Neural Code Generation: From and For Lightweight Language Models

The CLRS-Text Algorithmic Reasoning Language Benchmark

A Case Study of LLM for Automated Vulnerability Repair: Assessing Impact of Reasoning and Patch Validation Feedback

Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs

NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes

Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences

Enhancing Quantitative Reasoning Skills of Large Language Models through Dimension Perception

Large Language Models in Medical Term Classification and Unexpected Misalignment Between Response and Reasoning

InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models

An Overview Of Temporal Commonsense Reasoning and Acquisition

Advancing Transformer's Capabilities in Commonsense Reasoning

A Study of Situational Reasoning for Traffic Understanding