大模型推理能力

Intellecta Cognitiva: A Comprehensive Dataset for Advancing Academic Knowledge and Machine Reasoning

Ajmal PS, Ditto PS, Jithin VG

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learning

Jingyuan Selena She, Christopher Potts, Samuel R. Bowman, Atticus Geiger

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answering

Nirmalie Wiratunga, Ramitha Abeyratne, Lasal Jayawardena, Kyle Martin, Stewart Massie, Ikechukwu Nkisi-Orji, Ruvan Weerasinghe, Anne Liret, Bruno Fleisch

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Submitted to ICCBR'24

Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?

Bangzheng Li, Ben Zhou, Fei Wang, Xingyu Fu, Dan Roth, Muhao Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Work accepted by NAACL 2024

TaCo: Enhancing Cross-Lingual Transfer for Low-Resource Languages in LLMs through Translation-Assisted Chain-of-Thought Processes

Bibek Upadhayay, Vahid Behzadan

专题命中 推理评测 ：chain-of-thought(title,abstract)；分类 cs.CL、cs.AI

Enhance Reasoning for Large Language Models in the Game Werewolf

Shuang Wu, Liwen Zhu, Tao Yang, Shiwei Xu, Qiang Fu, Yang Wei, Haobo Fu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks

Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, Yoon Kim

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments NAACL 2024

Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification

Shan Chen, Yingya Li, Sheng Lu, Hoang Van, Hugo JWL Aerts, Guergana K. Savova, Danielle S. Bitterman

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 28 pages, 2 tables and 4 figures. Submitting for review

BaRDa: A Belief and Reasoning Dataset that Separates Factual Accuracy and Reasoning Ability

Peter Clark, Bhavana Dalvi Mishra, Oyvind Tafjord

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Added note about how dataset sampling was performed

ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification

Sehee Lim, Yejin Kim, Chi-Hyun Choi, Jy-yong Sohn, Byung-Hoon Kim

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Surfer: Progressive Reasoning with World Models for Robotic Manipulation

Pengzhen Ren, Kaidong Zhang, Hetao Zheng, Zixuan Li, Yuhang Wen, Fengda Zhu, Mas Ma, Xiaodan Liang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap

Saurabh Srivastava, Annarose M B, Anto P, Shashank Menon, Ajay Sukumar, Adwaith Samod T, Alan Philipose, Stevin Prince, Sooraj Thomas

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 37 pages, 10 figures

RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning

Congyun Jin, Ming Zhang, Xiaowei Ma, Li Yujiao, Yingbo Wang, Yabo Jia, Yuliang Du, Tao Sun, Haowen Wang, Cong Fan, Jinjie Gu, Chenfei Chi, Xiangguo Lv, Fangzhou Li, Wei Xue, Yiran Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 13 figures

Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables

Haisong Gong, Weizhi Xu, Shu wu, Qiang Liu, Liang Wang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by 38th Association for the Advancement of Artificial Intelligence, AAAI

Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning

Yiqi Wang, Wentao Chen, Xiaotian Han, Xudong Lin, Haiteng Zhao, Yongfei Liu, Bohan Zhai, Jianbo Yuan, Quanzeng You, Hongxia Yang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

DCR-Consistency: Divide-Conquer-Reasoning for Consistency Evaluation and Improvement of Large Language Models

Wendi Cui, Jiaxin Zhang, Zhuohang Li, Lopez Damien, Kamalika Das, Bradley Malin, Sricharan Kumar

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Models

Yuqing Wang, Yun Zhao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Data and results are available at: https://github.com/EternityYW/Gemini-Commonsense-Evaluation/

Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks

Melanie Mitchell, Alessandro B. Palmarini, Arseny Moskvichev

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Corrected Figure 3 (extra spaces were replaced by commas, which were lost in original formatting)

Journal ref Proceedings of the LLM-CP Workshop, AAAI 2024

Does Visual Pretraining Help End-to-End Reasoning?

Chen Sun, Calvin Luo, Xingyi Zhou, Anurag Arnab, Cordelia Schmid

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2023

Understanding Social Reasoning in Language Models with Language Models

Kanishk Gandhi, Jan-Philipp Fränken, Tobias Gerstenberg, Noah D. Goodman

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

WorldSense: A Synthetic Benchmark for Grounded Reasoning in Large Language Models

Youssef Benchekroun, Megi Dervishi, Mark Ibrahim, Jean-Baptiste Gaya, Xavier Martinet, Grégoire Mialon, Thomas Scialom, Emmanuel Dupoux, Dieuwke Hupkes, Pascal Vincent

专题命中 推理评测 ：reasoning(title)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms

Seungju Han, Junhyeok Kim, Jack Hessel, Liwei Jiang, Jiwan Chung, Yejin Son, Yejin Choi, Youngjae Yu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at EMNLP 2023 (long)

Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning

Sai Munikoti, Anurag Acharya, Sridevi Wagle, Sameera Horawalavithana

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 5 pages

Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Data

Mubashara Akhtar, Abhilash Shankarampeta, Vivek Gupta, Arpit Patil, Oana Cocarascu, Elena Simperl

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at EMNLP 2023 (Findings)

ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sense

Kankan Zhou, Eason Lai, Wei Bin Au Yeong, Kyriakos Mouratidis, Jing Jiang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments This is the camera-ready version of the paper that will be published in the EMNLP 2023 Findings (Singapore, 6-10 December 2023)

HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models

Yinghui He, Yufan Wu, Yilin Jia, Rada Mihalcea, Yulong Chen, Naihao Deng

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at Findings of EMNLP 2023

Journal ref Findings of EMNLP 2023

Reasoning about Ambiguous Definite Descriptions

Stefan F. Schouten, Peter Bloem, Ilia Markov, Piek Vossen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2023 Findings

BioPlanner: Automatic Evaluation of LLMs on Protocol Planning in Biology

Odhran O'Donoghue, Aleksandar Shtedritski, John Ginger, Ralph Abboud, Ali Essa Ghareeb, Justin Booth, Samuel G Rodriques

专题命中 推理评测 ：planning(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2023. Dataset and code: https://github.com/bioplanner/bioplanner

How to Enhance Causal Discrimination of Utterances: A Case on Affective Reasoning

Hang Chen, Jing Luo, Xinyu Yang, Wenjing Zhu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments accepted via EMNLP2023-main

NEWTON: Are Large Language Models Capable of Physical Reasoning?

Yi Ru Wang, Jiafei Duan, Dieter Fox, Siddhartha Srinivasa

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2023 Findings; 8 pages, 3 figures, 7 tables; Project page: https://newtonreasoning.github.io