大模型推理能力

MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Models

Yifan Wei, Yisong Su, Huanhuan Ma, Xiaoyan Yu, Fangyu Lei, Yuanzhe Zhang, Jun Zhao, Kang Liu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2023 Findings

Back to the Future: Towards Explainable Temporal Reasoning with Large Language Models

Chenhan Yuan, Qianqian Xie, Jimin Huang, Sophia Ananiadou

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 5 figures, code and dataset: https://github.com/chenhan97/TimeLlama

GPT-Neo for commonsense reasoning -- a theoretical and practical lens

Rohan Kashyap, Vivek Kashyap, Narendra C. P.

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning

Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, Asli Celikyilmaz

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models

Jaemin Cho, Abhay Zala, Mohit Bansal

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments ICCV 2023 (34 pages; see appendix for version changelog)

Counterfactual Reasoning for Bias Evaluation and Detection in a Fairness under Unawareness setting

Giandomenico Cornacchia, Vito Walter Anelli, Fedelucio Narducci, Azzurra Ragone, Eugenio Di Sciascio

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models

Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessica Wu, Joe Nudell, Joel Niklaus, John Nay, Jonathan H. Choi, Kevin Tobia, Margaret Hagan, Megan Ma, Michael Livermore, Nikon Rasumov-Rahe, Nils Holzenberger, Noam Kolt, Peter Henderson, Sean Rehaag, Sharad Goel, Shang Gao, Spencer Williams, Sunny Gandhi, Tom Zur, Varun Iyer, Zehua Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 143 pages, 79 tables, 4 figures

Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine

Thomas Savage, Ashwin Nayak, Robert Gallo, Ekanath Rangan, Jonathan H Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

NovPhy: A Testbed for Physical Reasoning in Open-world Environments

Chathura Gamage, Vimukthini Pinto, Cheng Xue, Peng Zhang, Ekaterina Nikonova, Matthew Stephenson, Jochen Renz

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Testbed website: https://github.com/phy-q/novphy

ARB: Advanced Reasoning Benchmark for Large Language Models

Tomohiro Sawada, Daniel Paleka, Alexander Havrilla, Pranav Tadepalli, Paula Vidas, Alexander Kranias, John J. Nay, Kshitij Gupta, Aran Komatsuzaki

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Submitted to NeurIPS Datasets and Benchmarks Track

Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models

Qingyu Tan, Hwee Tou Ng, Lidong Bing

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2023

Knowledge Graph-Augmented Korean Generative Commonsense Reasoning

Dahyun Jung, Jaehyung Seo, Jaewook Lee, Chanjun Park, Heuiseok Lim

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted for Data-centric Machine Learning Research (DMLR) Workshop at ICML 2023

Emotion Flip Reasoning in Multiparty Conversations

Shivani Kumar, Shubham Dudeja, Md Shad Akhtar, Tanmoy Chakraborty

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Paper accepted in IEEE Transaction on AI. 12 pages, 5 figures, 11 tables

Relational Temporal Graph Reasoning for Dual-task Dialogue Language Understanding

Bowen Xing, Ivor W. Tsang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE TPAMI). arXiv admin note: substantial text overlap with arXiv:2203.03856

Multi-Task Training with In-Domain Language Models for Diagnostic Reasoning

Brihat Sharma, Yanjun Gao, Timothy Miller, Matthew M. Churpek, Majid Afshar, Dmitriy Dligach

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to the Proceedings of the 5th Clinical NLP Workshop at ACL

Evaluating the Capabilities of Multi-modal Reasoning Models with Synthetic Task Data

Nathan Vaska, Victoria Helus

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models

Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, Dongkuan Xu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions

Mo Yu, Yi Gu, Xiaoxiao Guo, Yufei Feng, Xiaodan Zhu, Michael Greenspan, Murray Campbell, Chuang Gan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments arXiv admin note: text overlap with arXiv:2010.09788

Semantic-aware Dynamic Retrospective-Prospective Reasoning for Event-level Video Question Answering

Chenyang Lyu, Tianbo Ji, Yvette Graham, Jennifer Foster

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Prompt Engineering and Calibration for Zero-Shot Commonsense Reasoning

Chenkai Ma

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments To be published in the ICLR TinyPaper track

Multi-Viewpoint and Multi-Evaluation with Felicitous Inductive Bias Boost Machine Abstract Reasoning Ability

Qinglai Wei, Diancheng Chen, Beiming Yuan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Impact of Pretraining Term Frequencies on Few-Shot Reasoning

Yasaman Razeghi, Robert L. Logan, Matt Gardner, Sameer Singh

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Lila: A Unified Benchmark for Mathematical Reasoning

Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, Ashwin Kalyan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2022

Implicit Temporal Reasoning for Evidence-Based Fact-Checking

Liesbeth Allein, Marlon Saelens, Ruben Cartuyvels, Marie-Francine Moens

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments The 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023, Findings)

STREET: A Multi-Task Structured Reasoning and Explanation Benchmark

Danilo Ribeiro, Shen Wang, Xiaofei Ma, Henry Zhu, Rui Dong, Deguang Kong, Juliette Burger, Anjelica Ramos, William Wang, Zhiheng Huang, George Karypis, Bing Xiang, Dan Roth

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Published in ICLR 2023

DR.BENCH: Diagnostic Reasoning Benchmark for Clinical Natural Language Processing

Yanjun Gao, Dmitriy Dligach, Timothy Miller, John Caskey, Brihat Sharma, Matthew M Churpek, Majid Afshar

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Under review

Phy-Q as a measure for physical reasoning intelligence

Cheng Xue, Vimukthini Pinto, Chathura Gamage, Ekaterina Nikonova, Peng Zhang, Jochen Renz

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments For the associated website, see https://github.com/phy-q/benchmark

Utilizing Background Knowledge for Robust Reasoning over Traffic Situations

Jiarui Zhang, Filip Ilievski, Aravinda Kollaa, Jonathan Francis, Kaixin Ma, Alessandro Oltramari

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Camera ready version of AAAI 2023 workshop on Knowledge-Augmented Methods for Natural Language Processing

TRAC: A Textual Benchmark for Reasoning about Actions and Change

Weinan He, Canming Huang, Zhanhao Xiao, Yongmei Liu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Template Filling for Controllable Commonsense Reasoning

Dheeraj Rajagopal, Vivek Khetan, Bogdan Sacaleanu, Anatole Gershman, Andrew Fano, Eduard Hovy

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI