大模型推理能力

SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluation

Hangfeng He, Hongming Zhang, Dan Roth

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor

Jiali Chen, Xusen Hei, Yuqi Xue, Yuancheng Wei, Jiayuan Xie, Yi Cai, Qing Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACM MM 2024

VisScience: An Extensive Benchmark for Evaluating K12 Educational Multi-modal Scientific Reasoning

Zhihuan Jiang, Zhen Yang, Jinhao Chen, Zhengxiao Du, Weihan Wang, Bin Xu, Jie Tang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 89 pages, 70 figures

Inducing Human-like Biases in Moral Reasoning Language Models

Artem Karpov, Seong Hah Cho, Austin Meek, Raymond Koopmanschap, Lucy Farnik, Bogdan-Ionut Cirstea

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted to the 2nd Workshop on Unifying Representations in Neural Models (UniReps) at NeurIPS 2024

Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpus

Seungpil Lee, Woochang Sim, Donghyeon Shin, Wongyu Seo, Jiwon Park, Seokki Lee, Sanha Hwang, Sejin Kim, Sundong Kim

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Evaluating and Enhancing Large Language Models for Conversational Reasoning on Knowledge Graphs

Yuxuan Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance

Kai Xiong, Xiao Ding, Ting Liu, Bing Qin, Dongliang Xu, Qing Yang, Hongtao Liu, Yixin Cao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack

Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, Mikhail Burtsev

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024 Datasets and Benchmarks Track

JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking

Tong Niu, Shafiq Joty, Ye Liu, Caiming Xiong, Yingbo Zhou, Semih Yavuz

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?

Neeladri Bhuiya, Viktor Schlegel, Stefan Winkler

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 3 figures, EMNLP 2024 Main Conference

AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoning

Shirley Wu, Shiyu Zhao, Qian Huang, Kexin Huang, Michihiro Yasunaga, Kaidi Cao, Vassilis N. Ioannidis, Karthik Subbian, Jure Leskovec, James Zou

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2024 main conference

MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark

S Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, Dinesh Manocha

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Project Website: https://sakshi113.github.io/mmau_homepage/

StrucText-Eval: Evaluating Large Language Model's Reasoning Ability in Structure-Rich Text

Zhouhong Gu, Haoning Ye, Xingzhou Chen, Zeyang Zhou, Hongwei Feng, Yanghua Xiao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

LAR-ECHR: A New Legal Argument Reasoning Task and Dataset for Cases of the European Court of Human Rights

Odysseas S. Chlapanis, Dimitrios Galanis, Ion Androutsopoulos

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Published in Natural Legal Language Processing (NLLP) 2024 workshop

A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning

Yuanning Cui, Zequn Sun, Wei Hu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities

Lichang Chen, Hexiang Hu, Mingda Zhang, Yiwen Chen, Zifeng Wang, Yandong Li, Pranav Shyam, Tianyi Zhou, Heng Huang, Ming-Hsuan Yang, Boqing Gong

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 6 figures, 12 tables

Concept-Reversed Winograd Schema Challenge: Evaluating and Improving Robust Reasoning in Large Language Models via Abstraction

Kaiqiao Han, Tianqing Fang, Zhaowei Wang, Yangqiu Song, Mark Steedman

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

MMLU-Pro+: Evaluating Higher-Order Reasoning and Shortcut Learning in LLMs

Saeid Asgari Taghanaki, Aliasgahr Khani, Amir Khasahmadi

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to NeurIPS 2024, Safe Generative AI

Zero-shot Commonsense Reasoning over Machine Imagination

Hyuntae Park, Yeachan Kim, Jun-Hyung Park, SangKeun Lee

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 9 figures, EMNLP 2024 (Findings)

Connecting the Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using the New York Times Connections Word Game

Prisha Samadarshi, Mariam Mustafa, Anushka Kulkarni, Raven Rothkopf, Tuhin Chakrabarty, Smaranda Muresan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning

Jiachun Li, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Jun Zhao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2024 Findings

GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs

Navid Rajabi, Jana Kosecka

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to NeurIPS 2024 Workshop on Compositional Learning

ReFeR: Improving Evaluation and Reasoning through Hierarchy of Models

Yaswanth Narsupalli, Abhranil Chandra, Sreevatsa Muppirala, Manish Gupta, Pawan Goyal

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Paper Under Review

Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs

Zifeng Ding, Jingcheng Wu, Jingpei Wu, Yan Xia, Volker Tresp

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Findings of EMNLP 2024

A Case Study of Web App Coding with OpenAI Reasoning Models

Yi Cui

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning Factual Knowledge in Structured Data with Large Language Models

Sirui Huang, Yanggan Gu, Xuming Hu, Zhonghao Li, Qing Li, Guandong Xu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

New Curriculum, New Chance -- Retrieval Augmented Generation for Lesson Planning in Ugandan Secondary Schools. Prototype Quality Evaluation

Simon Kloker, Herbertson Bukoli, Twaha Kateete

专题命中 推理评测 ：planning(title,abstract)；分类 cs.AI、cs.LG

Comments Presented at Ndejje University Second Annual Research Dissemination Symposium 2024

PLUGH: A Benchmark for Spatial Understanding and Reasoning in Large Language Models

Alexey Tikhonov

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Wordplay Workshop @ ACL 2024

Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought

Zaijing Li, Gongwei Chen, Rui Shao, Yuquan Xie, Dongmei Jiang, Liqiang Nie

专题命中 推理评测 ：chain-of-thought(title,abstract)；分类 cs.CL、cs.AI

A Comprehensive Evaluation on Event Reasoning of Large Language Models

Zhengwei Tao, Zhi Jin, Yifan Zhang, Xiancai Chen, Haiyan Zhao, Jia Li, Bing Liang, Chongyang Tao, Qun Liu, Kam-Fai Wong

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI