大模型推理能力

ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning

Hosung Lee, Sejin Kim, Seungpil Lee, Sanha Hwang, Jihwan Lee, Byung-Jun Lee, Sundong Kim

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by CoLLAs 2024, Project page: https://github.com/confeitoHS/arcle

Steamroller Problems: An Evaluation of LLM Reasoning Capability with Automated Theorem Prover Strategies

Lachlan McGinness, Peter Baumgartner

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Look Globally and Reason: Two-stage Path Reasoning over Sparse Knowledge Graphs

Saiping Guan, Jiyao Wei, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted to CIKM 2024

HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning

Zhecan Wang, Garrett Bingham, Adams Yu, Quoc Le, Thang Luong, Golnaz Ghiasi

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted as a main conference paper at ECCV 2024 (https://github.com/google/haloquest)

RDBE: Reasoning Distillation-Based Evaluation Enhances Automatic Essay Scoring

Ali Ghiasvand Mohammadkhani

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Exploring State Space and Reasoning by Elimination in Tsetlin Machines

Ahmed K. Kadhim, Ole-Christoffer Granmo, Lei Jiao, Rishad Shafik

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 8 figures

A Critical Review of Causal Reasoning Benchmarks for Large Language Models

Linying Yang, Vik Shirvaikar, Oscar Clivio, Fabian Falck

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments AAAI 2024 Workshop on ''Are Large Language Models Simply Causal Parrots?''

SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning

Bin Wang, Zhengyuan Liu, Xin Huang, Fangkai Jiao, Yang Ding, AiTi Aw, Nancy F. Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Published at NAACL 2024. Code: https://seaeval.github.io/

Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models

Mosh Levy, Alon Jacoby, Yoav Goldberg

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2024

RVISA: Reasoning and Verification for Implicit Sentiment Analysis

Wenna Lai, Haoran Xie, Guandong Xu, Qing Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 6 figures, and 4 tables

Fake News Detection and Manipulation Reasoning via Large Vision-Language Models

Ruihan Jin, Ruibo Fu, Zhengqi Wen, Shuai Zhang, Yukun Liu, Jianhua Tao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language Models by Learning from Knowledge Graphs

Yifei Zhang, Xintao Wang, Jiaqing Liang, Sirui Xia, Lida Chen, Yanghua Xiao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models

Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang, Ming Liu, Bing Qin

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2024

Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models

Wenxuan Ding, Shangbin Feng, Yuhan Liu, Zhaoxuan Tan, Vidhisha Balachandran, Tianxing He, Yulia Tsvetkov

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reasoning Like a Doctor: Improving Medical Dialogue Systems via Diagnostic Reasoning Process Alignment

Kaishuai Xu, Yi Cheng, Wenjun Hou, Qiaoyu Tan, Wenjie Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 Findings

Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning

Bingchen Zhao, Yongshuo Zong, Letian Zhang, Timothy Hospedales

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments First three authors contributed equally. Dataset: https://huggingface.co/datasets/VLLMs/MIRB

MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, Wenhu Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments CVPR 2024 Oral

Eliciting Better Multilingual Structured Reasoning from LLMs through Code

Bryan Li, Tamer Alkhouli, Daniele Bonadiman, Nikolaos Pappas, Saab Mansour

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Multimodal Reasoning with Multimodal Knowledge Graph

Junlin Lee, Yequan Wang, Jing Li, Min Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 (Main Conference)

Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation

Chengwei Dai, Kun Li, Wei Zhou, Songlin Hu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Understanding Transformer Reasoning Capabilities via Graph Algorithms

Clayton Sanford, Bahare Fatemi, Ethan Hall, Anton Tsitsulin, Mehran Kazemi, Jonathan Halcrow, Bryan Perozzi, Vahab Mirrokni

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments 43 pages, 8 figures

MindMerger: Efficient Boosting LLM Reasoning in non-English Languages

Zixian Huang, Wenhao Zhu, Gong Cheng, Lei Li, Fei Yuan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning

Fangjun Li, David C. Hogg, Anthony G. Cohn

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Camera-Ready version for IJCAI 2024

SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge

Andong Wang, Bo Wu, Sunli Chen, Zhenfang Chen, Haotian Guan, Wei-Ning Lee, Li Erran Li, Chuang Gan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments CVPR

Humor Mechanics: Advancing Humor Generation with Multistep Reasoning

Alexey Tikhonov, Pavel Shtykovskiy

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments ICCC 2024

G-SAP: Graph-based Structure-Aware Prompt Learning over Heterogeneous Knowledge for Commonsense Reasoning

Ruiting Dai, Yuqiao Tan, Lisi Mo, Shuang Liang, Guohao Huo, Jiayi Luo, Yao Cheng

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoning

Zheqi He, Xinya Wu, Pengfei Zhou, Richeng Xuan, Guang Liu, Xi Yang, Qiannan Zhu, Hua Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Can't say cant? Measuring and Reasoning of Dark Jargons in Large Language Models

Xu Ji, Jianyi Zhang, Ziyin Zhou, Zhangchi Zhao, Qianqian Qiao, Kaiying Han, Md Imran Hossen, Xiali Hei

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Improving Language Model Reasoning with Self-motivated Learning

Yunlong Feng, Yang Xu, Libo Qin, Yasheng Wang, Wanxiang Che

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at LREC-COLING 2024

Evaluating Consistency and Reasoning Capabilities of Large Language Models

Yash Saxena, Sarthak Chopra, Arunendra Mani Tripathi

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI