大模型推理能力

MME-Finance: A Multimodal Finance Benchmark for Expert-level Understanding and Reasoning

Ziliang Gan, Yu Lu, Dong Zhang, Haohan Li, Che Liu, Jian Liu, Ji Liu, Haipang Wu, Chaoyou Fu, Zenglin Xu, Rongjunchen Zhang, Yong Dai

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Project Page: https://hithink-research.github.io/MME-Finance/

Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning

Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Jingdong Sun, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, Alexander Hauptmann

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments Accepted at NeurIPS 2024. 49 pages, 13 figures, Project: https://github.com/ZebangCheng/Emotion-LLaMA, Demo: https://huggingface.co/spaces/ZebangCheng/Emotion-LLaMA

LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages

Andrew M. Bean, Simi Hellsten, Harry Mayne, Jabez Magomere, Ethan A. Chi, Ryan Chi, Scott A. Hale, Hannah Rose Kirk

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Oral presentation at NeurIPS 2024 Datasets and Benchmarks Track. 10 pages, 5 figures, 22 pages supplemental materials

GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning

Yanbin Wei, Shuai Fu, Weisen Jiang, Zejian Zhang, Zhixiong Zeng, Qi Wu, James T. Kwok, Yu Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments NeurIPS 2024; Project Page: v-graph.github.io; Code: https://github.com/WEIYanbin1999/GITA/

CLR-Bench: Evaluating Large Language Models in College-level Reasoning

Junnan Dong, Zijin Hong, Yuanchen Bei, Feiran Huang, Xinrun Wang, Xiao Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 18 pages, 6 figures, dataset and evaluation framework will be opensourced

Message Intercommunication for Inductive Relation Reasoning

Ke Liang, Lingyuan Meng, Sihang Zhou, Siwei Wang, Wenxuan Tu, Yue Liu, Meng Liu, Xinwang Liu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments This work has been submitted to the IEEE for possible publication

Selective Vision is the Challenge for Visual Reasoning: A Benchmark for Visual Argument Understanding

Jiwan Chung, Sungjae Lee, Minseo Kim, Seungju Han, Ashkan Yousefpour, Jack Hessel, Youngjae Yu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 12 pages, 6 figures. Accepted as main paper in EMNLP 2024

Self-Contradictory Reasoning Evaluation and Detection

Ziyi Liu, Soumya Sanyal, Isabelle Lee, Yongkang Du, Rahul Gupta, Yang Liu, Jieyu Zhao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Belief Revision: The Adaptability of Large Language Models Reasoning

Bryan Wilie, Samuel Cahyawijaya, Etsuko Ishii, Junxian He, Pascale Fung

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Adaptive Reasoning and Acting in Medical Language Agents

Abhishek Dutta, Yen-Che Hsiao

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Evaluating Multilingual Long-Context Models for Retrieval and Reasoning

Ameeta Agrawal, Andy Dang, Sina Bagheri Nezhad, Rhitabrat Pokharel, Russell Scheinberg

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments To appear at MRL 2024

Can LLM Graph Reasoning Generalize beyond Pattern Memorization?

Yizhuo Zhang, Heng Wang, Shangbin Feng, Zhaoxuan Tan, Xiaochuang Han, Tianxing He, Yulia Tsvetkov

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 17 pages, 6 figures. EMNLP 2024 Findings. Code and data is publicly available at https://github.com/MatthewYZhang/NLGift

PizzaCommonSense: Learning to Model Commonsense Reasoning about Intermediate Steps in Cooking Recipes

Aissatou Diallo, Antonis Bikakis, Luke Dickens, Anthony Hunter, Rob Miller

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Findings of EMNLP 2024. The data is available at: https://github.com/adiallo07/PizzaCommonsense

Temporal Reasoning Transfer from Text to Video

Lei Li, Yuanxin Liu, Linli Yao, Peiyuan Zhang, Chenxin An, Lean Wang, Xu Sun, Lingpeng Kong, Qi Liu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Project page: https://video-t3.github.io

Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language Models

Philipp Mondorf, Barbara Plank

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments EMNLP 2024 main, 24 pages, 16 figures

Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?

Vagrant Gautam, Eileen Bingert, Dawei Zhu, Anne Lauscher, Dietrich Klakow

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Transactions of the Association for Computational Linguistics (presented at EMNLP 2024)

Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMs

Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, Enamul Hoque

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning

Tianxiang Hu, Pei Zhang, Baosong Yang, Jun Xie, Derek F. Wong, Rui Wang

专题命中 推理评测 ：CoT(title,abstract)；分类 cs.CL

Why context matters in VQA and Reasoning: Semantic interventions for VLM input modalities

Kenza Amara, Lukas Klein, Carsten Lüth, Paul Jäger, Hendrik Strobelt, Mennatallah El-Assady

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Efficient Driving Behavior Narration and Reasoning on Edge Device Using Large Language Models

Yizhou Huang, Yihua Cheng, Kezhi Wang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments Submitted for possible journal publication

What Are the Odds? Language Models Are Capable of Probabilistic Reasoning

Akshay Paruchuri, Jake Garrison, Shun Liao, John Hernandez, Jacob Sunshine, Tim Althoff, Xin Liu, Daniel McDuff

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments EMNLP 2024 (Main), 21 pages, 9 figures, 2 tables

A Joint-Reasoning based Disease Q&A System

Prakash Chandra Sukhwal, Vaibhav Rajan, Atreyi Kankanhalli

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 36 pages, 6 figures

Large Language Models are Limited in Out-of-Context Knowledge Reasoning

Peng Hu, Changjiang Gao, Ruiqi Gao, Jiajun Chen, Shujian Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Graph Reasoning with Large Language Models via Pseudo-code Prompting

Konstantinos Skianis, Giannis Nikolentzos, Michalis Vazirgiannis

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

EconLogicQA: A Question-Answering Benchmark for Evaluating Large Language Models in Economic Sequential Reasoning

Yinzhu Quan, Zefang Liu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

CKBP v2: Better Annotation and Reasoning for Commonsense Knowledge Base Population

Tianqing Fang, Quyet V. Do, Zihao Zheng, Weiqi Wang, Sehyun Choi, Zhaowei Wang, Yangqiu Song

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Profiling Patient Transcript Using Large Language Model Reasoning Augmentation for Alzheimer's Disease Detection

Chin-Po Chen, Jeng-Lin Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments accepted to EMBC 2024

Mothman at SemEval-2024 Task 9: An Iterative System for Chain-of-Thought Prompt Optimization

Alvin Po-Chun Chen, Ray Groshan, Sean von Bayern

专题命中 推理评测 ：chain-of-thought(title,abstract)；分类 cs.CL

Comments 13 pages, 2 figures, to be published in Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)

Can ChatGPT Make Explanatory Inferences? Benchmarks for Abductive Reasoning

Paul Thagard

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Linguini: A benchmark for language-agnostic linguistic reasoning

Eduardo Sánchez, Belen Alastruey, Christophe Ropers, Pontus Stenetorp, Mikel Artetxe, Marta R. Costa-jussà

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL