大模型推理能力 - arXivDaily 专题

2501.15595 2025-01-28 cs.CV 67%

SedarEval: Automated Evaluation using Self-Adaptive Rubrics

Zhiyuan Fan, Weinong Wang, Xing Wu, Debing Zhang

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)

Journal ref EMNLP2024

2501.07214 2025-01-14 cs.CV 67%

TimeLogic: A Temporal Logic Benchmark for Video QA

Sirnam Swetha, Hilde Kuehne, Mubarak Shah

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)

2312.03700 2025-01-10 cs.CV cs.AI cs.CL cs.LG cs.MM 67%

OneLLM: One Framework to Align All Modalities with Language

Jiaming Han, Kaixiong Gong, Yiyuan Zhang, Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, Xiangyu Yue

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by CVPR 2024. Code: https://github.com/csuhan/OneLLM

2501.04675 2025-01-09 cs.CL cs.AI cs.CV cs.LG 67%

Enhancing Financial VQA in Vision Language Models using Intermediate Structured Representations

Archita Srivastava, Abhas Kumar, Rajesh Kumar, Prabhakar Srinivasan

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2412.19260 2025-01-03 cs.CL cs.AI cs.LG 67%

MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes

Asma Ben Abacha, Wen-wai Yim, Yujuan Fu, Zhaoyi Sun, Meliha Yetisgen, Fei Xia, Thomas Lin

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This version has been updated with further clarification regarding the model size estimates that were mined from public articles only and provided to aid in contextualizing model performance. The authors cannot vouch for the accuracy of those estimates

2412.17754 2024-12-30 cs.SE 67%

ADC: Enhancing Function Calling Via Adversarial Datasets and Code Line-Level Feedback

Wei Zhang, Yi Zhang, Li Zhu, Qianghuai Jia, Feijun Jiang, Hongcheng Guo, Zhoujun Li, Mengping Zhou

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)

2310.13008 2024-12-20 cs.LG cs.AI cs.CL 67%

DavIR: Data Selection via Implicit Reward for Large Language Models

Haotian Zhou, Tingkai Liu, Qianli Ma, Yufeng Zhang, Jianbo Yuan, Pengfei Liu, Yang You, Hongxia Yang

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2412.04903 2024-12-17 cs.CV cs.AI cs.CL cs.LG 67%

EACO: Enhancing Alignment in Multimodal LLMs via Critical Observation

Yongxin Wang, Meng Cao, Haokun Lin, Mingfei Han, Liang Ma, Jin Jiang, Yuhao Cheng, Xiaodan Liang

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 19 pages

2410.16780 2024-12-11 cs.CL cs.AI cs.IR cs.LG 67%

Beyond Retrieval: Generating Narratives in Conversational Recommender Systems

Krishna Sayana, Raghavendra Vasudeva, Yuri Vasilevski, Kun Su, Liam Hebert, James Pine, Hubert Pham, Ambarish Jash, Sukhdeep Sodhi

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2412.02906 2024-12-05 cs.SE cs.AI cs.CL cs.LG 67%

Does Few-Shot Learning Help LLM Performance in Code Synthesis?

Derek Xu, Tong Xie, Botao Xia, Haoyu Li, Yunsheng Bai, Yizhou Sun, Wei Wang

专题命中推理评测：chain-of-thought(abstract)；分类 cs.CL、cs.AI、cs.LG

2412.01333 2024-12-03 cs.SE 67%

Can Large Language Models Serve as Evaluators for Code Summarization?

Yang Wu, Yao Wan, Zhaoyang Chu, Wenting Zhao, Ye Liu, Hongyu Zhang, Xuanhua Shi, Philip S. Yu

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)

2308.02490 2024-12-03 cs.AI cs.CL cs.CV cs.LG 67%

MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities

Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, Lijuan Wang

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2024. Code, data and leaderboard: https://github.com/yuweihao/MM-Vet

2403.18802 2024-11-08 cs.CL cs.AI cs.LG 67%

Long-form factuality in large language models

Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Jie Huang, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, Quoc V. Le

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024; 72 pages, 18 figures, 30 tables. Code at https://github.com/google-deepmind/long-form-factuality

2410.08081 2024-11-07 cs.LG cs.AI cs.CL 67%

Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning

Shuhe Wang, Guoyin Wang, Yizhong Wang, Jiwei Li, Eduard Hovy, Chen Guo

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2411.02810 2024-11-06 cs.CE cs.IR 67%

Leveraging Vision-Language Models for Manufacturing Feature Recognition in CAD Designs

Muhammad Tayyab Khan, Lequn Chen, Ye Han Ng, Wenhe Feng, Nicholas Yew Jin Tan, Seung Ki Moon

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)

Comments Paper has been submitted to The ASME Journal of Computing and Information Science in Engineering (JCISE)

2408.16725 2024-11-06 cs.AI cs.CL cs.HC cs.LG cs.SD eess.AS 67%

Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming

Zhifei Xie, Changqiao Wu

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Technical report, work in progress. Demo and code: https://github.com/gpt-omni/mini-omni

2405.00972 2024-10-30 cs.CL cs.AI cs.LG physics.chem-ph q-bio.QM 67%

CACTUS: Chemistry Agent Connecting Tool-Usage to Science

Andrew D. McNaughton, Gautham Ramalaxmi, Agustin Kruel, Carter R. Knutson, Rohith A. Varikoti, Neeraj Kumar

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2312.01678 2024-10-30 cs.AI cs.CL cs.DB cs.LG 67%

Jellyfish: A Large Language Model for Data Preprocessing

Haochen Zhang, Yuyang Dong, Chuan Xiao, Masafumi Oyamada

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by EMNLP 2024, a.k.a. "Jellyfish: Instruction-Tuning Local Large Language Models for Data Preprocessing''

2410.18976 2024-10-25 cs.CV cs.AI cs.CL cs.CY cs.LG 67%

CAMEL-Bench: A Comprehensive Arabic LMM Benchmark

Sara Ghaboura, Ahmed Heakl, Omkar Thawakar, Ali Alharthi, Ines Riahi, Abduljalil Saif, Jorma Laaksonen, Fahad S. Khan, Salman Khan, Rao M. Anwer

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 10 pages, 5 figures, NAACL

2406.12950 2024-10-21 q-bio.QM cs.AI cs.CE cs.CL cs.LG 67%

MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction

Yuyan Liu, Sirui Ding, Sheng Zhou, Wenqi Fan, Qiaoyu Tan

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2410.10818 2024-10-16 cs.CV cs.AI cs.CL cs.LG 67%

TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models

Mu Cai, Reuben Tan, Jianrui Zhang, Bocheng Zou, Kai Zhang, Feng Yao, Fangrui Zhu, Jing Gu, Yiwu Zhong, Yuzhang Shang, Yao Dou, Jaden Park, Jianfeng Gao, Yong Jae Lee, Jianwei Yang

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Project Page: https://temporalbench.github.io/

2406.19292 2024-10-15 cs.LG cs.AI cs.CL 67%

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data

Zheyang Xiong, Vasilis Papageorgiou, Kangwook Lee, Dimitris Papailiopoulos

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2405.06001 2024-10-10 cs.LG cs.AI cs.CL 67%

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit

Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Xianglong Liu, Dacheng Tao

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by EMNLP 2024 Industry Track

2410.03608 2024-10-07 cs.AI cs.CL cs.HC cs.LG 67%

TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation

Jonathan Cook, Tim Rocktäschel, Jakob Foerster, Dennis Aumiller, Alex Wang

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

2409.18216 2024-09-30 cs.AI cs.CL cs.LG 67%

MMMT-IF: A Challenging Multimodal Multi-Turn Instruction Following Benchmark

Elliot L. Epstein, Kaisheng Yao, Jing Li, Xinyi Bai, Hamid Palangi

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 24 pages, 16 figures

2409.12369 2024-09-20 cs.SE 67%

Program Slicing in the Era of Large Language Models

Kimya Khakzad Shahandashti, Mohammad Mahdi Mohajer, Alvine Boaye Belle, Song Wang, Hadi Hemmati

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)

2407.14944 2024-09-11 cs.CV 67%

Automatic Generation of Fashion Images using Prompting in Generative Machine Learning Models

Georgia Argyrou, Angeliki Dimitriou, Maria Lymperaiou, Giorgos Filandrianos, Giorgos Stamou

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)

Journal ref ECCVW 2024

2403.07832 2024-09-10 cs.RO 67%

DeliGrasp: Inferring Object Properties with LLMs for Adaptive Grasp Policies

William Xie, Maria Valentini, Jensen Lavering, Nikolaus Correll

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)

2408.16966 2024-09-09 cs.LG cs.AI cs.CL 67%

UserSumBench: A Benchmark Framework for Evaluating User Summarization Approaches

Chao Wang, Neo Wu, Lin Ning, Jiaxing Wu, Luyang Liu, Jun Xie, Shawn O'Banion, Bradley Green

专题命中推理评测：verifier(abstract)；分类 cs.CL、cs.AI、cs.LG

2409.03563 2024-09-06 cs.CL cs.AI cs.LG 67%

100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instances

Lorenzo Pacchiardi, Lucy G. Cheke, José Hernández-Orallo

专题命中推理评测：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Presented at the 2024 KDD workshop on Evaluation and Trustworthiness of Generative AI Models

1. 推理评测 10451 篇

SedarEval: Automated Evaluation using Self-Adaptive Rubrics

TimeLogic: A Temporal Logic Benchmark for Video QA

OneLLM: One Framework to Align All Modalities with Language

Enhancing Financial VQA in Vision Language Models using Intermediate Structured Representations

MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes

ADC: Enhancing Function Calling Via Adversarial Datasets and Code Line-Level Feedback

DavIR: Data Selection via Implicit Reward for Large Language Models

EACO: Enhancing Alignment in Multimodal LLMs via Critical Observation

Beyond Retrieval: Generating Narratives in Conversational Recommender Systems

Does Few-Shot Learning Help LLM Performance in Code Synthesis?

Can Large Language Models Serve as Evaluators for Code Summarization?

MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities

Long-form factuality in large language models

Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning

Leveraging Vision-Language Models for Manufacturing Feature Recognition in CAD Designs

Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming

CACTUS: Chemistry Agent Connecting Tool-Usage to Science

Jellyfish: A Large Language Model for Data Preprocessing

CAMEL-Bench: A Comprehensive Arabic LMM Benchmark

MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction

TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit

TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation

MMMT-IF: A Challenging Multimodal Multi-Turn Instruction Following Benchmark

Program Slicing in the Era of Large Language Models

Automatic Generation of Fashion Images using Prompting in Generative Machine Learning Models

DeliGrasp: Inferring Object Properties with LLMs for Adaptive Grasp Policies

UserSumBench: A Benchmark Framework for Evaluating User Summarization Approaches

100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instances